Take Judges Out of Hackathons

I was at a local hackathon last week. In the opening ceremony, previously winning ideas were presented in order to give us, the hackers, a general direction to work in. The next day the judges, who had only just arrived for our presentations, voted for the EXACT SAME idea that we had all been shown from that winning list. The hackers clapped and congratulated the team as any good sport does. In fact no one even mentioned the elephant in the room; the team had stolen a winning idea and re-implemented it as their own. The ironic part is that they were smart to do it. It did win them 1st place after all.

Winning ideas are recycled and daring ideas are ignored, at least by the judges. It’s a common story. In fact, this is the steady-state of modern hackathon economics. It goes something like this:

A hackathon usually begins with eager optimism. Cool workstations are propping up, people are mingling and teams are forming. At this point I’m usually very excited to see what will be built in the next 24 hours, and how I can be a part of it.

Then the hacking onsets. There is less chatter, more focus. Ideas turn into code like caffeine into energy. But as the caffeine wears off and the sun goes down, realizations kick in among the hackers. The design won’t work, it’s already been done, the scope is too broad. This is a beautiful and necessary stage in the materialization of any idea into reality. Big decisions are made here that will reflect the true values of the team. It’s also where projects tend to go south in a hackathon. Some common solutions I see teams make are to take out the backend (the functional bits), fake some numbers, or steal an existing ideas and call it your own.

By presentation time, I’m often embarrassed by my own project. Not because I didn’t finish what I planned, but because I compromised the values I began the night with. The sort of “hacker commradery” sentiment is also gone and replaced with a quiet competitive anxiety. I’m sad to say that many times at hackathons, I don’t even see the other group’s projects because I’m too involved in my own. But I do watch the finales, where judges decide between a subset of projects that all too often have made the same compromises that I did that last night.

So what inclines a hacker to compromise their work and turn an inspired idea into a hollow shell? One might argue that competition is the problem, and that we should all be working together. But competition breeds a diversity of ideas. Similarly, the 24-hour time constraint may cause a hacker to compromise on their original vision, but as I pointed out before that is actually a positive feature of a hackathon. In my ideal experience, I would finish a hackathon feeling humbled but accomplished. Most importantly, I would feel like I had spent the last 24 hours for some cause and not just for myself and a new Nintendo Switch. I think many hackers feel the same way. We love the prizes but that’s not why we spend our weekends hacking together weird ideas.

That is why I propose that hackers should be the judges of each others creations. Unlike external judges, hackers knows which ideas are repeats of old hackathons. We’re also the most sympathetic toward the process, as we ourselves are part of the event. We can tell which projects took real work, and which ideas are worth keeping.

Certainly having the hackers judge each other would change the social dynamic of the event itself. In taking part, you may feel like you’re being judged the whole time, every moment, even at that compromising point… hm. We may be less inclined to cheat our neighboring hackers than we would some external judges we have never seen before. And our neighbors are much less easy to fool than those judges. So we may be less inclined to fake data and steal ideas. We may also be more inclined to socialize — reach out to other teams and get to know one another. Of course there is a potential threat for collusion if enough teams agree to vote for one another. This form of cheating may be easier in smaller hackathons, but becomes increasingly difficult as the size of the event grows.

Companies may wonder why they should sponsor a hackathon if they don’t get to judge the results. Often what companies want is for developers to use their products in order to cultivate a developer ecosystem. They also want novel ideas that are only accessible with that creative touch. In a peer-judged sponsor challenge, teams would have more inclination for creativity to impress their peers. They would also have more fun with the product than if they were all trying to match a standard template. In effect, companies may actually get more out of a sponsorship by relinquishing their judgement authority.

Of course this is a pretty radical proposition and I don’t expect any hackathon event planners to change their format after reading this article alone. Instead I hope the content here can spark some thought and discussion on how we might better design a hackathon to incentivize exploration over exploitation.

If you are a hackathon planner and you’re looking for a specific change you can make right now to enrich the culture at your event, try this —add a “hackers’ choice” award to the closing ceremony. No need to call off the judges just yet. Maybe you don’t even need a prize. If voting is sufficiently easy, hackers may do it for their own expression. In this simple modification there is nothing to be lost, only to be gained.

Hackathons have become wildly popular since I began college five years ago. I have seen the art perfected and scaled with companies clamoring to sponsor. Still the core pattern has only solidified in that time. In some ways hackathons are like the think-tanks of the 1960s, which challenged the status quo and allowed unconventional ideas to become fully realized. But just like the once free-spoken think-tanks became inextricably entangled with corporate agendas, so is the hackathon at risk of losing its original authenticity and the “hacker” spirit.

As an undergraduate, taking part in hackathons shaped my perception of what could be done with technology and how. These events are a shining reminder that our skills can be used for more than a steady paycheck. It is proof that we can realize our own visions and actually make impactful things in a surprisingly short amount of time; if we’re only brave enough to try. I do hope this message continues to come through in the following years as hackathons assume an even larger role in the technologist culture.