Roadmap for the next two contests

[jeff.cameron]

It's time to start developing the next contest! Before we dive headlong into the task of writing a new contest, I want to propose a long-term strategy.

Some contributors have proposed a major rewrite of the contest systems. Other have argued that we should carry on with the system that we have now and continue to incrementally improve it. I propose a middle ground: let's take a few months to create a fundamentally bigger and badder contest, and in the meantime run another contest using the system that we have, making as few changes as possible.

The Upcoming Contest (Codename Epsilon)
We will rely as much as possible on the system that we have already built. We will avoid creating new features unless absolutely necessary. This will irritate people and impair the growth of the community but it will pay off when we release the bigger badder contest.

The system that we have is pretty modular. To launch a new contest, we need only the following:

• a game engine for a two-player one-on-one game
• starter packs for the game
• content for the website
• game visualizer

There will be a fair bit of work involved in running the contest once it's live, as problems always creep up. We will need to take care of the web server and maintain game servers and such. However, the hope is that we can run the contest using very little manpower.

Most of all, we need to avoid developing new features wherever possible. Community members that want to help with development should be introduced to the development efforts for Zeta.

The Bigger Badder Contest (Codename Zeta)
We will focus our development efforts on Zeta, the next generation contest system. In this message I will deliberately not give many details about what Zeta is going to look like because they haven't yet been decided! However, here are a few of my speculations:

• Contest instances should be trivial to set up on an Ubuntu machine. This would make it way easier for community members to get into the development effort, as people could launch their own contest instances. Setting up your own contest instance should be easier than installing Wordpress.
• Use a web framework for the website. Consensus is currently leaning towards Django. Even if you're not the biggest Django fan, you've got to admit that it's better than what we have right now.
• Multi-contest support. Make the system support multiple contests at the same time. We would launch a new contest every couple months like we do now, but the old ones would remain available. Eventually we would end up with a lot of contests running side by side on the same website. This opens the possibility of having a global leaderboard that takes into account your scores in all the sub-contests. Imagine Project Euler except every problem is a different AI contest.
• Let users upload new game engines much like they submit their bots right now. We would allow users to create their own contests through the site. After creating a contest you could upload a game engine, add starter packs, add content and tutorials, create a visualizer, etc. The higher quality user-created contests will attract more users of their own, while the crappy ones will die out. We can make good user-created contests easy to find by ranking the contests according to number of submissions in the last month or something.
• Coding in the cloud. Allow people to work on their entries using an in-browser IDE. Of course we would still let people upload code from their own local computers the way it's done now. An in-browser IDE would make getting started much quicker for beginners. You create your account, click Submit, then the next page you see is the IDE with the starter package code already loaded. A beginner can go from casual browser to editing code in 15 seconds.

So I guess my roadmap is to launch a new contest as quickly as possible with minimum effort, then focus on building a more ambitious AI contest website.

[jmcarthur]

In my opinion, Zeta looks to be in the complete opposite direction from where I would like to see this go. If we have multiple concurrent contests at a time, that would really devalue each individual contest. If we allow users to create and upload their own games for contests, that would devalue each game even more. I'd rather the contest be something like a biannual thing so that we can build up some hype and have huge contests (and hopefully with prizes or something).

As for the in-browser IDE idea... I just don't see the point. I understand that it's intended for beginners, but it still seems like a lot of work for very little gain.

I wouldn't mind seeing a rewrite, since we need to clean things up and fix some architectural flaws and stuff, but I don't want to change the direction of the contest like this.

[Zannick]

I have to agree with jmcarthur that having multiple contests ongoing rather dilutes the excitement and competition. While we could have people submit game engines (as contributors do), I would suggest instead that for a given contest we choose by some method one of the engines that have been submitted, and run the contest around it. This way we can spend our time improving just one contest. If there are multiple ones going on, and each has something small wrong about it (bug in the engine, not enough starter packs), and there aren't enough contributors to fix them, users are not going to have fun.

I have to disagree with the idea of prizes for two major reasons: first, it detracts from the idea of friendly competition, when there is something tangible to gain; second, our international userbase means there may be legal issues involved with sending prizes. The latter by itself doesn't mean we can't, if we're careful, but that is why I emphasize the first again. There are other, minor reasons I'm sure others have discussed elsewhere I don't need to repeat.

The way I envision the backend rewrite does not preclude the ability to add new features to the first upcoming contest. Namely, if something is created for the rewrite and it works, transplant it into the old contest framework. I only have one example at the moment, and that's the use of test cases, something that is buried in pre-Planet Wars code and unused.