I would go about this with an Elo-like model where each players' skill is normally distributed, and to get the probability that player A beats player B you plug the difference in skill into a logistic function. Working backwards from there, it should be possible to either find a closed-form posterior distribution, or a sampled approximation, for every bot's skill, given the entire game history (as long as the history only includes the current versions of every bot). Then you'd take the mean of each to do the ranking, but it'd be useful to know the standard deviation (or volatility) of the ranking as well. This is of course computationally expensive and requires a background in machine learning to implement, but this *is* an AI contest.
Statistics: Posted by a1k0n — Sun Feb 14, 2010 8:44 am