[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4586: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4588: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4589: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4590: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
AI Challenge Forums • View topic - Scoring system

It is currently Thu Jun 21, 2018 6:18 pm Advanced search

Scoring system

Share and discuss ideas for your entries here.

Re: Scoring system

Postby a1k0n » Wed Feb 17, 2010 10:53 pm

Right now I'll settle for just removing the 'U' map from the rotation. PLEASE.
a1k0n
Colonel
 
Posts: 90
Joined: Fri Feb 12, 2010 3:51 am

Re: Scoring system

Postby barxool » Wed Feb 17, 2010 11:08 pm

Why ?

Draws occurs on a lot of maps when top bots are playing now.
barxool
Lieutenant
 
Posts: 13
Joined: Wed Feb 10, 2010 9:22 pm

Re: Scoring system

Postby a1k0n » Wed Feb 17, 2010 11:38 pm

Yeah, but that one even the dumbest bots can force a draw on. With 'U', 'ring', and to a lesser extent 'toronto' the optimal strategy is "crash into your opponent in the center of the map". Draws are inevitable in the top half of the currently playing bots, and they hurt your ranking too much.

I dunno, maybe there should be a second tier of harder maps for the top 100? At the very least, maps with the center square blocked out.
a1k0n
Colonel
 
Posts: 90
Joined: Fri Feb 12, 2010 3:51 am

Re: Scoring system

Postby GarySWest » Thu Feb 18, 2010 12:24 am

GarySWest
Major
 
Posts: 34
Joined: Mon Feb 15, 2010 3:45 pm

Re: Scoring system

Postby jeff.cameron » Thu Feb 18, 2010 6:06 am

I think this is a great idea. I especially like the idea of using an ELO system. Perhaps I will implement this in the coming days. Anybody interested in making it happen faster by implementing it for me? Java is preferable, but pretty much any language works.

When we opened the contest a few weeks ago, we actually started off with a pairing algorithm that selected bots of similar strength, and a ranking algorithm based on the famous PageRank algorithm. However, it had some subtle quirks and we were getting complaints because it wasn't as clear why you had the ranking you did. We axed it fairly early on in favor of the current ranking system, which is totally straightforward and easy to understand.

Now we may have to do better, since the current ranking system is having trouble separating the top ten entries. What if we put up some alternate non-default leaderboards that use other ranking algorithms like ELO or my failed PageRank-based algorithm? If one of these leaderboards ends up being popular, we could even make it the new official leaderboard. Remember, you can help make this happen faster by writing the code for us! :-)
jeff.cameron
Contest Organizer
 
Posts: 91
Joined: Sun Jan 31, 2010 4:06 am

Re: Scoring system

Postby Janzert » Thu Feb 18, 2010 8:13 am

As mentioned I think would be a great, easy to implement way to do this. Bayeselo is already pretty well regarded by both the go and chess programming communities. It's now used as the more accurate rating system on , the primary place for computer vs computer go internet games.

Do you have a sample somewhere giving some input files and output desired?

Janzert
Janzert
Contest Organizer
 
Posts: 271
Joined: Sun Feb 07, 2010 1:59 am

Re: Scoring system

Postby jeff.cameron » Fri Feb 19, 2010 5:54 am

The input is a list of game records. Each game record has two player_id's, a winner, a loser, a flag to say if the game was drawn, and whatever other data you may need. The output is an ordered ranking of the player_id's, with optional other data, such as ranking scores, comments, win-loss-draw counts, etc.

I am just playing around with bayeselo. This is so cool! I am definitely going to use this with my own chess program to summarize the results of tournaments played with xboard/winboard. I will try to produce some sample rankings using actual game data from the contest, and see whether or not it makes sense. Hopefully I can get an ELO leaderboard up-and-running soon.

If it goes well, maybe we can try switching over the main leaderboard to an ELO-based system. This will only happen if it is really solid and is unlikely to generate a barrage of complaints. If we do this, we could even switch to a swiss-style pairing algorithm that tries to match players of similar strength. That way you get to see more games against bots of similar skill!

Anyone know of any other ready-to-go implementations of common ranking algorithms like glicko?
jeff.cameron
Contest Organizer
 
Posts: 91
Joined: Sun Jan 31, 2010 4:06 am

Re: Scoring system

Postby a1k0n » Fri Feb 19, 2010 2:31 pm

You could also use swiss-style pairing using the current score formula, since your rating is exactly the same as the original swiss points system. The initial matches on a new submission are the hard problem to figure out, but I guess you could binary search for the first eight games or so and then switch to a +/- 10 neighborhood. That would be least likely to generate a barrage of complaints, except to be fair you'd need to deprecate most of the existing games after switching.

I'm curious: does your challenging system constantly pick random pairings, or does it just happen when you submit something? Is there a bank of machines running games from a queue?
a1k0n
Colonel
 
Posts: 90
Joined: Fri Feb 12, 2010 3:51 am

Re: Scoring system

Postby Fritzlein » Fri Feb 19, 2010 3:14 pm

Fritzlein
Colonel
 
Posts: 81
Joined: Thu Feb 18, 2010 9:20 pm

Re: Scoring system

Postby Janzert » Fri Feb 19, 2010 4:18 pm

Janzert
Contest Organizer
 
Posts: 271
Joined: Sun Feb 07, 2010 1:59 am

PreviousNext

Return to Strategy

Who is online

Users browsing this forum: No registered users and 0 guests

cron