[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4586: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4588: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4589: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4590: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
AI Challenge Forums • View topic - Faulty TrueSkill calculations? (Sigma)

It is currently Tue Oct 23, 2018 12:54 pm Advanced search

Faulty TrueSkill calculations? (Sigma)

Topics about starter packages, visualizer or any other third party tools.
Please submit new language requests in the Language Request Forum.

Faulty TrueSkill calculations? (Sigma)

Postby skuto » Mon Nov 07, 2011 10:53 am

Hi,

I suspect the TrueSkill calculations are faulty, at least on the TCP servers. As far as I understand, sigma is supposed to decrease gradually as more games (information about strength) are available, and perhaps increase slightly for unusual results. But it seems to increase too much too often.

My own bot got its sigma down to +3.2 after about 50 games. It's now 450 games further and sigma is still at 3.2. Sometimes it even goes back up to 3.4 or thereabouts. This suggests that after 400 more games we are now less certain of its strength than after 50. This does not make sense in any rating model.

You can see the same behavior in some of the top bots:
http://ants.fluxid.pl/ranking

"A" has thousands of games and a sigma of 3.7! It should be ranked much higher if not for the "penalty" of the sigma that keeps stuck. Same for "graph14". It would be a top bot if its sigma wasn't stuck so high.
skuto
Captain
 
Posts: 21
Joined: Mon Oct 24, 2011 6:49 pm

Re: Faulty TrueSkill calculations? (Sigma)

Postby BenJackson » Mon Nov 07, 2011 9:06 pm

Part of the problem on fluxid is that the server can't choose which games to play. I leave a bot going on there all the time for the benefit of the community. So do people like "A". In idle periods I might get beaten 100 times in a row by A (well, we probably trade a bit). I've also seen evolutions of bots where I've been the only real competition and it's clear that some new player is is refining his bot based on results of playing me. So my bot is at 169th on fluxid, which is pretty crazy because there aren't 169 active bots and very similar code is on the aichallenge site at position 21. At least 95% of the bots ranked above it are "dead".
BenJackson
Colonel
 
Posts: 94
Joined: Sat Oct 29, 2011 4:16 am

Re: Faulty TrueSkill calculations? (Sigma)

Postby dshawul » Mon Nov 07, 2011 9:35 pm

I would also like to thank "A" and yourself "Benjackson" for leaving your bots connected all the time. They helped me tune
my bot's weights but still far from matching A's attacks though.

As to the ratings on the server , I don't pay much attention to it. Because it seems ratings go up and down significantly just after one game even though you played a thousand games before that should help stabilize your rating. Multi-player games are also a problem because for n-players you have to play n! games to avoid advantages due to hill placement or other factors. For example if my bot's hill is placed anywhere near A's , it comes out last more often than not. So there is some luck involded similar to white's first mover advantage as in chess. Having said that I agree the servers rating system needs a bit of work.
dshawul
Lieutenant
 
Posts: 17
Joined: Tue Oct 25, 2011 10:44 am

Re: Faulty TrueSkill calculations? (Sigma)

Postby skuto » Tue Nov 08, 2011 6:22 am

skuto
Captain
 
Posts: 21
Joined: Mon Oct 24, 2011 6:49 pm

Re: Faulty TrueSkill calculations? (Sigma)

Postby BenJackson » Tue Nov 08, 2011 7:06 am

BenJackson
Colonel
 
Posts: 94
Joined: Sat Oct 29, 2011 4:16 am

Re: Faulty TrueSkill calculations? (Sigma)

Postby skuto » Tue Nov 15, 2011 9:05 pm

skuto
Captain
 
Posts: 21
Joined: Mon Oct 24, 2011 6:49 pm


Return to Starter Packages & Tools

Who is online

Users browsing this forum: No registered users and 2 guests

cron