[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4586: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4588: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4589: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4590: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
AI Challenge Forums • View topic - ELO variability

It is currently Thu Jun 21, 2018 11:46 pm Advanced search

ELO variability

Code won't compile? Found a bug? Post here!

ELO variability

Postby dimkadimon » Tue Oct 26, 2010 12:41 am

Why is ELO jumping around so much? Last week the top bots had an ELO>4000 and the 100th bot was around 3400. Today the top bots have ELO<3800 and the 100th bot around 3200. Is there any reason for this? Is it because servers are slow and a lot of the top-ranked people are deciding to participate on the TCP server instead, hence lowering the average ELO?
dimkadimon
Major-General
 
Posts: 263
Joined: Wed Oct 06, 2010 11:34 pm
Location: Adelaide, Australia

Re: ELO variability

Postby siconize » Tue Oct 26, 2010 10:23 am

I dont believe "people are deciding to participate on the TCP server instead": They participate on both servers.
siconize
Captain
 
Posts: 27
Joined: Wed Oct 20, 2010 9:48 pm

Re: ELO variability

Postby karpizin » Tue Oct 26, 2010 6:38 pm

less games with the same reuploading rate lead to the situation when top-ranked bot actually are on 1xxx-2xxx-3xxx positions when they are "climbing up"

in other words, when you reupload your bot, your elo is gone and your opponents instead of getting much elo on winning you and less elo losing you actually get little elo winning you and negative elo losing you
karpizin
Captain
 
Posts: 24
Joined: Mon Sep 13, 2010 8:20 pm

Re: ELO variability

Postby Demerzel » Tue Oct 26, 2010 7:58 pm

Demerzel
Colonel
 
Posts: 72
Joined: Sat Oct 02, 2010 6:06 am
Location: themendios at gee mail

Re: ELO variability

Postby dmj » Tue Oct 26, 2010 8:03 pm

I believe, but am not certain, that the rating system is only given the games where each player has their most recent submission. So, when someone resubmits, those games that you won/lost against them do not exist any more for rating purposes. I think that is a big part of why the ratings change, even on days when there are only 5 games a day per player.... every resubmit causes many changes in the history.

It may be better, if impractical, to keep all submissions in the rating scheme. *But*, each submission is rated separately, and a person's "official" rating is based only on their latest entry.

Of course, I don't know if this is really how it works, and may be completely wrong.
dmj
Lieutenant-Colonel
 
Posts: 41
Joined: Thu Feb 11, 2010 11:36 am

Re: ELO variability

Postby siconize » Tue Oct 26, 2010 8:41 pm

siconize
Captain
 
Posts: 27
Joined: Wed Oct 20, 2010 9:48 pm

Re: ELO variability

Postby dimkadimon » Wed Oct 27, 2010 2:24 am

Another thing that really bothers me is that ELO can vary significantly (up to 200 points) for the same bot within a small amount of time. All it takes is a few (un)lucky streaks of wins (losses). This happens even for bots that have been running for weeks. Shouldn't the ELO change significantly less if a bot has been running for a long time?

So my question to everyone: how do you measure the strength of your bot if ELO is so unreliable? Personally I test against example bots and previous versions of my bot. Testing against example bots has very limited value for me, because I beat them always (except a few maps against the RageBot). Testing against my previous bots is very unreliable, because they have very few differences between them. So a small change can deceivingly look like a big increase in performance. Perhaps rank is a more reliable measure of performance. The problem with rank though is that the number of bots is constantly increasing. So a bot ranked in top 100 before is probably now in top 300. Perhaps it is better to convert rank to percentile, ie top K% of all bots.
dimkadimon
Major-General
 
Posts: 263
Joined: Wed Oct 06, 2010 11:34 pm
Location: Adelaide, Australia


Return to Technical Issues

Who is online

Users browsing this forum: No registered users and 2 guests

cron