[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4586: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4588: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4589: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4590: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
AI Challenge Forums • View topic - Why you should A/B test your bot, and how to do it

It is currently Thu Jun 21, 2018 9:47 pm Advanced search

Why you should A/B test your bot, and how to do it

Share and discuss ideas for your entries here.

Why you should A/B test your bot, and how to do it

Postby cobracom » Sat Nov 13, 2010 11:58 am

cobracom
Lieutenant
 
Posts: 14
Joined: Fri Sep 10, 2010 3:46 pm

Re: Why you should A/B test your bot, and how to do it

Postby lavalamp » Sat Nov 13, 2010 12:04 pm

I run two or more bots concurrently on the tcp server for this.
lavalamp
Colonel
 
Posts: 56
Joined: Wed Oct 06, 2010 9:10 pm

Re: Why you should A/B test your bot, and how to do it

Postby Mistmanov » Sat Nov 13, 2010 12:33 pm

Wow, that's pretty awesome. =)

I've seen many people run multiple bots on the TCP server, and I think it may by a more direct method of getting results (also because you simply play more games on the TCP server..), but I don't know how to run multiple instances of the tcp client on 1 pc -_-

But your solution is certainly a lot cooler than just running 2 bots. :D
Mistmanov
Colonel
 
Posts: 70
Joined: Fri Sep 24, 2010 6:50 pm

Re: Why you should A/B test your bot, and how to do it

Postby cobracom » Sat Nov 13, 2010 12:55 pm

I forgot to mention that I use a very similar technique to find my weak spots, for example whether my ELO score is lower or higher when the starting positions are close in comparison to when try are far away.
cobracom
Lieutenant
 
Posts: 14
Joined: Fri Sep 10, 2010 3:46 pm

Re: Why you should A/B test your bot, and how to do it

Postby iouri_ » Sun Nov 14, 2010 3:37 am

iouri_
Brigadier-General
 
Posts: 105
Joined: Thu Feb 11, 2010 4:16 pm
Location: Toronto, Canada

Re: Why you should A/B test your bot, and how to do it

Postby Celica » Sun Nov 14, 2010 6:46 am

Use a genetic algorithm to randomly mutate the constants.

I have made one, approx 100 lines of Python code.
It boosted my bot with some 200 rating points (after 14 hours of computation).
More to come!

I consider publishing it, but it may affect my rank negatively :-)
Celica
Lieutenant
 
Posts: 11
Joined: Fri Sep 10, 2010 9:52 am

Re: Why you should A/B test your bot, and how to do it

Postby cobracom » Sun Nov 14, 2010 7:56 am

@celica - yeah, I have used a genetic algorithm as well, but the important issue is how do you score each genome performance?
cobracom
Lieutenant
 
Posts: 14
Joined: Fri Sep 10, 2010 3:46 pm

Re: Why you should A/B test your bot, and how to do it

Postby Celica » Sun Nov 14, 2010 3:24 pm

Celica
Lieutenant
 
Posts: 11
Joined: Fri Sep 10, 2010 9:52 am

Re: Why you should A/B test your bot, and how to do it

Postby Mistmanov » Sun Nov 14, 2010 4:14 pm

Mistmanov
Colonel
 
Posts: 70
Joined: Fri Sep 24, 2010 6:50 pm

Re: Why you should A/B test your bot, and how to do it

Postby cobracom » Sun Nov 14, 2010 5:25 pm

yeah, had a similar experience - use GA to evolve some parameters, but when I uploaded the winning bot to the server, it didn't handle well real competition.
A/B testing works for me, especially now that a bot gets the required number of games in about 24h.
cobracom
Lieutenant
 
Posts: 14
Joined: Fri Sep 10, 2010 3:46 pm

Next

Return to Strategy

Who is online

Users browsing this forum: No registered users and 1 guest

cron