[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 483: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/bbcode.php on line 112: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4586: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4588: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4589: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
[phpBB Debug] PHP Warning: in file /includes/functions.php on line 4590: Cannot modify header information - headers already sent by (output started at /includes/functions.php:3765)
AI Challenge Forums • View topic - Final and timeouts

It is currently Tue Oct 23, 2018 9:27 pm Advanced search

Final and timeouts

Questions and discussions about the current Ant Game Rules.

Re: Final and timeouts

Postby ChrisH » Thu Dec 22, 2011 7:56 pm

lama3oid, you may have a legitimate gripe, or you may not, I don't have enough information to say for sure, but here are some additional things to take into consideration:

If there are certain kinds of games or maps on which you seem to timeout more often than others, then perhaps those situations cause your bot to execute some of it's code more than the situations where you don't timeout. Taking an example from my own bot, I know that when it gets involved in a lot of combat situations and I have a lot of other ants as well that the combat code will take up so much time that my path finding may not find a path for all the other ants. I spent a lot of time (maybe a couple of weeks) trying to make my code degrade gracefully and not timeout in those situations. But those situations only come up when the map has a lot of food and I'm playing against other bots that are really good at combat.

Even with all of that work on my code to make sure it doesn't timeout, it still does sometimes and those are probably caused by one of two reasons, either it has run out of memory (I have tried to avoid that as well, but my bot can be a memory hog when resolving really big battles, and my chosen language is garbage collected, so I can't control it completely), or because of a processing glitch on the server, which I understand does occasionally happen. Janzert has gathered statistics that indicate that top bots timeout roughly 1.5% of the time on the servers, which we can take as a baseline for server glitch frequency.

So just because your timeouts come in clumps, that doesn't mean they are caused by server problems. There are so many variables involved in this analysis that it is difficult to establish the root cause without knowing more information, but I would say you can only blame 1.5% of your timeouts on the servers, and the rest are due to something in your code, even if you haven't figured out what that is yet.
ChrisH
Colonel
 
Posts: 57
Joined: Tue Nov 30, 2010 8:54 pm

Re: Final and timeouts

Postby Zaph » Thu Dec 22, 2011 8:19 pm

Zaph
Colonel
 
Posts: 78
Joined: Sun Sep 05, 2010 9:00 pm
Location: Melbourne, Australia

Re: Final and timeouts

Postby lama3oid » Thu Dec 22, 2011 9:20 pm

ChrisH
Of course, everything you said makes sense, but I thought about that and didn't find the explaination.
for example, here:
http://aichallenge.org/visualizer.php?g ... 9&user=453
couldn't be any path-finding issue, I think this is clear.
but I timed out.
Let's say, this is because of the big number of battles (actually, it's not big, but let's assume that).
Then why I didn't time out here ?
http://aichallenge.org/visualizer.php?g ... 0&user=453
and why I did here ?
http://aichallenge.org/visualizer.php?g ... 2&user=453
let's say, in last game something happened with pathfinding. then what was that ?
http://aichallenge.org/visualizer.php?g ... 3&turn=296

and as for the assumptions: my pathfinding exits after max_num_of_steps*30 iterrations. example from log of how much time it's taking:
AStar => Iterrations limit reached. max way length: 30, iterrations: 901 time taken: 4
and I have very few places where max way length is more than 30. plus A* is very fast on open maps.

Yesterday I talked to amstan, so I already know that nothing's going to be done, but it's really disappointing. Think, I would finish at rank about 30-40, but now it would be good if I take at least #70.

Zaph
Just saw your post.
Of course, if I would know, that I'll face this problem, I'd use bigger threshold... 100ms or how much it would be needed. but even if this problem would appear before finals, I had no chance to see it since you all know about the game rate in these days. during last weeks I'd need to have my bot running on the server at least for a week just to get to the high rank battles and have at least few, so I would lose a lot of time that I finally spent for coding.
I did whatever I could to test it - played a lot of battles locally (with 8-10 same bots), in last few days tested on tcp servers (found them just before the finals, would have much smarter bot if I knew from the beginning :)
and additionally, I tried to avoid big performance-related code changes, since the old versions. but again - have no way to test it on live servers. But the problem is that I don't see any regularity here. I have many-ants games with 7-8 opponents without timeouts and then timming out in duel at 40+ ants. ant that happens on any type of map. and what's more strange, as I mentioned before, that timeouts seem to be somehow grouped.

By the way, thanks, guys, for your answers. Good to have somebody to discuss with and to hear other opinions.
lama3oid
Captain
 
Posts: 21
Joined: Wed Dec 07, 2011 7:20 am

Re: Final and timeouts

Postby Zaph » Thu Dec 22, 2011 10:28 pm

Zaph
Colonel
 
Posts: 78
Joined: Sun Sep 05, 2010 9:00 pm
Location: Melbourne, Australia

Re: Final and timeouts

Postby ChrisH » Thu Dec 22, 2011 10:46 pm

For the record, my bot starts a game with a 40ms time allowance for submitting all its orders. If it has all the orders ready before then it submits right away, but otherwise it will stop as soon as it sees the clock go past the turntime - 40ms.

But for extra safety it also records how much over the time allowance it actually went and adds that amount to its time allowance for the rest of the game. So, assuming the turn time is 500ms then if it took 465ms to submit orders on one turn, then it would stop after 455ms from then on. Then if it took 457ms on a later turn it would try to stop after 453ms from then on.

This method works well for dynamically dealing with the growth of ant population and also hard to know things like the overhead implied by the frequency of my time checks and delays due to garbage collection.

So if my bot times out it is either because it crashed due to running out of memory, the GC took longer than 40ms right at the end of a turn (which I don't think happens very often, maybe never, but I don't know that), or there is a server glitch that fails to schedule my bot's process in time for me to wake up and see the time running out before submitting orders.

And so far in finals it has timed out once in 51 games, not too far off the 1.5% rate of other top bots.
ChrisH
Colonel
 
Posts: 57
Joined: Tue Nov 30, 2010 8:54 pm

Re: Final and timeouts

Postby romans01 » Thu Dec 22, 2011 11:31 pm

It looks like different servers produce different results for the same bots. My bot has a lot of internal timeouts. I see it on its behavior.

Anybody knows, are the servers have the same CPU/RAM and etc. hardware?
romans01
Major
 
Posts: 31
Joined: Thu Oct 27, 2011 11:23 am

Re: Final and timeouts

Postby fourmidable » Thu Dec 22, 2011 11:54 pm

Personally, I've had a few episodes of timeouts with fourmidable. As it is a Java program, I initially blamed garbage collections, but careful debugging eventually found corner cases that would cause excessive time in each case. Note that I use a 125ms (25%) margin for submitting orders, which may explain why other have different experiences. I'd like to report that after careful analysis, I have zero unexplained timeout after running for two months, so from my perspective the servers have been impeccable, and I'd like to thank the whole organization for running things smoothly.

I also sympathize with lama3oid for having work hard to get a competitive bot, and have it under perform in the finals.
fourmidable
Cadet
 
Posts: 8
Joined: Sun Oct 30, 2011 11:55 am

Re: Final and timeouts

Postby qgazq » Fri Dec 23, 2011 12:19 am

Just my 2p, so far I've had a couple of timeouts which have been attributable to my bot.
I don't think I've had any in the finals.
I do however also leave quite a big margin, I'll break out at 100ms.
My bot is also pretty stupid so a lot of the time it doesn't even get to that limit.
However it only does the breaking on its exploring, code, the food and hill attack don't have any way of breaking.

So my conclusion is the servers can't be stealing _too_ much time, there might be 100ms here or there (which ok is quite a lot), but not enough that it can't be guarded against.
But I guess if you expect 500ms you should probably get most of it as runnable time. Would have been interesting if someone wrote a "time" bot which collected food, but would only move 1 ant per 10ms or similar so it would be easy to see how it varied. Obviously this would have to be coded in something without gc which didn't introduce its own random delays.

One odd one I did have earlier was that it "crashed" on the first turn.
Now given that its java I have no idea how it achieved that, surely it would just get an exception if anything which would cause a time-out. Unless crash can also mean malformed output?
I ran the first turn through it locally and nothing out the ordinary it gave the one ant a command as expected.
Didn't effect my ranking or anything, just odd.

Regards
QGazQ
qgazq
Cadet
 
Posts: 6
Joined: Thu Dec 22, 2011 10:12 am

Re: Final and timeouts

Postby McLeopold » Fri Dec 23, 2011 5:46 pm

After running some stats, I think both the python and java bots are having a hard time dealing with garbage collection.

These two languages might also be high in timeouts because the tutorial (with poor performing algorithms) was only written in java and python.

Even the top java bots have more trouble. Although, the top 2 bots are java, and 5 of the top 10 are java, so isn't an "insurmountable problem".
McLeopold
Contest Organizer
 
Posts: 262
Joined: Sun Sep 19, 2010 3:31 am

Re: Final and timeouts

Postby bluegaspode » Fri Dec 23, 2011 7:39 pm

Can you also check, if timeouts happen rather in turn 1+2 (for the better bots that implement time checking) or are they evenly spread?

Personally I only had trouble in turn #1 before the finals, never after.
I did many timings to find out the issue and it was not the map parsing in my code (I did use an optimized parser though) but everything else - even simple BFS for the visible area was factors slower then on all subsequent turns.
So I rather blame the JIT that when running code the first times compiles and optimizes it on the fly which makes a freshly started Java program always slow and then faster and faster.

I also think people shouldn't alway just blame the GC.
First of all: generational GC in Java is highly optimized. Allocating a lot of memory and releasing it the same turn actually does not need much time at all, because in a minor collection Java just throws away the Eden space and copies the rare survivors to the old generation (after some time).
In Ants I think there are either short lived objects per turn or those that live until the end of the program. You cannot have an easier and more optimized situation for the GC. Calling "free" in C for many objects will take more time than a Java GC in these circumstances.

Second: if one really wants to blame the GC it's easy as one just needs to fire up "VisualVM" - a tool that comes with every JDK - which will show you exactly how often and how long the GC pauses your program on average.
(It comes with a profiler as well by the way, so as a side effect one can find the spots that really matter).

Learn to use the tools available to you!

I implemented my cutoff after 400ms and now never timed out in the finals.
bluegaspode
Colonel
 
Posts: 51
Joined: Mon Nov 07, 2011 8:38 am

PreviousNext

Return to Game Specifications

Who is online

Users browsing this forum: No registered users and 0 guests

cron