Neural networks, anyone?

[Darhuuk]

Hm, only 50% match. Maybe I should have tried with my 75% matching after all . By the way, why didn't you train to reach the "optimal" outcome, instead of training to get the same outcome as a bot?

[DrClaes]

One word: time!

I did match 60% overall, though, not 50%

/C

[DrClaes]

Also, about "optimal play", I don't think that can be easily expressed on a per-ant basis: the optimal play for a group of ants might be to move towards configuration A 50% of the time and configuration B 50% of the time; if each ant were given a 50/50 chance of moving to their A or B position, the emergent result of that would not be the desired one...

I think my choice of a conservative and strong bot (xathis) as training data is reasonable and provides a fair approximation of locally optimal play.

And, again, I'm pleased that some reasonably complex behaviors actually do emerge from the local applications of the neural network...

[pkmiec]

I'm pretty clueless when it comes to neural nets (especially how to practically code them). I found some material online to help me get started: http://www.heatonresearch.com/online/in ... -edition-2.

Are there other good resources that demonstrate how to code neural nets?

DrClaes: One question I have is how did you know that a single hidden layer with 50-100 nodes was sufficient for this type of a problem? What would be the advantage / disadvantage of two or three hidden layers?

[DrClaes]

Pkmiec, I took the Stanford (free) machine learning course this semester, and recommend it to anyone interested in some up-to-date computer science with enough hands-on work to apply to your own problems. The course will run again after new year, at http://jan2012.ml-class.org/ and the neural network lectures and homework will give you the tools you need to train a neural network.

As for the application/implementation, it turned out that there are several matrix/linear algebra packages to choose from for most big programming languages. I program in Java and this was quite small matrix multiplication, so I went with the first one I tried, "JAMA". The extraction/abstraction of training data from JSON files and training/validation took the majority of the time I spent on the neural network part of my bot. Actually implementing it into my bot program was a matter of a couple of hours of work, since the matrix package was very easy to use.

My network architecture choice is based on suggestions given by professor Ng at the neural network course as starting points for a neural network problem: one hidden layer and approximately the same number of nodes in that layer as there are input nodes (0.5-2 times the number of input nodes)... if I'd had the time I would have put in some effort to see what would happen with more layers, but with two kids at home it just wasn't realistic. As I've understood it, and as has been mentioned previously on this thread, multiple layers can be more efficient in terms of computation, but can also take longer to converge during training (back-propagation).

[Learnbot]

Learnbot is controlled by trained neural networks. We got a late start, so our training was buggy, and the nets only got a few days of reinforcement learning before the submission deadline, and plus there were all kinds of fascinating research areas we are still sorting out. Still, it appears we will finish at roughly 90 percentile, which isn't too bad for a buggy neural net learning to play. We are continuing to develop for at least a few weeks, and then will let it learn for as long as it appears to be continuing to learn, and hope to produce a very strong ants program before we are done, albeit a bit too late for this competition. If we succeed you will see us on tcpants or similar server at some point.

[agent_smith]

@LearnBot: it would be interesting to see how things unveil.

@pkmiec: I was going to say http://www.ml-class.org . There seems to be an earlier course also taught by prof. Ng (what a surname, BTW ) here:

http://academicearth.org/courses/machine-learning

P.S. Kmiec would be a Polish surname, wouldn't it?

[DrClaes]

@Learnbot: I'm glad to hear about someone else's results with neural networks. Can you elaborate on your approach to the problem? You mention reinforcement learning, so I gather you haven't trained your network to approximate some sort of "correct" / "desirable" play as I did, but rather let the crude network try its hand at playing the game and used reinforcement to strengthen behaviors which produced favorable outcomes.

What input is given to the network? Is it handled per-ant? Does it handle both foraging / exploring and combat situations? Does the network receive information about the map which isn't strictly visible to individual ants, like how the other ants have traveled, location of enemy hills that are out of sight, et cetera?

An example game where my neural net combat module produced some nice-looking walling-off and engulfing tactics is http://aichallenge.org/visualizer.php?g ... 1&user=396 ... I think that if I'd given my ants their entire visual field as input and trained on that, I would have seen more successful emergent behaviors of "pincer movements" when enemies are surrounded...

An observation from my side is that my approach with neural networks is computationally really hard while training the network (offline), using all of (one core at a time, unfortunately) my high-end desktop CPU for 24+ hours to train the net which is playing in the finals, but computationally really efficient while playing... a few thousand float multiplications are really easy for modern computers to do, apparently.

Regards / Claes