Thursday, December 14, 2017

AlphaZero skepticism

Last week the Chess world learned of a new top dog, Deepmind's AI AlphaZero. It had bested (an older version) of Stockfish by the astonishing score of 28 wins, no loses, and 72 draws in a 100 game match. Two of the games looked like instant classics.

But some doubts started emerging. I didn't read the stories too closely, but I heard a few things via Twitter. Also, when I got to the club last Saturday, Paul had been looking at some of the games and had wondered why Stockfish had embarked on a risky looking piece sacrifice right out of the opening.

Here Stockfish 8 played 13 Ncxe5 fxe5 14 Nxe5, and eventually AlphaZero's extra material prevailed. Paul had found this perplexing and eventually stopped looking at the games. When Paul and I started discussing it, I had an additional piece of information, namely that the time control for the games had been one minute per move. (It seems that 13 Qc3 or 13 Be3 are both more sound.)

Anyone that has played around with computer programs has seen that sometimes a program will look at one move for quite some time before switching to another, better move. If the program is constrained by a time limit, however, it might choose a lone that it would later toss. That seems to be what happened here.

(Lars Bo Hansen tweeted:
My initial thought was "this cannot be good for White, my engine will easily refute White's play". Then I realized that the engine I wanted to consult was the one being crushed...Can't help feeling some concern for the future of mankind.
Turns out perhaps he should have tried it, and just let the engine "cook" a little bit longer that a minute!)

As it turns out, a fair number of people have expressed skepticism about the match, including someone that knows both AI and chess, IM Jose Camacho Collados. His article can be found here, and I recommend it to anyone interested in the topic. He covers several issues, including the time limit problem and hardware issues. Best of all, he includes links to other relevant articles in his footnotes. Check it out!

No comments:

Post a Comment