AlphaZero skepticism

Last week the Chess world learned of a new top dog, Deepmind's AI AlphaZero. It had bested (an older version) of Stockfish by the astonishing score of 28 wins, no loses, and 72 draws in a 100 game match. Two of the games looked like instant classics.

But some doubts started emerging. I didn't read the stories too closely, but I heard a few things via Twitter. Also, when I got to the club last Saturday, Paul had been looking at some of the games and had wondered why Stockfish had embarked on a risky looking piece sacrifice right out of the opening.

Here Stockfish 8 played 13 Ncxe5 fxe5 14 Nxe5, and eventually AlphaZero's extra material prevailed. Paul had found this perplexing and eventually stopped looking at the games. When Paul and I started discussing it, I had an additional piece of information, namely that the time control for the games had been one minute per move. (It seems that 13 Qc3 or 13 Be3 are both more sound.)

Anyone that has played around with computer programs has seen that sometimes a program will look at one move for quite some time before switching to another, better move. If the program is constrained by a time limit, however, it might choose a lone that it would later toss. That seems to be what happened here.

(Lars Bo Hansen tweeted:

My initial thought was "this cannot be good for White, my engine will easily refute White's play". Then I realized that the engine I wanted to consult was the one being crushed...Can't help feeling some concern for the future of mankind.

Turns out perhaps he should have tried it, and just let the engine "cook" a little bit longer that a minute!)

As it turns out, a fair number of people have expressed skepticism about the match, including someone that knows both AI and chess, IM Jose Camacho Collados. His article can be found here, and I recommend it to anyone interested in the topic. He covers several issues, including the time limit problem and hardware issues. Best of all, he includes links to other relevant articles in his footnotes. Check it out!

No Country for Old Men

GO players have been sent reeling in the last year. First came the AlphaGo program publicly beating one of the best GO players in the world early in the year. At the end of the year, that same program, temporarily disguised, tore through the rest of the GO community, winning 60 of 61 games online against all comers, including the best in the world. (The other game was drawn because of an internet connection failure.) Quote:

“Master” also claimed victories against a number of top Go pros including South Korea’s Park Jung-hwan and Japan’s Iyama Yuta, as well as beating China’s Ke Jie, who is currently ranked world number one, twice.

“When facing it, all traditional tactics are wrong,” commented Ke Jie after his defeat. Ke Jie had stated in December that he is currently not good enough to defeat AlphaGo.

GO had resisted the efforts of programmers for a very long time, unlike Chess, in which the programmers saw incremental improvement until they had surpassed human players. So we've had a long time to get used to this. But GO players are in shock, as AlphaGo seemed to come from nowhere to not only equal them, but surpass them by such a margin as to play at what appears to be a God-like level.

However, [GO master] Gu [Li] struck a different tone on Weibo (a Chinese microblogging site like Twitter), saying, “AlphaGo has completely subverted the control and judgment of us GO players. I can’t help but ask, one day many years later, when you find your previous awareness, cognition and choices are all wrong, will you keep going along the wrong path or reject yourself?” This uncertainty was echoed by GO master Ke Jie [ranked #1 in GO], who said, “After humanity spent thousands of years improving our tactics, computers tell us that humans are completely wrong. I would go as far as to say not a single human has touched the edge of the truth of GO.” [emphasis added = ed.]

Chess players haven't quite had the same reaction to being surpassed, but then Steinitz only started pulling back the curtain from the deeper truths of the game less than 150 years ago. That and our ever constant search for novelties and cooks probably saved us from such a deep crisis of faith.

Certainty is a shaky foundation upon which to build the edifice of one's self.

HT: Alice Maz, whose tweet alerted me to this.