This week the first four matches of Go pitting AlphaGo – the Google-developed “Deep Mind” Go-playing artificial intelligence – against Lee Sedol, currently considered by many to be the best Go player alive, have taken place. In the first match, AlphaGo emerged successful, shocking many, and had a number of interesting moments of play where Sedol tried moves he didn’t think the AI would be able to respond to, and later a particular move was made by AlphaGo that Sedol felt almost no human professional would ever play. The second match was also an AlphaGo victory, although it took longer to force Sedol to resignation than the first, and through-out, the commentators noted the “creative” and “unusual” nature of some of AlphaGo’s moves. In the third match, AlphaGo was once more successful – this means that it has won the series, and the implicit question of the series has shifted from “can AlphaGo win a single match?” towards “Can Sedol win a single match?”. In the fourth match, Sedol struck back – with AlphaGo making an apparently dubious move, and Sedol making a superb move, around the middle of the game – and made the series 3-1, with one more game to go. Having looked over all the commentary of the games – and I post this after match 4, with one still to play – I think there were a lot of interesting issues raised by these matches (or rather, the commentary around the matches) that needed assessing from a slightly more critical angle. AI is obviously a crucial topic in game development – and one that, long-term readers will know, is currently proving to be a complete nightmare in my own game development – and the discussion around these matches is surprisingly intriguing, and I think very relevant to understanding how we think about game AI, and what kinds of traits we’re looking for in a “good” game AI. We’ll be back to regular game development updates next week with another big step forward in finishing off the game’s AI (and I will be releasing the playtesting version on the weekend of the 26th/27th), but this week we will instead take a closer look at how these matches are being written about, thought about, and presented to the public.
Until very recently board games tended to dominate discussions of game-playing AI ability, and for years Chess was used as the absolute metric of AI ability. Once Deep Blue and subsequent chess-playing computers emerged clearly dominant over their human opponents, the idea of a “grand AI challenge” seemed to fade somewhat. Research in AI naturally takes place in a thousand different subfields, but research into other challenging games played globally at a professional level – poker and Go being apparently the dominant two – continued, but without quite the same public interest. As time went by, the greatest game AI challenge came to be considered an AI that could skilfully play Go – the game’s possibility space is far greater than that of chess, of course, and many suggested that an AI capable of beating the top human player(s) was most likely decades away in the future. For whatever reason, the poker-playing AI seems to have captured the imagination less – is it because it’s a game of incomplete information and therefore seen as “less skilled”? Is it because to the unskilled player it appears to be a game of luck? It is because it takes longer for the better player to emerge victorious? Is it because to the unskilled player there is simply less to visually look at than Go or Chess? It may be any one or several of these factors, but whatever the reason, the game of poker – which of course requires psychological and social, as well as mathematical, insight – was pushed aside in favour of an AI that can play Go well, which in game theoretic terms is essentially the same challenge as playing Chess, but just more so. Significantly moreso, actually: the classic quoted statistic is that there are more valid positions in Go than there are atoms in the universe, which outstrips the mathematical complexity of Chess by quite a margin.
I’d like to think about one particular question here: within the space of game AI developments (whether or not we accept the tacit claim that these are indicative of wider questions about AI and the overall ability of AI), are playing Chess and Go really the two greatest gameplaying achievements we can think about? I’m reminded of Demis Hassabis’ comment at the start of the competition that “the only game left after Chess is Go”. Firstly, it (and all the other discourses surrounding this match, I’m not singling Hassabis out here – he just happened to make an explicit comment that serves as good jumping-off point) is clearly oriented, at least in part, towards the broader reader. Everybody has heard of Chess, even if they don’t play it; fewer people (in the West, anyway) have heard of Go, and by positioning the two together like this as many media outlets have been doing in explaining the history of game AI, it feeds into popular assumptions about the difficulty of Chess. By positioning Go as even more difficult than Chess (the game many people consider to be hugely complex), the average person who knows nothing about Chess, Go, AI, or games per se, is immediately given an idea of the apparent scope of this achievement. I understand (of course) that conveying the noteworthiness of complex academic, technical or intellectual pursuits can be tricky, but this kind of statement and the wealth of associated commentary surrounding the matches that mentions Deep Blue vs Kasparov discursively positions game AI within an incredibly narrow space. The Nature paper reporting on AlphaGo does briefly acknowledge that Go may be the greatest challenge in “classic games” rather than all games per se, but that one comment aside, I can find little reference to games conceived more broadly than classic games of the Chess and Go variety. In both cases we are not actually talking about “games” as a massive sweeping category, but instead we are talking about board games, with extremely simple foundational rules, no particular requirement for rapid input or reflexes, and that take place with complete information. They don’t need their AI to read information from a rapidly-changing computer screen, or to deal with a far more complex set of rules and units (the “rules” and number of variables in Starcraft, for example, are vastly more detailed and numerous than Chess, and even more so than Go), or to rapidly or near-instantly respond to in-game actions and changes (a fairly generous timer tends to be used for Chess/Go AI matches), or extrapolate about incomplete information as in a card game or real-time strategy game featuring a fog of war. This isn’t even to mention other factors in the massive swath of human cultural items we call “games”, such as solving riddles, engaging socially with other human players or AI actors, exploring unknown virtual spaces, and so forth.
As many readers of this blog are likely aware, it was only a few months ago that an AI beat NetHack for the first time. Now, naturally I’m not suggesting that quite the same level of research and funding goes into beating NetHack as went into beating Go, but there are nevertheless a range of other frontiers of game AI design that aren’t “solved”, and solving procedurally-generated games and games with short-term randomness certain fall within that category. Even if we accept that AlphaGo’s victories demonstrate the absolute domination of AI calculation over human decision-making in simple-rule, large possibility space, complete information, no-time-pressure games, that’s only the slightest sliver of what games are at the moment. This lack of broader applicability is also very clear when we look at something like poker, which shifts and changes as the metagame changes. For most forms of poker I don’t think there exists a perfect and stable Nash equilibrium of gameplay when the “skill” (or not) of one’s gameplay choices is so heavily dependent upon the dominant metagame. In ~2010 the hyper-aggressive metagame meant that five-betting preflop with A5o was considered (by some) to be a strong and legitimate strategy, having been pioneered to great success primarily by young European players. By the time I quit playing professionally, I would say this particular metagame was approaching its demise, and although I don’t really keep up with the game now, I’m aware through comments from other professionals that this rather bizarre metagame has entirely faded. A poker AI needs to create models of its opponents’ playstyles, and ideally an even broader model of the overall poker metagame at the time; there are rarely moves as “obviously” strong or weak as in deterministic, complete-information games. Recently an AI that plays heads-up Limit Hold’em “perfectly” was announced, a very impressive achievement, although a very small subset of poker with a substantially reduced possibility space compared to No-Limit Hold’em Six-Max, or Seven-Card Stud No Qualifier, for example, and a particular niche of poker that is heavily mathematical, minimally psychological, and with very little variation in the metagame over time. Simply “playing the numbers” will not, in any poker game more complex than heads-up Limit Hold’Em, be sufficient, but such an approach is entirely adequate to games that share game theoretic characteristics with Chess and Go (and Shogi, Draughts, Othello, etc), which – as we’ll note – tend to be physical, historical games. When it comes to the generally far more complex computer games, AI struggles, and yet we define Go as the ultimate achievement in game-playing AI? We ultimately have to conclude that this is a very narrow sense of the word AI, and one rooted deeply in deterministic game theory rather than the far broader range of concerns that necessitate “skilled play” in contemporary games.
As hinted at above, there is also an additional cultural component here in how we regard board games and computer games. I’m sure we’re all aware of the struggle for games to be considered “high culture” – in the eyes of the average non-game-player, games are probably somewhere below television and above comic books, which is to say relegated to the absolute cultural dreck of the world. We know this to be a nonsense, of course, and I can honestly say a large volume of the most profound experiences I’ve ever had with any form of artistic media came from games, but there’s no doubt that games have it bad. Or, rather, that computer games have it bad. Few would doubt that an old, venerable and respected game like Chess or Go wasn’t a form of high culture, whereas I think few non-game-players would immediately recognize the cultural value of most computer games. If you’re looking for wider validation for the value of your AI to the non-specialist audience, where better to look than a slow, steady, refined, physical game that is thousands of years old, rather than one of these seemingly crass, transient, new-fangled, high-speed digital-game-thingies? Similarly, on a visual level, I think someone who knows nothing about Go can still reasonably take a guess at the complexity of that game simply by looking at the board and thinking briefly about the possibility space of each decision, whereas someone who knows nothing about your average fairly challenging computer game will be harder-pressed to “read” the game and see where the complexity and challenge lies. For all of these reasons I’m concerned that in the wake of these matches we may, to some extent, wind up foreclosing on public interest in other forms of game AI in public debate if the problem is believed to be solved, when in reality even the most modern games with eight-figure budgets ship with AI actors that are far removed in their abilities from even a moderately-competent human counterpart.
The victories of AlphaGo are certainly a massive achievement, but the point I’ve tried to argue here is that they are a massive achievement in a very particular sliver of the immense world of game AI, and that we should be very wary of, and think critically about, the kind of spectacular commentary being thrown around. Sure, we can build an AI to defeat the world’s most skilled player at Go, but unless I’ve missed something we don’t yet have AIs that have human-standard conversations in games, or can solve riddles and puzzles in “adventure” games, or can navigate their way across unexpected terrain with the trivial ease of a human being, handle any game thrown at it (the goal of “general game-playing AI”), defeat a top human player at Starcraft 2, or – maybe most compellingly to me – defeat top players at any one of the more complex and involved varieties of poker. Equally, I naturally understand that if one wants to get published in Nature, one will have to present one’s achievements as the most impressive or transformative thing developed in the field in question for years. And it is, indeed, hugely impressive. However, the world of “games” back when Chess AI was first programmed simply no longer exists. Far from classic physical board games being the ultimate challenge of game AI, they are now only a single subset of gameplaying AI, and one that handles none of the unpredictability, nor social or psychological engagement, required of so many modern games. Let’s treat AlphaGo’s successes as what they are – a tremendous victory for AI in the game of Go specifically, and in deterministic complete information games more generally – and not as the harbinger of perfect game AI, nor the inability of humans to ever compete with AI players, and certainly not as the highest achievement game AI could ever aspire towards.