The first ever Man-Machine Poker Championship was just completed at AAAI in Vancouver. Here is the New York Times article about it and here is the official tournament results page which has some interesting running commentary from some of the creators. The computer played two human players, Phil “The Unabomber” Laak and Ali Eslami, in four rounds of limit Texas Hold’Em Poker. They played duplicate matches, meaning the computer would play one human in one room, and in the other room would be playing the other human with the exact opposite sets of cards (i.e. the computer gets what the human in the other room got and vice versa). This was meant to remove some of the variability due to luck. Overall the humans won but the computer put up a good fight, getting a draw in the first round and a win in the second round. It looks like the humans were better at learning and figuring out how to adapt to the computer’s play than it was at adapting to the humans’ play.

I think this event is really cool. Poker is a much more difficult game to play than chess or checkers, which computers have already been very successful at. There’s no way to simply search to the end of the game to find the absolute best move because the opponent’s cards are hidden and unknown. In addition to all the probability and math involved in poker, there is the issue of trying to get a read on the opponent based on his actions, and trying to disguise your own hand with your actions, which adds a lot of complexity to the game. It is interesting that they competed in limit rather than no-limit Texas Hold’Em. Limit makes the action selection in the game much easier, as you simply have a discrete choice between a few actions such as fold, call, or raise. In a no-limit game, the player also has to choose the amount of money to bet, which can be any amount from the minimum allowed bet to all the money the player has. This addition of a continuously valued choice could make designing a no-limit computer player quite a bit harder, although the actual game of no-limit can sometimes be easier since you can bet a lot more, which can make bigger pots when you have good hands and make it easier to bluff people out.

I’ve thought about making my own computer poker player, and I actually have a couple different versions of code sitting around from my various attempts at it. It’s relatively easy for a computer to calculate exactly its probability of having the best hand as well as the probability of hitting a draw. You can calculate that a certain hand has say, a 75% chance of being the best hand right now. But this is something that professional poker players do fairly easily as well, but then they can also refine the probabilities much much more by getting a read on the player based on their facial tells as well as their bets. Trying to interpret this information is much harder for a computer. How do you tell a bluff from a bet? How do you predict the cards an opponent has based on the sequence of their actions?

It would be interesting to write a poker bot in a rigorous statistically valid way that treated each opponent action as an observation for a possible hypothesis and calculated probabilities of winning the hand. You could assign each action of the opponent to a hypothesis of the cards they have and have some probability that that action would have resulted from the opponent having those cards. That part could be specific to each opponent. Once you have a reliable idea of what the opponent’s cards are, you could calculate your probability of winning and compare it to pot odds and decide whether its worth it to be in. But even that wouldn’t account for trying to select your actions so they disguise your hand. So as you see, poker is a difficult and complex game. But a fun one. I may see about entering the computer tournament next year, this would be fun, but I have so many other interests as well.