The first ever Man-Machine Poker Championship was just completed at AAAI in Vancouver. Here is the New York Times article about it and here is the official tournament results page which has some interesting running commentary from some of the creators. The computer played two human players, Phil “The Unabomber” Laak and Ali Eslami, in four rounds of limit Texas Hold’Em Poker. They played duplicate matches, meaning the computer would play one human in one room, and in the other room would be playing the other human with the exact opposite sets of cards (i.e. the computer gets what the human in the other room got and vice versa). This was meant to remove some of the variability due to luck. Overall the humans won but the computer put up a good fight, getting a draw in the first round and a win in the second round. It looks like the humans were better at learning and figuring out how to adapt to the computer’s play than it was at adapting to the humans’ play.

I think this event is really cool. Poker is a much more difficult game to play than chess or checkers, which computers have already been very successful at. There’s no way to simply search to the end of the game to find the absolute best move because the opponent’s cards are hidden and unknown. In addition to all the probability and math involved in poker, there is the issue of trying to get a read on the opponent based on his actions, and trying to disguise your own hand with your actions, which adds a lot of complexity to the game. It is interesting that they competed in limit rather than no-limit Texas Hold’Em. Limit makes the action selection in the game much easier, as you simply have a discrete choice between a few actions such as fold, call, or raise. In a no-limit game, the player also has to choose the amount of money to bet, which can be any amount from the minimum allowed bet to all the money the player has. This addition of a continuously valued choice could make designing a no-limit computer player quite a bit harder, although the actual game of no-limit can sometimes be easier since you can bet a lot more, which can make bigger pots when you have good hands and make it easier to bluff people out.

I’ve thought about making my own computer poker player, and I actually have a couple different versions of code sitting around from my various attempts at it. It’s relatively easy for a computer to calculate exactly its probability of having the best hand as well as the probability of hitting a draw. You can calculate that a certain hand has say, a 75% chance of being the best hand right now. But this is something that professional poker players do fairly easily as well, but then they can also refine the probabilities much much more by getting a read on the player based on their facial tells as well as their bets. Trying to interpret this information is much harder for a computer. How do you tell a bluff from a bet? How do you predict the cards an opponent has based on the sequence of their actions?

It would be interesting to write a poker bot in a rigorous statistically valid way that treated each opponent action as an observation for a possible hypothesis and calculated probabilities of winning the hand. You could assign each action of the opponent to a hypothesis of the cards they have and have some probability that that action would have resulted from the opponent having those cards. That part could be specific to each opponent. Once you have a reliable idea of what the opponent’s cards are, you could calculate your probability of winning and compare it to pot odds and decide whether its worth it to be in. But even that wouldn’t account for trying to select your actions so they disguise your hand. So as you see, poker is a difficult and complex game. But a fun one. I may see about entering the computer tournament next year, this would be fun, but I have so many other interests as well.

Most people (or so it was believed) have three different types of cone cells in the eye to detect color.  Each type responds to a different set of wavelengths of light, and all the colors that we see are interpreted from the activations of the three different types of cone cells.  Now it turns out that many women (possibly more than 50%) have a fourth type of cone cell.  These women are able to detect a wider array of colors than other women or men with the standard 3 types of cone cells.  Pretty crazy.

Just a note that the researchers at the University of Alberta have now proved that their computer checkers player, Chinook, is unbeatable. The best anyone can hope to do against it is a draw. Checkers is now solved. I’m not sure how impressive it is that a computer has solved it, its requires the computer to be able to search through enough possible moves to find the correct move every time. It is cool that such a large and complex problem has been tackled, but I think it would be even cooler if the program had learned how to play on its own. Here are some opinions on the news from the New York Times: Is the Only Winning Move Not to Play?

I saw this article on Scientific American’s website about how our expectations relate to the effects of placebos. They did both a study with a placebo painkiller and a study using a gambling game. Subjects who had higher dopamine activity (and thus higher expectations of reward) during the gambling game were the same subjects who received more of an effect from the placebo. The conclusion is that your expectations of rewards are closely linked to the effectiveness of the placebo.

This is pretty interesting for a number of reasons. One is that it shows that an important aspect of having placebos work is simply being convinced that they will work. Looking at it from a reinforcement learning perspective, the dopamine activity can be viewed as the prediction of the reward signal. And then somehow expecting a higher reward signal can reduce the pain? Is the pain signal just a negatively valued feedback? What is the relationship between the expectation of reward and the actual reward received? Can simply expecting more or less reward affect the actual amount of reward received?

Here is a Wired magazine interview with Brian Eno.  I’m a fan of Brian Eno because of his work with Talking Heads, and now he’s doing some really cool new stuff.  He’s been working on generative art and music, basically having a computer create new images or sounds in some unique way to create new art and music.  It’s a really cool idea, where the artist may set the parameters of the program, provide the starting points for it, but then the art and music created will be new even to the artist.

I was reading an article in the NYT about the Mayans and the 2012 thing.  The Mayans thought that time ran in a loop.  I think some of string theory says that some of the dimensions could be loops, so this could be possible.  Maybe time is a giant loop, the big bang could be the beginning and then at some point in the future the universe could condense to a small point and the big bang would happen again.  One of the most interesting questions about time is why we only travel through it one direction, while we can travel freely in both directions in all the other dimensions.  I suggested before that maybe its just that our brains are unable to comprehend time all at once so we perceive it as flowing smoothly in one direction.  But if time is a giant loop, maybe it could also be that we can only spin around that loop in one direction.  Like how water going down a drain always goes one direction in the northern hemisphere and the other direction in the southern hemisphere, maybe time can only go in one direction around this loop.  Just an idea.