The first ever Man-Machine Poker Championship was just completed at AAAI in Vancouver. Here is the New York Times article about it and here is the official tournament results page which has some interesting running commentary from some of the creators. The computer played two human players, Phil “The Unabomber” Laak and Ali Eslami, in four rounds of limit Texas Hold’Em Poker. They played duplicate matches, meaning the computer would play one human in one room, and in the other room would be playing the other human with the exact opposite sets of cards (i.e. the computer gets what the human in the other room got and vice versa). This was meant to remove some of the variability due to luck. Overall the humans won but the computer put up a good fight, getting a draw in the first round and a win in the second round. It looks like the humans were better at learning and figuring out how to adapt to the computer’s play than it was at adapting to the humans’ play.

I think this event is really cool. Poker is a much more difficult game to play than chess or checkers, which computers have already been very successful at. There’s no way to simply search to the end of the game to find the absolute best move because the opponent’s cards are hidden and unknown. In addition to all the probability and math involved in poker, there is the issue of trying to get a read on the opponent based on his actions, and trying to disguise your own hand with your actions, which adds a lot of complexity to the game. It is interesting that they competed in limit rather than no-limit Texas Hold’Em. Limit makes the action selection in the game much easier, as you simply have a discrete choice between a few actions such as fold, call, or raise. In a no-limit game, the player also has to choose the amount of money to bet, which can be any amount from the minimum allowed bet to all the money the player has. This addition of a continuously valued choice could make designing a no-limit computer player quite a bit harder, although the actual game of no-limit can sometimes be easier since you can bet a lot more, which can make bigger pots when you have good hands and make it easier to bluff people out.

I’ve thought about making my own computer poker player, and I actually have a couple different versions of code sitting around from my various attempts at it. It’s relatively easy for a computer to calculate exactly its probability of having the best hand as well as the probability of hitting a draw. You can calculate that a certain hand has say, a 75% chance of being the best hand right now. But this is something that professional poker players do fairly easily as well, but then they can also refine the probabilities much much more by getting a read on the player based on their facial tells as well as their bets. Trying to interpret this information is much harder for a computer. How do you tell a bluff from a bet? How do you predict the cards an opponent has based on the sequence of their actions?

It would be interesting to write a poker bot in a rigorous statistically valid way that treated each opponent action as an observation for a possible hypothesis and calculated probabilities of winning the hand. You could assign each action of the opponent to a hypothesis of the cards they have and have some probability that that action would have resulted from the opponent having those cards. That part could be specific to each opponent. Once you have a reliable idea of what the opponent’s cards are, you could calculate your probability of winning and compare it to pot odds and decide whether its worth it to be in. But even that wouldn’t account for trying to select your actions so they disguise your hand. So as you see, poker is a difficult and complex game. But a fun one. I may see about entering the computer tournament next year, this would be fun, but I have so many other interests as well.

Just a note that the researchers at the University of Alberta have now proved that their computer checkers player, Chinook, is unbeatable. The best anyone can hope to do against it is a draw. Checkers is now solved. I’m not sure how impressive it is that a computer has solved it, its requires the computer to be able to search through enough possible moves to find the correct move every time. It is cool that such a large and complex problem has been tackled, but I think it would be even cooler if the program had learned how to play on its own. Here are some opinions on the news from the New York Times: Is the Only Winning Move Not to Play?

Here is a Wired magazine interview with Brian Eno.  I’m a fan of Brian Eno because of his work with Talking Heads, and now he’s doing some really cool new stuff.  He’s been working on generative art and music, basically having a computer create new images or sounds in some unique way to create new art and music.  It’s a really cool idea, where the artist may set the parameters of the program, provide the starting points for it, but then the art and music created will be new even to the artist.

I finally got around to looking at some of this stuff on Jeff Hawkins and his Hierarchical Temporal Memory (HTM) that people have been sending me.  Hawkins is trying to build a biologically plausible model of learning and memory in the brain that will supposedly be general across different tasks (i.e. it works for vision processing, audio, motor stuff, etc).  What he has come up with is called an HTM, and it is a hierarchical network of nodes that use belief propagation.  Here is a video from 2003 where Hawkins lays out some of his ideas about the brain and his belief that intelligence is better defined as an ability to predict the future.  This video is from 2006 and is a presentation were Hawkins describes the theory and implementation of the HTM system.

I think the idea of the HTM system is very cool.  I really think we should be looking to the brain for ideas on good learning and memory systems and doing it hierarchically makes a lot of sense.  The system Hawkins shows seems to be like a sort of hierarchical structure of Self-Organizing Maps (SOM), which is a method for unsupervised learning and clustering.  His method connects these different nodes using belief propagation so they all agree on the same thing.  And he has some unspecified method for detecting temporal sequences.  But otherwise it seems a lot like a SOM, grouping similar sets of inputs together and being able to provide the closest matching set from memory.  The results he shows on a simple object recognition task look very much like something you would see from a SOM or neural network.  It would be cool to see some results where the temporal sequencing or the hierarchical structure come into play.

Even though I think the results of the work are a bit weak so far, I still think it is pretty cool.  I’m still trying to figure out what direction I want to go in for my research and this gives me some ideas.  I would definitely like to work on something motivated by or inspired by the brain.  And it should be general enough to be applicable to many different problems.  It will be interesting to see what kind of results Hawkins gets out of the HTM as it gets extended and applied to more interesting problems.

The Edge just released the answers to their annual Question of the Year.  This year the question was “What are you optimistic about?”  A bunch of the answers said they were optimistic about optimism.  And a large number of responses were people saying they were optmistic about our chances of combating/surviving global warming.  But the ones I find the most interesting are the AI and brain related ones, so I’ll highlight a few of my favorites here.

AI pioneer Marvin Minsky says he’s optimistic about our chances of developing immortality.  He suggests that we may be able to download our conciousness to a computer in the near future and thus live on forever in silicon (something I’ve mentioned before).

AI researcher Rodney Brooks is optimistic about space flight in the next century.  He thinks there will be people living on other planets in our lifetime and cites all the private space ventures that are working on their own spaceship designs right now.  He emphasizes how important colonizing other planets is to increasing our species’ long-term chances of survival.

Autism researcher Simon Baron-Cohen has a really interesting piece about the rise of autism and the digital age.  He suggests the rise in the rate of autism in children is not something to be pessimistic about because many of these children will be very successful in the digital age.  Autism is a disease that makes it much more difficult for children to relate to others, but perhaps easier for them to relate to a computer.

Cognitive Scientist Donald Hoffman thinks we will soon have the answer to the mind-body problem.  He thinks most of the current theories are wrong, but because of our presuppositions about the brain and this problem. He foresees a shift in these presuppositions and a breakthrough in this area coming soon.

Computational Neuroscientist Terrence Sejnowski thinks there will be a breakthrough in understanding intelligence.  He thinks reinforcement learning may lead to this breakthrough as recent research suggests that dopamine in the brain may act as the reward signal in reinforcement learning.

Gloria Origgi talks about the impact of multi-linguism in Europe. Something I mentioned in my posts after reading Consciousness Explained was the effect that language has on our thinking and how it made me want to learn another language.  In Europe (and in most of the world other than the US), children are learning multiple languages all the time.  Origgis suggests this may lead to more cognitive flexibility in the children along with less intolerance in Europe.

Psychologist Sherry Turkle has some interesting thoughts on the objects that inspire young children to become scientists.  A girl braiding her doll’s hair again and again (recursion), taking apart toys and things.

Finally, artificial life researcher Steve Grand is optimistic about the strong possibility that we’ve got everything horribly wrong.  In many cases in the past, we have had things horribly wrong and then suddenly realize that all our assumptions are wrong (like believing the Earth is the center of the universe).  He suggests that two possibilities would be quantum theory and the theory of matter.  But then goes on to say that AI may be another likely one and that neuroscience has the answer (very similar to Donald Hoffman’s piece on solving the mind-body problem by fixing our presuppositions).

A question that I have been starting to get when I tell people I’m studying artificial intelligence is “Will robots take over the world?”  It even came up in a panel discussion of AI professors in one of my classes the other week.  The common response to this question is, “why would they want to?”  Just because there are some humans who would like to take over the world, there is no need for us to program these desires into robots.  For that matter, will we give robots desires and motivations at all?  Or will they be no more than a very advanced toaster?

I think the taking over the world question can be looked at in two ways.  First, there have been individuals who wanted to take over the world: Napoleon, Hitler, etc.  If we are actually creating robots with consciousness and desires and everything, then I suppose there is a chance one will have this desire as people have in the past.  The chances of us creating robots like this is probably slim, and if we do then all the rest of the robots would surely be trying to stop the one crazy one just like how humans would try to stop another human from taking over the world.

The second view of taking over the world is as a group.  Humans have essentially taken over the world.  We cover most of the land mass on the planet and do to it whatever we want (mostly pollute it terribly).  Other animals commonly expand their populations to the maximum that their environment can hold.  Again, if we gave the robots the ability to procreate and have offspring then this might happen.  But it is a very large step from having robots to do work for us to giving them consciousness and desires and the ability to procreate.

So no, I don’t think robots are going to take over the world in either sense of the question.  For this to happen would require us to create sentient, procreating beings which I don’t imagine happening any time soon.

In the book AI: The Tumultuous History of the Search for Artificial Intelligence by Daniel Crevier, Crevier brings up an interesting question: what is common to all forms of intelligence?  When we do eventually develop artificial intelligence, it will surely not be exactly the same as human intelligence.  And if do eventually meet some alien race, their intelligence would probably be different.  What aspects of the mind are required for intelligence to come about?  What would be common between these different types of intelligence and what would be different.

One interesting example is the android Data on the TV show Star Trek: The Next Generation.  He is an artifically intelligent lifeform, yet he has no emotions.  Is it possible to be intelligent without emotions?  Could intelligence develop on its own without them?  Emotions are important for motivation, in the book I just read, Creating Mind: How the Brain Works by John Dowling, Dowling says “Without emotions, why bother?” It’s very true that emotions are the driving forces behind our lives and I can’t imagine we would ever have developed into the intelligent beings we are today without them.

The most interesting project that I read about in AI: The Tumultuous History of the Search for Artificial Intelligence by Daniel Crevier was called Automated Mathematician (AM) and was developed by Douglas Lenat. The idea was to have the program learn and discover math principles and laws by playing around with different concepts on its own. Lenat wanted the program to learn by discovery, to learn about the world by playing with things it found interesting.

The program was had three principle components. First it had 115 initial ideas such as the concepts of sets, intersections of sets, etc. Next it had a set of 184 rules for manipulating these basic concepts. Basically heuristics like looking for inverses and extreme cases of things. Finally the program was given a metric to determine how good and interesting a discovery is so it would know which ones to follow or report.

The program rediscovered over 200 mathematic concepts including some that have not yet been proven or thoroughly researched. One of the most interesting was the programs interest in numbers that have many factors, sort of the opposite of prime numbers. This as something that was mentioned by a mathematician a while ago but never really followed up on.

Eventually the program slowed down and stopped discovering new concepts. This was likely because the program was limited by the manipulation rules and metrics that it was programmed with. The program would not be able to add new discoveries to its manipulation rules or change the metrics for what it finds interesting so it was unable to build on what it discovered. Lenat wrote a new program that could revise its discovery heuristics and used it both for computer chip design and for a contest called Traveller TCS. His new program bears a lot of similarities to the genetic algorithms now in use as the heuristics evolved through generations in a similar way.

More recently genetic algorithms have been used in the design of circuits and other areas, following up on AM’s initial success.

I just read AI: The Tumultuous History of the Search for Artificial Intelligence by Daniel Crevier. It was very interesting to see the different ways and methods that researchers approached AI as well as the problems that were faced in the early years of AI research. Here’s a quick look at one of the main problems encountered - the fact that understanding once concept or idea requires understanding of many or even all other concepts as well.

The main problem that was encountered is that it is nearly impossible to define a single concept without understanding many other concepts. For example, to understand the meaning of the word car you have to understand about wheels and driving and people and locations and distances, etc. This is what mathematician-philosopher Gottfried-Wilhelm Liebnitz said in the 17th century when trying to develop a method to formalize thought: “There is no term so absolute or detached that it contains no relations and of which a perfect analysis does not lead to other things or even all other things.” Which means that to understand any concept you need to understand many other concepts if not all other concepts.”

This problem manifested itself in many ways. First, early AI researchers were very limited by the amount of memory in computers at the time. One example is a project a student worked on to try to analyze sentences. Since understanding any one word requires understanding of many other words, there was nowhere near enough room in memory to attempt anything. In fact, he only had enough room in memory for definitions of about 4 words.

To avoid this problem, researchers worked on projects in limited problem spaces. In this way there was a limited number of concepts available and it was possible to write a program that could understand them all. At MIT they worked on a project called Micro Worlds where a robot would move colored cubes and pyramids around a room. For this all it had to understand was colors, shapes, and the robots functions. This program worked really well but it was very difficult to expand it to work in a more general environment.

Finally, even when memories increased enough to make things like the sentence understanding program more feasible, researchers were still trying to hand code these definitions and things into the program. However if understanding one concept could require understanding of all other concepts, programming them all in could take forever. Finally in the mid 1980s the researchers realized that it would be impossible to program all concepts by hand and the computer needs to be able to learn on its own to”completely automate the knowledge acquisition process.”

There was a good article on AI today in the New York Times. Basically they said that AI spring is starting (the 80s were the ‘AI Winter’) due to better computer technology and better understanding of how the brain works. It discusses some of the robot projects going on at Stanford and elsewhere. Here is the link: Brainy Robots Start Stepping into Daily Life