The most interesting project that I read about in AI: The Tumultuous History of the Search for Artificial Intelligence by Daniel Crevier was called Automated Mathematician (AM) and was developed by Douglas Lenat. The idea was to have the program learn and discover math principles and laws by playing around with different concepts on its own. Lenat wanted the program to learn by discovery, to learn about the world by playing with things it found interesting.

The program was had three principle components. First it had 115 initial ideas such as the concepts of sets, intersections of sets, etc. Next it had a set of 184 rules for manipulating these basic concepts. Basically heuristics like looking for inverses and extreme cases of things. Finally the program was given a metric to determine how good and interesting a discovery is so it would know which ones to follow or report.

The program rediscovered over 200 mathematic concepts including some that have not yet been proven or thoroughly researched. One of the most interesting was the programs interest in numbers that have many factors, sort of the opposite of prime numbers. This as something that was mentioned by a mathematician a while ago but never really followed up on.

Eventually the program slowed down and stopped discovering new concepts. This was likely because the program was limited by the manipulation rules and metrics that it was programmed with. The program would not be able to add new discoveries to its manipulation rules or change the metrics for what it finds interesting so it was unable to build on what it discovered. Lenat wrote a new program that could revise its discovery heuristics and used it both for computer chip design and for a contest called Traveller TCS. His new program bears a lot of similarities to the genetic algorithms now in use as the heuristics evolved through generations in a similar way.

More recently genetic algorithms have been used in the design of circuits and other areas, following up on AM’s initial success.

I just read AI: The Tumultuous History of the Search for Artificial Intelligence by Daniel Crevier. It was very interesting to see the different ways and methods that researchers approached AI as well as the problems that were faced in the early years of AI research. Here’s a quick look at one of the main problems encountered - the fact that understanding once concept or idea requires understanding of many or even all other concepts as well.

The main problem that was encountered is that it is nearly impossible to define a single concept without understanding many other concepts. For example, to understand the meaning of the word car you have to understand about wheels and driving and people and locations and distances, etc. This is what mathematician-philosopher Gottfried-Wilhelm Liebnitz said in the 17th century when trying to develop a method to formalize thought: “There is no term so absolute or detached that it contains no relations and of which a perfect analysis does not lead to other things or even all other things.” Which means that to understand any concept you need to understand many other concepts if not all other concepts.”

This problem manifested itself in many ways. First, early AI researchers were very limited by the amount of memory in computers at the time. One example is a project a student worked on to try to analyze sentences. Since understanding any one word requires understanding of many other words, there was nowhere near enough room in memory to attempt anything. In fact, he only had enough room in memory for definitions of about 4 words.

To avoid this problem, researchers worked on projects in limited problem spaces. In this way there was a limited number of concepts available and it was possible to write a program that could understand them all. At MIT they worked on a project called Micro Worlds where a robot would move colored cubes and pyramids around a room. For this all it had to understand was colors, shapes, and the robots functions. This program worked really well but it was very difficult to expand it to work in a more general environment.

Finally, even when memories increased enough to make things like the sentence understanding program more feasible, researchers were still trying to hand code these definitions and things into the program. However if understanding one concept could require understanding of all other concepts, programming them all in could take forever. Finally in the mid 1980s the researchers realized that it would be impossible to program all concepts by hand and the computer needs to be able to learn on its own to”completely automate the knowledge acquisition process.”

There was a good article on AI today in the New York Times. Basically they said that AI spring is starting (the 80s were the ‘AI Winter’) due to better computer technology and better understanding of how the brain works. It discusses some of the robot projects going on at Stanford and elsewhere. Here is the link: Brainy Robots Start Stepping into Daily Life

Here are a few more random thoughts that I’ve had from reading Daniel Dennett’s Consciousness Explained:

• Not only are regions of the brain multi-functional, but so are some of our genes. So when one trait or gene is selected and continued on, other aspects affected by that gene are affected as well, this is called collateral evolution. In Olivia Judson’s blog on the New York Times, she mentions a case were hunters bred foxes to be more friendly to humans, but in addition to that the foxes got shaggier ears and wider heads.
• Color was evolved as an imporatant way for us to spot predators and food. Much more than just the frequency of light reflected by something affect what color we see it as and the ability to see these differences evolved so we could spot a red fruit in a forest of green. But Dennett says that not only did we evolve to see that difference but also the apple evolved to be a color that we could spot (so we could spread its seeds). Dennett says “first there were various reflective properties of surfaces, reactive properties of photopigments, and so forth, and Mother Nature devloped out of these raw materials efficient, mutually adjusted “color”-coding/”color”-vision systems, and amont the properties that settled out of the design process are the properties we normal human beings call colors.” It’s interested to note that there are some humans who are red-green colorblind and there are some other species (some types of birds) who can see things in the UV range. Dennett says “Why is the sky blue? Because apples are red and grapes are purple, not the other way around.”
• The Baldwin effect - Dennett describes another aspect of evolution were “good tricks” are learned. Suppose there is some “good trick” that will help a creature in their environment. Creatures that are born with the trick are better off but others are able to re-wire their brains to learn it as well. If you assume that the amount of re-wiring needed depends on their genes, then animals that have genes that put them closer to learning the trick will be better off than animals that have a difficult time re-wiring for the trick (learning it). So genes that are close to the wiring of the brain for the trick continually get selected until eventually the gene that starts with this brain wiring could be selected. In this way the plasticity of the brain helps to speed up the evolution of the genes (this is known as the Baldwin effect).
• Memes - Memes are similar to genes, but they are instead ideas that try to reproduce themselves. Our brains and our cultures are were memes are stored and they are transferred through communication. Similar to the way life is just a vessel for the propagation of genes, our conscious minds may just be a vessel for memes to propagate. It’s interesting that the memes replicative power is based on its “fitness” rather than its contribution to our fitness. So memes or ideas that are harmful to us but spread easily are good replicators. This is a very interesting way to think about ideas, first proposed by Richard Dawkins.
• The concept of self is very abstract. It is important for us to be able to separate ourselves from the outside world so that we do not eat ourselves, etc. But in reality this is not such a clear line. There are lots of bacteria inside us that help us digest but are they really part of us? Dennett says that each normal person “makes a self. Out of its brain it spins a web of words and deeds.” He equates it to a snail building a shell or a spider making a web. Its just something we do. It is important for our self-preservation. And with the parallel thoughts (Multiple Drafts) theory of consciousness it is possible that a person could have multiple selves (multiple personality disorder) or even that one self could be spread across two people (in the case of the twins I mentioned in my last post).
• From what I understand about consciousness and the mind now, I find it hard to believe that we actually do have any free will. Our thoughts and mind are completely a result of physical and electro-chemical reactions in the brain. Do we really have any control over these reactions? Any control we would have would just be other electro-chemical reactions. It may eventually be possible to predict these reactions and fully predict the decisions of the brain. Our idea of free will may just be an illusion that these various electro-chemical reactions create for us.  It may seem that we have free will because the process that creates our decisions and thoughts is so complex.  An analogy would be the weather, surely the weather patterns and tornados and hurricanes and things could be predicted if we better understood the system and all the inputs, but since we don’t it seems quite random whether a tornado or hurricane will be created or not.

I’ve been reading Daniel Dennett’s Consciousness Explained and specifically looking at how his theory would be applied to creating artificial consciousness in a robot or software program.  Dennett makes a few key points that would have to be taken into account in developing artificial conscsciousness:

1. There is no central meaner, no executive decision maker.  Instead the brain is massively parallel, ideas come and go, actions and words are bubbling up everywhere and the best ones that fit best are the ones that get said or acted on.
2. Consciousness is effectively software running on the brains hardware.  The consciousness is a serial virtual machine running on the parallell architecture of the brain.
3. All of the parts of the brain are multifunction.  And different part’s functions overlap.  As Dennett says: “human engineers … design systems in which each element plays a single role, carefully insulated from interference from outside, in order to minimize the devestation of unforeseen side effects.”  He goes on to say the brain was designed by a process that thrives on “multiple superimposed functionality, something systematically difficult to discern from reverse engineering.”
4. The brain’s memory is not direct access memory (RAM) like a computer’s memory.  Instead things are brought back up and remembered by association and free thinking.  In addition, there is no definite boundary between program and memory as in a computer.
5. Many current researchers are working on modelling and researching the parts of the brain on the periphery, the parts that receive inputs from the vision or hearing.  But by working from the outside in, these researchers leave a lot to be explained by the eventual central point, where a lot of the consciosness may be in these peripheral parts of the brain.
6. The development of the brain and of life is very dependent on chaos.  Evolution depends on random variation in genes for it to select from, the brain was developed out of chaotic processes that hit upon chance processes and “serendipitous side effects.”

These are just a few of the ideas that I think need to be taken into account when drawing up a plan for artificial consciousness.

In Iowa this weekend Brad and I were discussing consciousness and the mind-body problem. Brad brought up a point that I had not thought about, that if the mind and consciousness are really all contained within the normal functioning of the brain, then all of our consciousness and our “selves” are created by the physical and chemical reactions of the brain. Not only does this make the boundary between yourself and the outside hard to determine, but it means we should eventually be able to develop a (complex) model of the reactions in the brain just like any other physics reactions (like 2 pool balls colliding). Assuming we know the starting condition of the brain and the experiences it is going to through then we should be able to determine how the brain will end up and the way it will react to said experiences. Which then provides questions about free will and things if we are able to predict how someone will think and act. The only randomness left would seem to be any effect of random quantum fluctuations.

An interesting example of a case of this is brought up in Consciousness Explained. Two twins, Greta and Freda Chaplin, who were in their 40s when the book was written and living in England. They act as one, finishing each other’s sentences, etc.. Dennett explains:

“Since these twins have seen, heard, touched, smelled, and thought about very much the same events throughout their lives, and started, no doubt, with brains quite similarly disposed to react to these stimuli, it might not take enormous channels of communications to keep them homing in on some sort of loose harmony.”

It seems that if two people start with very similar brains and experience the same things then they will end up with very similar brains. Which means that it should be possible to predict how one’s brain is going to develop based on their experiences. Does this mean once we understand the brain we’ll be able to predict the future? Well we can’t even predict weather patterns or the decisions of small animals or anything like that so I doubt its quite that simple. But an interesting idea.

I’ve been reading Consciousness Explained by Daniel Dennett.  In it he suggets that our perception systems work both ways with one expectation-driven side making hypotheses and the data driven side where senses are input confirming or disconfirming those expectations and hypotheses.  This fits nicely with my post on hallucinations from a few months back, where I suggested that hallucinations come from us fitting our sensory data to the wrong model or hypothesis.  Dennett suggests the same, saying that arbitrary patterns of confirmation and disconfirmation by the data driven side could cause detailed hallucinations.  His analogy is a party game where someone asks each person a question to figure out someone’s dream.  The trick is the people answer yes or no depending on the last letter of his question.  But through a series of arbitrary answers a story is built up based entirely on ideas of his own.