I finally got around to looking at some of this stuff on Jeff Hawkins and his Hierarchical Temporal Memory (HTM) that people have been sending me.  Hawkins is trying to build a biologically plausible model of learning and memory in the brain that will supposedly be general across different tasks (i.e. it works for vision processing, audio, motor stuff, etc).  What he has come up with is called an HTM, and it is a hierarchical network of nodes that use belief propagation.  Here is a video from 2003 where Hawkins lays out some of his ideas about the brain and his belief that intelligence is better defined as an ability to predict the future.  This video is from 2006 and is a presentation were Hawkins describes the theory and implementation of the HTM system.

I think the idea of the HTM system is very cool.  I really think we should be looking to the brain for ideas on good learning and memory systems and doing it hierarchically makes a lot of sense.  The system Hawkins shows seems to be like a sort of hierarchical structure of Self-Organizing Maps (SOM), which is a method for unsupervised learning and clustering.  His method connects these different nodes using belief propagation so they all agree on the same thing.  And he has some unspecified method for detecting temporal sequences.  But otherwise it seems a lot like a SOM, grouping similar sets of inputs together and being able to provide the closest matching set from memory.  The results he shows on a simple object recognition task look very much like something you would see from a SOM or neural network.  It would be cool to see some results where the temporal sequencing or the hierarchical structure come into play.

Even though I think the results of the work are a bit weak so far, I still think it is pretty cool.  I’m still trying to figure out what direction I want to go in for my research and this gives me some ideas.  I would definitely like to work on something motivated by or inspired by the brain.  And it should be general enough to be applicable to many different problems.  It will be interesting to see what kind of results Hawkins gets out of the HTM as it gets extended and applied to more interesting problems.

There was an interesting article on memory the other week in the New York Times.  Basically, people who are faster at remembering certain things are also the ones who forget other things the fastest.  So the key to a shart memory is actually being selective and only remembering what is important.  Now I finally have an excuse for my terrible movie memory (I generally can’t even remember the plot of a movie the day after I see it).  But by not remembering movies, I am much better at remembering things I learned in class or readings.  Maybe that’s not a bad trade-off…

The Edge just released the answers to their annual Question of the Year.  This year the question was “What are you optimistic about?”  A bunch of the answers said they were optimistic about optimism.  And a large number of responses were people saying they were optmistic about our chances of combating/surviving global warming.  But the ones I find the most interesting are the AI and brain related ones, so I’ll highlight a few of my favorites here.

AI pioneer Marvin Minsky says he’s optimistic about our chances of developing immortality.  He suggests that we may be able to download our conciousness to a computer in the near future and thus live on forever in silicon (something I’ve mentioned before).

AI researcher Rodney Brooks is optimistic about space flight in the next century.  He thinks there will be people living on other planets in our lifetime and cites all the private space ventures that are working on their own spaceship designs right now.  He emphasizes how important colonizing other planets is to increasing our species’ long-term chances of survival.

Autism researcher Simon Baron-Cohen has a really interesting piece about the rise of autism and the digital age.  He suggests the rise in the rate of autism in children is not something to be pessimistic about because many of these children will be very successful in the digital age.  Autism is a disease that makes it much more difficult for children to relate to others, but perhaps easier for them to relate to a computer.

Cognitive Scientist Donald Hoffman thinks we will soon have the answer to the mind-body problem.  He thinks most of the current theories are wrong, but because of our presuppositions about the brain and this problem. He foresees a shift in these presuppositions and a breakthrough in this area coming soon.

Computational Neuroscientist Terrence Sejnowski thinks there will be a breakthrough in understanding intelligence.  He thinks reinforcement learning may lead to this breakthrough as recent research suggests that dopamine in the brain may act as the reward signal in reinforcement learning.

Gloria Origgi talks about the impact of multi-linguism in Europe. Something I mentioned in my posts after reading Consciousness Explained was the effect that language has on our thinking and how it made me want to learn another language.  In Europe (and in most of the world other than the US), children are learning multiple languages all the time.  Origgis suggests this may lead to more cognitive flexibility in the children along with less intolerance in Europe.

Psychologist Sherry Turkle has some interesting thoughts on the objects that inspire young children to become scientists.  A girl braiding her doll’s hair again and again (recursion), taking apart toys and things.

Finally, artificial life researcher Steve Grand is optimistic about the strong possibility that we’ve got everything horribly wrong.  In many cases in the past, we have had things horribly wrong and then suddenly realize that all our assumptions are wrong (like believing the Earth is the center of the universe).  He suggests that two possibilities would be quantum theory and the theory of matter.  But then goes on to say that AI may be another likely one and that neuroscience has the answer (very similar to Donald Hoffman’s piece on solving the mind-body problem by fixing our presuppositions).

Here is a great video of a talk given by Richard Dawkins at Randolph-Macon Women’s College in Lynchburg, Virginia.  Jerry Falwell’s Liberty University is in the same town and its students showed up with all sorts of entertaining questions, making for an lively Q&A session.  Here’s the link for the video.

Here is another good link, a short Wired interview with Marvin Minksy and Daniel Dennett on the mind and emotions.

There’s an interesting article in today’s New York Times on free will.  It seems that the scientific evidence is starting to build up showing that we don’t really consciously make our decisions, more likely they are made by the brain in the subconcsious somewhere and then our consiousness is informed of it.  This has led to a debate over what free will is and whether we have it.  In my earlier post I said that although our decisions may be made by deterministic processes and the brain and could be predicted if we understood it, since it is our brain making those decisions, isn’t that still free will?  In the article there are some interesting mathematical discussions of Godel’s theorem and how even if our decisions are made through a deterministic process in the brain, it may remain too complex for us to ever predict what decisions we’ll make.

Here is a very interesting account by Scott Adams (the Dilbert creator) about his recovery from Spasmodic Dysphonia. As he explains it: “Essentially a part of the brain that controls speech just shuts down in some people.” Apparently depending on the context/type of speech being performed you are sometimes able to talk ok. For example, he was able to speak publicly, sing, or rhyme, but could not speak in normal conversation. Adams’ theorized that only one of the different brain pathways for speech were affected.  He tried to “re-map” the brain pathways or normal speech by practicing other similar methods of speech such as rhyming.

From everything I’ve read about the brain, this makes a lot of sense to me. All the functions of the body are controlled by the brain. And a lot of seemingly physical ailments may really be in the brain. And a change of thought processes or mental states could affect these ailments. It’s interesting that Adams was able to practice mentally and re-map his brain to function correctly.  But I think your mental state, confidence, happiness, etc can affect your health greatly.  I believe there has been some research showing that your mental state can help with blood pressure, hypertension, and other similar ailments.

Do we have free will?  This is something I’ve been thinking about a lot lately.  If (1) our mind and our intelligence are just emergent properties of the neural networks of our brain and (2) these networks are based on electrochemical reactions that follow basic rules then we should be able to predict all our actions and decisions (assuming that random quantum fluctuations do not come into play, but then our actions are random, not our free choice).  The reactions and interactions involved would be very complex but it seems at least that all our actions are predetermined on the basis of a series of chemical reactions .  Does this mean we do not have free will?

I believe we still do have free will.  Although it may be possible to predict what decision someone is going to make by looking at the current state of their brain and knowing the experiences they are about to encounter, are they not still making the decision freely?  The very chemical reactions we would watch to predict their decision are in fact their decision being made.  Although these reactions are predetermined on a set of physical laws, the very reactions themselves are us, they are our decision making processes.  They are based on the development of our brain through our life and our experiences.  I do not think the fact that these chemical reactions may be pre-determined takes away from the fact they are, in fact, us making these decisions.  Being able to predict someone’s choice does not necessarily mean they had no free will to make the choice.

I was thinking about how things will happen in the future when we are able to download our minds into a computer or robot or some other device to allow our consciousness to live on forever. When Gerard and I were discussing this a while ago he was complaining that the living entity would no longer be you. Imagine these two scenarios:

• First scenario: You are getting old and so as a backup “insurance” copy of yourself, you have your mind downloaded into a robot. You and the robot are both active for a few more years until you eventually die. However you also live on in the robot with all the same childhood memories, experiences, etc from when you were little.
• Second scenario: You are getting old so you decide transfer your mind to a robot so you can continue living. Your mind is downloaded to the robot, your body is put to rest, and you wake up in your new robot body. You have all the memories and experiences that you used to have, but now you have a shiny robot body.

To me the first scenario does not seem too pleasant, even though your mind lives on in the robot, “you” still die. Of course you also live on but the human version of you does still die and even when you decide to backup your mind to a robot you will still know you are going to die, just some copy of you will live. However in the second scenario, it seems like you are just getting a new body, like a snail changing shells. It doesn’t seem as bad, it seems like you can live forever and you won’t die. Even though that robot is the same and the only difference in the two scenarios is that in the second one the human body is killed much earlier. It’s weird how little changes like that can make such a big impact on how something seems or feels.

I just read Creating Mind: How the Brain Works by John Dowling. A lot of the book was just about basic brain workings, but I did find one part interesting. Apparently when we are born the brain is wired up for all possibilities, but there is a critical period where we lose any abilties that we do not take advantage of.

For example, the visual cortex is wired at birth with neurons to receive input from both eyes. If one eye is not functioning properly during early ages, the visual cortex ends up with vastly more neurons responding to the good eye. However if the eye is injured after this critical period, little change occurs. It is also difficult to reverse this effect afterward as well. Dowling theorizes that this happens because the neurons from the two eyes are “competing” for synapses in the visual cortex during this critical period. When one eye stops sending signals the other eye starts to win more synapses from the visual cortex.

It’s very interesting that during this critical period of development you literally must “use it or lose it.” They’ve found this in many different areas. If you don’t use your left eye during the critical period you will lose the ability to use it. When children are born they can easily discern all sounds and make all sounds but they lose the ability to discern and speak the ones they don’t use (apparently most adult Japanese speakers cannot discern l from r). This is the reason why it is so hard to learn a foreign language when you are an adult. In humans the critical period is from 6 months old to about 6 years old. Dowling suggests that we need to provide our children with full and rich environments so they don’t lose any skills.

Here’s a quote from the book to summarize:

“The conclusion is that the circuitry necessary to carry out complex neural tasks is formed during brain development. But at least some of it is labile, which means it has to be used if it is going to be retained.”

It’s scary that what we do at such a young age can have a such a profound effect on our brains. I wonder what abilities we are born with that we lose because they’re not used during that time….

In the book AI: The Tumultuous History of the Search for Artificial Intelligence by Daniel Crevier, Crevier brings up an interesting question: what is common to all forms of intelligence?  When we do eventually develop artificial intelligence, it will surely not be exactly the same as human intelligence.  And if do eventually meet some alien race, their intelligence would probably be different.  What aspects of the mind are required for intelligence to come about?  What would be common between these different types of intelligence and what would be different.

One interesting example is the android Data on the TV show Star Trek: The Next Generation.  He is an artifically intelligent lifeform, yet he has no emotions.  Is it possible to be intelligent without emotions?  Could intelligence develop on its own without them?  Emotions are important for motivation, in the book I just read, Creating Mind: How the Brain Works by John Dowling, Dowling says “Without emotions, why bother?” It’s very true that emotions are the driving forces behind our lives and I can’t imagine we would ever have developed into the intelligent beings we are today without them.