Microsoft recently demoed Project Natal at E3. It looks really impressive. Here’s a video. And here’s part of an article from Wired:

Later, Microsoft producer Kudo Tsunoda, who’d demoed this game to us, showed us what was going on “behind the scenes.” He brought up an image that showed what Natal’s two cameras could see. The standard RGB camera in the unit, he says, was watching the positions of our skeletons. Sure enough, crisp solid stick-figures appeared on screen, representing me and Baker. As we moved, the stick figures on screen matched our movements. It wasn’t picking up our individual finger movements, but it could tell when we bent our wrists, picked up our legs, tilted our heads, etc., not to mention tracking us as we moved around the room.

Meanwhile, an infrared camera was tracking our distance away from the television. As we moved forward, our outlines turned red, as we moved back they shimmered blue. By combining both of these sets of data, Natal knows exactly where you are and what you’re doing. And it knows when I step behind Baker, or if he walks in back of me, and it still tracks, separately, the parts of both of us that it can see.

Seriously, the motion capture we used at the hospital required you to wear a bunch of reflective balls, used 12 cameras placed around the room, needed constant calibration, and still would mess up, and miss markers sometimes. If this thing really works this well, it’s really impressive. I really want to see it in a room with lots of people and varying lighting conditions.