Kinect – an in-depth look at how it works

While Sony is busy working out how it can change videogames, Microsoft is working on how it thinks it can change the world. When developers such as Hideo Kojima talk about Kinect, they don’t just mention how it’ll work in their games; they talk about how security cameras can be smart enough to recognize trouble and how homes can react to your presence and even your mood. Kinect is the first truly tangible step towards Star Trek holodecks and is the Minority Report-style future of how humans will eventually interact with machines. But when jump-suited people somewhere in the future scoff a protein pill and look back at Kinect with their bionic eyes, they’ll find a bunch of Wii rip-offs, snowboarding titles and cuddling games.

Even the world’s biggest publishers and developers have only been working with Kinect for 18 months and the first generation of Kinect games are as patchy as the first generation of 360 games. Sure, there’s one Call of Duty and a Project Gotham 4, but there are more Perfect Dark Zeros and Quake 4s. It’ll be a full year before we see Kinect’s first Gears of War-style mega-games. For now, Kinect is pure potential; it’ll bring hand signals and voice control to squad-based shooters, head-tracking to flight sims and driving games, and will spawn games that are impossible to create on any other console. When the next Xbox is born, it’ll have Kinect in the box. For now, it’s the one 360 add-on that for many will be the must-own gaming tech this Christmas. So let’s take a look at the nitty-gritty of how Kinect actually works.

How laggy is it?
In a recent online interview, Rare’s Nick Burton pegged Kinect Sports’ lag between action and onscreen response at 150ms. Too slow for Street Fighter, but fast enough for Forza’s head tracking.

Does it need tons of space?
A potential problem is beaten by software. “We analyze and optimize for your playspace,” says Ubisoft’s Frederic Blais. “We’re thinking about countries like Japan where living rooms are small. There’ll be no problem with the play space for most.”

Can you sit down?
“Kinect can be used while sitting when developed with sitting in mind,” a Microsoft rep said recently, while Ubisoft’s Blais confirmed it could be set to track body parts in specific positions.

Is it expensive?
Retailers currently have it listed it at $150/£130 (including Kinect Adventures). There’s also a bundle of the new 360 and Kinect for $300/£250, but note that this is only the 4GB version, which means it’s made for the casual players – that’s not enough to install even one full game. Still, considering that same console retails for $250/£150, that’s a pretty hefty discount on the Kinect itself, assuming you don’t want Kinect Adventures.

Is it really not for the hardcore?
Forza 3’s Kinect upgrade shows how Kinect can be used in a high-end game for headtracking, while Child of Eden is the spiritual successor to hardcore favorite Rez and a game born for Kinect’s hands-free control. Other than that, we’ll have to wait and see how hardcore things get.

How it works

Ubisoft’s Kinect expert, Frederic Blais, on the limits of Kinect and hardcore gaming.

How did you start working with Kinect?

It’s been maybe a year and a half we’ve been working on the technology, way before Microsoft had it. I was on that camera (the Israeli 3DV camera that forms the foundation of Kinect) at the beginning – our CEO Yves Guillemot was convinced it would be really interesting for the market. I became the go-to technical guy at Ubisoft, just working on that technology and trying to do everything we could with it. Then Microsoft came along with something really cool – they added the skeleton tracking– so we had the perfect tools to do Your Shape: Fitness Evolved.

Does Microsoft provide special tools for working with Kinect?

Kinect has an SDK – a development kit – that has a skeleton tracking system and everything. Microsoft provides a complete interface, all based on DirectX – everything is clean, everything is easy to use. They provided a lot of support and we worked together to make it work. Microsoft’s skeleton tracking is the way we’re tracking everything in the fitness game and that’s directly from Microsoft’s own technology, but we’re pushing Kinect as far as we can with the depth signal, and that’s the edge we have right now compared to other projects.

Is Kinect hard on the 360’s CPU?

It’s not that expensive, in fact. Microsoft did something really great with the technology.

We heard a while back that it was chewing up an entire core…

Yeah, that’s not true at all. I don’t really know how much I can talk about it but it’s less than one percent (of the CPU’s power), or something like that. It’s pretty cool. We could apply Kinect to an action game and still put the same kind of action on screen. Of course, if you have a game that’s using every bit of every core then of course you haven’t got room for any other thing. The question becomes ‘is it a good idea or not?’, but yeah, you could do it. No problem about it.

Have you thought about using Kinect as a supplement to the 360’s controller?

Well, I know there’s some stuff we were thinking about. I think that technology is really cool when it’s faster than a controller, or you do something you can’t do with a controller. Those are the rules of Kinect design. Think about a hardcore game like Rainbow Six where you’re doing hand signs and stuff like that. It would be cool; it would be more immersive. We’ll have to see if hardcore gamers will be ready for that.

What’s the future of Kinect?

I think it’s all about having no controller in your hands. The Wii was obviously the first step in that direction – a controller that looks like a remote control – but I think the future is about having a complete hands-free solution for the living room. At Microsoft’s conference, they showed us how it could control movies and music and be used to chat to people all around the world. That’s where I’m seeing it utilized going forward – fitness, dance, entertainment, communication – all without anything in your hands.

Which version is right for you? It may not be so clear-cut

E3 is over. Battle plans are unveiled. Time to argue over who's going to come out on top!

Microsoft's newly angular funbox has improved, but makes some hardcore-unfriendly sacrifices