Microsoft tests custom hardware controls on Surface-style device

One big appeal of tabletop computers, such as Microsoft's Surface machine, is the ability to control everything via touch. But what happens when you need more precision than possible with a finger? Microsoft researchers have come up with a way to add customizable hardware controls literally on top of the existing experience.

Their project uses small hardware boxes with special tags on the bottom that are recognized by the computer's vision system when placed anywhere on the tabletop screen. The computer then projects "auras" around the edges of the boxes to let users assign controls of their choice to the physical knobs and buttons.

Microsoft Research is presenting the project at CHI 2009, a computer-human interaction conference in Boston this week. The work was spearheaded by Rebecca Fiebrink, a Princeton University graduate student who interned at Microsoft last summer.

It's one of 27 peer-reviewed Microsoft papers to be accepted for presentation at the conference -- giving the company bigger presence than any other organization there.

Microsoft researchers see the biggest opportunities for the tabletop-computing hardware controls in collaborative situations, where multiple people are sitting around a tabletop computer to work on a project. The prototype tabletop computer they used was larger than a Microsoft Surface, making that type of collaboration easier.

To demonstrate the technology, they used an audio-editing program called Ensemble that uses four boxes to give people precise control over different elements of the sound and instruments.

The approach "lets you control most of the stuff that you want to do on the surface, just the way you're used to, using your fingers, reaching out and touching things, still having that collaborative space," said Microsoft researcher Dan Morris, who worked on the project with Fiebrink and Microsoft researcher Meredith Ringel Morris. "But for the subset of things that really need precision, let's give you some high-precision controllers."

For example, to adjust the volume of a flute just slightly, a user could map that control to a specific knob and adjust it with the hardware. Other applications could include collaborative graphic-design software. Morris acknowledged the possibility of using the same approach to add a traditional keyboard and mouse to a tabletop computer, although that wasn't a focus in this case.

So where is this technology headed?

"This is purely a research project at this point," Dan Morris said. "You know how things work at Microsoft Research: We are always talking to product groups. In this case we've certainly shown demos to the Surface group and are talking about different types of directions this kind of thing might go, but there's no specific plan to build a product based on this concept."

READ MORE and COMMENT, more