A host of companies are looking to shake-up the ways we interact with computers. Using new motion sensing technology they aim for users to replace typing and mouse-clicking with some of the gestures and movements used in everyday life. WSJ's Jessica Vascellaro reports.A race to liberate computer users from the mouse is kicking into high gear, inspired by the potential of turning hands and other body parts into digital controllers.
The goal: to manage computers and other devices with gestures rather than pointing and clicking a mouse or touching a display directly. Backers believe that the approach can make it not only easier to carry out many existing chores but also take on trickier tasks such as creating 3-D models, verifying whether clothes fit, training athletes and browsing medical imagery during surgery without touching anything.
Such possibilities have spurred furious action at some high-tech heavyweights, including Microsoft Corp., MSFT -1.51%Apple Inc., AAPL +0.05% Google Inc. GOOG -3.64% and companies that make televisions, computers and other hardware. Microsoft, which helped ignite the action with the 2010 introduction of its Kinect accessory for gaming consoles, on Monday is updating the software that allows developers to build applications for the PC version of the product.
Leap Motion
Above, a graphics program using the Leap interface device meanwhile, smaller entrants are developing inexpensive gesture-based products to help bring the technology to a mainstream audience. A start-up called Leap Motion on Monday will begin taking orders for a $69.99 device that allows users to control any desktop or laptop computer by waving their hands and fingers in the air. The gadget, which resembles a pack of gum, is designed for particularly precise actions. Where devices like Microsoft's Kinect track larger movements—like the action of swinging a tennis racket—Leap Motion says its technology can let a person write words or draw a picture, as it tracks motions within fractions of millimeters. David Holz, Leap's chief technology officer, demonstrates how the Leap allows users to rotate and zoom in on a digital globe with one motion, something that would take several steps using a mouse or a touch screen.
How it Works
The Leap is a series of small camera sensors enclosed in a metal case slightly larger than a jump drive, with hard black plastic film on the top.
The device emits a stream of infrared light particles, which bounce off objects within the four-cubic-foot sensor zone.
"It really makes you feel connected," says Mr. Holz, during an interview in the company's bunker like basement office in San Francisco. By contrast, existing tools that control computers "don't work like real life.”Not every interaction with a computer will necessarily be enhanced by gestures. But new interfaces have consistently defined winners and losers in the technology industry—such as Apple's pivotal backing of the mouse on the Macintosh in the 1980s and touch-screen applications since 2007 with its iPhones and, later, iPads. Apple hasn't publicly tipped its hand, but a person familiar with the matter said it has been studying gesture-based technology for years. It has patented an invention for allowing users to manipulate three-dimensional interfaces without touching a screen. Apple declined to comment. Backers believe gesture-based technology, along with voice-activation technology, will inspire entirely new kinds of devices and applications, such as robots that recognize and interact with humans based on their movements or physical therapy training programs that can tell whether someone is performing exercises correctly.