viewing Tony Stark wave his hands to manipulate projected constructs is an ever-approaching reality — at least in terms of gesture-tracking. lift — a prototype built by a team from UC Irvine and FX Palo Alto laboratory — is able to track up to ten fingers with 1.7 mm accuracy!
Lift’s gesture-tracking is achieved by using a DLP projector, two Arduino MKR1000s, and a light sensor for each digit. Lift’s design allows it to work on practically any flat surface; the projected image acts as a grid and work area for the user. As their fingers relocation across the projected surface, the light sensors feed the information from the image to the Arduinos, which infers the location of each finger and translate it into a digital workspace. Sensors may also be mounted on other objects to add functionality.
So far, the team has used lift as an input device for drawing, as well as using it to feign gesture controls on a conventional laptop screen. The next step would be two or much more projectors which would allow lift to function fully and efficiently in three dimensions and directly interacting with projected media content. Can it also operate wirelessly? Yes. Yes, it can.
While we don’t have Tony Stark’s hologram workstation quite yet, we can still play Tetris, fly drones, and mess around with surgical robots.