Published On: Sat, Jan 24th, 2015

Hands On With Microsoft’s HoloLens

One second, you’re looking at the flat surface of a real wooden table. Then, you’re gazing straight through it, past small trees, tiny confused zombies, and layers of earth into a deep hole filled with animated lava.

Microsoft's new HoloLens

It’s not a peyote-induced hallucination, but a genuinely impressive illusion created by Microsoft’s (MSFT, Tech30) new augmented-reality goggles, HoloLens.

In this case, it was a 3-D game of Minecraft taking place, on, over, under, behind and inside real furniture and walls. As you move your head and body around, the illusion moves completely with you in 3-D, remarkably affixed to the real world objects.

Microsoft previewed the brand new piece of hardware on Wednesday. HoloLens is Microsoft’s foray into the virtual reality market. In its unique spin on VR, Microsoft has developed goggles that offer an augmented — or “mixed” — reality experience. Unlike Facebook’s (FB, Tech30) Oculus Rift, which completely blocks out the outside world to fully immerse the wearer in another reality, HoloLens keeps one foot (and both eyes) firmly planted in the real world.

The lenses of the goggles are transparent, your view of the space around you only selectively blocked by digital images that can mingle with real objects. For example, an interior designer can move around a real room and rearrange 3-D pieces of furniture too see how they will look, even placing a virtual vase on a real shelf.

To manipulate something in the HoloLens world, you use hand gestures, which are picked up by cameras on the front of the device. The single wiggle of an index finger can drop a flag on the surface of Mars or select a color for your 3-D sculpture. The cursor is always at the center of your view, moving when you move your head. Microphones pick up voice commands that bring up menus.

The goggles are wireless and don’t need to be tethered to any device, so while they’re meant for home or office use, they wearer is able to roam freely. The speakers play “spacial” sound, so a noise might seem like it’s coming from behind or beside you, adding to the virtual reality experience.

After an on-stage demonstration of the sleek, wraparound black glasses, Microsoft gave reporters four hands-on demos of the device at its Redmond, Washington headquarters. (Notably, the goggles I tested were not the finished product Microsoft demonstrated on stage — they had jumbles of exposed electronic elements connected to a box that hung around my neck).

The demonstration scenarios attempted to show real examples of practical uses for the technology.

Video games. Gamers tend to be enthusiastic early adopters of experimental immersive technology, and the stunning Minecraft demo seemed like the most natural and realistic use case. I do not usually play Minecraft but would absolutely start, and maybe forget to stop, if I had a HoloLens.

Customer service. A Skype call demo was to walk me through fixing a broken light. The person on the other end of the call could see what I saw (real wires, tools, a hole in the wall) and could annotate my view with colorful lines and arrows. She even doodled a little diagram on a patch of blank wall. Remote customer support is a commonly imagined use for smart glasses, including Google Glass. While a neat thing to have, it’s nothing a phone or PC running video chat wouldn’t be able to accomplish.

Design. HoloStudio is a creative app that is designed for anyone to easily tinker with, like a beefed up MS Paint. A palette of colors and shapes floats in front of the artist who can create simple or complex 3-D models. Final designs can be ordered online, and a printed version of your 3-D purple skyscraper will be delivered to your door.

Virtual reality. At NASA’s Jet Propulsion Laboratory, scientists regularly dig through images of Mars collected by the Curiosity Rover. They’ve collaborated with Microsoft to create a program that places them inside a 3-D rendering of Mars using HoloLens. Whether you look up or down, turn left or right, or walk forward or back, the landscape shifts accordingly so you feel like you’re taking a stroll on the red planet.

The Mars demo was one of the few times I wished I was using full virtual reality goggles. They had reasons for wanted to keep reality involved — JPL scientists are working not playing, and they need to use other tools like computers at the same time. But as impressive as it was, the mixed-reality version of Mars looked thin and desaturated.

Full and partial virtual reality look different because of the underlying technology. HoloLens doesn’t display images on any screen, but projects light directly onto the retina, imitating the way light travels from real objects. The augmented view is contained in a rectangle directly in front of your face. Oculus Rift shows its images on a screen in front of the eyes and extends farther out and around for a more encompassing view.

The virtual reality space is still mostly filled with buzz and not-ready-for-prime-time products. But VR has been anointed the next big thing by a tech industry hungry for something fresh and exciting to woo customers.

Overlaying digital 3-D creations onto the real world is cool, but are there practical uses are there for this type of technology?

Microsoft hopes so. HoloLens was by far the most interesting announcement at a daylong event for Windows 10, and one Microsoft believes can help get customers excited about its products again.