How Wayfair Built Its Augmented Reality App
The online furniture retailer used Google’s Tango platform and clever engineering to create a virtual showroom in consumers’ homes.
This program was produced by the Marketing Department of WIRED and Ars in collaboration with CA Technologies.
Mike Festa won online furniture retailer Wayfair’s hackathon so soundly that the company granted him his own development team. Festa, who is now director of the company’s R&D lab, Wayfair Next, had tinkered with Google’s Tango motion-capture platform a couple years prior to the competition, but he had not done much with the technology. When his employer challenged staff coders to come up with new ideas to improve operations, such as reducing returned items, it clicked: an augmented reality app for furniture buying.
Online-only retailers save a lot of money and gain huge competitive advantages by not operating brick-and-mortar stores. Buildings require rent, staff, insurance and on-site inventory. Being a lean, online-only operation means cutting out almost all of that.
But for stylish purchases like furniture—unlike, say, books from Amazon—digital product specs and renderings leave unanswered questions for many consumers. How does that end table fit with the sofa? Would that dining set look good in low light? Will that loveseat seem bulkier than in the pictures?
“[Augmented reality] won't be mass market today—but in a few years we think it will be. In the long run, it’s going to end up being a crowded marketplace.”
— Mike Festa, Director of the Wayfair R&D Lab
That’s where many online retailers like Wayfair hope augmented reality will bridge the gap. “It won’t be mass market today—but in a few years we think it will be,” Festa says. “In the long run, it’s going to end up being a crowded marketplace.”
The company has an Android-only app that uses the phone’s camera to overlay what a piece of furniture would look like in a customer’s home. The prospective buyer can move the piece or walk around the room to see the arrangement from alternate angles.
Festa concedes there is still a lot of work to do before the experience mimics an actual showroom. First, phones running augmented reality applications burn significant battery power processing the overlays and images of the room. Keeping the virtual furniture correctly oriented requires the phone to continuously monitor and correct for position and orientation. If someone walks into the frame, the phone needs to sort out what’s in front of or behind the virtual furniture.
“At this point, it's more of a software problem than a hardware problem,” Festa says.
Like most graphical rendering issues, it’s the final details of accuracy that prove most challenging. The Wayfair team wrote code to work with the camera’s orientation sensors to ensure the image positions properly in space. Rooms also have specific lighting, from the sliding door to the yard, track lighting, a lamp. And for a piece of virtual furniture to feel at home, the shadows and coloring need to match. Festa’s team is continuously optimizing that “uncanny valley” of furniture realism.
If a customer is pleased with their renderings, Wayfair keeps the purchase just a tap away. That ease, the team hopes, may even be what motivates customers to endure the not-quite-real experience.
And Festa’s team isn’t finished creating buying options. The team recently released an app for the Oculus virtual reality headset. Novel (and pricey) VR hardware hasn’t yet entered the mainstream, but Wayfair can count on the fact that millions of people have Android smartphones to preview furniture in augmented reality.