The Agile Way to Test Software? Continuously.

Popular on-demand alcohol-delivery firm Minibar delivers a new and improved app with continuous testing.

The app development process has evolved to become more agile, and so too must the testing used to vet those apps. Not long ago it could take months for application updates to reach consumers. Microsoft once let three years go by between releases of Microsoft Office. But times have changed, and apps are updated regularly now, sometimes even on a daily basis.

For Minibar—a popular on-demand alcohol-delivery company that launched in 2014 and now serves more than 35 cities throughout the United States—customer expectations for speed drive the need for continuous testing.

"We're always thinking about how to make the experience faster and more convenient. Our mobile shopper, in particular, is looking to find what they want and check out quickly, so we're constantly thinking about how to design an experience that's easier for them," says Lara Crystal, co-founder and co-CEO of Minibar, who works closely with the developer team. "Continuous UX/UI testing helps us ensure that we are designing and building the best customer experience using feedback at every stage of development."


Automated Testing, With a Continuous Twist

While automated testing remains an important tool for app development and troubleshooting, it only evaluates individual components of software at critical points. With traditional automated testing, Minibar's development team could, for example, test its "add to cart" functionality or the speed of its check-out process. But given the company's speed of development, these isolated tests were no longer sufficient.

For companies like Minibar that have embraced the agile mindset, continuous testing—the process of executing ongoing automated tests as part of the software delivery pipeline—is the agile way to test.

In Minibar's case, continuous testing became the only viable option when it recently relaunched its mobile app with a sleeker design and a faster, more intuitive shopping experience. But like any fast-paced IT operation, Minibar's development team couldn't afford the bottlenecks associated with testing automation.

“Our dev team was responsible for testing all new features throughout the redesign," Crystal explains, adding that continuous testing was the only way to ensure that all features, from sign-in tabs to the checkout button, were ready for release.

For Minibar, using continuous testing presented clear advantages. For one, there's less risk associated with every software release because apps are vetted more thoroughly, rather than at individual milestones. There are also less testing-caused delays, because continuous testing happens simultaneously with development. And, when speed and seamlessness of experience are key metrics, as they are for Minibar, it pays to look at the app experience as a whole.

A Continuous Feedback Loop

Continuous testing saved Minibar's development team time and eliminated the risk of shipping shoddy updates, which freed the team to supplement its internal tests with direct feedback from customers.

“There's a variety of ways for us to measure the effectiveness of the overall experience [on our app]. The most straightforward is a star rating that consumers provide on every order, as well as the NPS surveys they complete," explains Lindsey Andrews, Minibar co-founder and co-CEO.

That feedback often reveals surprising improvement opportunities, undetected by testing. One example? Practicality trumps flair.

“Customers are busy doing a million things at once, and Minibar Delivery is there to make their lives easier by eliminating an errand. So while a subtle, stylized button might look better, a bigger, more obvious button works better for helping customers move through the shopping experience," Crystal says.

Learn how continuous testing could help your development team improve its apps and services, without sacrificing speed or quality: Continuous Testing for Continuous Delivery: What Does It Mean In Practice?

Maria Minsker
By Maria Minsker | September 13, 2018

Want to create an agile business?

We’ve got even more insights, guidance and tips.

Read more >