The Truth About A/B Experimentation

THAT Conference 2016
Day: Wed, Aug 10   Time: 2:30 PM   Location: (map)
Level: 300  Primary Category: Testing  Secondary Category: Data/Storage
Tags: product development, experimentation
There are plenty of blog articles singing the praises of using A/B experimentation to iteratively make product improvements, but what details are they missing? After spending the last five years running thousands of experiments across the web, mobile, and email platforms at Groupon, I've learned firsthand about the successes and pitfalls of experimenting at scale. Turns out that experimentation is not simple, nor is it a solved problem. That said, getting experimentation right is the difference between organizations being driven by data versus being driven by a CEO's latest whims.

Attendees of this talk will walk away knowing:

* What is A/B experimentation, and how does it work?
* What happens when all the "low-hanging fruit" optimizations are gone?
* What are the flaws with Frequentist and Bayesian methods, and how did we solve them?
* What are the organizational hurdles that prevent the adoption of experimentation best practices?