Conference archive


Monday, June 6, 2016 - 8:30am to 12:00pm

A/B Testing: Improvements through Continuous Experimentation

“The goal of a lean startup is to learn what is valuable to a customer” (Eric Ries, The Lean Startup). Do you know what is valuable to your customers? Can you measure which improvements result in more engagement, more sales, or more long-term use of your product? A/B testing―or experimentation―is one way for software teams to learn more about how customers use software, and which changes drive customer engagement and satisfaction. In this fun and interactive tutorial, Alan Page begins with the basics of experimentation, including everything you need to build a minimal experimentation platform, elaborates on the details of running valid and trustworthy experiments, and progresses through advanced experiments and statistical analysis in order to give you everything you need to get started and dive deep into running your own A/B experiments. And, so you can avoid many of the mistakes he (and countless others) have made in this area, Alan shares the common traps, pitfalls, and dangers of A/B testing and statistical errors.

Alan Page
Alan Page has been building and testing software for nearly 25 years. He was the lead author on the book How We Test Software at Microsoft, contributed chapters for Beautiful Testing  and Experiences of Test Automation: Case Studies of Software Test Automation , and recently published a collection of essays on test automation in The A Word. He also writes about a variety of software engineering subjects on his blog at
Alan joined Microsoft as a member of the Windows 95 team, and since then has worked on a variety of Windows releases, early versions of Internet Explorer; Office Lync and Xbox One. Alan also served for two years as Microsoft’s Director of Test Excellence. Currently, he’s working on a brand new collaboration application for Microsoft.