Stop Intermittent Test Failures that Slow the Continuous Delivery Pipeline
Intermittent failures ruin the whole premise of continuous delivery (CD). In fact, they undermine any software QA process, since they break the fundamental assumption that if your test suite passes, you are good to deploy. Automated testing can run orders of magnitude more tests than manual testing but may result in an ever growing backlog of tests that fail intermittently for reasons that aren’t understood. Whether this is embarrassing, annoying, or scary depends on your sensitivity to risk. No matter the level of your pipeline’s sophistication, you need to reproduce the defect before you can begin to address an intermittent failure. Greg Law describes how capturing the actual execution of your tests makes this possible in a way that logging cannot. Debugging an exact replica of the failure allows you to go back in time to any line of code that executed and to inspect the full program state at that point. This makes test failures totally reproducible, allows them to be fixed, and restores trust in the CD process.
Greg Law is the cofounder and CEO of Undo. He is a coder at heart but likes to bridge the gap between the business and software worlds (sadly though, most of his coding these days is done on airplanes). He is happiest working at the OS/application and hardware/software boundaries. With twenty years’ experience in the software industry, Greg has programmed for and managed at companies including the pioneering British computer firm Acorn, as well as fast-growing start-ups NexWave and Solarflare. He left Solarflare in 2012 to run Undo. In his spare time, Greg catches up on email.