Testing and Measurement in DevOps: Find Solutions—Not More Problems

The promise of DevOps is to deliver new features faster following today’s best practices. However, blindly automating the delivery pipeline by installing Jenkins, Chef, and Docker without adapting test approaches will cause a great number of deployments to fail. While the tester’s role and testing are critical for the success of DevOps, the tester’s objective changes—from finding more defects to understanding the patterns that make deployments fail. Then, the job is to automate the detection of these patterns through quality gates into the pipeline. Using examples from Capital One, Verizon, and others, Andreas Grabner explains which technical metrics—# of SQLs, # Memory Allocations, # of Service Calls—to capture while testing in order to identify bad coding and architectural patterns earlier. In the DevOps world, you are no longer measured by number of tests created, executed, and problems reported; you are measured by your collaboration with development and operations, and the success rate of your team’s deliverables.