Agile + DevOps East 2020 Concurrent Session : DevOps From a Different Dataset: What 30M Workflows Reveal About High Performing Teams

Conference archive


Wednesday, November 11, 2020 - 11:45am to 12:45pm

DevOps From a Different Dataset: What 30M Workflows Reveal About High Performing Teams

What can we learn about DevOps best practices by looking at data from a SaaS with 800k dev users, 30K orgs and 30m+ builds a month, particularly when compared with surveys where users opt-in? Takeaways include: 
- Teams using CI are fast: 80% of all workflows finish in less than 10 minutes, compared to teams not using CI at all.
- They stay in flow: With CI 50% of all recovery happens in under an hour, and 50% of orgs recovered in 1 try.
- The average team using CI achieves high-performing results: 50% of orgs start six workflows per day across all of their projects, and at the highest end these numbers rise to 39 workflows on the default branch, 74 workflows per project, and 250 workflows across all projects for an org.
- Contrary to other reports, the median time to recovery is 17.5 hours. This implies that when engineers get a failing signal at the end of their day, they wait until the following day to resolve it.

What is driving this disparity between reported and observed behavior? Our hypothesis is the survey designs itself. Our data covers all workflows for all branches, not just on the primary application.


Michael Stahnke is VP, Platform at CircleCI running SRE, Security, and tooling. Prior to this, he worked at Puppet running PE, Platform engineering as well as SRE. He is an established author where he has co-authored two State of DevOps Reports and most recently, a Data-Driven approach to continuous integration. He is also a popular speaker and has attended various DevOps Days, CTO Summits, Puppetize conferences and more. He also founded the package repository EPEL and wrote a book on SSH in 2005.