Recently, Rob England (best known for his The IT Skeptic blog) and Gene Kim (co-author of The Phoenix Project, the forthcoming DevOps Handbook, and organizer of the DevOps Enterprise Summit, “DOES”) interviewed each other about DevOps. In our last article Gene asked Rob about how a skeptic came to be a fan of DevOps.
In this article, Rob gets to ask some skeptical questions of Gene.
Rob: Are there holes in DevOps? What doesn’t it address (yet)?
Gene: I think one of the most obvious gaps in DevOps is addressing the widespread gap in automated testing, especially for legacy code. As Michael Feathers once quipped, his definition of “legacy code” is any code that doesn’t have automated testing — which makes me think of how many of us have friends who are writing legacy code today as we speak!
In my opinion, one of the largest gaps in is the proper integration of our testing and information security objectives into our daily work. But there are so many great case studies emerging on how people have dramatically increased their quality of their code by creating and running automated tests into their daily work, including Google, CSG, HP, etc.
One of my favorite phrases from DevOps Enterprise Summit 2015 came from Bill Shinn of Amazon — he optimistically described that in most cases, it’s now straightforward to convince information security that DevOps can be just as secure, and likely even more secure, than traditional software development methods.
He observed that:
“DevOps is all about bridging the gap between Dev and Ops. In some ways, the challenge of bridging the gap between DevOps and auditors and compliance officers is even larger. For instance, how many auditors can read code, and how many developers have read NIST 800-37 or the Gramm-Leach-Bliley Act? That creates a gap of knowledge, so we’ve got to figure out how.”
I’m super excited about extending the work we did on the DevOps Audit Defense Toolkit to help bridge this gap — my belief is that it’s easier to teach DevOps practitioners about audit than it would be to teach auditors about DevOps. And if there’s anything that the DevOps community has proven to be incredibly good at, it’s this type of boundary-spanning work.
Rob: How might DevOps fail? What could relegate it to the dustbin of history, or as a fringe activity that never went mainstream?
Gene: Like in the previous year, we asked each presenter and attendee at the DevOps Enterprise Summit what their top obstacles were in their DevOps transformation. It was genuinely surprising to me how nearly two-thirds of the issues were along these lines:
● How do I to change to a culture that values improvement, end-to-end collaboration, and technical excellence?
● How do I get my leadership onboard, especially more stodgy and conservative leadership?
● How do I get started and create my first win, especially when we don’t have enough time and dedicated resources right now?
● How do I accelerate DevOps adoption in my organization?
The bad news is that these definitely aren’t technology issues — it means that technology leaders who are driving DevOps transformations in their organizations have to develop the same level of leadership and political sophistication that you’d expect to see someone driving a business process re-engineering initiative.
But in my mind, this convinces me, beyond a shadow of a doubt, that these technology leaders are the next generation of business leaders.
I loved how you described the progress of DevOps as “relentless and remorseless” — without a doubt, people will figure this out, and it will result in winners and losers in the marketplace.
Rob: How is the term “DevOps” misused? Is it being taken in the wrong directions?
Gene: There certainly is a sense of danger when existing $12 billion software product suites are suddenly being recast into a DevOps-relevant umbrella. But, in so many cases, that might be an appropriate step — after all, what does it mean when automated testing tools aren’t just for testers to run, but for everyone in the value stream?
I think this recasting of roles of how everyone participates in the technology value streams is exciting and has the potential to dramatically improve the outcomes we get from software development and operations. Vendors, as well as consultants, are an important part of mobilizing changes.
Quite frankly, the fact that DevOps is so discussed confirms that it is relevant to the problems that people are having — I’ll take relevance over irrelevance every time.
Rob: Skeptics seek the evidence. Is there enough data to support DevOps, and where do we need more?
Gene: At the risk of oversimplifying, science requires both theory-building and theory-testing. Like you, I’ve always sought to find empirical evidence that substantiated which “best practices” actually resulted in improved outcomes — since 2013, we’ve collected survey responses from over 20,000 respondents to better understand what high performance looks like, and what practices improve performance.
In summary, we found that high performers were significantly more agile and reliable than their peers:
• Agility metrics
– 30x more frequent code deployments
– 200x faster code deployment lead time
• Reliability metrics
– 60x higher change success rate for production deployments
– 168x faster Mean Time To Restore Service
I love the DevOps Enterprise Summit, because it helps us take findings like this to surface experience reports of people using these practices, share their outcomes, and do further theory-building.
As a community, we’re not just reporting experiences, but testing practices with an underlying causal model. We are sharing lessons learned and creating the conditions where we can help each other, often from wildly different domains.