Have a basic IT Change Management process, or planning to implement one? Change Management should implement changes faster and reduce the risk of business impact. I’ll show you how the right metrics get you there.
What does ‘Success’ look like?
In IT Operations, we talk about Critical Success Factors (CSF). 
Which sounds grandiose, but is really nothing more than a short list of things that must go right for a process (or service) to be successful.
CSFs deal with why. Why are we doing Change Management? What does the customer need, and what are we hoping to achieve?
CSFs are outcomes (not metrics), but I can’t tell you what outcomes your company needs in Change Management – that’s where you come in.
A few questions to help you discover what they might be for your company:
- What do our customers complain about (related to Changes)?
- Are we meeting customers’ needs for speed and quality? (and how do we know)
- Do we get a lot of change-related calls to the Service Desk?
- Are there business initiatives that are or will exceed our current capability to manage changes?
Keep them short and easy for non-IT people to understand.
Think of things like:
- Reduce negative business impact of changes (value)
- Timely and successful changes (effectiveness)
- Consistent and effective handling of changes (risk)
Don’t take on too many. Choose the one or two that are most important to your company.
Now, how do we know if we’re getting closer to, or further away from these outcomes?
Are we there yet?
Metrics that help us know if we’re moving in the right direction are what Service Management calls Key Performance Indicators (KPIs). Again, I can’t tell you what KPIs your company should use.
But, here’s some common Change Management KPIs that may be helpful:
- Number/percent of failed changes
- Number of incidents caused by changes
- Service downtime caused by changes
- Number/percent of unplanned/emergency changes
- Number/percent changes that met customer’s established requirements.
- Average time to implement changes
I’m a fan of having few – like two – so it’s really clear what’s important. Simple and clear are good things.
Look for KPIs that speak action – what actions do we need to take? If a KPI isn’t triggering or validating active tactical decisions, time to consider other, more action-oriented ones.
What gets Measured gets Done
- Customers want business value.
- Critical Success Factors help us know what that looks like.
- Key Performance Indicators help us know if we’re getting there.
A quick example.
Let’s say your business is tracking customer engagement on the new portal. They’re having weekly action planning to adjust their strategy. They want IT changes implemented on the same time frame. Money is literally on the line and there’s no room for error.
They want faster and less failed changes, so, let’s make that our single CSF:
- Faster implementation with less failed changes
Appropriate KPIs, might be:
- Average time to implement changes
- Number and percentage of failed/rolled back changes
Make these the ONLY metrics you gather and report! Crank up the focus on the KEY metrics that support the CRITICAL business outcomes. Turn down the volume on everything else.
You discover that average implementation time is four weeks, but the business wants IT to match their weekly action/implementation cadence. Four weeks just doesn’t cut it.
KPIs are about action – start by looking closely at changes that are taking greater than three weeks to understand why. Make adjustments to shorten the process, and monitor the KPI to make sure you’re moving in the right direction.
Measuring IT Change Management helps us focus on delivering the right business outcomes. Start by making sure you understand what success looks like from the business perspective. Identify things you can measure that help you take the right actions to achieve success.
Keep it simple and clear. Celebrate success.
And don’t forget to enjoy the journey.
If you’re getting started with Change Management, be sure to check out How to Implement Basic IT Change Management, and How to Mature a Basic IT Change Management Process.
What’s your experience with measuring Change Management?
Photo Credit: Javier Kohen via Compfight cc
Photo Credit: Unhindered by Talent via Compfight cc
Great article!
Hello. What is the normal /average succes/failure ratio when you push a change in prod ? Any number ? Thanks jbp@zone-communications.net
Hello Jacques,
Well, that’s the million dollar question, isn’t it?
The problem is, there’s far too many variables for a straight answer. The whole point of DevOpps is a continuous flow of changes seamlessly into production with zero defects. This is accomplished through shifting the focus away from individual changes to the overall process of delivering change. How changes are prepared, with tremendous emphasis on collaboration between the various IT disciplines and the customer. Streamlining and simplification of the production environment, with strong emphasis on automation of the build and test practices. All designed to increase both the velocity and quality of changes.
Comparing an organization to others can be difficult and problematic for assessing your change capability. One organization’s ‘successful change’ is another ‘failed change’, which perhaps brings up the more important element of your question – what’s the customer impact of changes – what percentage of changes achieve the results the customer anticipated, in the required timeframe, without unintended consequences. That is a meaningful metric that should be first baseline established in your organization. Armed with that baseline, then, your customers determine if improvement is required, and to what degree. That answer drives the need for improvements in the change capability – not a comparison against other organizations’ success rates.
You can always find an organization with a better or worse change success rate then you are currently experiencing. What’s important is are you meeting your organization’s need.