Technical Debt: The real cost of deferred maintenance

Forgone IT maintenance comes at a cost. For most companies, it’s a never ending battle. Here’s five tips for getting out of technical debt.

Falling behindTechnical Debt

A company’s IT infrastructure is complex and requires constant attention to remain stable, secure, and up to date. Like everything else in life, there’s the eternal balance between bleeding-edge-current and falling behind.

Both come with a cost.

But, let’s be honest, most of us IT people know that far, far more frequently we deal with postponed maintenance and upgrades in favor of delivering new project and business-facing services. It’s easy to forgo maintenance, and frankly, most short-term delays are tolerable and make a great deal of sense.

Sadly, we all know that ‘postponed’ actually means ‘never’. You know how it goes – just get it up and running – we’ll fix it later. But ‘later’ never seems to come. And the further we get behind, the more fragile the infrastructure becomes, and we increasingly rely on heroics to keep old servers and systems running far past their expected life.

It’s come to be known as ‘Technical Debt’ – an interesting term, because it has some parallels with financial debt – something many of us know a bit about.

Finance 101

It goes something like this (keeping in mind that I’m not an accountant, and I’ve never played one on TV): Organizations borrow money to fund current opportunities. In taking out these loans, they get the benefit of having money they don’t possess, in exchange for taking on what finance people call ‘debt service’ – basically loan payments. The general idea is that whatever they do with the borrowed money now will produce more income than the loan payments later.

Follow me?

On a balance sheet, ‘debt service’ – the payments you must pay against debt incurred – is sometimes expressed as a percentage of revenue. This is where it gets interesting, because the more debt you take on, the higher the percentage of your income that goes to servicing debt. Debt service is essentially a zero-gain use of money.

Theoretically, you can take on so much debt (don’t get me started on politics!) that your entire income is spent paying loans, leaving no income to fund anything new. Obviously this hamstrings an organization to the point they’re so buried in debt they become ineffective.

Deferred maintenance

Meanwhile, back in the world of IT, when needed maintenance, upgrades and infrastructure investments are deferred, technical debt increases. The extra time and effort required to keep an aging infrastructure limping along consumes resources that could otherwise be spent enabling the business.

In simplistic terms, technical debt shows up on the keep-the-lights-on vs. project balance. The further in technical debt you go, the more it takes just to “keep the lights on”. The number and severity of incidents increase. Systems become “fragile”, and simple troubleshooting of one problem can trigger others.

Just like in finance, organizations can get so far behind that they consume all or most of their IT resources just keeping things running. Because business usually takes keeping-the-lights-on for granted, they increasingly view IT as unable (or unwilling) to deliver on meaningful projects, and has an increasingly dim view of IT.

It’s particularly demoralizing to IT staff who are unable to provide the level of service they’re capable for reasons outside of their control. They look bad when they’re unable to implement even the simplest of changes without problems; when they’re unable to fix a problem without causing a couple new ones in the process.

Freedom from debt

I’ve never seen an IT shop that didn’t have some technical debt. It comes with the territory.

But what do we do about it? How do we ‘get out of debt’?

Turning to none other than Money Magazine, Kara Brandeisky gave her Seven Ways to Free Yourself From Debt – For Good.
Let’s see how some of them apply to technical debt:

1. Know what you owe.

The connection between “those things IT people always complain about” and business impact is hard for many organizations to grasp. Constantly lamenting about “things that should be done”, or a business case for something that appears to be entirely an IT internal, self-serving thing often fall on deaf ears. Worse – it can add to the general perception that IT just doesn’t get it, and only want to focus on ‘IT things’.

Make a list of areas needing updating, keep it current, and make it part of the regular discussion among IT and business leadership. Incidents, especially highly visible, painful ones can be tied (where appropriate) to specific debt, and remediation recommended as the way to address the inevitable “how do we avoid this in the future?” question.

But, just like financial debt, don’t expect instantaneous results, and don’t make it the centerpiece of every discussion.

2. Follow the 10×10 Rule

With a clear understanding of your debt, and the business impact – identify small elements of debt that are directly related to significant projects or efforts. Add these “10%” debts as project risks and propose remediation as a project risk mitigation strategy. For instance – if new applications are targeted to be released onto debt-identified servers, suggest that the project include system upgrades to reduce risk of implementation risks and post-implementation capacity challenges.

Obviously this should be done with integrity, and not tie unrelated items together on the off chance they’ll somehow be approved. (As is common in politics, but let’s not start in on that, shall we?)

Over time, these barely-noticeable project additions can add up to significantly reduce technical debt. They also can help change the culture, making needed updates part of the normal flow, even if ever so slowly.

3. Reduce Your Rates

If you must pay debt, why pay any more than you must, right? We all know you can’t do everything, so be very intentional about where you incur your technical debt. You should target less business-critical systems. If you don’t have a formal business impact analysis, you’ll have to make informed guesses on which systems, and always validate those assumptions with your customers.

Use incident data to identify which systems have the most customer-impacting incidents – in quantity, severity and impact. For instance, if your internal file servers are on unsupported hardware, but haven’t had any significant incidents, it’s probably a better place to incur debt (forgo maintenance) than a public-facing web server that hosts the company’s order processing system.

4. Stop lending so much money to the IRS

Overpaying your taxes month-to-month and waiting for the end of the year to get some of it back just doesn’t make sense.

In the same way, don’t make technical debt an all-or-nothing deal. If you can’t do all the maintenance you’d like, try to do at least some. Rather than forgo maintenance completely, try extending the cycle – daily tasks deferred to weekly, weekly to monthly. You get the idea. Try holding a tight line on no maintenance – which is way over paying.

5. Ask for help

For many of us, asking for help feels like admitting weakness and failure. We also don’t want to always be complaining about how everything’s falling apart.

But you also know if you don’t ask, you won’t get.

So ask.

Ask vendors, partners and providers. Believe it or not, they’re not only in it for the money. Well, OK, actually they are, but they also know that happy customers are repeat customers, and that’s good for business.

Vendors have resources. Ask, for instance, if they can provide some gratis integration help during an upgrade. Ask if they can provide some design consultation, or other services that can be provided over the phone in small time segments.

Use your network, both inside and outside the company. Leverage the knowledge and expertise of others who have addressed the issues you’re facing. Don’t limit your thinking to what you can accomplish with only your limited resources. While you’re at it, ask if you can borrow other resources during key implementations or upgrades.

Maybe use Service Desk staff occasionally for field work, or PC lifecycle labor where possible. Make it a win/win – staff gets the opportunity to grow and develop, and you get additional resources.

And don’t forget to ask your up line management for the resources you need. Ask in a ‘what’s-in-it-for-me’ sort of way. Be specific about the problem the resources will address, and how it will reduce risk and business impact.

In it for the long haul

Sadly, there’s no instant solution. Getting out of any type of debt requires time and constant attention. There’s an art to the ongoing conversation around needed improvements and the business impact of not making them. You must be clear and consistent about where investments are needed, and how those investments will address specific business needs – stability, supportability, capacity, incident reduction, and accelerating change implementations (and reducing the likelihood of collateral issues).

While technical debt is a reality of IT life, it does’t have to cripple your organization. Start taking steps to reduce your debt today!

How about you? What have you done to manage technical debt?

Photo Credit: bionicteaching Flickr via Compfight cc