A Lesson Hard Learned: Hiding Behind A Service Level Agreement

But there is no joy in Mudville- mighty Casey has struck out.ServiceLevelAgreement

Ernest Lawrance Thayer penned those infamous words to describe the shocking strikeout of the odds-on favorite of The Mudville Nine. Down by two at the bottom of the ninth. Two outs; up to bat comes The Mighty Casey. The crowd goes wild.

Casey is popular with the home team fans because of his track record. Obviously he has come through many times before. When he advances to bat with the winning runners on base, the future seems clear – Casey hits a home run, the runners score, and the Mudville Nine wins five to four.

But after much fanfare, and watching two pitches hit the catcher’s mitt for strikes one and two, Casey takes an epic swing for the stands.

But, alas, there’s no joy in Mudville, my friends. Casey has struck out.

A hard learned lesson

Service Level reporting shows how well a service is performing, right? A service with high service level performance is generally popular with the home team crowd.

I learned a very important lesson early in my career. I was the Network Operations manager with a major corporation. My domain included a world-class wafer fab, where a brief network outage caused major problems for the manufacturing process. Mind you, it was a very brief outage – less than a couple minutes in length. But the follow up meetings and root cause analysis, and the ‘what are you doing so this never happens again’ continued for several weeks.

Frankly, it seemed a bit out of proportion for the nature of the outage. After many meetings, I helpfully added something along the lines of “…for as impactful as this outage was, let’s keep in mind that we are still well within the SLA…” The words hadn’t stopped reverberating in the conference room before I realized I’d made a painful mistake for which I was going to pay.

Appealing to the Service Level Agreement was not only unsuccessful in helping my customer to put the brief outage into perspective, it served to prove just how “IT”, (and therefore out of touch) I really was.

I struck out.

I underwent a crash course on the cost of scrapped work in progress, lost hours of production recovering a database, and restarting a hundred-million-dollar production line. The impact was enormous to the business.

I’ve never forgotten that outage. I recall it like it was yesterday. I’d like to believe the managers who invested in my education have completely forgotten the whole episode. But somehow, I’m afraid they are telling the same story elsewhere about the clueless IT manager who tried to hide behind a  Service Level Agreement.

When it really counts

FedEx used to have the slogan – “when it absolutely, positively has to be there overnight”. ‘FedEx’ is synonymous with a perfect delivery record. It’s why you pay the premium price… because it absolutely, positively has to be there. No fail.

The problem with SLAs is they’re cold, hard, statistical math. They don’t tell the story. Had my network outage happened when there was no production running, it may not have even been noticed. Likely no one would have cared.

But that’s not how the story goes. They did care, because the network absolutely, positively had to be up to support production. And it failed.

In a way, Casey got a bum rap. By all measures, he was a superstar player with a stellar record. By the numbers; an undisputed champion. But that’s not how he’s remembered.

Thankfully, Casey is a fictional character, because he’s gone down in infamy as the didn’t-cut-it-when-it-counted would-be hero.

But you and I aren’t fictional.

Business Value

Part of the problem is right in our name “IT” – ‘information’ and ‘technology’. No mention of what we’re to do with either. No surprise, then, that we often go on blissfully ignorant of the business value of the IT services we deliver. Clearly I had a different focus than my early career educators.

I mean, seriously, a world class manufacturing application that doesn’t recover gracefully from a minor network blink, one so short it didn’t even show up in network monitoring?

Yes. Seriously.

Big impact. Not a “minor glitch”.

Anything you can do to ‘walk a mile’ in your customers’ shoes will help.

Do you understand your business’ critical functions? Have you done business impact analysis – so you truly understand the value of business processes and the IT services that support them?

Spend time with your customers. Step outside the IT bubble, and look at your services from your customers’ perspective. Recognize that, like it or not, its not what you do 99.9% of the time, it’s what you do when it absolutely, positively has to be there.

That’s what you’ll be remembered for; make it count.

Photo Credit: adwriter via Compfight cc