When critical IT suppliers fail, the impact can be severe

The collapse of 2e2 is a warning to all IT organizations.

As IT evolves into a multi-sourced, supplier driven model, how many companies understand the risks?

One of the big stories in corporate IT this week has been the troubles of the IT service provider 2e2.  2e2 are a supplier of a range of outsourcing, resourcing and support services. As liquidators moved in, staff were cut and services ceased. It’s horrible for the staff, of course, and I wish everybody all the best. For customers, particularly datacenter customers, the situation is uncertain.

Increasingly, the role of IT within a large organization is to be a broker of services, driven by a range of internal functions and external suppliers. This trend continues to drive growth in the outsourcing market, which Gartner now estimates to be in excess of quarter of a trillion US dollars.

This hybrid model means that a typical IT service, as understood by its customers and stakeholders, will be dependent on both internal capabilities, and the performance and viability of multiple external suppliers. The collapse of 2e2 is a reminder that suppliers sometimes fail. When this happens, not all organizations are prepared for the consequences.

A failed service can kill a business

A harsh truth:  The failure of a critical business service can kill a profitable multi-billion-dollar company in a matter of months.  It has happened before, and will happen again.

One of the biggest examples of this is the billing system failure that caused Independent Energy to collapse.  A British darling of the dot-com stock market boom, Independent Energy was a new energy supplier, operating a multi-sourced supply chain model to compete with large post-privatization incumbents.

The model was initially a big success.  Floating in 1996 for 15 million pounds, it had risen in value to over 1 billion pounds (approx USD$1.6bm at current rates) by 2000.

However, in February of that year, the company was forced to admit that it was facing serious problems billing its customers, due to serious problems with systems and processes. Complex dependencies on external companies and previous suppliers were compounded by internal IT issues, and the effect was devastating.

The company simply couldn’t invoice the consumers of its products. The deficit was significant: the company itself stated that it was unable to bill some 30% of the hundreds of millions of pounds outstanding from its customers.

The company itself seemed otherwise to be healthy, and even reported profits in May, but the billing problems continued.  Months later, it was all over:  in September 2000, Independent Energy collapsed with at least 119 million pounds of uncollected debt. The remains of the company were purchased by a competitor for just ten million pounds.

The impact on customers

For 2e2’s customers, the immediate problem is a demand for funding to keep the lights on in the datacenter. The biggest customers are reported to have been asked for GB£40,000 (US$63,000 immediately), with smaller customers receiving letters demanding GB£4,000 (US$6,300). Non payment means disconnection of service. Worse still, there is the additional threat that if overall funding from all customers is insufficient, operations might shut down regardless:

But the complications don’t end there. The infrastructure in the 2e2 datacenters is reportedly leased from another supplier, according to an article in the UK IT journal The Register. Customers, the article claims, may face additional payments to cover the outstanding leasing costs for the equipment hosting their data and services.

A key lesson: It’s vital to understand the services you are providing

The events we’ve discussed reinforce the importance of understanding, in detail, your critical IT services. The Service Model is key to this, even in simple examples such as this one:

Even for this simplistic example, we see some a number of critical questions for the organization. Here are just a few:

• How do I know which equipment in the datacenter belongs to us, which belongs to the customer, and which is leased? Recently, a number of companies experienced devastating flooding as result of the storm which hit the USA’s Eastern Seaboard. Many are now struggling to identify their losses for insurance purposes. This can cause a serious cashflow hit, as equipment has to be replaced regardless of the fact that payouts are delayed.

• What happens if our cloud-based archiving provider gets into difficulties? In this situation, the immediate impact on live service may be limited, but in the medium and longer term, how will billing and vital financial record keeping be affected?

• Our client tool is dependent on a 3rd party platform. What risks arise from that? A few days ago, Oracle released a critical fix which patched 50 major security holes.  Updates like this are nothing unusual, of course.  But there are many examples of major security breaches caused by unpatched platforms (the Information Commissioner’s Office recently cited this error in its assessment of the Sony Playstation Network failure, adding a £250,000 fine to the huge costs already borne by Sony as a result of the collapse). Of course, there are other risks to consider too: How long will the supplier continue to support and maintain the platform, and what might happen if they stop?

The required understanding of a service can only be achieved with effective planning, management and control of the components that make it up. Is this the most critical role of IT Service Management in today’s organization?