Musing about the ROI of IT

Yup, it’s time for another Very Provisional Post.

There’s something I don’t get about IT and ROI. Something fundamental. And that thing is: How can we possibly use the tools of a very old paradigm to solve the problems of a very new paradigm?

I guess this is something I’ve been musing about for fifteen years, after reading Paul Strassmann’s The Business Value of Computing.

I guess this is something I’ve wrestled with every time I’ve had to stand up and be counted during budget rounds at the various institutions I’ve worked in. And I’ve been in many such rounds, particularly since 2001, where the tone of the budget discussion was “Go South, Young Man“. And I wasn’t that young either.

I guess it is what was at the back of my mind when I read Nicholas Carr’s article in the Harvard Business Review in 2003, when I read his book a year later, and even when I spent time discussing various aspects of the issue with Andrew McAfee.

I guess I’m getting stupider as I grow older. You see, what gets me is this:

Ever since I read the Strassmann oeuvre, I’ve watched computing grow more distributed, more networked; I’ve seen a move towards more “enterprise architecture”,  more middleware, more platforms. I’ve watched a substantial increase in complexity.

This increase in complexity manifests itself in many ways:

  • requirements capture has gotten harder as we made the historical silos merge and coalesce
  • estimation has gotten harder, since everything now connects with everything else
  • testing has gotten harder, particularly regression and end-to-end testing
  • delivery has gotten harder and slower as silo spaghetti entangled us
  • fault replication has gotten harder, and as a consequence so has bug-fixing
  • and everything has gotten harder as the enterprise boundaries began to extend and even disappear

As IT professionals, we’ve recognised this and tried to simplify the chessboard, exchanging pawns, pieces and even queens:

  • using component architecture and reuse to speed up delivery
  • using publish-subscribe bus architectures and adapter frameworks to reduce the number of interfaces
  • using time-boxing  to ease requirements gathering
  • using fast iteration models to  make the gathering process more accurate
  • using increasing standardisation and rationalisation to simplify all this
  • using consolidation, virtualisation and service orientation to derive at least a modicum of value out of Moore’s Law during all this
  • using agile methods in general to speed up all of this

I’ve watched all this happen, watched us learn. But.

During all this time, I haven’t really seen changes in the way we account for our IT investments and expenditures. I’ve seen papers about changes, particularly those suggesting a move towards option theory; I’ve seen articles about such changes: I particularly liked the SMR proposition of Big Bets, Options and No-Regrets Moves. I’ve taken part in long arguments about the processes we use to price and value investments in IT.

But, unlike the IT environment during that period, I haven’t really seen changes in the way we measure the ROI of IT. Just 50-year old lipstick on 500 year old pigs. 

This was a problem in 1987. A bigger problem in 1997. And it’s an absolute killer in 2007.

You see, we’ve moved on. There have been various convergences, convergences of standards, of techniques, even of devices. The opensource community has had its effect, commoditising aggressively up the stack.  We’ve seen telephony become software, we’ve seen the disaggregation and reaggregation of hardware, software and services. [Much of my disagreement with Carr is about timing, not direction. ].

Today we have a new challenge. What Doc Searls calls The Because Effect.

In the past, we could claim there was a direct causal relationship between the investments made in IT and the returns, positive or negative. We had siloed systems so we somehow managed to shoe-horn what we did into 15th century mindsets. As everything became more connected, we couldn’t find the causal relationships any more, so we started wondering whether Strassmann et al were right. Yet we knew they couldn’t be, we could sense the productivity gains, the cycle time gains, the quality gains, even if they were later sacrificed. After all, there were many sacrificial altars: vendor lockin, vendor bloat, the politics of projects, the tragedies of e-mail and spreadsheet, the system of professions.

Last week I was at a conference where there was much discussion about agile methods, and the issue of agile-versus-cumbersome-accounting came up. You know something? I’ve yet to work in a place where people were happy with the finance system. Ever. This, despite finance being one of the first places to be “automated”. I don’t wonder why, I know why. Just ask Sig.

Now things will get harder still. The Because Effect is something we live with already. We make money with X because of Y.  X and Y aren’t unknowns we’re solving for. In many cases, Y is a commoditising infrastructure which enables or disables our ability to derive value out of X, the edge application.

Using traditional ROI techniques, we may drive investment away from both X as well as Y over time, as we continue the shoe-horning madness.  That’s why I read what McAfee and Brynjolfsson researched, why I read what Carr researched. Our measurement tools aren’t up to the job. And the consequences could be tragic.

Just musing. And looking forward to the comments and flames.

12 thoughts on “Musing about the ROI of IT”

  1. JP
    Would it be a useful analogy to say that in the Strassmann model we are seeking to apply deterministic Newtonian principles, whereas what we need are probabilistic quantum principles?
    A similar transformation has taken place in literary criticism, where canonical Leavisite judgments have given to way to Iser and reader response theory. The overthrow of supposedly authoritative doctrine.

  2. I like David’s literary metaphor better than the physical one, because it provides an interesting lens for viewing the distinction between product economy and service economy. One of the key problems is that all of the bullets in both of JP’s bullet lists are all about production, which implies that we are talking about products. Indeed, the very ROI model is product-oriented: If I lay out X dollars today to purchase widget W, then can I measure how much richer I shall be after, say, a year’s time than I would have been had I not purchased W? There are probably plenty of widgets that can be subjected to that kind of reasoning, but I do not think that account for a substantial portion of the IT economy.

    So let us consider a modest proposal (in the Swiftest sense of the word): The proper place for IT management is under Site Services! (Before everyone gets indignant about “professional values,” one of my friends from the UK told me that, during the Second World War, the shortage of medical help was so drastic that the military did its own training for emergency medical services; and, unless I am mistaken, the best trainees were the ones who had experience as garage mechanics. So let’s talk about the work, rather than the masks!) Now I have to confess that I have no idea how any large business budgets for Site Services and even less idea of whether the corporate bean counters even ask about the return on those particular expenditures; but I think there may be a useful object lesson there, even if it is not a very dignified one.

    To think out of the box, you have to GET out of the box. I have yet to encounter anyone involved with Site Services at the “CxO” level (for the usual values of X: I, K, F, E, etc.). Yet the “value that matters” from IT, whether internally or externally, keeps coming back to rendering or facilitating services; and Site Services tends to be the only part of the budget systematically examined in terms of paying for services rendered.

    This is why we should examine that literary metaphor, at least in passing. Leavisite criticism concentrates on the text as product as an object for evaluation. Iser introduced the idea of reading as a relationship between author and reader (which may even extend beyond completing the reading of any particular text). Leave it to a good literary critic to understand a service relationship better than the economists and technicians!

  3. JP, It’s a good book you are reading. I agree that in 2007 we are at a crucial point in the industry. My opinion is that there is a painful disconnect between the technology industry and real customer priorities.

    I’m very tired of hearing about how technology is driving business innovation today and every company needs a skyscraper high stack of SOA infrastructure to get competitive. Why? Because it is always coming from technology company executives. Of course technology is an enabler that allows business plans to be implemented where they could not have been without it.

    The transportation system of the US is amazing but we don’t walk around saying it drives business innovation. It’s a valuable piece of infrastructure. Dell and Amazon wouldn’t be here today without it. But let’s not go overboard.

    ROI matters but I think it is a bit of a red herring. What business executives need is much closer alignment between what IT can do for them vis a vis their business initiatives. During the same conference you attended there was a presentation given that touched on “business application architectures” but it was from a pure technology perspective.

    I could go on as this is a current research topic but wanted to say I think you are really onto something!

  4. I find it difficult to get excited about the “ROI of IT”. Why? Because it seems to miss a findamental point which is that on its own IT cannot deliver ROI; it is an enabler for people to do things smarter, cheapers, faster etc. On its own it delivers nothing. It has to be used by people to derive value. Hence my view is that ROI needs to be built around all of the components that deliver the ROI for any business imperitive that has an IT component. The challenge is for IT executives to articulate their contribution to organisational value building.

  5. I’m intrigued by where David and Stephen have taken the conversation, guess I will follow it at The Rehearsal Studio….

    MNB, the route you take is one I’ve been on a few times, seems to lead nowhere (as Kris suggests).

    We can isolate many things, even salesmen and products, and say that they by themselves don’t make money. Which is a good thing, because it’s true. Teamwork is essential. Brand plus market access plus platform plus infrastructure plus enablers plus supporters and all that jazz. But this network is hard to shoe-horn into the silo approach.

  6. Without wishing to revert to a previous stage of the argument, it is worth pointing out some severe limitations that Carr’s book imposed upon itself. Understanding these limitations may save us driving up a cul de sac.

    On p xii Carr’s book defines IT as excluding people and information. This definition means that Carr’s argument applies only to boxes, screens and wires, with software marginal. In that case we can all agree that they matter less, can we not?

    On p xiv Carr’s book states that only by becoming an infrastructural network can IT ‘deliver its greatest social and economic benefits.’ Since we all agree this is precisely what’s happening, it follows that despite Carr’s title, IT now matters more than ever, but not quite in the old way it used to. Once again, hands up if you dissent.

    I believe I first heard the arguments about measurable benefit from IT recited soon after I became a trainee programmer in 1962. I am not convinced that the level of discourse has risen much since. We need a new paradigm and must risk being regarded as airy-fairy theorists to fabricate one. Bearing in mind what JP has posted and the subsequent comments, I believe a vital distinction is between procedural necessities and behavioural necessities. I believe changes in areas of thought that are seemingly remote from IT, such as quantum theory (pace Stephen’s distaste) and literary criticism, have a part to play. I will plough on.

  7. Stephen, this is Doc’s Snowball effect in operation. It can be immensely satisfying and immensely frustrating, depending on your point of view. I love seeing it happen. So I shall watch with interest, and participate when appropriate.

  8. Incentives are a bigger problem than accounting, but related to it. Is there any way to reward IT staff for their part in putting together future-usable infrstructure after they have moved on to other responsibilities? More here.

  9. ROI for IT most IT projects is total bullshit. Now that I have my opinion clearly stated, let me expound on my personal experiences that lead me to this conclusion:

    I am a risk manager for an insurance company. One project I am leading is building what might loosely be described as an enterprise risk data mart. Basically we want to be able to make better forecasts both for the short term and the longer term. To do that we need to be able to see all the moving pieces in one system. As part of the standard op procedure the IT folks who are on my project have to prepare and file ROI and CBA analysis for the project. I’m with the business unit, the business unit has decided that we need this system in order to better understand our risks and make good projections. We have the budget, we have approval. But the f’ing IT guys are required to spend hours of their time preparing B.S. statistics about ROI and CBA to feed up their command stream. What a royal waste of time!

    I’m with you, JP. I don’t get it. I don’t understand how to calculate the ROI on an IT system designed to support decisions by the business. Any numbers created by the exercise are just fantasy. As you state, the metric is just broken.

    But what IS the right way to allocate finite resources among infinite wants? How does one create proper incentives to efficiently allocate IT dollars?

    I like that this is becoming more of an economic problem and less of an IT problem!


Let me know what you think

This site uses Akismet to reduce spam. Learn how your comment data is processed.