Thinking about predictability: More musings about Push and Pull

Chichen Itza photograph courtesy JuanRojo

If you’ve been following this blog for long, then you’ll probably know that I’ve been interested in a number of themes to do with information and its implications on business structures and process. You will also know that every now and then, I use the arenas of food and music to illustrate points or issues.

Of late, much of my time has been taken up in reading (and re-reading) The Power of Pull, and I’ve written a couple of posts on the subject. Messrs Hagel, Seely Brown and Davison have really made me think about things; they’ve answered some of the questions I’ve had for a while, and raised a number of new ones.

In parallel, I’ve also been delving into collapse theory, sparked off by Clay Shirky’s excellent post on the collapse of complex business models. That led me on to digging deeper into Joseph Tainter; I ordered and read The Collapse of Complex Societies straightaway; if any of you is interested in collapse theory, I would strongly recommend you read it; it is also really worth reading a long book review that Professor Tainter did, headlined Collapse, Sustainability and the Environment: How Authors Choose to Fail or Succeed.

Clay has also been responsible for putting a third, connected, stream of thinking into my head, via Kevin Kelly, another must-read person. The Shirky Principle. [Incidentally, you can rest assured I will be reviewing Clay’s upcoming book, Cognitive Surplus. I’ve already ordered it; how nice it was to see that the book was actually being released same-day US and UK. I am so sick of artificial scarcities.]

“Institutions will try to preserve the problem to which they are the solution.” — Clay Shirky

So that’s where my head is right now. Thinking about institutions built fundamentally around “push” principles trying to survive in a “pull” world. Thinking about institutions that are designed that way through no fault of their own; these principles have been the basis of management and economics thinking for the best part of a century. Thinking about institutions that “try to preserve the problem to which they are the solution”. Thinking about institutions that grow more and more complex over time, and then collapse.
Which leads me to the main point of this post. Predictability.
In the Power of Pull (pg 37) the authors articulate a number of “instincts, assumptions and beliefs” that make up “the philosophy of push”:
  • There’s not enough to go around
  • Elites do the deciding
  • Organisations must be hierarchical
  • People must be molded
  • Bigger is better
  • Demand can be forecast
  • Resources can be allocated centrally
  • Demand can be met
I am particularly taken with the Demand can be forecast/Demand can be met statements. For some years now, I have been bemused at the things institutions do in order to make something fundamentally unpredictable into something somehow more predictable.
Through sheer serendipity I was at Burroughs Corporation when the firm, and the industry, began to realise that the money was moving from hardware to software; I was at Data General when the firm, and the industry, began to realise that the money was moving from proprietary to open; I was at Cap Gemini when the firm, and the industry, began to realise that IT services were moving from national to global scale; I was at Dresdner Kleinwort when the firm, and the industry, began to realise that financial services were moving from analog to digital.
In each case, there was a significant shift taking place in the industry, one that was going to transform the entire market radically. In each case, there was a clear institutional response. And in each case, the response was expected to provide a predictability of outcome that was required to be of unusual precision for such an emergent venture in such a changing environment.
I saw the same thing when it came to “agile” methods and practices. Everybody seemed to understand what the methods and practices were; and then, for some strange reason, everyone expected some incredible level of precision in planning and outcome which militated against any concept of agility; this is what I was referring to as the “planning horizon” problem in a recent post.
These behaviours caused a level of cognitive dissonance in me, and I’ve therefore been thinking about it for some time now. Right now I don’t want to go into why people want this predictability, suffice it to say that it appears to be a Push principle as articulated in the book. What I want to do is concentrate on the things people do in order to achieve this state of predictableness.
The easiest route to guaranteeing predictability is to control the market, have a monopoly. Many companies have tried this in the past, many continue to try this, despite legislation and regulation to prevent this. It is easier to do this in some jurisdictions rather than others; surprisingly, it would appear that some companies in “developed” countries seem to be as adept in sustaining monopolies while pretending to be otherwise, as their emerging-nation counterparts: the processes get more sophisticated and get called lobbying rather than bribing, but the outcome’s the same. Nevertheless, global monopolies are hard to sustain: artificial scarcities (eg region coding on DVDs) get met by artificial abundances (eg chipping of DVD players).
If you can’t have a monopoly, the next thing you do in your quest for predictability is to try and control your customer, make it hard for the customer to leave you. This is not surprising, since financial backers (quite possibly the ones wanting the predictability in the first place) are trained to ask “where’s your lock-in” as part of Due Diligence 101. Overt ways of controlling the customer start looking anti-competitive; as a result, the practice has begin to ease, although there are worrying signs that it will resurge in the guise of customer data import and export.
If you can’t lock in your customer, then you do the next best thing. You convince yourself you have predictability. You use a variety of financial and reporting tools to sustain this pretence, often under the guise of “risk management”. [In this context, I would strongly recommend you read Michael Power’s The Risk Management of Everything, a wonderful little tract that you can download for free here.] Sometimes when I look at everything that happened in the financial markets, I cannot help but feel that the same principles that Clay Shirky speaks of in The Collapse of Complex Business Models applies in modern financial markets. The problem with this approach is that you land up having no defence for “black swans”. In fact I would go so far as to state that the process of pretending predictability is one that systemically creates and increases black swan risk.
If you can’t convince yourself, then it all begans to unravel. Because you’re left with the temptation to convince others of the predictable nature of business while not being convinced yourself. Why? Because “push” organisations work on this basis of precision in predictability. The techniques people use to “convince” others in this respect are immoral at the very least and illegal at the extreme.
Institutions, particularly “push” institutions,  have grown up on the mother’s milk of predictability, of smoothed-out forecasts, of a level of precision that is no longer sustainable in a “Pull” world. They’ve resorted to using an extensive variety of techniques to try and hold on to that predictability; some of them are sustainable in the short term, some of them are downright illegal, none of them have any real value in the long term.
Which brings me to the question that’s keeping me awake. Can a traditional Push institution really become a Pull institution, bearing all this in mind? is that transition, that hybrid state,  really possible? how long can one institution uphold two sets of radically different principles under one style of management?
I tried to imagine a Push institution outsourcing work to a Pull institution, to try and understand what the working agreement would look like. And I didn’t like the answer.
Comments and advice welcome.

11 thoughts on “Thinking about predictability: More musings about Push and Pull”

  1. Pingback: Exectweets » jobsworth at 05/04/10 12:09:21

  2. And now to consider something truly scary, take the ideas of Pull (which I am also in the middle of reading right now) and try to apply them to traditional educational spaces like schools… We fall directly under Push. No doubt in my mind. Encouragement or will to change? Very little. At a time in our history when learning and education need to be central to the success of nations, this is scary stuff indeed.

  3. One of the scariest habits I've seen in what you would describe as “push” organisations is that the reliance on predictions increases when higher precision is employed – as if somehow a more precise number is more accurate :)
    As for a “push” becoming a “pull” (outside of Hugh Lofting's books), or working comfortably with one … I think the discomfort level would be too high; it will take disruption for the change to happen, and it is more likely to be new entrants rather than the incumbents who surf that wave …

  4. Institutions, particularly “push” institutions, have grown up on the mother’s milk of predictability, of smoothed-out forecasts, of a level of precision that is no longer sustainable in a “Pull” world. They’ve resorted to using an extensive variety of techniques to try and hold on to that predictability; some of them are sustainable in the short term, some of them are downright illegal, none of them have any real value in the long term.

    Brought a smile to my face remembering the straight blue line rising to the right of Tarek's “sales credit” forecasts at DrKW (and the fact that he didn't have to talk to any product people to make these numbers up. Actually come to think of it since in reality they were artificial in any event, his forecasts actually probably made a lot of sense in a wonderful tautological way…)

    You convince yourself you have predictability. You use a variety of financial and reporting tools to sustain this pretence, often under the guise of “risk management”.

    Indeed. And a manifestation of the point made by Col. Jessep in A Few Good Men (paraphrasing): most people working in most large organizations simply “can't handle the truth!” The smarter ones know they are living in denial and the less smart, well as they say ignorance is bliss. Or at least less stressful…

    And ultimately why the most significant and fundamental changes in our economies, societies and cultures arise from outside the establishment. Where the wild things are.

  5. Clarence, education is definitely in Push territory! The Learner Generated Contexts Group have been talking about Organisational Achitectures of Participation in Education for some time (BJET39/3) and blog about it from time to time on http://architectureofparticipation.wordpress.com/. We argue that to develop “Adaptive Institutions working across Collaborative networks” you need three elements.
    1) Developmental e-maturity, to create responsive infrastructures,
    2) collaborative quality improvement, (like webactions) to enable adaptive (real-time) processes
    3) networked Public Value, to enable system change to be recognised
    Working with some Uni’s in the UK and looking at these issues it is clear that none of these elements are in place or, worse, even thought about. Maybe the £500m cuts in HE funding might force a change in this complacency. Like Ecles College’s webactions the most adaptive institutions are FE Colleges, who have a long history of e-enabled practice in the UK.

  6. Pingback: The Obliquity of Policy « The Heutagogic Archives

  7. A number of related ideas are sparked off by this post. Forgive me if they are jumbled. Your comment on “Time Horizon” and “Predictability' are I believe inter-related. See Elliott Jaques In Praise of Hierarchy http://www.lib.uwo.ca/programs/generalbusiness/… the basic premise being that the higher up in the hierarchy, the longer out the time horizon they have to look over. So in a way, hierarchy has been a way of codifying time into organisational structure. Organisational responsiveness to environmental change has always maintained that those closest to the source of the change should be allowed to “organically” respond to that change, and the more codified the response mechanism, the more formal the organisational structure, culture etc. This created tensions and integration mechanisms between different parts of the organisation that had to respond to different levels of environmental change. http://faculty.babson.edu/krollag/org_site/org_… . Predictability (in the application of resources to outcomes) was thus the application of strategy+structure+environmental = fit. (uh hum, taking some liberties, but you get the idea). Structure of course, in turn, affects your perception of the environment, and industry background, affected the Lens through which you analysed and perceived the environment and its threats (top team composition), so in a way, every firm created an 'appropriate story'. Lean Production, Lean Enterprise, and The Networked Organisation just took apart all the elements and re-examined them at granular, unit, and modular levels (sources of variation, production time, pull through supply chain systems). What isn't commonly dwelled upon, is that education, cross skilling of workers, management systems, and two way responsibilities between a team and its leader, were equally revolutionary. Workers also kept very, very detailed records of their tacit knowledge in little notebooks tied to the machines and tools they operated (this machine bends maximum degree 24, not spec of 25 etc.). So this released increased granularity of data for design use. Cross organisational team structures released the flow of data (and intentions) across formerly isolated functional silo's. But key to all this was the ability to also understand the theory of constraints; the set up times of key change over process points. A lot of this lean manufacturing theory, the actual thinking, not its codification, has not made it through to other industries it seems. I guess the overall point (I am labouring no doubt) is that many or not all the elements involved in the analysis have always existed. We just need to constantly re-examine them?

  8. Great post – thank you. A couple of days ago I listened to two talks by David Harvey, leading Marxist geographer, on the crisis in Marxism & the end of capitalism. He talks about the spheres of action where change occurs – you might find his thinking really of interest here as I am particularly interested in his challenge to conceptualize what society can look like outside of a market economy and the necessary shifts in vision, in mental conceptualizations that we need to change the way the world currently works.

  9. Love the blog JP. I have just finished reading Freedom from Command and Control by John Seddon. I agree with the majority of what he is saying there which included that we should train on what is frequent and predictable and be flexible to deal with what is not predictable or not common. This includes “pulling” knowledge in for the less frequent issues hence learning as you work. With the number of books you have read I am sure you must have seen this one so you will probably have a better handle on what he is saying.

    I think the problem with traditional Command and Control businesses (most of them) is that this way of managing is so engrained it would take a brave senior management to try and make the necessary changes.

    The book recommends that a ‘clean’ room or office is set up and the new methods used there with people being trained in the new ways before joining that team. Then hope you don’t get reorganised before the resulting savings have been achieved!

    Unfortunately it is much easier for new entrants to the market (competition) to work on their systems than the rest of us.

Let me know what you think

This site uses Akismet to reduce spam. Learn how your comment data is processed.