Dennis Howlett commented on a recent post of mine, On Control, where I was musing over Big’s relationship to Control Failure, and arguing that we needed a Better But Not That Big approach. One of the things Dennis said was
“Big often means complex. So how do you propose to solve the complexity issues?”
That stayed in my mind; I have tremendous respect for what Dennis has to say, it’s not often that I am able to connect with a tech-savvy open-minded articulate accountant :-).
Almost coincidentally, I was reading The Origin Of Wealth by Eric Beinhocker, as recommended to me by a couple of readers of this blog. [Thank you Dave Bridgeland and Tommi Vilkamo]. [And yes it’s true, I read every comment, try and reply to as many as I can, and try and learn from the comment. Otherwise why bother?].
I’m still reading it. [I read maybe a dozen books in parallel, usually in multiple passes as well. First pass skim-read, take a few notes along the way. Determine where to focus serious reading time the next time around. Second pass do the serious reading. Take notes on what I didn’t understand, what I need to dig deeper into, not necessarily in the book either. Third pass follow up and clean up, get the ideas clear in my head.]
And it was while I was reading the Beinhocker book first-pass that I was given the opportunity to revisit some of the stuff that Scott Page had written. I’d come across Scott before, I think it was Erik Brynjolfsson who first pointed me at him; I’d read Scott’s Computational Models from A to Z some time ago, and some of his stuff on Path Dependence more recently; I couldn’t link to that, you can find it on his homepage as a pdf.
I quote from Beinhocker:
Page proposed that there are two dimensions to the difficulty of an economic coordination problem. One dimension is how hard it is to decompose the problem into chunks that can be solved in parallel. [……] The second dimension is the number of steps that need to be done sequentially. […….] Page posited that organisations evolve to match the nature and difficulty of the probelms they are trying to solve. In rough terms, if the problem can easily be chunked into parallel tasks and doesn’t require much sequencing, then the organisation will reflect this simplicity and tend to be broad and flat. If the problem cannot be easily divided up and has numerous sequential steps, then the organisational hierarchy will tend to be narrow and deep.
And I think that’s where one of the major problems of Big and Complex lie.
There is considerable organisational inertia to task redefinition, de-sequencing and re-sequencing, as people desperately try and hold on to fiefdoms and power bases built around particular task definitions and sequences. As a result, things are done sequentially where they don’t need to be done sequentially. This false sequencing yields an unsolvable complexity and an immense amount of wasted energy, repeat work, even completely unnecessary work. Which in turn demotivates the workers and reduces task completion quality and increases task completion time. Workers aren’t stupid, soon you have apathy.
And I guess apathy is the hierarchical organisation’s equivalent of anarchy.
Big is necessary for Complex. But only when Complex itself is necessary. What I worry about is how often Complex is a construct of past hierarchies rather than a genuine need to solve a problem. I’ve heard phrases like “Don’t Allocate, Isolate”, “Don’t Automate, Obliterate” for a few decades now. I’ve even read HBR and related articles with similar titles that long ago. So why doesn’t it happen? Because of Unnecessary Complex.
As Einstein said, we need to keep things as simple as possible. But no simpler. So where there is a need for Big, Big can and should stay.
When you take into account the fruit of Moore’s Law and Metcalfe’s Law and Gilder’s Law, when you take into account the sheer power of the web and virtualisation and consolidation and service orientation, when you overlay all that with distributed computing and the grid and P2P, then maybe some of these hub-and-spoke approaches are ripe for obliteration. Just maybe.
Thanks again to Dennis for forcing me to think harder about this. And I look forward to more challenging comments, that’s how I learn.
Here’s a somewhat orthogonal view, from a technology architect involved in complex projects. His view in essence is that ‘some things just take time’. But it’s worth a look.
http://www.jroller.com/page/cpurdy?entry=gestation
You are working your way into the cause of disruptive technologies.
The big company had to be big. It was the only was they would be able to build the “Big Widgets†that were needed. I am thinking of PBX vendors, because that is where my experience lies. It took a huge company with 1000’s of engineers to build a 500E switch.
The technology changes, Moore’s law progresses and some grad student writes Asterisk, a full featured PBX.
The big company won’t change. They have their hierarchy to protect. The smaller faster company whose structure fits the reality of today wins.
Thanks for the ping JP – hint: I’m not like any accountant you’re likely to meet :)
Beinhocker is onto something here which I’d not thought about. I’ve trawled the various links on this and it intrigues me. If nothing else, I’ve added to his royalty fees.
This slots very well into the ongoing and developing argument about what constitutes then next generation of applications. Is it socially led? Is it edge? Is it viral? Is it all three with innovation at the core around delivery models as the general economics of tech change?
Absolutely worthy of additional thought.
Now you’ve got me at it – thanks.
So your “masochistic” read is sometimes of value? :-) we should meet sometime. I’ll ping you to find out where you’re to be found.
JP – Read my ‘about’ page – I’m very easy to find and contact
Agree with Doug’s point above. Indeed it is an underlying pillar of AmazonBay – the virtue of bigness for financial services firms is no longer even debated it seems, at least in the mainstream of industry thinking on the subject. However, as JP points out above much of the big-ness found in large organisations serves no more purpose than to perpetuate itself for its own sake. A veritable Truman Show. This obviously opens up an opportunity for (small) firms with disruptive technologies and business models to prosper.
I share your interest in both control and complexity but have a different take on the matter, which has developed as a consequence of my passionate attempts to revive an interest in liberal education. I would like to refer you to three entries in my own blog that touch upon what I feel are some fundamental issues that have to be addressed (but may not be resolved). The first has to do with prevailing educational philosophy. It strikes me that as education becomes more specialized, it also becomes more preoccupied with the agenda of CONTROLLING THE WORLD, without necessarily devoting much attention to the hard truth that CONTROL ALWAYS HAS CONSEQUENCES. Without trying to get too Zen (or, for that matter, too Heideggerian), I would argue that in many situations, particularly complex ones, BEING IN THE WORLD (and adapting to all the vagaries that the world throws at us) is more important than control. I discussed this matter in my June 13 blog entry:
http://blog.360.yahoo.com/blog-Mff23hgidqmHGqbcv.lfskakEtS6qLVHUEMFUG4-?cq=1&bid=72&yy=2006&mm=8&l=66&u=70&mx=77&lmt=5&p=12
Next, since you like to read so much, I would like to invoke my favorite Henry Miller quotation (from TROPIC OF CAPRICORN): “Confusion is a word we have invented for an order which is not understood.” (I put this sentence at the top of the notes for the first university course I ever taught back in 1971 at the Technion, and I still live by it!) I believe that the lesson here is that confusion in the face of complexity is a fact of life; and we should approach it the same way we approach my first point about “being in the world.” In my blog entry, I even took the radical position that we should embrace the chaos behind the confusion, because it stimulates the mind to seek out perceptual categories and try to understand the order that has not yet been understood:
http://blog.360.yahoo.com/blog-Mff23hgidqmHGqbcv.lfskakEtS6qLVHUEMFUG4-?cq=1&bid=68&yy=2006&mm=8&p=68
This then leads to my third point, which is to question whether or not the very concept of “organization” carries as much weight as it did back in the bad old days of scientific management. I have tried to illustrate the point with the uncomfortable truth that the phrase “terrorist organization” is probably oxymoronic. It seems as if the most successful terrorist attacks recently have come from highly-distributed loosely-coupled networks of INDIVIDUALS, whose interactions seem to consist primarily of resource management and coordination (the latter generally on relatively small scales). Thus, when I read about such concepts as “organizational inertia to task redefinition,” I cannot help but wonder if such problems are due to the fact that organizations, as we know them, are no longer up to the task of what I call “being in the world.” This is probably one of my more controversial turns of thought:
http://blog.360.yahoo.com/blog-Mff23hgidqmHGqbcv.lfskakEtS6qLVHUEMFUG4-?cq=1&bid=72&yy=2006&mm=8&p=72
I hope you enjoy these. Since my blog is on Yahoo! 360, I am happy to invite anyone to join my group who would like to be informed of my unkempt thoughts on a more regular basis!
Great blog, JP – it’s a good way to think about organization inertia and how it interferes with simplicity. But your proposed dream/hope of using technology (virtualization, grids, p2p etc) to solve a human created problem of Bigness Instilling Complexity seems off. I wish it were so simple. The core issues of control and power in many humans, and the almost Pavlovian training young managers get to love Complexity so that it enables Control and Power is not easily shaken…
However, Dennis is on to a great point. I have always believed that inertia is one of the biggest barriers to success and so in understanding this Big / Complex behavior, lies the opportunity for a younger company to attack the soft under belly of the larger beasts in their markets…