Recently I wrote about meeting Doug Engelbart for the first time, courtesy of a dinner invite from Tom Malone. Before dinner, I had the opportunity to hear Doug speak at a Center for Collective Intelligence seminar at MIT. As you would expect, he covered a lot of ground very quickly, and I won’t attempt to document all of it here.
One particular thread, however, intrigued me so much I feel driven to share it here, to see what people think. The quotes are as close to verbatim as I could make them; apologies for any inaccuracies or misrepresentations:
Tackling a large-scale problem requires a strategic rather than a tactical approach. The paradigms that shape our individual and collective perceptions of big problems, and of their possible solutions, they tend to evolve much more slowly than the problems themselves, due to the inherent complexities of the big problems. [….]Far too many of the possible improvement steps will change the design environment for other improvement candidates. So we have to depend on an evolutionary process. We can learn to facilitate and accelerate that process.
What struck me about that line of thought was its potential applicability in aiding root cause analysis of complex systems problems. More and more, systems environments are growing more and more complex, as wave upon wave of device descend on our network shores; boundaries of the enterprise keep getting stretched, both in the supply chain as well as in the customer chain; the impacts of globalisation and disintermediation, of offshore and outsource, continue to be felt.
While all this has been happening, there have been a few other shifts as well. Each enterprise has tended to become multivendor in itself, with a greater number of hybrid environments; if anything, this has been accelerated by the opensource movement. Then, as telephony becomes software, there is also aÂ movement of intelligence from the core to the edge of the network, and this tends to aid customer-to-customer interaction at that edge. As a result, looking at any given large enterprise, we tend to see the following characteristics:
- Sharp increases in device population and proliferation
- Steady creeping-out of the enterprise boundary
- An increase in hybrid environments
- Significant extensions to supply and customer chains
- Greater and more complex electronic relationships
- All happening over a global footprint
- All happening 24 x 7
When something goes wrong, it isn’t always that trivial to work out the root cause. The more complex the problem, the more likely we are to apply some sort of serial process to solving it. And maybe that’s where collective intelligence should meet Agile. Where we use the power of well-established knowledge bases and tie it up to the experience of a large collective in order to focus on a problem, then use an accelerated evolutionary process to iterate through the possible solutions, taking care to avoid “changing the design environment for other improvement candidates”.
Initially, I used to think about Agile somewhat narrowly, keeping to the bounds of systems development. Over the years, I’ve come to realise that, more than anything else, Agile is a mindset and a business strategy. It is only more recently that I’ve come to consider Agile as a problem-solving tool and an aid to bug-fixing, but I wasn’t sure about that. Now, having heard what Doug had to say, I’m beginning to think there is something to it after all.
I’d love to hear your views, particularly from people who use a combination of agile methods and collective intelligence techniques to solve complex systems problems.
An aside: In a many-sided marketplace, is every participant a fractal representation of the marketplace itself? Sometimes when I look at a large enterprise, it seems to behave like an open software platform in its own right. Just a thought…. for some time now I’ve been intrigued by the fractal nature of organisation structures, and at the comic ways we use to try and de-fractalise them by PowerPoint and memorandum.