Skip to content


Thinking about streams of information at work

At school and at university, I was reminded by teachers not to allow the knowledge I’d accumulated to constrain unduly my thinking about the future. There was something liberating about the mere process of trying to understand that knowledge could be considered a constraint, a liberation that continued throughout my life, evinced at different times and in different ways.

Early on, it was a personal fascination with the concept of time, triggered by an experience every child in India goes through: finding out that the hindi word for yesterday: “kal” was the same as that for tomorrow: “kal“. While still at school, as with most of my fellow students in the Sciences stream, the thoughts and writings of Richard Feynman entered my life. His teachings on The Character of Physical Law, more particularly the chapter on The Distinction of Past and Future, influenced my conceptions of time even further, giving me a sense of its irreversible nature. Most 13-14-year olds have a blank-slate approach when it comes to absorbing ideas, and so it was when we were first able to think about what Einstein had been saying about relativity, allowing us to view time as a dimension.

Then came university, where I read Economics, and then work, where none of this appeared to matter. The nearest I came to thinking further about all this was in my late twenties, when I went through a long period of regular dreaming, often lucid, often with repeating themes. [The commonest theme had me in flight: I'd slowly lengthen my stride and then gently take off, more gliding than flying, able to keep myself airborne for a minute or so, soaring and banking using my arms as wings, never flapping, unable to hover.].  They were dreams rather than nightmares, relaxing me, letting me feel rested and refreshed; this, coupled with their lucidity, meant that I tended to remember my dreams. And occasionally, very occasionally, I would experience something in “real life” that seemed, if I stretched it enough, to be something I’d experienced before in a dream. But I “knew” it wasn’t possible and so I dismissed it. Sort of. It didn’t stop me from reading Michio Kaku from the mid-1990s onwards, starting with Hyperspace/Parallel Universes. But the Feynman in me ruled, time continued to be seen as something irreversible.

One other principle stayed with me, influenced by some of the sayings and quotations I’d been attracted to over the years: Einstein saying that we couldn’t solve problems by using the same kind of thinking we used to create the problems in the first place, Einstein suggesting that common sense was the collection of prejudices one has by age eighteen, someone (occasionally credited to Schopenhauer) saying that talent hits a target no one else can hit; genius hits a target no one else can see. In each case, I was reminded of what my teachers had said to me, about not allowing my “knowledge” to constrain my “thinking”. Easier said than done.

Over time, I understood more about cognitive biases and anchors and frames, a learning that was accelerated by conversations with colleagues like Sean Park, Malcolm Dick and James Montier while at Dresdner Kleinwort. So it should come as no surprise that in my roles as deputy CIO and CIO, and as chief scientist, in multiple organisations, I kept taking the “don’t let the past predict the future” tablets, religiously, systematically, every day. That didn’t always endear me to everyone, but it helped me keep my thinking fresh. It was the reason I kept wanting to connect with people outside the organisation at least as much as I spoke with people within the organisation. It was the reason that I’ve always tried to support a graduate intake program in firms I’ve worked in, one way of ensuring that fresh thinking is allowed to enter an organisation.

Which is why I loved the Wayne Gretzky quote about playing where the puck is going to be, not where the puck is or was. Which is why I loved the Steve Jobs quote, in his Stanford Commencement Address in June 2005:

Again, you can’t connect the dots looking forward; you can only connect them looking backwards. So you have to trust that the dots will somehow connect in your future. You have to trust in something – your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all the difference in my life.

In the words of Stephen Stills: Don’t let the past remind us of what we are not now. [That’s taken from my all-time number one favourite song, Suite: Judy Blue Eyes. You can hear a sample, containing the quote, here…. and even buy the MP3 download if you so wish.

It is with all this in mind that I spend time thinking about streams of information. For most of my adult life, these streams have been about the past. Transactions that had already happened. We spent a long time studying the fossil remains of human activity in order to try and predict what the future would look like, a mongrel form halfway between scatology and eschatology.

More recently, we’ve been able to “life-stream”, sharing our current activities and our “status” with others, aided and abetted by near-ubiquitous connectivity, ever-smarter devices and digital frameworks that support the social networks. In the past, we were only able to capture things that had happened. We were used to calling things that had happened “transactions” and so we called the analysis of those records “transaction processing”. We’re able to look now at what people are doing, within “activity streams”, and, because we share our activity streams within social networks, we call the study of this “social media monitoring”.

But we’re on the cusp of something way way more exciting, in a classic William Gibson future-is-here-but-unevenly-distributed kind of way: we’re beginning to what we intend to do.

Doc Searls, a good friend, was the first person I heard using the term “Intention Economy” to describe this. And I’ve signalled my intention to him, by pre-ordering his book on the subject, due May next year.

Esther Dyson, another good friend, when talking about the future of internet search, complimented Bill Gates on saying “the future of search is verbs”….. now that gets interesting, really interesting, when you consider verbs as having tenses. Tenses that help segment the continuum of time. Past. Present. And future.

The future.

When I was at university, one of the things I studied in classical economics was the works of Jean-Baptiste Say. And one of the ways in which his “Law” was paraphrased, originally by John Maynard Keynes, was as follows: Supply creates its own demand.

We’re soon going to be able to signal our intentions in ways that we could never have done before.

Over time, those signals will become more sophisticated, more evolved, more nuanced.  Social norms will be formed, telling us what we can or can’t do with our signalling of intent. The semaphoring of intent will slowly come to include disinformation, the false-carding,  feints and dummies, elaborate ways of disguising intent in order to further some other intent. With that will come the need to watch for, and to recognise, the digital “tells” in the world of supply-and-demand poker. On both sides.

And over time, the systems and processes required to interpret and assimilate those signals into actionable information, this too will evolve.

I cannot wait.

Posted in Four pillars .


13 Responses

Stay in touch with the conversation, subscribe to the RSS feed for comments on this post.

  1. Daen de Leon says

    This business of “the future” is not one that corporations generally understand. I remember, as a callow youth, when working at J P Morgan (first at Angel Court, then 60 VE — yes, I’m old enough to remember JPMSL’s tenure at Angel Court …) questioning some of the assumptions in the company-wide Value-at-Risk report that Dennis Weatherstone was so proud of having on his desk by 4:15 every business day. They were, to be blunt, pulled out of somewhere unsavoury. I’m pleased that Nassim Taleb finally said to a broad audience what I was unable to say to my management, albeit 20 years too late. What we learnt in 2007, and what I suspected in 1990, is that the past is an extremely poor indicator of the future when it comes to the unexpected. But whenever I raised that issue, when it came to the choice of volatility parameters in our equity options pricing modules, I was sarcastically asked, “So how would you do it, genius?”. Which rather surprised me: that I didn’t have a solution wasn’t the issue; the issue was that we were pinning the future profitability of the company, and indeed its continued existence, on a risk management tool whose choice of operating parameters was controversial, at best. In the end, the groupthink that led to the Cult of the VaR resulted in JPM Chase needing a $25bn bailout. That’s what over-reliance on the past can do to you.

  2. JP says

    Agree. That’s why I liked where prediction markets were heading, but we’re yet to see the next steps. Serendipity meets black swan.

  3. clive boulton says

    Streams at work provisioned by Cloud technologies predicting at unprecedented scale could be a boom just ahead. But are information technologies also creating new demand for supplies of predictions unregulated for their monopoly effects…

    Newly bankrupted MFGlobal proved underestimating risky bets is fatal hubris (Bear, Lehman, AIG). On the other hand, Logitech’s CEO commented on a massive misstep losing $100M on GoogleTV, using simulation streams could have predicted product market readiness?

    http://www.theverge.com/2011/11/10/2553406/logitech-ceo-google-tv-cost-us-dearly-no-revue-replacement-coming

  4. Ken L says

    JP, talking of signals… Are you aware of Sandy Pentland’s work with social signals??? His book ‘Honest Signals’ (www.amazon.co.uk/Honest-Signals-Shape-World-Bradford/dp/0262162563) is fascinating exploration of this topic. I would agree with you that as we gain greater insight into these emerging signals, the advantage may reside with societies who have cultures that respect ‘listening’ rather than ‘talking’…

  5. JP says

    @Clive, I will be following up this post with more details on implications for “manufacture”, “inventory” and “priicing”. I tend to stop posts at around the 1200-1500 word mark, allow some of the ideas to sink in, generate some comments to help me think more, and then launch into the next post.

  6. JP says

    @Ken, yes, I was a founder/supporter of Tom Malone’s Center For Collective Intelligence, and used to meet Sandy there regularly, and at Davos.

  7. Guy Stephens says

    Great post and resonated on a number of different levels for me. I look at the use of social media within customer service, and as I journey along this path, themes or ideas emerge every so often. The idea of ‘activity streams’ is one and how, I believe, this may fundamentally change the way decision-making takes place, and by extension, the implications of streams on how customer service is/will be provided. The other is the idea or use of ‘verbs’ (hadn’t necessarily thought about the idea of tenses). If we look at what Facebook is doing, and I think this is very rudimentary, then the space between human behaviour and customer service becomes inextricably linked. I am still at the early days of my thinking, but as I wrote above, eveyr so often, a bit more of the jigsaw is revealed, and this post has certainly revealed a liitle bit more for me. Thank you.

  8. JP says

    Glad you liked it Guy. The verbs and tenses framework definitely helped me think about this space

  9. Brian Driggs says

    Could it be we’re already seeing the nascent predictors of behavior in “tools” such as Klout or Peer Index and their ilk? Their apparent preference for treating marketing departments with more respect than the individuals whom they so self-aggrandizingly label “authoritatively” aside, aren’t such evaluations an attempt to predict the future? Could this one day evolve to (super computer-) powered algorithms capable of calculating the actions people are likely to take, given specific timeframes?

    What’s more, I find it interesting how difficult it seems to think about predicting the future without some reliance upon the past. There’s that bias you mentioned (as well as the genius of the solving unseen problems)! Common boilerplate reminds us, “Past performance does not guarantee future results,” yet we often base our forecasts almost entirely on past performance.

    Very interesting, JP. Thank you for sharing these thoughts.

  10. JP says

    Brian, glad you liked it. It’s amazing just how difficult it is to shift our past-predicts-future mindset, it’s very deeply ingrained in us. like reward/punishment systems.

Continuing the Discussion

  1. Quora linked to this post on November 15, 2011

    Are verbs the future of search?…

    Is Bill Gates right, are verbs the future of search? Esther Dyson, another good friend, when talking about the future of internet search, complimented Bill Gates on saying “the future of search is verbs”….. now that gets interesting, really interesting…

  2. TWICs, XIV linked to this post on November 18, 2011

    [...] Confused of Calcutta: Thinking About Streams of Information at Work (11.13.11) What’s more, I find it interesting how difficult it seems to think about predicting the future without some reliance upon the past. There’s that bias you mentioned (as well as the genius of the solving unseen problems)! Common boilerplate reminds us, “Past performance does not guarantee future results,” yet we often base our forecasts almost entirely on past performance. [...]

  3. The persistence of (organisational) memory – confused of calcutta linked to this post on November 22, 2011

    [...] we can leave behind. Because of the power of streams of information at work. Now, it is possible for the “record” to show everything that was discussed, not just [...]



Some HTML is OK

or, reply to this post via trackback.