Musing about collective intelligence

I was intrigued by the possibilities that opened up in my mind as a result of reading this story: Captains’ logs yield climate clues: Records kept by Nelson and Cook are shedding light on climate change.

Diaries and journals, both amateur as well as professional, have always been a rich source of historical information.

As we move further into the digital age, the information in such diaries and journals is slowly becoming richer and richer. Meta-information is becoming easier to collect and to embed. Dates and times, places and people, environmental factors of every type.

Thirty years ago, when film was primarily analogue, all we had was frame numberings. Then we had auto date and time, followed by some crude tagging ability. Now, in the digital film world, everything is auto dated and timed and geolocated; everything is easily tagged, and in community as well, with folksonomy.

The same is happening with voice. With sound.

The same is happening with text.

And as a result, the social objects that connect us in conversation are themselves getting enriched and embedded in similar ways: commentaries and reviews, photographs and videos, sights and sounds.

Just musing. Something I’ve been working on for a few years now, yet something where I received a fresh insight by reading that story. So I thought I’d share it. And await comments.

14 thoughts on “Musing about collective intelligence”

  1. Agree. And unlike until recent times, its not just the elite who get to capture their musings.

    The democratization of information capture will have repercussions on how we view this time period in retrospect. Biographers view our friendfeed streams to construct a character and activity profile. Archaeologists would not longer muck about with dust of the earth but use data mining tools to dig deep into the bowels of data stores.

    Very interesting line of thought, thanks!

  2. Good post and I like the comment above too – interesting take on how archaeologists will find out about us given the ethereal nature of data.

    It’s the meta information that enables humans and machines to talk to each other, and that’s essentially how I see web 2.0. Some point to mobile as web 3.0 and that makes sense – I posted a while back on ideas about geo-sensitive ‘hyperaudio’ which would essentially enable audio through geotagging (

  3. I read that with interest as well, partly as that sort of old data fascinates me (especially if it involves ships and sailing :) ) but also because it should help set up a fact base for the whole Global Warming debate, which – in my view – seems to have too much raw emotion and posturing on both sides.

    Similarly I recall reading about grapes for wine being grown in Roman times in York.

  4. As the cost of collecting “democratised” information continues to drop, as the number of devices we can use to collect and transmit that information grows, as communications get cheaper and tend towards ubiquity, we will see this mushrooming of metadata.Metadata that takes the emotion out of many arguments.

  5. well, yeah, it is heading towards omniscience .. but the yogis already covered this, being is everywhere, inseparable from you/me/all. so your research is just another name for self-discovery, but taking the looong way around

  6. Gregory, I think the magic is not in what the yogis and others may have discovered, and continue to discover. That is not new.
    What is new is our ability to share what is discovered, learn from it, improve it, continue to learn from it.
    There is no longer a holy of holies.
    That is a key difference.

  7. @ JP

    Where is the verification in this vast sea of data? At least in the blogosphere, many opinions are expressed, unhindered by data. And those also happen to be opinions that do not welcome debate and surprisingly, still find many readers.

  8. hmm . Hello from Aaron and me in a Lyons bar ( to provide some meta data ;-) not sure I know what you are asserting so hopefully meet for a beer one day ….

  9. Shefaly, the problem of verification is a critical one, and about to go ballistic. We’re going to have billions of devices connected to the internet, trillions of readings of various elements of information, all condensed to actionable levels. And all gameable, unless we get the verification right.

    A subject of a different post, to follow soon.

  10. Thought you might be interested in this: climate change scientists went back to the paintings of constable, and in particular to his notebooks, as he spent so much time outdoors drawing clouds. They were able to make estimates of percentage cloud cover in that time based on the observations.

  11. @ Shefaly – Verification is found in the aggregation and comparison of the raw data to extract patterns and meanings from it. As your example of the blogosphere shows, the problems of deception and falsification arise when we are presented with interpretations, data to which somebody else has already attached their meanings.

    @JP – might this be another context to apply your thoughts on Open Source ? The value that is created when we share raw data and allow others to generate their own interpretations of it.

  12. Gordon, this is just a classic implementation of Linus’s Law. Given enough eyeballs….
    Too often we refuse to see the value of open inspection. As you say, P2P inspection could be an effective way of solving Shefaly’s verification problem.

Let me know what you think

This site uses Akismet to reduce spam. Learn how your comment data is processed.