In earlier posts towards the tail end of last year and early this year, I committed to writing a number of posts on filtering. The background is simple:

• soon, everything and everyone will be connected
• that includes people, devices, creatures, inanimate objects, even concepts (like a tweet or a theme)
• at the same time, the cost of sensors and actuators is dropping at least as fast as compute and storage
• so that means everything and everyone can now publish status and alerts of pretty much anything
• there’s the potential for a whole lotta publishing to happen
• which in turn means it’s firehose time
• so we need filters
• which is why the stream/filter/drain approach is becoming more common
• and which is why I want to spend time on all this during 2014, starting with the filter

So here goes.

1. Filters should be built such that they are selectable by subscriber, not publisher There should be no publisher-level filters. Allow the firehose to happen. We know how to solve the firehose. What we don’t know is how to solve a much bigger problem: what to do when there are filters at publisher level. Once you allow this, the first thing that happens is that an entry point is created for bad actors to impose some form of censorship. In some cases it will be governments, sometimes overtly, sometimes covertly; at other times it will be traditional forces of the media; it may be generals of the army or captains of industry. The nature of the bad actor is irrelevant; what matters is that a back door has been created, one that can be used to suppress reports about a particular event/location/topic/person. If we keep making sure that it’s not easy to filter at publisher level, the bad actor is left with the strain of large-scale filtering of firehoses. Not easy.

2. Filters should intrinsically be dynamic, not static In keeping with the firehose that’s being filtered, the act of filtering should itself be one of flows and not stocks. There is a place for canned filters, to support trend analysis, pattern recognition, predictive analytics. But the norm should be that the subscriber can reset filters anytime without any loss of time or value.

3. Filters should have inbuilt “serendipity” functionality Have you ever chosen “random” when presented with a choice of things to look at, to listen to, to read, to follow? It’s a simple insurance policy to take out in order to avoid digital bigotry or heretical thinking or tunnel vision or herd instinct or groupthink or whichever other buzzphrase a la mode excites you. You must have something that takes you outside the pattern of what you do normally. And you must be able to switch that something on at will. The StumbleUpon approach is useful, but since you can “train” it you run the risk of “filter bubble” in Eli Pariser terms. In fact any publisher-level filter can create a filter bubble. Which, at its worst, allows someone else to determine what you can see/touch/listen to/engage with.

4. Filters should be interchangeable, exchangeable, even tradeable I should be able to give someone else my collection of filters; similarly, someone should be able to give me their filter set. Their transient filter set, nothing permanent as I said earlier. The idea is that one person is given an opportunity to engage with the firehoses of the digital universe while “walking in someone else’s shoes”. So I should be able to view news as if I was a 21-year old Iranian. Not by selecting the publisher-side filter for “21 year old Iranians” but by being able to exchange filters with a real live person who has those characteristics. Again, we need to watch for static, hierarchical filters and avoid them like the plague.

5. The principal filters should be by choosing a variable and a value (or range of values) to include or exclude The variable could be anything. Place. Time. Person. group. Topic. Temperature. Degree of wetness. Humidity. Blood pressure. Relative density. Weight. State. Number or count. Size. Type. Part number. SIC or NACE code. Tag. Hashtag. Label. Length. Material. Language. Species. Duration. Anything and everything. And filtered again, if needed, by the associated value. Hotter than. Lighter than. Higher than. Containing. And then filtered again for inclusion or exclusion.

6. Secondary filters should then be about routing This is where the concepts behind If This Then That come into their own. The universe that IFTTT represents is one of conditional filtering and routing. The filtered information, having passed the conditions set, needs to go somewhere. Devices now form part of the world of filters. A person who has a laptop, tablet, phone and wearable does not want the same filters for each, the same notifications to each. For one thing, the social conventions for each form factor are different; for a second, the readability and “actionability” will differ as well. So we will use IFTTT and similar constructs to filter by notification type, intensity, device, perhaps even recipient time of day and location.

7. Network-based filters, “collaborative filtering” should then complete the set Collaborative filtering is also critical. Show me the tweets that are trending with my friends that I haven’t seen yet. Let me know the restaurants frequented by people in my network who like spicy food and who’ve posted on TripAdvisor about those restaurants in the last six months; make it relevant to my location and the current time.

So that’s a starter set, seven principles that inform me when I think about these things. I shall expand on each in days to come. In the meantime, keep your observations, advice, questions and comments flowing, choosing whichever means or channel you prefer. Comment here. Respond to the link in Twitter, Facebook, Google+, LinkedIn. WhatsApp me if you want. Talk to me via @jobsworth. If you don’t like any of these, then I suppose you can email me via [email protected] but be warned that I look at email rarely and that too only under duress.

Last January, around this time, as people came back from hibernation into the plastic warmth and sparkle of CES, we heard about luggage that can tweet. Hello, I’m here. In Rio. While you wait for it. In Riyadh.

Trakdot was one of the stories I latched upon at that time.

You could put a Trakdot into your sharkskin suitcase and bingo, not only could it tell you where it was, it could also tell you when it approached you. Via SMS if needed. [Incidentally, no sharks were harmed in producing the sharkskin suitcase, it’s not real sharkskin, just in case you were getting ready to complain].

A year on the story gets better. Why bother with mock sharkskin when you can have the real thing? Real sharkskin, still worn by the shark, while it’s alive and breathing?

Apparently over 300 sharks have been tagged and equipped to let bathers and swimmers know of their proximity. And many other things besides.

Everything is a node on the network.Even sharks.

Everything can publish, everything can subscribe.

Which means firehoses and filters.

That’s why all this publishing and subscribing takes place against thresholds we set. Like location. Or size. Or temperature. Or time. Or whatever. In whatever combination. Static as well as dynamic.

Over the next few weeks, as I spend more time writing about filtering in an age of streams and drains and filters, the tweeting shark is a good place to start.

What a difference a year makes.

httpv://www.youtube.com/watch?v=AmGgcqjrxMc

We’re not gonna take it
Never did and never will
We’re not gonna take it
Gonna break it, gonna shake it,
Let’s forget it better still

We’re Not Gonna Take It: The Who (Townshend, 1970)

We live in inventive times.

Sputnik 1 went into space a few weeks before I was born; shortly before I entered my teens, man had landed on the moon. While mechanical calculators had been around since the 17th century, solid-state “pocket” calculators only emerged in the early 1960s. The radio sets and amplifiers I grew up with were, similarly, valve-based; there was something magical about how they glowed, how the air smelt different as they warmed up. By the time I was thirteen, I’d been given a solid state “transistor” radio by a friend. The typewriters I grew up with were massive mechanical devices. While electric typewriters had been invented at the turn of the previous century, it was only in the early ’60s that the “golf ball” showed up, and “electronic” typewriters took a decade longer.

The first e-mail was sent during my early teens; the first mobile-phone call around the same time; by the time I was in university, the Commodore PET (“for the masses, not the classes”) had started shipping. I touched my first PET in 1978 or so. The countertop microwave oven came out before I turned 10. Compact cassette tapes began to be commercially produced around 1964; the Sony Walkman didn’t turn up for another fifteen years, shortly after I completed university.

TV’s transition to colour programming took place in the US in 1965, and in Europe a couple of years later. I was 10. It would take a further decade before TV became available in India; I’d actually left the country before I saw colour TVs there.

The first patient to receive a fully implantable heart pacemaker did so shortly after my first birthday; the world’s first heart transplant took place shortly after my 10th birthday. The laser was invented the day after I was born.

By the time I was 12, Woodstock had happened. In the twelve years preceding, the Beatles, the Who, the Rolling Stones, the Grateful Dead, Simon and Garfunkel, Bob Dylan, Leonard Cohen, The Band, Crosby Stills Nash and Young, Traffic, Cream, Blind Faith, The Allman Brothers, Jim Croce, the Doors, Chicago, Jefferson Airplane, Ten Years After, Jethro Tull, Peter Paul and Mary, Gordon Lightfoot, Janis Joplin, Stevie Wonder, Led Zeppelin, Joan Baez, Joni Mitchell, Van Morrison, were just part of the incredible talent parade emerging. And we’d started our painful march across formats, as the recording industry sought to take advantage of a short-term blip in performance art.

The world’s first successful commercial jet service began before my first birthday; while there’d been a handful of hijackings since 1929 or so, hijacking went mainstream from around 1958, to peak in 1969.We’d just begun to understand about DNA; I was still at university when the first successful nucleotide sequencing was completed.

I was 6 when ideas about the internet began to surface, 10 by the time ARPANET was approved, 12 by the time the first message was sent; protocols like FTP and TCP-IP emerged as I left school.

You get my drift. I was born into an age of invention and innovation and disruption, an age that continues apace today. But.

The focus has all been on the “supply side”

Invention, innovation, disruption. Popular terms. And for some reason, loaded on to the supply side of the equation. Inventors and innovators and disruptors thought up stuff and made stuff and provided stuff and sold stuff. It was all about them. And yet, while a tiny handful of them succeeded, many failed; the failures were orders of magnitude greater than the successes. And we looked to the successes. And cheered.

There was also a tendency to make invention and innovation and disruption the progeny of individuals. It made things like patent law easier, I suppose, and created wonderful bottlenecks in the creative process that allowed money to be minted. Sometimes history has been kind to us and shown us that actually this wasn’t the case, that much of invention and innovation happens through the sharing of ideas, sometimes cooperatively, sometimes competitively. But most of the time we were allowed to believe that the magic was in the sole inventor. Libraries of books have been written to tell us this is the case; and over the last forty years, similar libraries of books have been written to tell us that this is not the case.

All this is changing. We’re meeting a new class of inventor, a new class of innovator, a new class of disruptor.

The Disruptive Customer

“You are old, Father William,” the young man said,
“And your hair has become very white;
And yet you incessantly stand on your head—
Do you think, at your age, it is right?”

“In my youth,” Father William replied to his son,
“I feared it might injure the brain;
But now that I’m perfectly sure I have none,
Why, I do it again and again.”

[An aside: One of my favourite parodies, Lewis Carroll as inventive and disruptive as ever. Or perhaps I should call him innovative rather than inventive, since he took an existing “technical principle”: Robert Southey’s The Old Man’s Comforts and How He Gained Them: and parodied it. If you read Southey’s version, you’d be surprised at how pragmatic and sensible copyright law was before it was corrupted into the nonsense of today.]

The point of this post is simple. No invention, no innovation, no disruption, none of this takes place without the customer.

It is only through adoption that we hear the tree of invention, of innovation, of disruption, make a sound

No adoption, no invention. No innovation. No disruption. Nothing without the customer’s adoption.

The Maker Generation is upon us. The tools of production are more and more in the hands of the customer. 3D printing will make this happen even faster. Which is why new roles have emerged, “platform” roles. Platforms that enable connections and interactions, conversations and transactions.

When customers become inventors and innovators, the scale of disruption changes, and changes radically. More people are involved in inventive activities, more trials and experiments carried out, more “failures”, more learning, more “successes”. The cost of finding out what someone else has done, learnt, shared is low and continues to get lower. The pace at which such learning takes place is increasing, and again radically. The places it takes place in are also changing. It’s happening all the time. Everywhere.

When we design platforms, let’s bear that in mind. It is the customer who disrupts. We make it possible, we enable it, we may even catalyse it. But what we do is nothing unless there’s a customer present.

As everything gets connected, as everyone gets connected, as we see these enabling platforms emerge and evolve, the customer will invent more, innovate more, disrupt more. What customers want of us is what they’ve always wanted of us: get rid of the friction, get rid of the latency, take out the barriers to entry, let them in.

Successful platform providers will understand that, and as a result

design to enable while getting out of the way

That’s what I was trying to say four or five years ago, when I was working on my “Design for Loss of Control” theme. And it hasn’t changed.

Design to enable. And get out of the way. More to follow, as I concentrate on platforms and control and filtering and getting out of the way.

You gotta go where you wanna go
Do what you wanna do
With whoever you wanna do it with

You gotta go where you wanna go
Do what you wanna do
With whoever you wanna do it with

The Mamas and The Papas, Go Where You Wanna Go (John Phillips, 1966)

[Incidentally, I’ve never seen an earlier example of the marauding apostrophe. Not just errant and extant but hunting in pairs. I am told that the group became grammatically correct after their first album. They had to become politically correct even earlier: the toilet on the album cover had to be covered over, apparently. Read all about their cover shenanigans at the Dunhill site.]

I have an Indian passport. That in itself is not surprising, given that I was born and brought up in India. But I left there 33 years ago, and have lived in the UK since.

When I left the country, my mother asked me not to change my nationality; she was concerned that with the then political climate, I may find it hard to visit India at short notice if I changed nationality. So when I qualified for UK citizenship, I didn’t do anything about it. Which may not be a big deal for most people. But for a regular traveller like me, it adds capsaicin to the merest of journeys across borders. Take a look at the visa differences shown below, courtesy of the fantastic site VisaMapper.

If you’re a UK citizen, this is how the world looks in terms of visas:

If you’re a US citizen, things are about the same, except for Brazil. Dilma Rousseff is less than happy about being spied on by her neighbours.

But if you’re Indian, many of the leaves that are green turn to brown. Incidentally, if you haven’t heard Simon and Garfunkel perform that song, just click and enjoy.

I love travelling, and have been privileged to work in jobs that require regular travel. And, given the brownness of the Indian visa situation, I’ve had to get a lot of visas. [I was a stamp collector as a child. I never thought I’d continue to be one this long ….. using a passport rather than a traditional stamp album].

Over the years that’s led to a number of interesting situations:

In 1981 I was snowbound in Russia, managed to barter my duty-frees and escape from Sheremetyevo’s transit area — I couldn’t bear to think of being cooped up there for perhaps six days. I made it to Red Square, along with a couple of Swedes who were heading back home from Thailand and who’d seen what I was up to. But we were apprehended there (while sampling the country’s vodka, of course) and were very closely scrutinised back at the airport till our plane left. Very closely indeed.

In 1985 I had one day to get a Greek visa prior to going to Spetses on vacation. I’d left it late. The guy at the Embassy was very helpful, but couldn’t help me. My passport was four months out of date, and I hadn’t noticed. Which meant getting a new passport of an evening in order to give me even the smallest chance of joining my wife in Spetses. To make matters a little more interesting, I was due to fly out of Belfast. [Long story]. We made the flight.

In 1998 I turned up at Frankfurt airport duly visa-ed, but with a minor problem. I planned to go there on day X, and later had to change my plans to arrive there on day X-1. An early meeting had been added to my schedule. Tough. Because my visa didn’t actually start till the next day. So I had to spend an hour at the airport cooling my heels with only the security staff for company. But once Cinderella time came around, I was let in.

There have been many shenanigans since, many close shaves. Part of the spice of life. At least one of them involved going to the Mexican visa official’s home one late evening in Austin, Texas, so that I could get to the Yucatan with my family. Another involved being let in to the Irish Republic on an emergency visa given at the airport, once the officer had heard me refer to my daughter as Orla. [Which is not surprising, given that’s her name].

But I came close to utter disaster a few weeks ago. One of the challenges of having so many stamps in your passport is that you run out of pages very quickly. You collect additional booklets like political donors collect awards. And each of those booklets has valid visas, for different countries. So my right to enter the UK indefinitely is in one passport, my 3 year Schengen is in another, and my 10 year US, my 5 year Japanese and 2 year Irish, for example, are in a third, the “current” one.

As life would have it, I’d been travelling so much that I forgot my “Schengen-visa-holding-but-otherwise-full-while-valid” passport en route Munich. Turned up there. Patted my pockets, my carry-on bag. Nothing. Nada. Left at home. UK visa? Check. US visa? Check. Schengen? Oops.

The border guard was very nice. No Visa? No Entry. I tried to explain. Have Visa. Just Not With Me. He smiled, agreed. And repeated what he’d said earlier. No Visa. No Entry. Brainwave time. Since I “had” the visa, except it was at home, on my desk, would he let me in if I got someone to take a photo of that visa and email it to me? Then I would Have Visa. And Enter.

He hummed. But he didn’t haw. He actually agreed with my proposal, on one condition. The photo would have to be emailed to him directly, not via me. So I got my daughter to put her iPhone to use. And I was let in that morning. That really exemplifies Germany for me, a culture that is far more about spirit than about letter when it comes to stuff like this.

Worse was to come. Just before Christmas, we decided to take a short break in Dubai. I’ve taken them before, and the hotel has always been able to sort my visa out in short order.

Not this time. Days passed and no visa. Nobody understood why. I pulled every string I could, and in the end managed to fly there visa-less. And when I got there, I found out why. Someone with a name vaguely similar to mine (not close, not close at all) was on a security blacklist; no one could decide what to do. So I was in limbo. Until I presented myself, they saw me and my family, and all was well.

Go where you wanna go? Chance would be a fine thing. There is something very broken with the visa system, especially when you see the crazinesses and anomalies something like VisaMapper shows up. So bizarre that it must be true.

When Greece won the European Championship a decade or so ago, Otto Rehhagel, the coach, is rumoured to have been given a truly wonderful bonus. The right to park anywhere in Athens, anytime.

People hanker after many things. Fame, fortune, good looks, talent, whatever.

Me? I’m a simple man. All I want is an instrument that lets me enter 200 countries at will on short visits. Have visa will travel. [And sometimes, usually inadvertently, but at least once advertently, have no visa will travel anyway].

Where do you go to, my lovely? Peter Sarstedt, 1969

Where do the children play? Cat Stevens, 1970

Where have all the flowers gone? Pete Seeger, 1955

Platforms are a bit like Jabberwocky: to paraphrase Alice, they seem very pretty, but they’re rather hard to understand. I chose the songs above for a number of reasons: because I like them; because they answer the question “where” with answers that have to do with much more than just “place”; and because I could demonstrate some of the value of platforms like Wikipedia and Youtube simply and effectively by so doing. [Incidentally, Wikipedia needs your help to stay ad-free. Please donate. Now].

Platforms simplify interactions by removing frictions and latencies. By helping people connect and interact and transact, they exhibit what a good friend, Sheldon Renan, called “netness“. Sheldon describes netness as:

Another way of looking at platforms is by using the metaphor that another good friend, Doc Searls, gave us, when he spoke of the Giant Zero. People use platforms to do things they cannot do as effectively, as quickly or even as enjoyably elsewhere. With as little friction as possible, with the lowest possible latency. Simply. Easily. Where, when and how they want it to be.

Which brings me to my first point:

Platforms enable interactions with a minimum of fuss and bother

Now then. I use Wikipedia to try and explain my references, and use the Web at large when that is not possible. I use YouTube to connect to popular musical performances. I’m interested in many things that are essentially analog in characteristic: books, vinyl albums, photographs, memorabilia, art, recipes, musical instruments, things to do with Calcutta or India or the Raj, scientific instruments, things to do with printing and publishing, cricket, chess, it’s a long list. Those interests mean that I spend time discovering the right platforms: my alphabet goes abebooks, barneby, cricinfo, discogs, epicurious….

Why these particular services? Because they have the right “content”. And that matters. Content matters. When I worked in telcoland, people used to talk about “attach rates”. People tended to buy telecommunications and connectivity products and services from the provider who had the most interesting/compelling/comprehensive content. Which forced telcos to enter the content business or risk being disintermediated and pushed deeper into infrastructure and utility with consequent impact on competitive intensity and margins. Content rightsholders have therefore, not surprisingly, been at the forefront of copyright and intellectual property battles every time a new platform has come along and disrupted the living daylights out of part or all of their business. I say “rightsholders” because the people who get the money aren’t necessarily the people you think get the money. Take the music business for example. Some of you may remember this diagram from The Root a few years ago:

But that’s a whole ‘nother story, saved for another day. Incidentally, I love what I’m hearing about Iron Maiden. They tracked where the illegal downloads were happening: the majority in South America, principally in Brazil. And, as Business Insider put it, quoting Citeworld:

Rather than send in the lawyers, Maiden sent itself in

When it comes to platforms, content is king. But

Platforms are about networked content, not broadcast content

Platforms create value because of the “networked content” rather than with the “broadcast content”

To a platform, “everything is a node on the network”

As everything gets connected, the platforms that will succeed are the ones that get “netness”

In 2014, platforms will start to focus on filters

Nobody goes there any more. It’s too crowded.