Musing about search

Like most people, I’m not particularly interested in “search”. I’m interested in “find”. Particularly when I’m not entirely sure what I’m looking for. For various reasons I was thinking about search today, and remembered something I’d read a while ago.
Many years ago, during the heady days of the last Web boom, Mary Modahl (currently on the board of Yankee Group) wrote a book called Now Or Never: How Companies Must Change Today To Win The Battle For Internet Customers.

It’s a good book. And in it, Modahl recounts the tale of a car salesman in Somewheresville, USA. He had his own dealership, ran a solid and very profitable business. For many years. People came from all over the place to his dealership, because he was such a good salesman.

Now he knew that the number one item his customers wanted was a white pickup truck. But Head Office kept sending him green pickup trucks. And his customers would come looking for white pickups, and leave with green pickups. Somewhere deep inside the bowels of Head Office, someone would notice a spike in the sales of green pickups, and with the customary flash of brilliance associated with such people,  raise the production targets for green pickups, override all the salesman requests for white pickups, and send them even more green stuff.

But that was then. Today, the customer checks on the web first, makes contact with the car dealers, and then only goes to dealers that have white pickups for sale. So now, through no fault of his own, the salesman is behind the eight ball. He never gets the chance to use his dazzling selling skills, because the customers figured out he hasn’t got what they want. And the reason he doesn’t have what they want? Not because he didn’t know — he did — but because someone else was interested in what was sold to the customer and not what the customer wanted to buy.

The Intention.

[My apologies to Mary Modahl and to anyone else associated with the book if you feel I have misquoted; it’s six or seven years since I read the book, and the quote’s a paraphrase from memory].

I use the story to try and explain where I would like to see search going, particularly in an enterprise context.

I want to see much more of “Did you find what you were looking for?”

I want to understand why some people find what they are looking for faster than others.

I want to see the routes people take to do the finding.

Because somewhere in all that, somewhere among the steps and the categorisation and the differentiation, there is expertise. Some of the expertise is in the use of the search tool. But most of it is a way of looking into a person’s head and distilling the expertise contained, in a manner that it can be shared. And that’s when you have magic in the enterprise and in the classroom.
Search is about finding. And the path of discovery is about learning. And about expertise. The next generation of search will be about intention and how to capture and refine and improve the process.

Musing on IPR and DRM

Imagine there was a significant risk of a life-threatenic epidemic of some medical condition or the other. Imagine there was a vaccine or perhaps an antidote for this condition. Imagine there was only one patent-holder. Imagine that the short-term demand for the medicines far exceeded the supply capacity of the patent holder.

Thankfully people far more learned than I’ll ever be have considered scenarios like this, and as a result the concept of compulsory licences exists. In simple terms, a government or equivalent body can overturn the rights of patent holders in such situations and grant licences to their competitors, using a Malcolm Quantum Energy approach to the problem. Malcolm loves isolating a problem and then giving it some Tipping Point level of energy and resource to see if innovative solutions emerge. He’s wise beyond his hair.
It nearly happened in the US with anthrax in 2001. It nearly happened in many parts of the world with bird flu. I suspect it probably happened in some form or shape with SARS; weak versions of the process have already been used for HIV and for AIDS. The TRIPS legislation, inadequate and almost-dangerous in many respects, nevertheless tries to do something about this from a world trade perspective.

So we have some sort of sledgehammer for the lock-in of patents, but these are for physical things and so they are easyish to understand. There are many problems to do with the way compulsory licences work; these relate to import and export and localisation and recompense to patent holders and a host of other things. But at least there is a sledgehammer. And we are learning. And we will get better.

Until people started talking about trying to patent software, my primary interest in patents came from pure curiosity, and was concentrated on “functional” and “life-saving” medicines. I couldn’t for the life of me understand how anyone could justify making something as essential as a life-saving drug hard to get, particularly by the use of artificial monopolies and their subsequent monopoly rents.

Even then I was no more than an active bystander, an interested observer. Until people tried to patent software. Then, as I began to perceive the unholy messes that could be caused, I started getting interesting in everything to do with patents. So that I could learn to do the right thing.

My family business was in journalism, and we had a printing press. At school and in church we had “cyclostyle” machines which we used to churn out various types of pamphlet and journal. We had dozens of typewriters at home. And over the last 20 years, I have seen the technology and costs of reproduction and transmission of the reproduced copies improve dramatically, with photocopiers and fax machines and scanners and digital cameras.

The same happened with music. My girlfriend had a massive Akai reel-to-reel tape system, and used it to record things. I grew up with cassette tapes and recordable CDs through to today’s iPod Generation.

The same thing happened with video. And with cameras in general.
Yet I bought books, I didn’t photocopy them. I bought prerecorded tapes. I bought CDs and DVDs by the cartload. And I never let anyone copy them. Never made a copy of them either. I’ve never bought anything from iTunes, though my children have. I use iTunes as a means of transferring music I already own on to my iPods.

I am not unique. There are many people who care about IPR and DRM because of the things that go wrong, not because they want to cheat “the system” or authors or musicians. They want to do the right thing.

We have a lot of things wrong with the “system” of IPR and DRM. They are no longer fit for purpose, they create behaviours and consequences that are diametrically opposed to the original intent of the rules and regulation. Innovation is hampered. Productivity is impeded and reduced. Incentives are used for intermediation and lock-in rather than disintermediation and freedom. Costs rise instead of dropping. The wealth created by the passage of Moore’s Law and Metcalfe’s Law and Gilder’s Law is frittered away, to a point of absurdity.

Patents are hard to write, to protect. Prior art is hard to discover. The process is creaking to a point of gridlock. The “system” is being corrupted and arbitraged to a point beyond absurdity.
That’s why we have to figure out how to do the right thing.

Let me take a simple example. Let’s say Clarence Fisher or Judy Breck come up with a scheme to create a global market for podcasts of lessons. That children are encouraged to trade their favourite podcasts, issued under a Creative Commons label. That the children contribute the podcasts, that they use social networking and collaborative filtering. That this trade happens internationally, similar to the legendary Grateful Dead tapes. [I quote from Wikipedia: The Grateful Dead allowed their fans to tape their shows like several other bands during the time. For many years the tapers set up their microphones wherever they could. Naturally the best sound was in front of the sound board. The eventual forest of microphones became a problem for the official sound crew. Eventually this was solved by having a dedicated taping section located behind the soundboard, which required a special “tapers” ticket. The band allowed sharing of tapes of their shows, as long as no profits were made on the sale of their show tapes. Recently, there was some dispute over what recordings archive.org could host on their site.] [Told you that Jerry Garcia influenced my opensource thinking!]

Wouldn’t it be great if there was a Dick’s Picks of favourite lessons on podcasts?

Then imagine that these kids, worldwide, needed media players in order to play these podcasts. Imagine that Apple or Microsoft or Real or Someone New dominated the media player space.

And imagine that the media players had bad DRM built into their DNA, so that the kids couldn’t play the Creative Commons-licensed podcasts. Imagine the media players were themselves locked in to specific and expensive devices. Imagine that the media player producers and the device producers and the connect providers and the “content” providers all banded together and made sure that the DRM chain was pure and ensured complete lock-in.

Imagine the kids who couldn’t listen to or watch the podcasts. Just because of bad DRM. What a shame that would be.

And this is not science fiction. It is happening. Now.

We have to learn to do the right thing. 

[An aside. We’re going to see a lot of legal activity on this front. This story (thanks to Cory), challenging what a digital copy is, who made the copy, what the original was, who owned the original, and so on, is just the beginning. Every enterprise has its hands full trying to prove that Person A is associated with Password B and Second-Factor authentication C and worked at device with IP address D and downloaded/altered/deleted E.  It is not as easy as it should be. Even in controlled work environments. ]

We have to learn to do the right thing.

What would happen if the Patent Office burnt down?

Many years ago, we were trying to decommission a particular application that was way past its sell-by date. Step One in decommissioning was to price it and plan it. And it didn’t matter what we tried, there was some sort of Conway’s-Law-Meets-Organisational-Inertia in operation, and the immune system kicked in, and the answers were always the same, regardless of the application in question: 18 months and 2 million bucks. [I guess it was similar to the business plans floating around during the First Net Boom, every one of which said Three Years and 75 million bucks :-) ]

And then something happened. There was a flood in the machine room, the hardware was not replaceable, things were beyond redemption. And maybe 300,000 bucks and six weeks later, the application had been successfully decommissioned.

That made me think. Maybe the way to decommission systems was to pretend there was a flood. Necessity and mother and invention and all that.

When I first got involved with the mess that is IPR today, I mused occasionally “What would happen if all past patents just disappeared, and we had to start afresh? What would the “new” IPR look like? Why?”

So it was with some amusement that I read recently that we had already experienced something similar. I’d bought a book some months ago while travelling in the US, and it had entered the dark world that was my library, never to be seen again. Until this week, when I’ve been working on packing the library up, that is.

It’s a fantastic book. Called How Invention Begins, it’s written by John H Lienhard, Professor Emeritus of Mechanical Engineering and History at the University of Houston. I was first attracted to Lienhard when I saw a review by him of Stephen Jay Gould’s The Mismeasure of Man, another subject close to my heart.

In the book, Lienhard recounts the case of the US Patent Office: officially opened in 1790, the building was “hopelessly outdated and overcrowded” by 1836, and burnt down soon after. “The fire destroyed the office and everything in it”. And a new Patent Office arose from those ashes, rebuilt from scratch. Barely 20% of the original patents could be reconstructed.

But that’s almost an aside, stated here only to make the point that we’ve had a period of “patent amnesty” not that long ago.

Lienhard’s book has really challenged me. He describes the process of invention much like Doc Searls would talk about snowballs in blogs: a series of small events, with many different and disconnected people participating, somehow serendipitously coming up with coherent inventions over time. Or maybe it’s David Weinberger and Small Pieces Loosely Joined, the way Lienhard describes it.
He makes a number of fascinating points, things I have thought about but not really comprehended, I needed to read his prose to get to my Meringue Moment.

Four key points stand out:

One, that invention is a response to a community want, a communal hunger for something, a passion that creates a Zeitgeist that must be obeyed.

Two, that the invention is actually made up of a whole lot of small pieces, whose ideas came from different people, whose attempts at converting ideas into inventions came from different people, whose experiences of such attempts were shared openly most of the time, and that a whole heap of time tended to pass before a coherent invention came out in response to the Zeitgeist.

Three, that we tend to celebrate the Light-At-The-End-Of-The-Tunnel, somehow imbuing heroic values to the last person in that chain, often not knowing what went before, sometimes not caring.

Four, this need of ours, to name and label and date and time the “invention” and its “inventor”, is contrary to what really happened, misinformed at best and corrupt at worst.

There are many other ideas in the book, it will take me time to explore and digest them. But what I have read and understood so far supports everything I espouse about collaboration and teamwork and sharing and co-creation of value and serendipity and community.

More later. Probably within a week, I plan to watch as much of the Ryder Cup as I can while packing and lazing.

The Because Effect and the future of Marketing and IPR and maybe even the Net

I have often wondered why we spend so much time wrestling with the issues related to Marketing in the 21st Century, with reframing IPR and DRM, and with “net neutrality“. Why we land up with starkly polarised views and start calling each other names. Blefuscu and Lilliput all over again.

And recently, I’ve begun to think it’s about Doc’s Because Effect. More precisely, it’s about people unable to cope with the Because Effect.

What we are seeing are people who are running scared, paralysed, sometimes not willing, often not able, to make the move from making money with something to making money because of something. So they try and build walls around whatever it is they made money with, and try and build laws that will keep those walls in place, and try and claim that anyone who disagrees is a pinko lefty criminal tree-hugging utopian no-hoping loser.

Take a look at what Peter Drucker said in the context of marketing:

  • It is meaningful to say that “product A costs X dollars”. It is meaningful to say that “we have to get Y dollars for the product to cover our own costs of production and have enough left over to cover the cost of capital, and thereby to show an adequate profit.” But it makes no sense at all to conclude, “… and therefore the customer has to pay the lump sum of Y dollars in cash for each piece of product A he buys.”

What Drucker is referring to is the fallacy inherent in directly correlating what a customer pays for with what an enterprise wants, even sometimes needs,  to receive. Drucker continues:

  • Rather, the argument should go as follows: “What the customer pays for each piece of the product has to work out as Y dollars for us. But how the customer pays depends on what makes the most sense to him. It depends on what the product does for the customer. It depends on what fits his reality. It depends on what the customer sees as ‘value’

That’s what part of the Because Effect is about. Google doesn’t make money from search. It makes money from advertising. So Google gets their “Y dollars”, and “how the customer pays depends on what makes the most sense to him”.

You can see it as part of Christensen’s Innovator’s Dilemma. People didn’t buy encyclopaedias, they assuaged guilt to the tune of $1500. So when they could buy a PC and a disk containing an encyclopaedia for the same $1500, they “paid it, depending on what made the most sense to them”.

You can see it in the kids who pay $4.99 for a ringtone yet won’t pay $15 for a CD.

You can see it everywhere.

Back to Drucker. He said:

  • Price in itself is not “pricing”, and it is not “value”. But this is nothing but elementary marketing, most readers will protest, and they are right. It is nothing but elementary marketing. To start out with the customer’s utility, with what the customer buys, with what the realities of the customer are and what the customer’s values arethis is what marketing is all about. But why, after forty years of preaching marketing, teaching marketing, professing marketing, so few suppliers are willing to follow, I cannot explain.

What Drucker cannot explain is what I have found hard to understand. And why I am Confused.

Enough about Marketing. Let’s take patents. One way of looking at patents is that they provide an inventor with an artificial monopoly rent. This monopoly rent allows the inventor to create a difference between the cost of production and the price charged to the customer. In effect, the patent allows the inventor to make money with a product rather than because of it. [For a short time, as With moved to Because].

This was reasonable, a short-term artificial basis for inventors and their firms to recoup their investments and their costs of capital. First-mover disadvantage was neutralised and everyone was happy.

But.

In the old days, most of research was done in universities and academic institutions. They thrived on knowledge, on sharing knowledge, it was their life-blood. And, for the most part, they had no patents. Yet they shared ideas. Sometimes the ideas needed considerable capital injections before they could see the light of day as inventions, and so we had patents. In this context you could argue that the monopoly rent was used to offset the costs of research. Perfectly reasonable.

But.

Even then, there was a considerable time delay between the issuance of a patent and the commercialisation of the invention. In many cases the commercial product came into existence around the time the patent was ending anyway. Which meant that the artificial monopoly rent disappeared at the right time.

But.

Now cycle times have changed, the gestation period between concept and patent and commercial product is small. The capital costs have changed, much of the need is intellectual rather than land or capital. Production costs have changed in a digital borderless innovating-on-the-edge networked non-hierarchical world.

Yet.

Patents are still pending while the commercial product meanders towards obsolescence. There are offensive patents, defensive patents, frivolous patents, spam patents, shyster patents trying to take public domain things private, more patents than the population of China. Patents trying to create monopoly rents in order to defray nothing at all.

And so we have high costs to do with patents. More patent lawyers than the population of India and China. An expensive meaningless discovery process. An inability to define what the thing is, an inability to prove or disprove prior art. And guess what, someone needs to recoup all these costs.

Have we reached a stage where the true reason for a patent’s monopoly rent is actually to recoup the costs of the patent process and nothing more? I wonder.

It’s back to The Because Effect. Customers will pay because they get value. It is up to them how they pay. And firms that understand that will land up recouping their costs because they have happy customers. Firms will not recoup their costs via lock-ins and artificial monopolies. Especially costs that shouldn’t exist in the first place. Costs to do with production and distribution. And patents. And windfall expectations.

The same happens when people argue about who is going to pay for Stevens’ Tubes. [By the way, talking about Senator Stevens, have you seen this? Or, in the context of IPR, this? Or, in the context of DRM, this? Laughable.]

A coda: Peter Drucker on Marketing. Again.

….That after twenty years of marketing rhetoric consumerism could become a powerful popular movement proves that not much marketing has been practiced. Consumerism is the “shame of marketing“.

Consumerism is the shame of marketing. And IPR and DRM.

Heated arguments

This is a post about chillies and about endorphins. If you’re into either, read on, Macduff. I’m into both.
The kernel for this post was a story by Mark Frauenfelder on Boing Boing recently, revisiting an article he’d written some time ago called The Cult of Capsaicin.

In the excerpt, Mark quotes Dave De Witt, a seasoned (yes, pun intended) veteran of books on the chili pepper. Dave in turn refers to a Yale study in the 1970s that classified people into supertasters, tasters and nontasters, based on the number of taste buds per square something-or-the-other.

I’ll come to the argument and the controversy shortly; first, some scattered seeds to help those who aren’t yet into chillies big-time.

Believe it or not, there’s actually an index, the Scoville Scale, to measure the “hotness” of food, particularly chili peppers. To give you an idea of how hot hot can be, the Naga Dorset pepper, currently claimed as the world’s hottest chillie, has a Scoville rating of between 876K and 970K. Contrast that with the habanero, a measly 100K to 350K, or with your common-or-garden jalapeno, a miserly 2.5K to 10K.

There is some evidence to suggest that chillies only grow in hot climates, and that one of their key roles is to help humans living in such hot climates to cool down. The logic goes: Eat chillie. Sweat buckets. Allow prevailing wind to operate on sweaty body. Cool down.

Now that’s not everyone’s cup of tea (although the same logic is used to explain why people drink hot tea in hot climates, to cool down….).

There is also considerable research into the protective and curative properties of the chillie (assuming that this public domain good hasn’t already been illegally patented to death); one of the more unusual bits of research suggest that some aspect of capsaicin, the active compound present in chillies, helps produce anti-carcinogenic behaviour in the human body. But that’s a deeper subject. If you are interested in the rich history, culture, science and folklore of the chillie, I would recommend you read Amal Naj’s Peppers. It’s an excellent first book on the subject.

Now to the main course of this post. Frauenfelder’s story set off alarm bells in my head, as I saw an old controversy rear its ugly head. The controversial concept is simple:

  • Some people can eat hot chillies. Others can’t. The difference can be explained by the number of tastebuds.

The 1970s Yale study seemed to endorse this position, given the likelihood that people could be classified into super-tasters, tasters and non-tasters.

Wrong.

This is another nature-versus-nurture debate.

Man’s ability to endure chillie heat has nothing to do with tastebuds. It has to do with training the mind.

You see, chillies are a fantastic con. Let me explain.
When you drink a cup of really hot coffee or tea, you damage some tastebuds. If you lived in the US I guess you may even sue, on the basis that you couldn’t read Warning: Hot or something like that.

The damage that a hot liquid does to your tastebuds is physical. Unless you’re desperately unlucky, the damage is temporary; for a few days, a part of your tongue is over-sensitive to heat and cold, and you struggle. But soon you mend.

As against this, look what happens when you eat a really hot chillie. Brain gets message: I AM ON FIRE. Sends out the fire engines. Every mucous membrane known to mankind rains on the chillie’s parade. Brain  then says to itself, OK, damage limitation done. Now let’s see what else needs to be done. Oh yes, we took some serious heat there, better send out some heavy-duty painkillers. The brain then dispatches a cartload of endorphins. Shortly after, the body calms down and all is well.

Now you can train your body to be more efficient about this process. Eat chillies. Miss out the fire-engine bit. Do not pass Go. Go straight to collect endorphin consignment. Live happy.

This is not to say there isn’t a physical issue to do with chillies. Capsaicin is a major irritant. So you’re messing with death if you don’t wear gloves while chopping up really hot chillies. Don’t even think about removing contact lenses or visiting the men’s (or ladies’ ) rooms until and unless you’ve really washed your hands AND worn gloves.

The irritant aspect of capsaicin is one to be feared, especially with sensitive areas like the eyes, nose and for that matter regions further south. But most people don’t tend to eat chillies using those parts of the body.

For the mouth and the tongue, the irritant aspect is manageable and trainable. And the endorphin kick is enjoyable.

It’s only anecdotal and completely unscientific, but I know people who are serious foodies, with highly developed senses of taste, smell and flavour, who can and do eat heavy-duty chillies. At least one is also a pretty good judge of wine.

So for me I don’t buy the argument that chillie tolerance is connected to having a shortage of taste buds.

Any comments?