I guess my last post intrigued quite a few people, judging by the number and nature of the comments. In some ways we come back to the Nurture Versus Nature theme, and to the meaning of “expertise”. [An aside, I am still ploughing through The Cambridge Handbook of Expertise and Expert Performance, all 900 pages of it, including the index. If you’re interested in the issues surrounding expertise, it looks well worth a read. More later.]

Four themes stand out from the comments:

• One, that the web has many bad librarians: bad in the context of their anchors and frames, bad in the context of their biases and gatekeeping, bad in the context of the expertise they possess (or not, as the case may be)
• Two, that the profession of librarianship has understood the need to change with the times, and that they are looking at the web and to the web quite seriously
• Three, the time dimension associated with the World Live Web is one that has magical possibilities and horrendous potential for problems as well
• Four, we need to look at the social space and engagement and experience as closely as we look at the classification and finding and retrieval, and within this we need to understand more about the relationships and trust bonds between searcher and find-enabler

So thank you everyone for your comments, I continue to be blessed. Let me try and take some parts of the argument a bit further.

First on gatekeeping and expertise and Damn Fools.

Paul Miller gave me some very useful insights into what librarians are doing today. I was particularly taken with this:

The library of Library 2.0 exults in integrating the consistency and cohesiveness of formal classification systems with the more fluid granularity of the folksonomy.

There’s the rub. How to take historical taxonomies and ontologies and mold them into such a shape that they can be enriched by the Wisdom of Crowds, while managing to keep out the Madness of the self-same Crowds. Capture the passion, connect with the perseverance and patience, discard the Damn Fools and their biases and anchors and frames.

This can be done. This will happen. But only if we allow the passion of the amateur to flourish at the same time as the professionalism of the “expert”. There’s a Long Tail aspect somewhere in this, where we need to move out of the Hit Culture and understand that Search and Find become real when you deal with the outliers, the low-frequency requests and responses. That’s where expertise comes into its own. In the niches, in the nooks and crannies.

Paul Simmons had some very interesting thoughts about “unused” pieces atrophying and being greyed out, whether it’s software functionality or links or information or books or whatever. There’s something about the idea that I like, that intrigues me, yet I push back. Because it’s to do with the time dimension. There may be some things that come into play only once in a blue moon, but that doesn’t make them less important. In fact at blue moon time they may be critical, far more important than anything else. I think that holds true for software, for links, for information. And for books.

I fully accept Stephen Smoliar’s points about the role of credentials and of domain knowledge in conversations, and those of John Dodds and Anant in similar vein. My point was more to do with keeping the barriers to entry as low as possible, so that the folksonomy is also allowed to flourish. This is not easy to do, but should be easier with the web and with modern tools. Much of this is borne out in Paul Miller’s piece and the pdf he links to.

John Dodds’ other point about the paucity of fluid multidimensional skills required by a “livebrarian” is also of particular interest to me. With the tools we have today, we can attach time and location and context information much more “richly”, thereby improving the whole search process, something that Simon Hollins referred to in an earlier comment. We have the technology.

Once we move into an n-language n-data-type n-discipline-or-genre n-location model, I think we will also need to move into an n-person-distributed-librarian model. We have many historically disenfranchised people, some illiterate, some unable to see or speak or hear, some unable to move, many unable to understand more than one language. The web allows us to re-enfranchise everyone, but only if we get the “livebrarian” right.

Which means we have one more social shift to deal with, that of moving from deterministic request-response to probabilistic and context-aware and heuristic.

The Livebrarian becomes an aggregated “recommender”, seeking to use both formal as well as folksonomic information, enrich the request with context and location, then summarise the recommendation for the searcher. I cannot help but think that this is best done by providing wetware librarians with software assistance and peer-generated collaboratively-filtered information.

What I have seen so far in web-enabled “Ask and I shall give you advice” services is less than promising. Even Damn Fools are better. Which is why I am glad that the profession of librarianship takes this issue seriously, their domain expertise is necessary. But it is not sufficient, not until we get the value of the Wisdom of Crowds and the movement away from the Hit Culture.

I haven’t yet had the opportunity to try out Dan Pett‘s recommendation on Enquire.

Paul Anderson‘s comment on the futility of polarised debate with respect to all this is also important. We need to be comfortable with grey when pushed towards black and white. We need to be comfortable with Analogue when we are pushed towards zero and one. We need to be comfortable with Mu.

Often polarisation takes places as a pendulum response to an entrenched situation. Over time, the polarisation becomes less intense and a sustainable change takes place. Isn’t that how paradigms shift?

A coda. When I was a kid I used to read Dennis The Menace by Hank Ketcham. I found it hilarious. And in this context, two of the strips come to mind:

In one, Dennis is standing in the middle of a shopping mall. He gets asked by a friendly policeman if he was lost. And he replies “I’m not lost. I’m here. It’s my parents who are lost”.

In the second, Dennis is looking at a dictionary. When told that he needs to know how to spell a word in order to find the word, he looks bemused and says “But then what do I use to find out how to spell it?”. Or words to that effect.

I think we should also look at our children and at Generation M, to see how they search and how they find. They don’t have our biases and corruptions. Or expertise :-)

More later.

Following my recent post about search, there were some very interesting comments. Some suggested the emergence of new tools that are better at helping us find what we are looking for, by providing richer context and colour to the information. Some suggested that as we get better at defining who we are and what we are doing, as we get better at role and context definition, we will get better at finding things. Some suggested that the fault, dear reader, is not in our stars (or other wild cards) but in ourselves; that we should get better at defining what it is we are looking for. Some likened search to library visits, and moved from there to the role of librarians and the social engagements that take place, and on to the motives.

Great comments for which I am truly grateful, and there is work for me to do in following them up.

But in the meantime. I’ve been musing.

There are a number of critical differences between the physical libraries of yore and the digital library that is the web. I think there is a way of categorising them:

• Time. Libraries are static. The web is live.
• Shape. Libraries have books and magazines and CDs and DVDs and tapes and a few other things. The web has all of these, sound, picture, video, text.
• Location. Libraries are physically located in particular places. The web is everywhere and global.
• Scale. Libraries contain a discrete and finite number of items. The web is infinite.
• Classification basis. Libraries rely on Dewey and its extensions. The web relies on tags.
• Nature. When you take a book out of a library, it is with you and not with the library. When you take something out of the web, it is still there.
• Speed of change. Libraries measure their purchases and their culling and their weeding in months. The web does it in seconds.

I could go on, but that’s not the point.

The point is that the web is live.

So we need livebrarians. Part bookseller, part journalist, but primarily librarian. Librarian of something that is live.
And guess what? We have livebrarians. All over the place. Every webmaster is a livebrarian, every blogger is a livebrarian, every creator of “UGC” is a livebrarian.

• They do a number of librarian-like tasks which may not be that well understood or appreciated.
• They categorise, using tags. This helps others find them.
• They point out where things are by linking to them.
• They go out looking for new things and make sure that the new arrivals are shown as such.
• They take care of the old things and prune the stock as needed.
• They even review things and comment on them, much like you have “staff picks” in libraries and bookshops

The libraries we have are new, a different paradigm. The tools we have are not yet fully fit-for-purpose, we’re still building out the library. But the tools are getting better. The librarians we have are a different breed, but they exist.

And our readers are different as well. Now they can tear things out of books, scribble on them, mix the pages up, throw them up in the air to see if they land buttered side first.

And they can write as well.

Kids are allowed to make noise. In fact everyone’s allowed to make noise. There are no SILENCE signs in the web.

We have to get better at using the tools we have. Particularly with tags and with microformats.

We have to get better at telling people what new tools we need. Because we’re the authors, we’re the borrowers, we’re the lenders and we’re the librarians. If not us who?

And we have to ensure that our new libraries have no termites or woodworm or silverfish or damp or dry rot.

Otherwise called bad IPR and bad DRM.

It’s happened again.

Over twenty years ago, I heard a poem at a poetry reading in London. Loved the poem. But had no idea what it was called, what the first line was, who wrote it. You’ve probably been in rooms like the one I was in, all smoky and echo-ey and dim and  dingy, with “announcements” as incoherent as the ones you hear at airports and railway stations. Sound, yes. Fury, occasionally. But ultimately signifying less than nothing.

But there was something about the poem I really liked, and for years I’ve been looking randomly for it. Needle: meet haystack.

So you can imagine my delight when I found the entire poem at a site called Kitabkhana. How did I come by the site? Well, Devangshu, an old friend, schoolmate, trivia team partner and chess teacher (yes Devangshu, you taught me more than you may remember; my thanks to you), visited ConfusedofCalcutta and left a comment. And soon we were in contact, I found out what he was doing and where he lived. And it transpired that Nilanjana, his wife, blogged. I went to take a look, liked what I saw, and linked to her.

And today, while looking through what’s new on the blog, I found the poem. Thank you Nilanjana. I reproduce it in its entirety here:

Learn by heart this poem of mine;
books only last a little time
and this one will be borrowed, scarred,
burned by Hungarian border guards,
lost by the library, broken-backed,
its paper dried up, crisped and cracked,
worm-eaten, crumbling into dust,
or slowly brown and self-combust
when climbing Fahrenheit has got
to 451, for that's how hot
your town will be when it burns down.
Learn by heart this poem of mine.

Learn by heart this poem of mine.
Soon books will vanish and you'll find
there won't be any poets or verse
or gas for car or bus - or hearse -
no beer to cheer you till you're crocked,
the liquor stores torn down or locked,
cash only fit to throw away,
as you come closer to that day
when TV steadily transmits
death-rays instead of movie hits
and not a soul to lend a hand
and everything is at an end
but what you hold within your mind,
so find a space there for these lines
and learn by heart this poem of mine.

Learn by heart this poem of mine;
recite it when the putrid tides
that stink of lye break from their beds,
when industry's rank vomit spreads
and covers every patch of ground,
when they've killed every lake and pond,
Destruction humped upon its crutch,
black rotting leaves on every branch;
when gargling plague chokes Springtime's throat
and twilight's breeze is poison, put
your rubber gasmask on and line
by line declaim this poem of mine.

Learn by heart this poem of mine
so, dead, I still will share the time
when you cannot endure a house
deprived of water, light, or gas,
and, stumbling out to find a cave,
roots, berries, nuts to stay alive,
get you a cudgel, find a well,
a bit of land, and, if it's held,
kill the owner, eat the corpse.
I'll trudge beside your faltering steps
between the ruins' broken stones,
whispering "You are dead; you're done!
Where would you go? That soul you own
froze solid when you left your town."
Learn by heart this poem of mine.

Maybe above you, on the earth,
there's nothing left and you, beneath,
deep in your bunker, ask how soon
before the poisoned air leaks down
through layers of lead and concrete. Can
there have been any point to Man
if this is how the thing must end?
What words of comfort can I send?
Shall I admit you've filled my mind
for countless years, through the blind
oppressive dark, the bitter light,
and, though long dead and gone, my hurt
and ancient eyes observe you still?
What else is there for me to tell
to you, who, facing time's design,
will find no use for life or time?
You must forget this poem of mine.

-- Gyorgy Faludy

There’s something haunting yet lyrical about the poem that stayed with me even though I could not remember much else about it. And the serendipity of finding it still amazes me. Can you imagine googling for a poem without knowing author, title, first line, in fact any line, date, and so on?

It feels even weirder to know that I read Faludy’s obituary only recently, and still didn’t make the connect.

[Note re copyright: I understand that openDemocracy are the publishers and copyright holders, and that I am able to share excerpts from the article on a Fair Use non-commercial basis with attribution. Thanks are therefore due to openDemocracy]

Like most people, I’m not particularly interested in “search”. I’m interested in “find”. Particularly when I’m not entirely sure what I’m looking for. For various reasons I was thinking about search today, and remembered something I’d read a while ago.
Many years ago, during the heady days of the last Web boom, Mary Modahl (currently on the board of Yankee Group) wrote a book called Now Or Never: How Companies Must Change Today To Win The Battle For Internet Customers.

It’s a good book. And in it, Modahl recounts the tale of a car salesman in Somewheresville, USA. He had his own dealership, ran a solid and very profitable business. For many years. People came from all over the place to his dealership, because he was such a good salesman.

Now he knew that the number one item his customers wanted was a white pickup truck. But Head Office kept sending him green pickup trucks. And his customers would come looking for white pickups, and leave with green pickups. Somewhere deep inside the bowels of Head Office, someone would notice a spike in the sales of green pickups, and with the customary flash of brilliance associated with such people,  raise the production targets for green pickups, override all the salesman requests for white pickups, and send them even more green stuff.

But that was then. Today, the customer checks on the web first, makes contact with the car dealers, and then only goes to dealers that have white pickups for sale. So now, through no fault of his own, the salesman is behind the eight ball. He never gets the chance to use his dazzling selling skills, because the customers figured out he hasn’t got what they want. And the reason he doesn’t have what they want? Not because he didn’t know — he did — but because someone else was interested in what was sold to the customer and not what the customer wanted to buy.

The Intention.

[My apologies to Mary Modahl and to anyone else associated with the book if you feel I have misquoted; it’s six or seven years since I read the book, and the quote’s a paraphrase from memory].

I use the story to try and explain where I would like to see search going, particularly in an enterprise context.

I want to see much more of “Did you find what you were looking for?”

I want to understand why some people find what they are looking for faster than others.

I want to see the routes people take to do the finding.

Because somewhere in all that, somewhere among the steps and the categorisation and the differentiation, there is expertise. Some of the expertise is in the use of the search tool. But most of it is a way of looking into a person’s head and distilling the expertise contained, in a manner that it can be shared. And that’s when you have magic in the enterprise and in the classroom.
Search is about finding. And the path of discovery is about learning. And about expertise. The next generation of search will be about intention and how to capture and refine and improve the process.

Imagine there was a significant risk of a life-threatenic epidemic of some medical condition or the other. Imagine there was a vaccine or perhaps an antidote for this condition. Imagine there was only one patent-holder. Imagine that the short-term demand for the medicines far exceeded the supply capacity of the patent holder.

Thankfully people far more learned than I’ll ever be have considered scenarios like this, and as a result the concept of compulsory licences exists. In simple terms, a government or equivalent body can overturn the rights of patent holders in such situations and grant licences to their competitors, using a Malcolm Quantum Energy approach to the problem. Malcolm loves isolating a problem and then giving it some Tipping Point level of energy and resource to see if innovative solutions emerge. He’s wise beyond his hair.
It nearly happened in the US with anthrax in 2001. It nearly happened in many parts of the world with bird flu. I suspect it probably happened in some form or shape with SARS; weak versions of the process have already been used for HIV and for AIDS. The TRIPS legislation, inadequate and almost-dangerous in many respects, nevertheless tries to do something about this from a world trade perspective.

So we have some sort of sledgehammer for the lock-in of patents, but these are for physical things and so they are easyish to understand. There are many problems to do with the way compulsory licences work; these relate to import and export and localisation and recompense to patent holders and a host of other things. But at least there is a sledgehammer. And we are learning. And we will get better.

Until people started talking about trying to patent software, my primary interest in patents came from pure curiosity, and was concentrated on “functional” and “life-saving” medicines. I couldn’t for the life of me understand how anyone could justify making something as essential as a life-saving drug hard to get, particularly by the use of artificial monopolies and their subsequent monopoly rents.

Even then I was no more than an active bystander, an interested observer. Until people tried to patent software. Then, as I began to perceive the unholy messes that could be caused, I started getting interesting in everything to do with patents. So that I could learn to do the right thing.

My family business was in journalism, and we had a printing press. At school and in church we had “cyclostyle” machines which we used to churn out various types of pamphlet and journal. We had dozens of typewriters at home. And over the last 20 years, I have seen the technology and costs of reproduction and transmission of the reproduced copies improve dramatically, with photocopiers and fax machines and scanners and digital cameras.

The same happened with music. My girlfriend had a massive Akai reel-to-reel tape system, and used it to record things. I grew up with cassette tapes and recordable CDs through to today’s iPod Generation.

The same thing happened with video. And with cameras in general.
Yet I bought books, I didn’t photocopy them. I bought prerecorded tapes. I bought CDs and DVDs by the cartload. And I never let anyone copy them. Never made a copy of them either. I’ve never bought anything from iTunes, though my children have. I use iTunes as a means of transferring music I already own on to my iPods.

I am not unique. There are many people who care about IPR and DRM because of the things that go wrong, not because they want to cheat “the system” or authors or musicians. They want to do the right thing.

We have a lot of things wrong with the “system” of IPR and DRM. They are no longer fit for purpose, they create behaviours and consequences that are diametrically opposed to the original intent of the rules and regulation. Innovation is hampered. Productivity is impeded and reduced. Incentives are used for intermediation and lock-in rather than disintermediation and freedom. Costs rise instead of dropping. The wealth created by the passage of Moore’s Law and Metcalfe’s Law and Gilder’s Law is frittered away, to a point of absurdity.

Patents are hard to write, to protect. Prior art is hard to discover. The process is creaking to a point of gridlock. The “system” is being corrupted and arbitraged to a point beyond absurdity.
That’s why we have to figure out how to do the right thing.

Let me take a simple example. Let’s say Clarence Fisher or Judy Breck come up with a scheme to create a global market for podcasts of lessons. That children are encouraged to trade their favourite podcasts, issued under a Creative Commons label. That the children contribute the podcasts, that they use social networking and collaborative filtering. That this trade happens internationally, similar to the legendary Grateful Dead tapes. [I quote from Wikipedia: The Grateful Dead allowed their fans to tape their shows like several other bands during the time. For many years the tapers set up their microphones wherever they could. Naturally the best sound was in front of the sound board. The eventual forest of microphones became a problem for the official sound crew. Eventually this was solved by having a dedicated taping section located behind the soundboard, which required a special “tapers” ticket. The band allowed sharing of tapes of their shows, as long as no profits were made on the sale of their show tapes. Recently, there was some dispute over what recordings archive.org could host on their site.] [Told you that Jerry Garcia influenced my opensource thinking!]

Wouldn’t it be great if there was a Dick’s Picks of favourite lessons on podcasts?

Then imagine that these kids, worldwide, needed media players in order to play these podcasts. Imagine that Apple or Microsoft or Real or Someone New dominated the media player space.

And imagine that the media players had bad DRM built into their DNA, so that the kids couldn’t play the Creative Commons-licensed podcasts. Imagine the media players were themselves locked in to specific and expensive devices. Imagine that the media player producers and the device producers and the connect providers and the “content” providers all banded together and made sure that the DRM chain was pure and ensured complete lock-in.

Imagine the kids who couldn’t listen to or watch the podcasts. Just because of bad DRM. What a shame that would be.

And this is not science fiction. It is happening. Now.

We have to learn to do the right thing. 

[An aside. We’re going to see a lot of legal activity on this front. This story (thanks to Cory), challenging what a digital copy is, who made the copy, what the original was, who owned the original, and so on, is just the beginning. Every enterprise has its hands full trying to prove that Person A is associated with Password B and Second-Factor authentication C and worked at device with IP address D and downloaded/altered/deleted E.  It is not as easy as it should be. Even in controlled work environments. ]

We have to learn to do the right thing.