I hear you say “Enough already!” to the A-list-blogger-as-gatekeeper debate; so no more on the subject.
What I’d like to do instead is open up debate on a question that kept bugging me throughout that debate:
If we believe in a Long Tail World, then why do we insist on looking at that Long Tail World through the eyes of a Hit Culture?
Discussions about ranking are in some form or shape related to a Hit Culture. Ever since search engines have been available, I have seen papers suggesting that Big-Gets-Bigger or Rich-Get-Richer. If you’re popular, then you get to the top of the rankings, which makes you even more popular. And so on and so forth.
That was the received wisdom. But something about it didn’t make sense to me. Soon after reading Cluetrain in 2000, I’d had the pleasure of meeting Chris Locke in Bangalore and again in London, and he was talking about communities that did “left-handed organic gardening” as being an intrinsic part of web communities and of the nascent blogosphere.
Around the same time, I started following Amy Jo Kim’s work on web communities, directly as a consequence of reading her book, Community Building On The Web. And she in turn influenced a lot of my thinking about how web communities work; I was particularly intrigued by her discussions on how subgroups emerge and why they should be encouraged to emerge. I quote from her site:
- If your goal is to build a robust, large-scale community, then fostering member-run subgroups should be an integral part of your community strategy. Whether they’re set up by the community staff, or created by the members themselves, these small groups are where people will form their deepest relationships and strongest loyalties. That’s why it’s crucial to understand how these groups evolve, and make sure that you cultivate a fertile environment within which they can take root and grow.
I took these member-created subgroups to be the same thing as Chris’s Organic Gardening sites; micromarkets with microconversations involving people who had a very specific narrow-focus interest binding them together.
By this time, I’d already become a fan of Steven Johnson’s after reading Interface Culture in 1997, so by the time Emergence came out, aided and abetted by Lazslo-Barabasi’s Linked and Bloom’s Global Brain (both, incidentally, referred to me by Gary Casey!), I was getting very comfortable with the idea that lots of little and specific and healthy markets were where the action was, and that all this represented the disaggregation of the Hit Culture.
You can see that I was ready for The Long Tail, especially since I’d also been exposed to power laws and Zipf curves a few decades earlier while at university.
Yet I kept seeing Hit Culture attitudes, particularly to do with search engines and ranking, and more recently in the A-list gatekeeper discussions. This intrigued me, and continues to intrigue me. [Which is why I do apparently strange things like look at Youtube’s Most Linked, Most Viewed, Most Discussed and Top Rated All-Time video lists, to see what’s happening. And for sure I see Long Tail and not Hit Culture.]
Bearing all this in mind, I was fascinated by an article in the New Scientist. Headlined Internet Search Engines Go on Trial, what was of particular interest to me was a study it cited, done by researchers at Indiana University. You can access elements of the study here.
I quote from the excerpt to the study, titled Topical Interests And The Mitigation Of Search Engine Bias:
Search engines have become key media for our scientific, economic, and social activities by enabling people to access information on the web despite its size and complexity. On the down side, search engines bias the traffic of users according to their page ranking strategies, and it has been argued that they create a vicious cycle that amplifies the dominance of established and already popular sites. This bias could lead to a dangerous monopoly of information. We show that, contrary to intuition, empirical data do not support this conclusion; popular sites receive far less traffic than predicted. We discuss a model that accurately predicts traffic data patterns by taking into consideration the topical interests of users and their searching behavior in addition to the way search engines rank pages. The heterogeneity of user interests explains the observed mitigation of search engines’ popularity bias.
Surprisingly, I could not sign up to a full subscription to PNAS online, there is a forced offline step and associated time delay. [Which gives me time to figure out whether I already have access as a result of some other academic/professional body.] So I cannot claim to have read the entire article. But I will, soon.
I am truly fascinated by this, because it allows me to get rid of one of those things that made me Confused.
And I genuinely believe that is how the blogosphere tends to work. Not hits, not ranking-linking-vicious-circles, but heterogeneous long-tail interested communities. Comments welcome.
Fortunato and co also gave a paper at the WWW2006 conference called “The Egalitarian Effects of Search Engines”. This quote from it makes a related point:
“the net effect of search engines on traffic appears to produce an egalitarian effect, smearing out the traffic attraction of high-degree pages … search engines lead users to visiting about 20% more pages than surfing alone … contrary to intuition and prior hypotheses, the use of search engines contributes to a more level playing field, in which new Web sites have a greater chance of being discovered and thus of acquiring links and popularity. ”
The full paper is freely availabe here:
http://arxiv.org/PS_cache/cs/pdf/0511/0511005.pdf and their “Decoding the structure of the WWW: facts versus
sampling biases” is also interesting:
http://arxiv.org/PS_cache/cs/pdf/0511/0511035v1.pdf
And I suppose that my commenting on this post, because someone I know and trust said “hey, you should read Confused of Calcutta” (he was quite right), and then thinking, “I seem to remember quoting Fortunato in some contract reasearch I did last year for the ESRC”, and then finding another Fortunato paper, and putting the links into this post, is an example of exactly the sort of thing that you (and Fortunato) are talking about.
Seb
Absolutely, Seb. You made my day. And it’s still early in the UK! You’ve catalysed me into writing a related post, which I shall do sometime today. Welcome to the conversation.
Fractals. It’s all about fractals. One community’s hit is another’s tail. And most of humanity spends their life zooming in and out as their mood or moment dictates. One moment I’m reading ‘Confused’ alongside a few thousand (?) others and the next I’m watching the World Cup Final with half of humanity (and I don’t even like football!) – and both are worthwhile. Fractals and power laws. (How’s that for hubristic reductionism?!)
Very interesting bit of research and a very positive conclusion for the web in general. Puts the entire SEO industry into a questionable light too…
Am keen to avoid confirmation bias though ;-) so I guess the question is whether their model of search and browse can be extrapolated to the way people navigate the blogosphere which, as Seb notes above, is more typically by recommendation (either explicit or implicit via links).
This means the way bloggers (esp. the A list) recommend and link to other blogs is critical — it’s the ‘pagerank’ of the blogosphere. Who knows whether this human, non-algorithmic process is more or less equitable than Google’s. Hopefully Fortunato et al will figure it out…
I believe that “we insist on looking at the Long Tail World through the eyes of a Hit Culture” because that is basically the mindset we bring to our commercial endeavors. I write as someone who basically “lives on the Long Tail.” I have abandoned bookstores because just about any book I want to buy is a low-demand item; and the Internet is just better at selling those things than bookstores are. On the other hand I believe that it is highly unlikely that television distributors (cable or satellite) will ever accommodate my Long Tail needs. Here is my argument:
Opera has taken a real beating with television distribution (at least in the United States). Not counting what Public Television has done, in the earliest days of cable, the Arts & Entertainment channel offered a fair amount of opera and was, for the most part, sensible about commercial breaks. Then Bravo came along, around the time that A&E was beginning to pull away from content that was too “heavy” and offered opera WITHOUT commercials. When I left for a four-year stint in Singapore, Bravo was still a really good source for “high culture” content; but, by the time I returned, Bravo had commercials and was no longer showing “heavy” content. Meanwhile, the closest A&E got was “Breakfast with the Arts.” Then a new channel called Ovation was launched, and they jumped in with an impressive library of opera on video. Ovation bit the dust after a couple of years of not being able to sell time for commercials, “Breakfast with the Arts” has gotten far more “light-weight,” and Bravo now sustains itself with reality programming. I am sure that there is still a community of opera lovers out on that Long Tail, but it is easier to buy opera DVDs on Amazon than to expect to see anything through your cable. (At least Comcast has allocated one of their audio channels for opera music!)
Sounds to me like there is an opportunity for a business there. Opera seems to be a classic Long Tail deal. I like opera, but not to the extent I could call it a passion. Anyone else interested in taking up Stephen’s challenge?
JP, there is one fly in the ointment, at least on this side of the pond. It’s called the Musicians’ Union! I had some discussions with an ensemble here in San Francisco that specializes in contemporary music (the “serious” kind, not hip-hop). They discovered that most of the audience cannot remember the names of any of the composers they have heard the morning after the concert (talk about being way out there on the Long Tail)! Needless to day, there would be all sorts of ways to use the Internet to familiarize the general public with these composers, including Web sites with audio links and podcasts; but there is a whole morass of problems surrounding royalties and licensces from ASCAP or BMI. Ironically, the Philharmonia Baroque has been doing some very innovate things with both their Web site and a volunteer podcaster (at least I cannot imagine that he is being paid for his efforts); but the Union may yet push back on this. Perhaps the Long Tail runs into trouble when, even though the pie is very small, you still have people fixated on fighting over the size of their own slices!