Sharon makes some interesting observations in a post headlined Jimmy versus Jack. Examples:
- Wikipedia is a great resource, but I have been even more fascinated by a model of dealing with knowledge that could dispense with the elitism inherent in peer review. Wikipediaâ€™s open editing model sounds so wonderfully subversive. But now that Wikipedia has a dominant web presence, itâ€™s finding that allowing the masses to have free reign over knowledge has its downsides.
- I have argued […….] that peer review, though an imperfect system, may be the best system we have for dealing with scienceâ€“at least as it pertains to science funded by government. Iâ€™ve often doubted my own conviction about this argument, and had secretly hoped that Wikipedia offered some alternativeâ€“if not for funding scienceâ€“then at least for propagating science that might be unfairly quashed by peer review.
- In the final analysis, my issue […..] isnâ€™t even whether […..] belongs to […….], but who gets to determine that classification. Wikipedians, [……], Jimmy Wales, or perhaps peer review?
- I donâ€™t have the answer, and neither does Wikipedia,……
My italics. My elisions. My emboldening. And, if necessary, my bad.
Peer reviews can often be like benchmarking. Everyone accepts the intrinsic value, and yet everyone tends to use the process only when they can be sure that the answer is one that suits them.
I’m interested in Sharon’s observation that there is an elitism inherent in peer reviewing, particularly in scientific academia. More than just interested, intrigued. I sense that all her statements are in the same ball-park as my concerns with gatekeepers.
Humour me as I go for a wander. Take the origin of the word “bankrupt”. Wikipedia has this to say:
- The word bankruptcy is formed from the ancient Latin bancus (a bench or table), and ruptus (broken). A “bank” originally referred to a bench, which the first bankers had in the public places, in markets, fairs, etc. on which they tolled their money, wrote their bills of exchange, etc. Hence, when a banker failed, he broke his bank, to advertise to the public that the person to whom the bank belonged was no longer in a condition to continue his business. As this practice was very frequent in Italy, it is said the term bankrupt is derived from the Italian banco rotto, broken bench (see e.g. Ponte Vecchio). Others rather choose to deduce the word from the French banque, table, and route, vestigium, trace, by metaphor from the sign left in the ground, of a table once fastened to it and now gone. On this principle they trace the origin of bankrupts from the ancient Roman mensarii or argentarii, who had their tabernae or mensae in certain public places; and who, when they fled, or made off with the money that had been entrusted to them, left only the sign or shadow of their former station behind them.
I’ve looked at the Shorter Oxford and at Skeat, and they both endorse what I have always thought. The term is understood to have derived from the Italian banca rotta around the middle of the 16th century. Which is consistent with imagery of the merchants of Lombardy breaking the moneychanging bench of one of their peers when he let them down, so that he could not transact any business in the marketplace.
Peer review. Of a sort. Peer-driven action as a result of some community value or more or principle being broken. [I don’t buy the argument, suggested in the Wikipedia entry, amongst others, that the failing banker broke his own bank.]
The rules appear to be simple and consistent. Anyone is allowed to speak. But you cannot trash the monarchy or seek to overthrow the government. Apparently.
So you can speak about pretty much any subject you like. And there are no bouncers or gatekeepers about.Â But there are regulars about. Regulars who listen to anyone and everyone, quick to heckle, quick to clap. Who are these regulars? In the context of the All Comers market, their peers.
Destruction of the tools of trade, as in the Lombardy bankrupts. Ejection from the place of trade, as in defenestration. Heckling and jeering, as at Speakers Corner. At some level of abstraction, I guess that even jury trials are a form of peer review.
All of them have their strengths and weaknesses. The bulk of the weakness comes from being able to game the system, which happens as soon as you introduce some formal selection process for the peers, or (much worse) some barriers to selection or entry.
Peer review processes work best when the peers are the men on the Clapham omnibus, or even with Twelve Angry Men.
But we need to keep the selection criteria for peers as open as possible, and the barriers to speaking/publishing/sharing as low as possible.
I need to understand more about how elitism comes into play in scientific reviews. Any offers out there?