Thanks for all your comments and tweets re my earlier post. Some of you solved the “unGoogleAble” question. Others commented on what they’d been doing with the Prime Numbers in Arithmetical Progression question. And a number of you engaged in conversation with me across a variety of platforms. It helps me think. And learn. For which I’m grateful. I hope it’s been of some use to some of you. I want to touch upon a few of the answers and comments, to see if that helps me articulate where I was going more effectively.
First, Nicole Simon’s question. Why would an UnGoogleAble cricket question teach her anything? And if it didn’t, was she a bad student as a result? Or was I a bad teacher? I promised to get back to Nicole when wrapping up this stage of the conversation. I was trying to establish a framework for constructing a decent question in our post-Google world, and suggesting three things:
- it shouldn’t be easy to find via Google
- it should teach the answerer something
- it should be fun
To explain this, I chose two exemplar areas, cricket statistics and prime numbers, each esoteric in its own way, each valuable only to the passionate amateur, particularly one already interested in the mysteries of cricket or primes. For my sins I am a member of both those sets. There was another reason I chose those particular areas. In both cases there was a lot of data on the web, available to the public while remaining not easily GoogleAble. “Open” data. I wanted to find a way of demonstrating what could happen when open data meets the curiosity of the passionate amateur. The cricket question would have been pretty much impossible without access to the data; the prime numbers in series question would have continued its regal and exclusive status of being made available only to those students talented enough to participate in mathematical “olympiads”. Open data has a wonderful democratising effect, accentuating and embellishing the level-playing-fieldness of the internet. Combined with passion and perseverance and patience, the possibilities are boundless, as human beings unleash their creativity in using the tools of the age to work on freely accessible data to test conjectures, solve problems, build new products and services. So Nicole, my answer to you is that I failed. I could not get good enough examples, generic enough examples, to make the post work by itself. I needed the previous post to try and catalyse some of the examples via the comments and answers given, as you will see. I hope as a result you can take something of value away from this post.
Which then brings me to my second point, the comments/answers given by Obiwan Kenobi and Daen de Leon; Obiwan gave me the perfect example of someone who took my question and then worked the example out in detail, and then proceeded to share the workings with everyone else. The answer was not important. It was the sharing of how he got there, which you can see in his comment. He created a query on Cricinfo Statsguru that required no programming knowledge, just an ability to select filters and sort criteria and views; he then proceeded to copy-paste the output to his tool of choice, then came up with the answer. I’m not quite sure why he needed the copy-paste step, all I did was to inspect the same query output, the same 206 lines over 2 pages, looking for duplicates in the right-hand-most column, managing to avoid a couple of red herrings along the way. Obiwan’s example is useful in helping set out what is needed for open data to be valuable. One, people need to know what’s out there: Obiwan knew of the existence of Cricinfo Statsguru, and we may soon need specialised open data directories. [Yes, someone can write an “app” for that, but please please in HTML5 rather than as a captive device app!]. Two, people need simple tools to manipulate that data, and the ability to learn how to use those tools: Obiwan knew how to use the filters in the Statsguru query builder. And three, Obiwan needed the stimulus of my question to be bothered to do it in the first place: new classes of teacher will emerge to provide such access, training, stimulus. While Daen didn’t give us a blow-by-blow account of how he solved the prime number problem, he did two important things: one, he shared a big clue, that he could work out the constant interval using just the limit and the number of terms. That in turn would allow other readers to consider the role of the interval in the answer, while still allowing for some personal discovery as to how to use that information. But the most important thing that Daen did was to share something else …. “I’m proud to say I figured out the interval….”.
Daen was doing something really important there, sharing with us the joy he felt when he discovered why something must be true. That’s my third point. The joy of discovery as part of the road to mastery. Kathy Sierra, one of the earliest commenters on my earlier post, is someone I have a lot of time and respect for; to me, what Daen did is the fundamental reason why all the current superficial attempts at “gamification” will fail. All achievements involve learning. The joy of learning is in achieving mastery. In knowledge work, that mastery begins when you have a little light-bulb go off in your head and you say “aha” or “eureka” or “gadzooks”…… because you know why something is true. When a process is standardised, stable and easily repeatable, knowing how may have been enough. Today, many processes are what my friend Sig calls “barely repeatable”, so exception handling becomes the norm, an unsustainable position. So now it’s no longer enough to know how, you need to know why as well. Gamers know something about nonlinear worlds, about patterns rather than processes, about the value of knowing why and not just how, about working in a peer network and using that peer base to select teams and tasks and tools. Gamers have a lot to teach us at work. “Gamification” will remain lipstick on a pig until the designers have the revelation, the road-to-Damascus moment of understanding the joy of mastery. That’s what people like Kathy are trying to get the world to understand, and the world needs to listen.
So what am I saying? It is in the knowledge of the Why that the foundation of the How is laid. The What is just an instance of the How. [Yes, there’s a Who’s on First gag somewhere there: What doesn’t Matter. What matters is How. How Doesn’t Matter if you don’t know Why…..] It used to be said that the person who knows how will always find work….working for the person who asks (and learns) why. We are fast approaching a world where all the Whats are commoditised. Open. Easily accessible. Published “content” may be seen to be a “what”. And if you’re in the business of making money off content, then you’re going to do everything in your power to protect your business, which means resisting the commoditisation of the what. That’s what most publishing industries face, a commoditisation of the very thing they made their money on, the “what”. The “how” continues to be valuable, so there’s a market for the tools that publish the content, that simplify access, that help massage and manipulate and mutate that content. The “why” is even more valuable, particularly because it is often scarce, tacit, hard to share. But that’s changing.
Which is why I write posts like this one.
Cricinfo’s StatsGuru is good but it’s not data. That’s why I started http://cricdata.org – to provide an open, RESTful, Googleable API to cricket data.
It’s a one-man project, so it’s going to be biased towards answering the questions I find interesting. But I hope to give it enough flexibility to support general queries in a way that allows it to be embedded, mashed up and consumed in any way anybody wants.
It’s my small corner of an important battlefield.
That came out more anti-StatsGuru than I intended. For the record, I think StatsGuru is great at what it does.
@Dominic – why is it not data? I think it is certainly data, but not intelligence (being the BI guy I am)
Btw, I am not sure my previous comment was recorded – if not, please approve this.
I went through the copy/paste process only to eliminate human error. I did what you initially did – go through the list manually. But then I realized a bit later that I counted one of the false positives – which I wouldn’t have, if I had the tools available. So I relied on the tools, by resorting to copy/paste.
@Obiwan Kenobi – I just meant it’s designed to be an end point, not a source. There’s no XML, CSV or JSON version, and you scrape the HTML at your own risk.
What an incredible read. Thank you for sharing, JP.
The only cricket stats of any concern to me is how many set up camp outside my window on nights I cannot sleep, but the revelation of how ‘gamification’ *should* work, paired with the relationships between why, how & what are worth all the confusing mathsport discussion necessary to bring the topic to light.
It seems the single greatest obstacle I struggle with as a publisher is communicating the value of why to our audience. We repeat over and over again why we have dedicated the better part of two and a half years to finding and sharing edifying stories meant to inspire and promote interaction on a richly rewarding, global scale, yet the vast majority simply consume the information and move along.
Still, when those precious few “get it,” and we share in those light bulb moments, it is worth all the struggle in the world. We’re not out to inspire everyone, just one in a million.
Your blog is becoming as much of a source of insight into information as Derek Lowe’s is on drug discovery. Thanks for sharing with us, JP.
I remember reading a book sometime back (Mother of Demons by Eric Flint) which included a new religion as a sub-plot. The main tenet of this religion was that questions are all important and answers only matter to the extent that they lead to further questions. Obviously the sect was being targeted by the established, dogma-based religions in an obvious analogy with the scientific method.
The point being that the process of answering is more important than the answer itself – which is what I understand you to be saying.