Category: Filtering

Thinking lazily about notifications and alerts: Part 3

[Note to readers: For those coming into this cold, I wrote Part 1 and Part 2 early in January; this one, Part 3, will be followed by a handful more in weeks to come].

I think of sensors this way: every piece of equipment capable of sensing something and sending me information about that something is in effect extending my own set of senses. A security camera outside the house extends my eyesight. If it is capable of night vision, it extends my ability to see. If it is linked to a network and allows me to “see” from afar, it allows me to sense remotely. If it has the capacity to retain what it sees and allows me to query it at a later date, it allows me to “see” into the past, to travel in time as well and in space.

KKlaughsm.jpg

Kevin Kelly (one of my favourite authors)

Sensors extend our natural senses. As I mentioned in an earlier post, this is very much in keeping with Kevin Kelly’s assertion that technology speeds up evolution. [I love having a reason to re-read essays in his Technium. Something I would commend you do, it is well worth the while].

It’s not just about enhancing an existing sense or extending the sensing capacity over distance or over time. We are also on the threshold of “sensing” new things, things we could not sense before, things that haven’t existed for long. Nobody thinks twice about our current ability to “sense” (via satnav apps) the nearest ATM, the nearest fuel selling unit, the nearest electric car charging facility.

Yup, our smart devices, and their ecosystems of hardware extensions (sometimes as wearables) and software apps, they’re all part of our rapidly expanding sensory network. Some form of Extra Sensory Perception is now reality.

It doesn’t stop there. It’s now been a few years since I first read about how we’re bringing sensors closer to home, not just by wearing them or hanging them off our smart devices, but by implanting them. Cochlear implants that, while improving our sense of hearing, also throw in, for the heck of it, an ability to sniff out wifi signal and compass direction, just to give a few examples. I guess it will only be a matter of time before people gain “x-ray vision” or an ability to see through walls. Time to dust off that William Gibson mantra again. The future’s already here, it’s just not very evenly distributed.

Some of our technology-enabled extra-sensory ability to solve time and distance issues have existed for a while. The security guard in a room somewhere looking at a wall of screens, checking for exceptions. The recording capacity attached to surveillance cameras. All ways to “sense” something and receive that sensing in a different place or at a different time.

Prior to that, the technology used to do the remote sensing was cruder. Mountain and Mahomet time. Since the remote place couldn’t come to you, you went to the remote place physically. Before video cameras, security guards used to do the rounds the old-fashioned way. Expensive, inefficient but reasonably effective. Most of you must have seen films set in the Second World War where prisoners of war escape only when the security guard is at the aphelion of his personal orbit vis-a-vis the would-be escapees.

We’ve seen this movie before. Before telephones we couldn’t speak over any real distance. Telephones conquered the distance problem, but not the time one. If the party you wished to speak to wasn’t there, tough. No tickee no washee. Then came a time when people took messages for you if you weren’t there. And left them by the phone, so you had to get to the phone to see them. Then came answerphones that looked like tape recorders, attached to the phone. The message taking facility had become a little more automated. Then you could query those messages remotely. And the capacity to hold messages increased rapidly. And then the message could call you.

That telephone-message movie is now playing out for pretty much every form of notification or alert there is.

Firehoses.

Firehoses of notifications and alerts.

So they need filters before they can be rendered useful.

Notification Class 3, Houston We Have a Problem, is a filtering mechanism. Conditional, with thresholds that can be set in advance and changed at will. Some of you will be used to IFTTT, If This Then That. Houston We Have a Problem needs IFTTT for notifications, nothing more. You don’t want the firehose. You want to know only if some exception condition is met or breached. If the doorbell rings and the only person in the house is my aged aunt who doesn’t hear too well, then make the bell ring more loudly, or make a pre-agreed light fitting flash. You get my drift. Establish the threshold for something that can be sensed remotely, then establish the notification process that is triggered when the threshold is passed.

There is a lot more to be done with the notification process in such contexts, in terms of the devices alerted, the timing of the alerts, the route taken (sound, light, movement, etc), whether the alert is persisted for later inspection, whether time series of the alerts are themselves persisted, whether there is a need for acknowledgement of the notification (and that’s a notification in itself).

Imagine the drudgery of building maintenance and you can see just how efficient it can become. No need to inspect the bulbs, batteries, toilets, rubbish chutes, whatever. Remote sensors will tell you when a threshold condition is breached. Maintenance by exception.

Enough on Class 3 for now. The next class of notification is I Am Here. The best way to think of this one is as the answer to the question “Dude, Where’s My Stuff?”. There’s a lot of motion in life, lots of stuff moving around. And people want to know about the state of that thing in motion. Are you on the train, on your way home yet? Did you get my letter? Where’s the dress I ordered? Where’s Kevin? Where’s Wally? Did you get my cheque? (No, not the one’s that’s “in the post”, along with Billy Bunter’s Postal Order or the homework that the dog ate).

Fundamentally, I Am Here signals the current position of something in motion. The requester of the notification wants to know that Elvis has Left the Building. That the Package Has Arrived. That the Train has Passed Clapham Junction. Whatever.

So many notifications, so little time. Filters needed. We need to be able to subscribe to the notifications sensibly, so there’s a pub-sub approach needed. We need to have some way of signalling conditional routing, not just about the thresholds to be tested against, but also the routes taken by the notification, the timing of the notifications, and the devices to which the notifications will be sent.

That’s enough on I Am Here for now. I will deal with further classes of notification, and start building on the subscription models and threshold conditions in later posts.

 

 

 

 

 

 

Thinking lazily about notifications and alerts: Part 2

This is the second in a series on notifications and alerts, building on what I started sharing earlier today, as promised.

First, a musical interlude.

Someone’s knocking at the door, somebody’s ringing the bell/ Do me a favour/Open the door/And let them in.

Mum, the kettle’s boiling/Daddy, what’s the time/Sis, look what you’re doing/Can’t you see/The baby’s crying

We get used to receiving and processing notifications while we are still children. Doorbells ringing. Kettles boiling. Hearing footsteps approach, learning which ones are friendly, recognising the patterns made by parents and siblings.

And yet the one I remember most vividly is to do with what’s exemplified in the photo below:

kayarakhet31.jpg

[My thanks to VikalpSangam whose photo of Mohan, above, helps make my point].

It’s  a very strong childhood memory, one that is almost as strong as sensing the presence or return of a parent. You see, I wasn’t much of a sleeper. [Never was. And now that I’m approaching 60, it looks like I never will be. The power of habit]. As I lay awake, tossing and turning in the sweltering heat of a Calcutta night, I’d hear a strange sound. A sound I came to treat as a friend. The sound made when a stout stick gently hits a lamppost. A strange sound indeed.

We used to live on a street called Hindustan Park in the 1960s, in Ballygunge in Calcutta. I don’t know the precise history behind the phenomenon, but what I can remember is this. The local darwans, who performed roles of building manager, maintenance man and security guard, used to take turns to walk around the neighbourhood while their colleagues dozed. The one doing the walking would signal his presence and doing of the rounds by twanging the occasional lamppost with his danda, his lathi.

It seemed to me that his lamppost-striking action achieved many outcomes: it alerted his colleagues that he was doing his rounds; it probably alerted would-be burglars as well, but more of that in another post; most importantly, it made me feel that God was in his Heaven and that All was Well with the World. It wasn’t just about presence being signalled over distance, it was about the sense of security implied by that presence.

Which brings me to the first class of notification: All is Well.

There’s a rhythm, a pulse, a cadence, to the All is Well notification. It’s a repeating signal. It’s like hearing the sounds a baby makes while asleep. It’s like seeing an ECG at a hospital bedside. There is no need for alarm, no warning threshold has been breached, no action needs to be taken. But its absence is often a signal for action, for investigation.

As the number of sensors continues to grow exponentially, and as we get better at joining the data collected and creating value via insights, we will learn to build baselines for many such things. Initially these baselines may well pertain to single senses, but as we learn and adapt we will build multisensory baselines as well. We will describe whole environments in multisensory ways: a child’s bedroom; a person working out in the gym; a restaurant kitchen in full flow; a factory floor full of robots; a street with a mixture of driven and driverless vehicles. For each of these, we will have established when All is Well: the temperature, the energy consumption, the heart rate, the breathing sounds, the ambient noise, whatever.

It’s important to distinguish the All is Well notification from all others; I think it would be a mistake to assume that people only want a variant of “management by exception” reporting. It’s like the mother wanting to check that the child is still alive, asleep, relaxed. There’s a wellbeing signal, a Linus’ security blanket involved, and this should not be confused with exception management.

Right now it may not be obvious why we should concern ourselves with the All is Well notification when in the context of enterprise software. But I think it’s only a matter of time. One of the implications of hyperconnectedness appears to be that by the time we find out something’s wrong, it’s too late to avoid damage. Our early warning systems will learn to become more sophisticated in order to deal with the problems of connectedness.

It’s worth taking a leaf out of David Agus’s book in this context. Marc Benioff introduced me to David some years ago, and I found his line of thinking very instructive, concentrating on wellness rather than illness. The human body is just one great example of complex adaptive systems in operation, and there is much we can learn from people like David as a result; it is then up to us to adapt that learning to the enterprise context. With the Second Machine Age  of Brynjolfsson and McAfee now upon us, we have to get a move on in understanding how to keep notified of a state of wellness at work and at play, as collectives and as individuals.

So there’s a lot of work to be done in fleshing out the All is Well notification. How to form the new baselines. How those baselines move from being single-sense to multi sensory. The role of time series in all this. The increasing march of robotics, of augmented reality, of hybrid operating environments. The likely arrays of unintended consequences we will face as we go through the learning: the world-ruled-by-algorithm issues identified by people like Kevin Slavin,  the problems caused by poorly designed filters as described by people like Eli Pariser, modern versions of Asimov’s Three Laws, we have all that and more to face and to adapt around.

That’s only the beginning, as we then learn more about which notifications to receive on which devices, and when;  how those notifications will announce themselves when they arrive; when the receipt of a notification has legal standing.

It’s only the beginning.

In my next post I shall be dealing with the next two classes of notification: the Houston, We Have a Problem class and the I Am Here class. After that I shall try and wrap up the remaining classes of notification quickly, so that I can concentrate on the filtering/subscription processes.

Some of you have suggested that I should hold these thoughts and write a book around them. I’d rather share and learn via places like this one here, even if the conversations are with just a small number of people. It’s not that I don’t want to write a book. I do and I will. Sometime. But not about this. More to the point, I want things like this to be discussed openly so that we can all learn. Maybe I have the most learning to do. I will find out soon enough, even if only in private.

Feel free to engage via Twitter or LinkedIn or Facebook or even here, if it’s not too retro for you. Use e-mail only if you absolutely must, we haven’t had that spirit here since 1969.

 

Thinking lazily about notifications and alerts

There are some things  that I can sense for myself. I can see something, hear something, touch something, smell something, taste something, provided it happens within the range of the sense needing to be used.

With the necessary training, I may be able to add granularity to what I sense. Some people can gauge distance with great precision, assess the weight or height of an object. Sometimes this is done with senses working in combination: a smell may impart information about taste.

Most of the time, granularity is added by our using tools: measuring jugs, scales, thermometers and the like. The dashboards we build at work are just examples of measuring tools, often providing granularity to something that a skilled person can “sense”. For example, a good sales manager will know how she is doing against target, but use a dashboard to add granularity to that knowledge.

Ever since I first heard Kevin Kelly speak of technology as something that can speed up evolution, that concept has fascinated me. The internet of things may mean many things to many people: what intrigues me the most about it is the idea of “more sensors, more actuators”. When those sensors are networked together, then terms like collective intelligence and “global brains” start gaining prominence. When the datasets collected by those sensors are labelled usefully, then terms like “big data” become more meaningful.

Connecting sensors together allows us to conquer “distance”, to sense things that are happening elsewhere. It’s been a few decades since terms like “the death of distance” were used to describe what the internet and the web represented. As mobile devices proliferated and became “smarter” this trend continued.

As it became possible to persist the data collected by these myriad sensors, and to make that data available across networks, we began to talk about our new-found ability to “shift time”. A trend that was probably first seen in messaging (centrally held paper-based phone messages, then paging services, then locally held voicemail, then remotely accessible voicemail, and so on) managed to work its way into other spheres of activity. The television industry was the one that felt it quite deeply, as people decided to record stuff to watch later; it then became only a matter of time before video on demand became normal and TV went “nonlinear”.

Whenever I think of notifications and alerts, I tend to view them with all this in mind. That the notification or the alert is made possible because I can “sense” more. That the sensing is taking place while I am able to shift time and/or place.

There’s one more thing. Filters. Something which regular readers will know I’ve spent quite some time on before, here, in this series, and even earlier in this series.

When you have sharp increases in the number of sensors available, when these sensors allow you to sense things far away, when the event being sensed may have taken place earlier, then the ability to filter notifications and alerts becomes very important.

I’ve been spending time thinking about this, and plan to share my thoughts here over the next few weeks, looking for criticism, for feedback, for advice. Stay tuned.

 

 

On firehoses and filters: Part 2

Note: This is a follow-up to an earlier post on the subject, written in May this year. You may find it worth the while to read that one first. But if you don’t feel like it, no problem. This post is readable standalone.

I love the very concept of publish-subscribe: if you search for the term in this blog, you’ll find I’ve written maybe a dozen posts on the subject over the last six years or so. So I thought it would be worth talking about filters and firehoses in that context.

So let me start with the three “laws” of information filtering that I laid out in that earlier post: (if you want to know why, please read the post; I’ve linked it it earlier in this post)

  1. Where possible, avoid filtering “on the way in”; let the brain work out what is valuable and what is not.
  2. Always filter “on the way out”: think hard about what you say or write for public consumption, why you share what you share.
  3. If you must filter “on the way in” then make sure that the filter is at the edge, the consumer, the receiver, the subscriber, and not at the source or publisher.

Yes, as Clay Shirky put it, we don’t live in an age of information overload; rather, we live in an age of filter failure.

So everyone’s been looking for better filters.

Which is fine.

What’s not fine is when we expect the publishers to do the filtering. Because that allows bad actors to come and spoil the party, whether they’re bad corporations or bad governments.

It’s not the job of the publisher to do the filtering. It should not be the job of the publisher to do the filtering.

But there is a job for the publisher to do. And that is to provide the tools by which subscribers can filter.

Let me expand on this. [Incidentally, when I started using Google+, I raised this issue under the guise of asking for “circles” to be built by publishers as well as by subscribers…. with some interesting discussions and comments as a result].

We live in a world of publish-subscribe, and this world has three facets.

There is an infrastructure, allowing individuals and aggregations of individuals to publish stuff, and to subscribe to stuff.

We exist as publishers, sharing stuff on this infrastructure.

We also exist as subscribers, sharing stuff on this infrastructure.

So now let me look at what I want to do as a subscriber. I want to choose whom and what I “follow” or subscribe to. Most of the time, I expect to be allowed to follow whatever I choose. Sometimes the publisher places a restriction on following, permission is needed. You may just have to ask for permission, register in some form or the other. In some cases registration alone is not enough, there is a gatekeeper who decides whether you qualify. And in extreme cases there is a paywall as well.

Subscriptions can therefore be open or closed, paid or unpaid. But they remain subscriptions. Choices made by the subscriber.

As a subscriber I now receive information. And I’d like that information to have certain characteristics. One, I want that information available to me wherever I am, whenever I am, whatever device I am using. [And I don’t want to have to pay multiple times for the same information as it goes through format transformation]. Two, I want to be able to annotate that information, add notes, tags, links, images, whatever. Three, I want to be able to share that information, via twitter, facebook, google+, chatter, whatever; as in the case of opensource licences, it makes sense that I have to share-alike, share the information with the same constraints under which it was shared with me. Four, I want to be able to filter that information very granularly: for example, I may want to follow a person on twitter, but only for her music tweets; I may want to follow someone else on Google+, but for everything but his music posts. Five, I want to be able to persist the bookmarks, tags, shares, links, pointers, whatever, somewhere, so that I can recreate, “play back”, the shared information, for a specific date or range of dates, and with specific filters.

Simple, isn’t it?

So what should I do as a publisher? Even simpler. As a publisher I need to be able to share what I share in such a way that subscribers can do what I’ve described.

And infrastructure providers have an even simpler job: all they have to do is to provide the tools that publishers and subscriber need to do what I’ve described.

For too long, we’ve kept looking at all this from the viewpoint of the publisher. The publisher in each of us has to work much harder at publishing in a way that makes it easier for others to subscribe. To filter us out when needed. To find us easily when needed. To aggregate us, synthesise us, annotate us, edit us to shreds. Platform-independent, location-independent, device-independent. As private as the information requires us to be, and no more. As public as the information requires us to be, and no more.

So when I tweet, I don’t want to restrict what I tweet about. I want to be able to tweet about food, about cricket, about etymology, about idiocy, about work, about me, about my beliefs, whatever. I want to be able to blog about all these as well. And write books about all this, speak on the topics, and so on.

And I want to be able to do all this in such a way you can find me when you want to, block me when you want to, block me by subject, block me for a time, only follow me for a narrow subset of what I do. It has to be your choice. And the infrastructure I use has to be able to do this. It has to be able to let me do all this, so that you can do what you want out of it.

Which is where the fun begins. Because you don’t think of all this as a winner-takes-all arms race. Because we don’t this of all this as an arms race where we have to choose between the Betamax of Google+ and the VHS of Facebook.

Each of us, as subscribers, will choose how and where and when we will subscribe.

Each of us, as publishers, will choose the environment and infrastructure that most suits us to do what the subscribers want.

So federation and sharing between social networks is unavoidable. A multiplicity of such networks will exist, for cultural, technical and style reasons. One size will never fit all, when we’re seven billion people.

And each social network will come with its tools for sharing, for publishing, for subscribing, for filtering, for helping filter.

For helping filter.

Which is what this post has been about. What do I have to do in order to make it easier for you to find me or block me, find this post or block it, save it or share it, add to it or shred it?

Because you will decide. You will decide whether it’s Betamax or VHS or both or something else as well.

We can only fix filter failure by providing subscribers with better filters, by providing publishers with tools that allow subscribers to filter better.

More later.