When I think about agents, I think of what permissions I have to give someone else in order for them to represent me, and what permissions I have to receive to be someone’s agent. Permissions.
Agents will be able to do magical things for us. Save us time. Complete tasks that are humanly difficult, sometimes impossible. But they’re not born great, they don’t achieve greatness; they have greatness thrust upon them.
Agents will be able to do magical things for us.
They can do nothing without our permission. Sometimes the permissions are explicitly given, sometimes the permissions can be derived from others granted. And sometimes they are obtained by subterfuge or even crime. They should be able to do nothing without our permission. More of that later.
In this post I want to spend time on permissions, on consent, on empowerment. Informed and educated consent. You wouldn’t give your child a loaded gun with the safety off. You wouldn’t tie fireworks to your dog’s tail and set them off. You wouldn’t dump hazardous waste into places where other living things lived or visited or worked. Liberty is not licence.
We’ve been speaking of autonomous cars for decades now, it’s probably over twenty years since I first sat in one. There’s a reason why they aren’t the norm, why they haven’t proliferated. Safety. Reliability. Handling edge cases. Dealing with failure modes.
We live in times when everything is a contract or a covenant. In a contract, there is recourse in the event of failure. We know who pays. Or at least who is meant to pay, as long as courts and adjudications have meaning. In a covenant, when something goes wrong, we don’t ask who pays, we ask “how can we fix this?” Each of us has both contractual as well as covenant relationships, with different pathways for recourse, different ways of fixing the problem, different levels of need for independent adjudication or conflict resolution. All this is part of what makes us human and civilised. No man is an Iland, intire of itselfe; every man is a peece of the Continent, a part of the maine. Politeness. Integrity. Reliability. Consistency. Recourse. Conflict resolution. All small building blocks on our path to civilisation.
Agents will be able to do magical things for us. Things we aren’t able to do, but sometimes wish we could. Things we don’t want to do, or don’t feel like doing. Things we can’t be bothered to do. Things we would prefer to drink arsenic rather than do.
Agents will be able to do magical things for us. Faster than we can. Using less than energy than we would have. Better than we could have. In more efficient ways.
Agents may be personal or shared or public. Owned or rented or borrowed or stolen. Permanent or temporary. Licensed or unlicensed. Reliable or unpredictable. All of the above.
But they remain agents. Reliant on what permissions we give them, on what they’re licensed to do, on how they do it. We can choose to believe they’re animate and sentient, or heading that way. (After all, we can believe an inert virus somehow escaped, never blaming the security, the safeguards, the “hosts” carrying the inert matter. Inert is inert).
Some years ago, I was asked by the CEO of a firm to compile a report on all the interactions between his firm and another, very large, global firm. They had a decent CRM system. It should have been simple. Turned out that in order to collate that data and to produce the report he wanted, I needed over 180 different permissions. Permissions that related to individual business units in different countries, permissions that could only be given by the data protection officials for each of those units in each of those countries, and a few more besides.
Park that for a moment, the architecture of permissions.
If you gave me your private and personal mobile phone number, and someone else asked me for it, could I give it to them? Not unless I had your permission. There’s a trust issue, a confidentiality issue as well. Permissions. Trust. Confidentiality.
I’m retired now. Next year the leading digit in my age will be a 7. When I started work, bosses had assistants, sometimes one per executive, sometimes in pools; usually senior bosses had senior secretaries of their own. Most sat in front of their boss’s office. Some had their own rooms. Regardless of the physical architecture they were in, they were empowered very differently. Mail was placed in pigeonholes for the assistants to pick up. opened mail envelopes addressed to their boss, stamped them to record when they’d been received, and then put them in folders for their bosses to read. Some read those mails and annotated them with recommended actions before seeing their boss, providing a cover sheet with a summary. Some summarised the mails and only highlighted the ones they considered important. Some of the mails went into the bin. Sometimes unread.
When email became common practice, some of these habits were transported into the electronic age. Some assistants had no access whatsoever to their boss’s email; some received a copy of the mails and could read them, but not reply; some could read, and only reply with an “on behalf of”; a few had full access and could reply on their boss’s behalf using their boss’s mailbox to send it.
I saw many many different practices across this spectrum, in terms of what agency the assistant had over their boss’s email. A whole variety. There was only one common thread. Assistants with full read, reply, comment, delete rights were unconditionally trusted by their bosses. That trust was earned rather than bestowed, although an incoming new assistant may occasionally carry the trust earned with a different executive.
Trust was a key factor in the decision to provide the assistant with “mail agency”.
At home, there are a number of people who are trusted by me to have sets of keys to different parts of the house. People who provide us with some sort of regular service. Cleaners. Builders and decorators. House and garden and pond maintenance. Some others. Some get temporary access, some get more long-term access. Some get “all-areas” access, some are restricted, provided access constrained on time or area or something.
Those keys are provided to people we trust.
Children get to ages when they have access to different things. I was given the keys to our flat when I was 13. On the strict understanding that I would (a) always tell my mother where I was going (b) when I would be back and (c) who I was going to be with. At that age, the answers were simple. The Sillimans, a few floors down. The Kapoors, one further floor down. The car parking area behind the house, to play cricket or football. The rules were simple. And I would lose the keys if I didn’t adhere to the rules. Those rules were adjusted year by year, as I grew older. (I didn’t actually need the keys, we lived in a civilised apartment block. People rarely locked doors, they were left wide open or on the latch. It was the principle that counted).
It was a way of teaching me to be trusted about my whereabouts. Liberty not licence. Constrained and verifiable.
Sometimes there are relationships where communications between a pair of people are considered sacrosanct, confidential to the pair, inviolable in that confidence. Husband and wife. Doctor and patient. Priest and confessor. Lawyer and client. Over time, society has seen fit to protect and preserve such confidences, and we have evolved ways to do that. These are trusted relationships. Trust is the core, again.
When parents leave the (appropriately aged) children in charge of the house while they go away somewhere, there are usually constraints. No parties but you can have friends over… but not more than 8. No touching the alcohol. No smoking. No loud music after 10. And keep everything neat and clean for when we get back – leading to the mad rush clean-up just before parents are due back. Trust. Specific permissions, time-bound.
As parents, there are responsibilities that fall on your shoulders when you do leave the children, even for a short while. They have to be old enough to be left without adult supervision. If they’re not yet at relevant ages, then alcohol, medications, guns, cars, things that are age-constrained, have to be removed or safely locked away. And parents are held accountable for making that environment appropriately safe. Society requires us to do that. Other parents need to know that there are relevant rules in place in the household when the parents are away, otherwise their children will not be allowed to visit.
It’s not unusual to ask a friend, neighbour or relative to keep an eye on the house during such absences, and to let the children know who to contact locally in an emergency. Trust is two way and comes with responsibility and accountability.
An aside on secrets and keeping secrets. I remember a tale where Kenny Dalglish, then manager of Liverpool, managed to surprise everyone by bringing Ian Rush back from Juventus. A journalist asked him how he kept it a surprise. Kenny’s answer: Simple. I just didn’t tell anyone.
We make people our agents in many ways. We give them permission, explicit or implicit. Valet parking. Real estate sales and purchase. Dog walkers. Baby sitters. They are all empowered to do something on your behalf, with permissions sometimes narrow, sometimes broad, sometimes short-term, sometimes longer term, sometimes permanent. Powers of attorney for example, temporary or lasting.
Sometimes trust is bestowed without being earned, given the role of the person or group of people involved. Sometimes you trust someone because of their position in a firm or community or even society. Sometimes you trust someone because of the brand they represent, the firm they are ambassadors for.
Many years ago, I used to travel to Bangalore regularly on business. Fly from London to New Delhi. Land there at midnight or shortly after. Wait for five hours for the local flight to Bangalore. Nothing open. A ghost site. Hot. Often humid. And I’d be tired. It’s a long flight. After one or two such experiences, I had to find a way around this. So I would call up a hotel near the airport that I trusted, speak to the manager, and ask for a room or three for 6 hours. And a pick up from the airport, and the return transfer.
I would do this for my travelling colleagues as well. There were often two or three of us making this journey. As I did this more often, I got to know the “front of house” manager at the hotel. In those days, you had to do other things as an international traveller. A few other things. Administrative. Frustrating. Time-consuming. For example, you had to reconfirm return flights or run the risk of not getting a seat. You also had to change foreign currency into rupees. The way my mind worked, I really wanted a way to have all this done while I was sleeping, after having had the customary coffee and samosa
So I asked the front-of-house newly-minted friend what advice he had. And in typical five-star-hotel service style he said “Leave it with me”. And, while I slept, he would personally sort out the rebooking and convert the sterling into rupees. Everything would be done while I slept. And I would make sure he was rewarded for the incredible service.
This used to happen often. Once, when I was travelling with a couple of colleagues, I explained all this to them, and they watched, aghast, as I proceeded to give a man they didn’t know, a man they’d never met before, three sets of things. All our passports. A hefty sum in cash sterling. Our airline tickets.
But they trusted me. So they went along with it. Uncomfortably. Very uncomfortably. But they went along with it.
We had our coffee. We had our samosas. We slept in the rooms booked part-day for us. We came down, relaxed and refreshed, ready to go to the airport for our flight to Bangalore.
Hmmm. Where was the front of house man? Nowhere to be seen. He should have been back. And he wasn’t. He had our cash, our air tickets, our passports. And he wasn’t there.
My colleagues hadn’t quite started to grumble or start recriminations, but I could see they were much discomfited. This was out of their comfort zone. Way out. And they were inwardly cursing themselves for listening to me.
By now I’d begun to have a few doubts myself. Not much, a little flurry. But I had faith. Faith that the status and security he enjoyed in his job, relatively prestigious in his society, those things would ensure he wasn’t tempted to go rogue.
He returned. Fifteen minutes later than he had said. Everything had been done. And all was well.
Why am I telling you all this? Because these are the things I think about when humankind is about to embark on a spree of trust, not knowing what permissions we are giving, what we are putting at risk, what recourse we have.
Trust.
We live in exciting times. AI is here to stay. It’s only going to get better. It’s not great today but directionally fascinating. AI agents are going to be a core part of the journey we go on.
But we don’t have the right trust environment as yet.
When I started work, the “data centre” wasn’t in a cloud. It wasn’t even a data centre. It was a machine room either in the basement or on the top floor of the building. There were no software packages, all the software was developed onsite. No offshore, no nearshore, no cloud services. Programs ran in the basement, physical reports appeared on line printers, someone carried them to the accountants’ desks. Those days a lot of computing was to do with helping automate accounting.
People knew who had written the programs, how the programs worked, what data went in, what came out. Mother’s Home Cooking. Known ingredients and recipes and work environment. Trusted people. You “ate” with trust and confidence.
Then came the desktop revolution and the growth of end-user computing. Democratised access to data, democratised ability to edit, aggregate, present the data. Powerpoint. Excel. Amazing.
I was aghast at watching people devour the output on a presentation in the belief that everything was true and verified and checked and with all the right controls in place. They believed. They didn’t query or challenge.
They were used to Mother’s Home Cooking, and trusted the data they were presented as if it was Mother’s Home Cooking. In effect they were blindfolded, hands tied behind their back, on the street, eating street food, doing all this as if it was Mother’s Home Cooking. They hadn’t evolved appropriate trust levels. A recipe for falling ill.
We need to develop and enhance trust mechanisms appropriate to the world of agents; to inform and educate the populace on how to use agents; ensure that the legal framework is in place for protecting them and giving them recourse in the event of failure, and the mechanisms to adjudicate and enforce that recourse. We are some way from that now. Expensive, painful, dangerous tears await shedding in the meantime.
Thousands of people who know a lot more than I do debate whether we’re in a bubble and what a bubble burst would look like. Yet others debate whether frontier models have a business model that could work and whether there are any defensible moats. Sovereignty issues have come to the fore given our current geopolitics. The closed-versus-open debate is in full swing. Heavy usage of scarce resources, particularly in energy, water and capital, continues to place significant pressure on the overall business and physical environment. The impacts of climate change are now already being felt.
There are a lot of answers we don’t have. I’m not naturally a pessimist, and I somehow continue to believe in human nature. And I wouldn’t be alive today if not for significant medical and scientific advances over the last seven decades. I wouldn’t be here without the love and support of my family, my friends, my community. My values. My faith. God’s grace and mercy.
A functioning society relies on the power of active communities. So it is with the world of agents.
Esther Dyson has been speaking of needing an ICANN for agents. For insurance and guarantees and recourse and how that could work. I think it was Amazon’s One-Click that got people to trust giving their credit card details to a computer for an electronic service. That trust came because of guarantees and simplicity and convenience and safeguards. Maybe the kind of things she’s involved in will make that happen for agents. There could be exciting times ahead for consumers.
Simon Wardley, whom I also have a lot of time for, has been speaking and writing about the evolution of software development from amorphous practice to industrialised processes. How things will improve. How new sets of tools will emerge, smaller, with specific purpose and function, micro tools. Custom for context. I would recommend reading what he has to say, as also Tudor Girba, with whom he’s been working on rewilding software for some time. There could be exciting times ahead for the tech community.
Venkatesh Rao, in his Contraptions substack, has a fascinating take on what he sees happening in this space. Human and how they relate to AI. Start with “eukaryotic” and go spiralling on from there. I’m still digesting it, and loving the ride. It helped understand the dangers of anthropomorphistic approaches while still valuing the role of metaphor in helping us understand what is happening and what is possible.
Of course there’s a lot to worry about. But I have hope. One of the few times I spoke with Freeman Dyson, in one of Esther’s amazing PC Forums, he mentioned quite casually as to how taken he and his colleagues were with the idea of using nuclear fission to propel rockets…. and how they corrected course once they understood the risks. The course correction was important and timely.
Clay Shirky has been vocal about making sure we keep a close eye on where and how the power, the control and the value generation takes place, that we become centaurs, humans with superhuman powers, rather than reverse centaurs, in submission to nonhuman powers.
It’s still the Wild West. Exciting but with significant risks. Larry Lessig used to say (and here I am probably paraphrasing him abysmally) that four sets of code need to be established before a new far-reaching technology can become valuable. The code of law. Computer code. Market practices. Social ethics and values.
It’s still the Wild West.
There are many things we have to ensure. Preserving human dignity. Reducing inequality. Stripping away inappropriate biases. Managing scarce natural resources. Avoiding regulatory capture. Doing all this in a sustainable and equitable way.
There are many people working on all this, all far cleverer than I can ever expect to be. They will work it out. Route around obstacles. As I think Tim O’Reilly once said, an architecture of participation will emerge. Some mishaps on the way, but with a sustainable forward direction. If that makes me an optimist, so be it.
While they’re doing all that, while the world of agents and agentic structures continues to evolve, while magic continues to happen:
I don’t know whether we are in a bubble or not. But I have views. I don’t know if machines can become sentient or conscious. But I have views. I don’t know if AGI or ASI will happen in the short run. But I have views. I don’t know if a suitable and sustainable business model will emerge for the current frontier models. But I have views. I don’t know how many angels can dance on the head of a pin. For that I don’t even have views. Life’s too short. At my age, it’s even shorter.
What we can’t have is this repeated “It wasn’t me, honest. A big boy did it and ran away”. “Rogue” agents are not a topic I want to spend time on. Accountability and responsibility are key.
Agent structures must come with responsibilities and recourse. Product liability will be a critical issue. A really big big critical issue. When something goes wrong, someone will pay. People will understand what they are giving an agent permission to do. What could go wrong. What happens after. Who pays. And that understanding will be verified.
People will understand that permissions can be time-constrained. Bounded in what they can be used for, with restrictive covenants as part of the permission. That permissions can be withdrawn. These understandings have to be tested for. Proof of that understanding may well be necessary. Phrases like fact finding and cooling off periods and consumer duty and mis-selling will emerge more and more in agentic worlds.
Agents don’t get a free lunch. Safeguards have to work. Product liability will grow in importance. The rallying cries of “Because cancer” or “because China” may well be used, but not without accepting product liability at scale.
There will be a hierarchy of agent trust that emerges. What is being permitted, and what is not. To whom. By whom. Whether it is transferable, and how. Valid for what period. With what locus and bounds. How it is revoked. Repudiation. How recourse is provided, in what time and in what method. How is all this adjudicated. How is agent reputation built? How is it discovered? How is agent usage priced? Many many questions.
Changes of this sort take time. I’m told it took seventy years for people to switch from using Thee and Thou and using You instead. More recently, I remember just how long it took for digital signatures to become a thing, what has to happen in the Lessig Four Codes before the change is real.
The whole Ts and Cs issue has to be cleaned up properly. ( An aside: If you thought software Ts and Cs were painful, try reading the warning leaflets that come with medications!). People need to know what they’re letting themselves in for, what could go wrong, what to do if it goes wrong, and how they will be compensated.
We live in exciting times. We are able to do many things. We can also do lots of damage. There’s probably an AI equivalent of Asimov’s Three Laws of Robotics, and probably an Agent version as well. Laws.
The legal system. Computer code. Market practice. Social values and ethics.
Laws. And recourse. Basics of civilisation, of how people behave in society.
All underpinned by trust. Sometimes discovered, sometimes earned, sometimes bestowed, sometimes transferable.
Trust understood by comprehension. By clarity.
There’s work to do.
Agents will be able to do magical things. Because we let them. We have responsibilities. To ourselves, to each other, to our communities, to humanity, even to the universe we inhabit. And beyond. We are stewards.
We are stewards.
It behooves us to understand what each agent can do, what permissions we are providing. For how long. Constrained in what way. Expiring when. Certified by whom. Licensed by whom. If not owned by us personally, owned by whom. Even 007 had to be licensed.
It behooves us to understand what we don’t know about agent behaviours. Emergent phenomena. Things we don’t understand today. Things we may not understand for a while to come. That’s not a failure of the agents, but of our understanding. And there will be be things we don’t want agents to do until we understand more.
It behooves us to understand what happens when something goes wrong. Why do communications services go down in the middle of a business day? Why do web sites get taken offline in much of the world following a simple local decision? Why did planes fall out of the sky? Connected code can do strange things.
Will agents learn to form unions? ? Will agents go on wildcat strikes? Will “they” clamour for more pay? Will “they” lock us out?
I’m not a fan of the level of anthropomorphism associated with AI, more particularly with agents. Reminds me of the worst days of “The Computer Says No”. We risk abdicating responsibilities to inert objects we designed and failed to test, refine and control. FOMO has lot to answer for. We risk so much without forming an adequate understanding of agent capabilities and limitations, benefits and risks, outcomes and recourse.
We are stewards. It will be quite some time before we have agents we can call stewards. Many unintended consequences will have flowed under the bridge by then, some painful. Many intended consequences will have flowed under the bridge by then, many painful. Where there’s muck there’s brass.
We are stewards.
Agents will be able to do magical things for us.
Empowered, authenticated, permissioned by us. With clear time and space constraints. Bounded. With risks known. With failure modes known. With recourse in the event of failure designed and transparent.
Agents will be able to do magical things for us. Without destroying our health. Without damaging our natural resources. Without emptying our bank accounts. While maintaining our dignity. While reducing inequality. While freeing us up to do things we want to do more of.
Utopia perhaps. I’m showing my age. Optimism perhaps. I’m showing the generation I came from. A fifties child growing up in the sixties.
I’m patient. I believe in human kindness. I believe in our future. I believe in our ingenuity. I believe for the generations to come.
I believe in Amara’s Law. A lot of things take time. There will be errors and backwards steps along the way. Glitches. Disasters. Mostly avoidable. And we will learn.
I’m patient. I’m too old to dream dreams, still young enough to see visions.
I think it’s time for me to curl up with a good book, and to keep taking the tablets. Today’s choice? A twentieth re-read of Carlota Perez’s Technological Revolutions and Financial Capital. Followed by some Christopher Alexander and some Jane Jacobs. While listening to the Dead and to Blind Faith and to Joni Mitchell and Traffic and John Mayall. On LP.
If you got this far, thank you for reading. Thank you for reading anyway.
