When I think about agents, I think of what permissions I have to give someone else in order for them to represent me, and what permissions I have to receive to be someone’s agent. Permissions.
Some years ago, I was asked by the CEO of a firm to compile a report on all the interactions between his firm and another, very large, global firm. They had a decent CRM system. It should have been simple. Turned out that in order to collate that data and to produce the report he wanted, I needed over 180 different permissions. Permissions that related to individual business units in different countries, permissions that could only be given by the data protection officials for each of those units in each of those countries, and a few more besides.
Park that for a moment, the architecture of permissions.
If you gave me your private and personal mobile phone number, and someone else asked me for it, could I give it to them? Not unless I had your permission. There’s a trust issue, a confidentiality issue as well. Permissions. Trust. Confidentiality.
I’m retired now. Next year the leading digit in my age will be a 7. When I started work, bosses had assistants, sometimes one per executive, sometimes in pools; usually senior bosses had senior secretaries of their own. Most sat in front of their boss’s office. Some had their own rooms. Regardless of the physical architecture they were in, they were empowered very differently. Mail was placed in pigeonholes for the assistants to pick up. opened mail envelopes addressed to their boss, stamped them to record when they’d been received, and then put them in folders for their bosses to read. Some read those mails and annotated them with recommended actions before seeing their boss, providing a cover sheet with a summary. Some summarised the mails and only highlighted the ones they considered important. Some of the mails went into the bin. Sometimes unread.
When email became common practice, some of these habits were transported into the electronic age. Some assistants had no access whatsoever to their boss’s email; some received a copy of the mails and could read them, but not reply; some could read, and only reply with an “on behalf of”; a few had full access and could reply on their boss’s behalf using their boss’s mailbox to send it.
I saw many many different practices across this spectrum, in terms of what agency the assistant had over their boss’s email. A whole variety. There was only one common thread. Assistants with full read, reply, comment, delete rights were unconditionally trusted by their bosses. That trust was earned rather than bestowed, although an incoming new assistant may occasionally carry the trust earned with a different executive.
Trust was a key factor in the decision to provide the assistant with “mail agency”.
At home, there are a number of people who are trusted by me to have a set of keys to different parts of the house. People who provide us with some sort of regular service. Cleaners. Builders and decorators. House and garden and pond maintenance. Some others. Some get temporary access, some get more long-term access. Some get “all-areas” access, some are restricted, provided access constrained on time or area or something.
Those keys are provided to people we trust.
Children get to ages when they have access to different things. I was given the keys to our flat when I was 13. On the strict understanding that I would (a) always tell my mother where I was going (b) when I would be back and (c) who I was going to be with. At that age, the answers were simple. The Sillimans, a few floors down. The Kapoors, one further floor down. The car parking area behind the house, to play cricket or football. The rules were simple. And I would lose the keys if I broke them. (I didn’t actually need the keys, we lived in a civilised apartment block. People rarely locked doors, they were left wide open or on the latch. It was the principle that counted).
It was a way of teaching me to be trusted about my whereabouts. Liberty not licence. Constrained and verifiable.
Sometimes there are relationships where communications between a pair of people are considered sacrosanct, confidential to the pair, inviolable in that confidence. Husband and wife. Doctor and patient. Priest and confessor. Lawyer and client. Over time, society has seen fit to protect and preserve such confidences, and we have evolved ways to do that. These are trusted relationships. Trust is the core, again.
When parents leaves the (appropriately aged) children in charge of the house while they go away somewhere, there are usually constraints. No parties but you can have friends over… but not more than 8. No touching the alcohol. No smoking. No loud music after 10. And keep everything neat and clean for when we get back – leading to the mad rush clean-up just before parents are due back. Trust. Specific permissions, time-bound.
As parents, there are responsibilities that fall on your shoulders when you do leave the children for a short while. They have to be old enough to be left without adult supervision. If they’re not yet at responsible ages, alcohol, medications, guns, cars, things that are age-constrained, have to be removed or safely locked away. And parents are held accountable for making that environment appropriately safe. Society requires us to do that. Other parents need to know that there are relevant rules in place in the household when the parents are away, otherwise their children will not be allowed to visit.
It’s not unusual to ask a friend, neighbour or relative to keep an eye on the house during such absences, and to let the children know who to contact locally in an emergency. Trust is two way and comes with responsibility and accountability.
An aside on secrets and keeping secrets. I remember a tale where Kenny Dalglish, then manager of Liverpool, managed to surprise everyone by bringing Ian Rush back from Juventus. A journalist asked him how he kept it a surprise. Kenny’s answer: Simple. I just didn’t tell anyone.
We make people our agents in many ways. Give them permission, explicit or implicit. Valet parking. Real estate sales and purchase. Dog walkers. Baby sitters. They are all empowered to do something on your behalf, with permissions sometimes narrow, sometimes broad, sometimes short-term, sometimes longer term, sometimes permanent. Powers of attorney for example, temporary or lasting.
Sometimes trust is bestowed without being earned, given the role of the person or group of people involved. Sometimes you trust someone because of their position in a firm or community or even society. Sometimes you trust someone because of the brand they represent, the firm they are ambassadors for.
Many years ago, I used to travel to Bangalore regularly on business. Fly from London to New Delhi. Land there at midnight or shortly after. Wait for five hours for the local flight to Bangalore. Nothing open. A ghost site. Hot. Often humid. And I’d be tired. It’s a long flight. After one or two such experiences, I had to find a way around this. So I would call up a hotel near the airport that I trusted, speak to the manager, and ask for a room or three for 6 hours. And a pick up from the airport, and the return transfer.
I would do this for my travelling colleagues as well. There were often two or three of us making this journey. As I did this more often, I got to know the “front of house” manager at the hotel. In those days, you had to do other things as an international traveller. A few other things. Administrative. Frustrating. Time-consuming. For example, you had to reconfirm return flights or run the risk of not getting a seat. You also had to change foreign currency into rupees. The way my mind worked, I really wanted a way to have all this done while I was sleeping, after having had the customary coffee and samosa.
So I asked the front-of-house newly-minted friend what advice he had. And in typical five-star-hotel service style he said “Leave it with me”. And, while I slept, he would personally sort out the rebooking and convert the sterling into rupees. Everything would be done while I slept. And I would make sure he was rewarded for the incredible service.
This used to happen often. Once, when I was travelling with a couple of colleagues, I explained all this to them, and they watched, aghast, as I proceeded to give a man they didn’t know, a man they’d never met before, three sets of things. All our passports. A hefty sum in cash sterling. Our airline tickets.
But they trusted me. So they went along with it. Uncomfortably. Very uncomfortably. But they went along with it.
We had our coffee. We had our samosas. We slept in the rooms booked part-day for us. We came down, relaxed and refreshed, ready to go to the airport for our flight to Bangalore.
Hmmm. Where was the front of house man? Nowhere to be seen. He should have been back. And he wasn’t. He had our cash, our air tickets, our passports. And he wasn’t there.
My colleagues hadn’t quite started to grumble or start recriminations, but I could see they were much discomfited. This was out of their comfort zone. Way out. And they were inwardly cursing themselves for listening to me.
By now I’d begun to have a few doubts myself. Not much, a little flurry. But I had faith. Faith that the status and security he enjoyed in his job, relatively prestigious in his society, those things would ensure he wasn’t tempted to go rogue.
He returned. Fifteen minutes later than he had said. Everything had been done. And all was well.
Why am I telling you all this? Because these are the things I think about when humankind is about to embark on a spree of trust, not knowing what permissions we are giving, what we are putting at risk, what recourse we have.
Trust.
We live in exciting times. AI is here to stay. It’s only going to get better. It’s not great today but directionally fascinating. AI agents are going to be a core part of the journey we go on.
But we don’t have the right trust environment as yet. Thousands of people who know a lot more than I do debate whether we’re in a bubble and what a bubble burst would look like. Yet others debate whether frontier models have a business model that could work and whether there are any defensible moats. Sovereignty issues have come to the fore given our current geopolitics. The closed-versus-open debate is in full swing. Heavy usage of scarce resources, particularly in energy, water and capital, continues to place significant pressure on the overall business and physical environment. The impacts of climate change are now already being felt.
There are a lot of answers we don’t have. I’m not naturally a pessimist, and I somehow continue to believe in human nature. And I wouldn’t be alive today if not for significant medical and scientific advances over the last seven decades.
Esther Dyson has been speaking of needing an ICANN for agents. For insurance and guarantees and recourse and how that could work. I think it was Amazon’s One-Click that got people to trust giving their credit card details to a computer for an electronic service. That trust came because of guarantees and simplicity and convenience and safeguards. Maybe the kind of things she’s involved in will make that happen for agents. There could be exciting times ahead for consumers.
Simon Wardley, whom I also have a lot of time for, has been speaking and writing about the evolution of software development from amorphous practice to industrialised processes. I would recommend reading what he has to say, as also Tudor Girba, with whom he’s been working on rewilding software for some time. There could be exciting times ahead for the tech community.
Venkatesh Rao, in his Contraptions substack, has a fascinating take on what he sees happening in this space. Start with “eukaryotic” and go spiralling from there. I’m still digesting it, and loving the ride. It helped understand the dangers of anthropomorphistic approaches while still valuing the role of metaphor in helping us understand what is happening and what is possible.
Of course there’s a lot to worry about. But I have hope. One of the few times I spoke with Freeman Dyson, in one of Esther’s amazing PC Forums, he mentioned quite casually as to how taken he and his colleagues were with the idea of using nuclear fission to propel rockets…. and how they corrected course once they understood the risks.
Clay Shirky has been vocal about making sure we keep a close eye on where and how the power, the control and the value generation takes place, that we become centaurs, humans with superhuman powers, rather than reverse centaurs, in submission to nonhuman powers.
It’s still the Wild West. Exciting but with significant risks. Larry Lessig used to say (and here I am probably paraphrasing him abysmally) that four sets of code need to be established before a new far-reaching technology can become valuable. The code of law. Computer code. Market practices. Social ethics and values.
It’s still the Wild West.
There are many things we have to ensure. Preserving human dignity. Reducing inequality. Stripping away inappropriate biases. Managing scarce natural resources. Avoiding regulatory capture. Doing all this in a sustainable and equitable way.
There are many people working on all this, all far cleverer than I can ever expect to be. They will work it out. Route around obstacles. As I think Tim O’Reilly once said, an architecture of participation will emerge. Some mishaps on the way, but with a sustainable forward direction. If that makes me an optimist, so be it.
While they’re doing all that, while the world of agents and agentic structures continues to evolve, while magic continues to happen:
I don’t know whether we are in a bubble or not. But I have views. I don’t know if machines can become sentient or conscious. But I have views. I don’t know if AGI or ASI will happen in the short run. But I have views. I don’t know if a suitable and sustainable business model will emerge for the current frontier models. But I have views. I don’t know how many angels can dance on the head of a pin. For that I don’t even have views. Life’s too short. At my age, it’s even shorter.
What we can’t have “It wasn’t me, honest. A big boy did it and ran away”. Rogue agents are not a topic I want to spend time on. Accountability and responsibility are key.
Agent structures must come with responsibilities and recourse. Product liability will be a critical issue. When something goes wrong, someone will pay. People will understand what they are giving an agent permission to do. What could go wrong. What happens after. Who pays.
People will understand that permissions can be time-constrained. Bounded in what they can be used for, with restrictive covenants as part of the permission. That permissions can be withdrawn. These understandings have to be tested for. Proof of that understanding may well be necessary. Phrases like fact finding and cooling off periods and consumer duty and mis-selling will emerge more and more in agentic worlds.
Agents don’t get a free lunch. Safeguards have to work. Product liability will grow in importance. The rallying cries of “Because cancer” or “because China” may well be used, but not without accepting product liability at scale.
There will be a hierarchy of agent trust that emerges. What is being permitted, and what is not. To whom. By whom. Whether it is transferable, and how. Valid for what period. With what locus and bounds. How it is revoked. Repudiation. How recourse is provided, in what time and in what method. How is all this adjudicated. How is agent reputation built? How is it discovered? How is agent usage priced? Many many questions.
Changes of this sort take time. I’m told it took seventy years for people to switch from using Thee and Thou and using You instead. More recently, I remember just how long it took for digital signatures to become a thing, what has to happen in the Lessig Four Codes before the change is real.
The whole Ts and Cs issue has to be cleaned up properly. ( An aside: If you thought software Ts and Cs were painful, try reading the warning leaflets that come with medications!). People need to know what they’re letting themselves in for, what could go wrong, what to do if it goes wrong, and how they will be compensated.
We live in exciting times. We are able to do many things. We can also do lots of damage. There’s probably an AI equivalent of Asimov’s Three Laws of Robotics, and probably an Agent version as well. Laws.
The legal system. Computer code. Market practice. Social values and ethics.
Laws. And recourse. Basics of civilisation, of how people behave in society.
All underpinned by trust. Sometimes discovered, sometimes earned, sometimes bestowed, sometimes transferable.
Trust understood by comprehension. By clarity.
There’s work to do.