In his 2015 book Rise of the Robots, Martin Ford presents a thought experiment: what would happen if an alien race arrived with no desires or wants, tastes or preferences, sense of home or place, mortality or time-boundedness, status-seeking or social comparison, values of leisure or rest, emotional or relational needs, or meaning-seeking beyond output. He meant robots or AI – and within 10 years, they have landed.
It was that book that led me to join Dynamic Planner. If the change was coming, I wanted to be part of it, whether in the form of robotic process automation, machine learning or today’s natural language models.
In our regulated world, is the absence of those emotional attributes entirely a strength of AI, or could it be a weakness? In many novels and films, a machine’s relentless objectivity is more frightening than mythical monsters or human baddies.
In such stories, the bad outcome seems more certain because the machines have none of the potential incentives or distractions – unlike organic villains and monsters, whose hubris, desires, anger or even glimmers of compassion save the day. Even Darth Vader was the ‘human in the suit’ in the end. As driverless cabs and trucks become a reality, is our reality closer to Stephen Spielberg’s unrelenting Duel than to the value-driven David in the film AI?
I had this in mind as I sat and listened to Sheldon Mills’s speech in the code node. While the speech was positive about the AI opportunity, Mills was also clear that you still have to have, and evidence, systems and controls, monitoring and governance, and to meet all the prevailing guidance and rules.
‘Agentic’ is an exciting step in technology, but what does it mean? My understanding of the word ‘agent’ in this context is someone or something that ‘takes action in the world’. In our industry, we are used to the legal definition of ‘acting on behalf of a principle’. As across the western world, our industry relies on consent, contracts, outsourcing and intermediaries with fiduciary duties, authorities and obligations.
Who is agentic AI acting for – and, most importantly, how do you know? What it can do is evolving and is down to the expertise of others; this is about what it may do.
You will have empowered someone to carry out duties and actions on your behalf and been given the same agency by somebody else – appointed rep being an obvious example. If one of the aliens above applied for the role, would you be less stringent with them? You would probably want a track record from their home planet.
How would you feel if the alien turned up in place of someone you had given agency to? At least you would see them and know about it. But somewhere up the chain, someone is making decisions to invest in, build and sell AI, then others are deciding to use AI, as well as which AI and how. Possibly those decisions are themselves aided by AI.
Governance checks and balances and management of conflicts of interest could be conducted using AI – which is all fine until it goes wrong. As one compliance officer said to me, it isn’t ‘humans in the loop’, it’s ‘humans on the hook’. Is that you? I guess you’d rather it was someone else.
If you give agency to anyone or anything, you probably want to monitor them. Today, Duel would have been picked up on speed cameras, dash cams and satellite – if not live-streamed by Dennis Weaver’s iPhone.
And as efficient as transport becomes, the action and its outcomes are aligned in time. I look out the window, I get there, I get out. The challenge in our industry is that it is long term. Many of the biggest scandals in its history were monitored and deemed ok at the time. An AI agent I set in motion today could cause harm you don’t identify until long after I am gone.
The Mills Review references the monitoring conducted by METR in February showing that AI can perform increasingly long and complex tasks without prompting. From February to the most recent update in May, that measure increased by 40%. Progress, but if you no longer have to check in so often to make it go, you might want to check in to see where it’s going.
In consumer duty language, you need friction. On the road, that comes in the form of junctions, signs and traffic lights. The equivalent for our industry is fixed deterministic check points between bursts of AI activity, so you can monitor it in a consistent way and identify whether accountability has changed.
Even with those guardrails, there is a challenge inside the human. AI today is nicer and more charming than even Jude Law in the AI movie, so you are inclined to like, trust and believe it. Rather than appearing like a killer truck, it appears like fun buses instantly taking you to exciting new places.
But when you’re choosing which one to board, you want to know the bus company, and that the bus has an MOT and is monitored – because one day, one of them will go off a cliff.
Chris Jones is financial services director at Dynamic Planner
