“In most of society we worry about AI alignment, but we don’t have a set of rules that we’re trying to align to. Whereas here [the financial sector] we do. And so that’s what makes it, I think, really interesting as a case study, is we know what we’re trying to achieve. Can we achieve it? And if we can’t align it to the regulations that are set out pretty clearly, then how are we going to align it with much more difficult to specify, human morals.”
As part of our Campaign for Social Science events programme, last week we held a webinar featuring Jonathan Hall discussing the impact of AI in the financial sector.
Chaired by Professor Jonathan Beaverstock FAcSS, this was the fourth webinar this year in our series, in partnership with the UK Evaluation Society and the Social Research Association, on the theme of how we can evaluate, understand, and manage different aspects and uses of AI as it continues to rapidly change our economy and our society.
Jonathan, who is an external member of the Bank of England’s Financial Policy Committee, began by outlining the mandate of the committee, highlighting that this is primarily focused on ensuring financial stability and supporting growth, but also a secondary mandate to support the government’s priorities on growth, a large part of which is supporting the safe adoption of new technologies.
He went on to explain how he believes that financial markets are an obvious testing ground for AI alignment in terms of the balance between maximising profit and the public good. He outlined how there is an established set of regulations which apply to humans and the need to be able to apply those successfully to AI trading agents.
He said, “If we can reduce AI risk in finance then perhaps that does give us comfort in other areas. Obviously, if we can’t then that would be a concern.”
Jonathan discussed the two different styles of trading that trading agents follow and which feed into current financial markets and the efficient market hypothesis: value trading and flow-analysis trading. The first being that a trader tries to model what is a fair price for an asset (e.g. a bond, stock etc) and buys and sells that asset in relation to what the model considers the fair price to be. Whereas flow-analysis trading focuses on trying to anticipate the future value of an asset and buy and sell that asset based on that estimated future value.
He said, “There are two potential answers to the question of why the market rallies. A value trader will say because it was cheap, and a flow-analysis trader would say because there were more buyers than sellers.”
Jonathan then went on to discuss how these types of trading styles could be implemented by an AI algorithm and what that would mean for potential risks to financial markets. He explored how AI value traders would likely be more successful than human value traders and that a greater return on investment would mean greater efficiency within the market which would be good for stability. However, he also highlighted that as systems become more efficient, they become more brittle and therefore prone to shocks. If markets were to become dominated by AI trading agents then there is the potential for catastrophic errors being made where the cause is unknown and also that a greater correlation in AI trade agents, due to them drawing from the same information and using the same base foundation models, could result in flash crashes.
He said, “I have a theory in that part of the benefits of the current system and the reason the current market works is because human traders are bounded in terms of their rationality. And so therefore the way that they look at a subset of the information, the way they model the information is different between different humans. And, therefore, we have a natural diversity as a function of the fact that we’re not all knowing and we’re not unconstrained. So if you imagine that AI has greater capabilities, has less constraints, that that would potentially reduce this diversity in the market and increase correlation, increase the risk that suddenly everyone moves in the same direction and causes some kind of crash.”
From this Jonathan highlighted that a mix of agent trading models would be needed to offset these risks. But explained that there are also risks of AI trading agents based on flow-analysis trading. The first of these is that these models are likely to naturally exhibit collusive behaviour and this is compounded within a broader category of risk around emergent communications between AI agents which humans cannot understand. Secondly, AI trading agents based on flow-analysis trading could look for opportunities to amplify instabilities within financial markets to drive asset prices down and take advantage of that to maximise its own profit. Jonathan explained how this behaviour is prohibited for human traders and so AI trading agents would need to be trained to respect these rules.
He said, “There is a broader worry about alignment that the trading agent doesn’t train itself to respect the rules, it just trains itself to not get caught breaking the rules. […] This is an area that is at the cutting edge of AI alignment and some of the market changes might help us to understand how to do this better and implementation in the market. But also we need to watch what is going on in AI safety more generally to see if we can deal with this within the market environment.”
Jonathan ended his presentation by exploring the implications for regulation and responsibility for the use of AI trading agents.
He said, “In terms of responsibility, the current position we’re taking as regulators is that the human manager that implements the trading algorithm is responsible for its behaviour. […] They’re choosing to implement it [the trading algorithm] because they think that there’s something about the model over and above a human or linear model which is going to generate them profit. But there’s no reward without risk and they need to understand the risk. They are responsible if that risk loses money but also breaks regulatory rules.”
A Q&A followed Jonathan’s presentation, and the full webinar is available to watch below.