Overview – AI models in the insurance sector

The use of AI models in the insurance sector raises significant concerns, according to Nikhil Rathi, Chief Executive of the Financial Conduct Authority (FCA). Speaking at the StepChange Connected conference in Leeds, Mr Rathi spoke about the potential of AI to hyper-personalise insurance offerings. Although he recognised that this development may be beneficial for businesses and some customers, it could also result in certain customers becoming uninsurable or increasing discrimination due to algorithmic bias.

These developments highlight the need for businesses to consider regulatory expectations regarding the use of AI. While the AI legal and regulatory landscape still remains relatively unchartered in the UK, Rathi’s comments could signal a cautious approach from regulators. This mirrors the EU’s AI Act, which illustrates a regulation-heavy stance towards the integration of AI. In the EU market, where the use of AI systems is classified as being high risk, such as automated risk assessment and pricing of life and health insurance and assessment of credit rating, more stringent regulatory obligations apply.

Insurance providers and other businesses operating in regulated sectors must navigate a balance between innovation and consumer protection. While the FCA recognises that AI can be a useful tool in creating tailored products, ultimately businesses must be transparent and fair. Again, this aligns with the principles highlighted in the EU AI Act of accountability and preventing algorithmic discrimination. There is, amongst other things, a risk of AI systems using historical data to perpetuate biases and accentuate unfairness; automated decisions using AI without an adequate degree of transparency could not only alienate customers, but also spark regulatory and reputational risks.

The FCA has advocated for the need for collaboration within the financial services industry, to ensure that consumers are protected while nurturing innovation. There is a need to consider not only compliance with current and potential legal and regulatory responsibilities, but also the ethical implications of using AI in a way that will impact consumers. As the use of AI technology evolves rapidly, innovators are likely to be ahead of regulators. This puts greater onus on businesses to use AI systems in a responsible way. While use of the technology may be permitted, financial services providers should consider whether the implications are unfair for customers.

Many insurers operate in the EU market, of course, and will expect to adopt similar approaches in the UK. The EU’s AI Act explicitly mentions certain uses of AI systems within the insurance sector which are considered high risk. Legislation in the UK does not currently provide this level of detail, however Mr Rathi’s comments suggest the FCA shares concerns that certain uses of AI could cause unfairness and harm to consumers. It further indicates that, particularly in high risk circumstances, there is an expectation that the use of AI technology will be subject to human review. It is crucial that insurance providers and other financial services providers are legally compliant and also ethically aligned with consumer protection principles and that these are not jeopardised by the use to automated systems. We can draw parallels from the EU’s robust approach, while recognising that the UK market is not itself, at least for the time being, subject to equivalent legislative oversight.

The Prudential Regulatory Authority (PRA) and the Bank of England published a joint response earlier this year to the government’s white paper on AI. Their response aligns with Mr Rathi’s comments and welcomes the government’s proposed principles-led approach to regulating AI. They noted that this complements the existing framework for regulators. The PRA also stated that it will undertake further analysis as to AI’s potential impact on financial stability. In addition to the principles of transparency, fairness and accountability, the FCA and PRA have an obligation to promote the international competitiveness of the UK economy. Regulators, and the businesses that operate under them, will need to carefully consider how these competing objectives are balanced.

Tim Ryan – Partner, DAC Beachcroft

Hannah Clements – Solicitor, DAC Beachcroft