A new tool developed through social science research challenges harmful biases in AI systems.
Summary
As artificial intelligence, data science and machine learning systems are being used more and more, the expertise of social scientists is fundamental to ensuring new technologies can fit with the world around us and have the greatest positive impact on our daily lives.
Research by Professor Sandra Wachter and colleagues identified an incompatibility between European notions of discrimination and existing work on algorithmic and automated fairness. To counteract this, they developed a tool to detect discrimination in AI and machine learning systems which aligns with the European Court of Justice’s gold standard for discrimination assessment.
Making fairer decisions: developing the ethical use of AI
“Using our bias test, practitioners around the world can now test for biases and make better and fairer decisions.”
Professor Sandra Wachter
The challenge
Although artificial intelligence, data science, machine learning and robotics are all closely associated with STEM subjects and skills, the expertise of social scientists is fundamental to ensuring new technologies can fit with the world around us and have the greatest positive impact on our daily lives.
Artificial intelligence bias creates new challenges for establishing discrimination as it can appear more abstract, subtle and intangible, making it more difficult to detect and prove. With the increasing use of AI and machine learning systems making decisions about our lives, there is a pressing need to ensure that such algorithms and systems are fair, ethical and transparent. In part, this requires legal and technical communities to collaborate to create legally sound solutions for fairness and non-discrimination in automated systems.
The research
Professor Sandra Wachter and colleagues at the University of Oxford analysed non-discrimination law and jurisprudence of the European Court of Justice (ECJ) and national courts and found there was an incompatibility between legal notions of discrimination and technical measures of algorithmic fairness.
This analysis showed that EU non-discrimination law did not, by design, provide a framework suitable for testing for discrimination in AI systems as many of the concepts fundamental to bring a legal claim require normative or political choices to be made by the judiciary on a case-by-case basis.
To counter this, Sandra and her colleagues proposed a new test for ensuring fairness in modelling and data driven decision-making which aligned with the ECJ’s gold standard for assessment of prima facie discrimination. The statistical test, ‘conditional demographic disparity’ (CDD), allows for considerations of fairness to be built into automated systems while still enabling the contextual approach to judicial interpretation.
“Many researchers around the world have been aware of that problem [AI discrimination] and they have been developing a range of bias tests. But we were the first ones to show that the majority of them do not live up to the standards of European non-discrimination law. This prompted us to develop our own bias test.”
Professor Sandra Wachter
The impact
This research has been implemented by Amazon, which is considered to be the largest online retailer in the world, in their AI accountability toolkit on fairness detection. Released by Amazon’s machine learning service, Sagemaker, the tool uses ‘conditional demographic disparity’ as its baseline for testing fairness in automated systems and is available for use by Amazon Web Services customers.
“This metric is useful for exploring the concepts of direct and indirect discrimination and of objective justification in EU and UK non-discrimination law and jurisprudence”
Amazon