Photo: Pixaby/Geralt (CC0 - free for commercial use)
Artificial Intelligence (AI) technology is increasingly being used by businesses, governments, and other institutions to augment many fields of human endeavor. Leveraging the best of human values and ability alongside artificial intelligence brings enormous benefits to society (e.g., Seeing AI; Translator; SwiftKey; disease control; biodiversity; sustainable farming; water security; healthcare; etc.). However, civil society groups, governments, and others are rightly asking questions regarding the risks to human rights (e.g., unfair bias, consequences to privacy and freedom of association, etc.). People will not rely on technology they do not trust. Society needs to come together to consider these questions, explore solutions, and deploy AI that puts people first, protects human rights, and deserves the public’s trust.
Modern AI often involves the use of machine learning to pursue a defined purpose. Training data relevant to that purpose is processed through mathematical methods to detect patterns in the data and develop a model to make predictions or recommendations about new data. For example, an employer might use existing data about workers to train and develop an AI model to make recommendations on hiring decisions on future job candidates.
Satya Nadella, Microsoft CEO, penned an article on the partnership between humans and AI, noting that the most productive debate isn’t whether AI is good or evil, but about “the values instilled in the people and institutions creating this technology.” He reflected on six principles and goals that the industry and society should discuss and debate, which were further explored in The Future Computed: Artificial Intelligence and Its Role in Society: (1) fairness; (2) reliability and safety; (3) privacy and security; (4) inclusiveness; (5) transparency; and (6) accountability. All are relevant to the protection and advancement of human rights, and fairness and inclusiveness specifically pertain to the risk of unfair bias.
After all, AI does not decide the purpose for which it is deployed. Human institutions decide what an AI model should achieve, optimize or maximize. Does the defined purpose account for the relevant context and considerations? For example, if an AI model that makes hiring recommendations is purposed by humans to focus solely on sales maximization, would its recommendations lean towards candidates that pursue sales at all costs (including the use of questionable or unethical sales tactics) and against candidates with more holistic skills and approaches that build customer satisfaction and loyalty? Is the purpose illegal or unfair? AI should not be used to discriminate based on race, religion, gender or other protected classes (e.g., in employment, housing, and lending decisions). Even when illegal or unfair bias is not the intended purpose, humans need to assess the risk of misuse or unintended consequences. For example, an AI model that predicts whether someone is a member of a protected class could be used to find and offer applicable services to members of the class, or misused to exclude those members from important benefits (e.g., housing, employment, etc.).
In addition, the quality of the training data in machine learning is pivotal to the accuracy and fairness of the resulting model. Is the data appropriate, relevant, accurate, accurately labeled, diverse, and representative? Does it reflect existing human bias? Is it missing relevant data and training a model that makes biased predictions? For example, if an AI model for hiring recommendations is trained solely on the data of existing and past employees, and those employees are not diverse (e.g., predominantly young white males), the resulting model would likely be unfairly biased against candidates who are older, racial minorities, and female.
One might assume that the way to avoid unfair bias is to exclude sensitive data such as race, gender, or other status. But that is not true. In a research project using AI to predict the risk of death from pneumonia, the AI learned that having asthma lowers this risk. However, this favorable outcome only occurs because asthma patients receive faster and more intensive care than patients without asthma. If this model were used to predict who needed hospitalization, it would result in poor outcomes for asthmatics, because the model would predict high survival rates and falsely deduce that asthmatics do not need hospitalization. However, removing the asthma data would only make the problem worse.
There can be significant correlation among variables in any data set. If one variable is removed, remaining variables correlated with it can still affect the learning of the AI and make the problem harder to detect. It is important to train AI with all variables, including the ones that may cause problems, so those problems can be identified and addressed after training. Furthermore, in testing an AI model, using diverse test data, including sensitive parameters such as race, gender, age, etc., would help assess whether the model makes fair and unbiased predictions.
Even after an AI model is launched, human overseers need to assess the quality and risk of unfair bias in ongoing new training data. Given the pivotal importance of the training data to the risk of bias, explanation of the nature of the training data can help address concerns over transparency and fairness of an AI model.
But as society aims to improve the fairness of AI with diverse training data, society needs to be thoughtful about squaring AI’s need for vast amounts of data with fundamental privacy principles that require minimizing the personal data used and retained and limiting the ways in which that personal data is used in the future. This tension between AI and data protection laws requires society to grapple with several difficult questions. For example, what is a valid legal basis for processing data used in training AI models? How can advancement in the use of de-identification techniques such as differential privacy (a technological solution that adds noise to mask data that would otherwise be considered identifiable) mitigate the risks to peoples’ privacy while enabling broader AI development?
Human decision making without the help of AI is not necessarily fairer or more transparent. The human mind may be the ultimate “black box.” When asked to explain their decisions, people may not tell the truth due to intentional or unconscious bias. On the other hand, training an AI model using existing data infused with human bias will result in an AI model that continues the human bias. In other words, the bias in AI is often the bias of humans. But understanding the nature of the training data, testing an AI model with diverse test data, reviewing the model’s predictions for bias, and other tools could help improve fairness in ways humans-only decision systems could not. Instead of choosing between humans-only systems and AI systems, leveraging the best of human values and ability as well as artificial intelligence promise greater progress in fairness, transparency, and accountability, and will be critical to building strong public trust.
*** This article is part of a series on technology and human rights co-sponsored with Business & Human Rights Resource Centre and University of Washington Rule of Law Initiative.