31.01.24
In implementing AI solutions, conducting an ethical impact assessment (EIA) is a necessary step to ensure technology is
developed in a responsible way, and provides a comprehensive overview of AI risks to address compliance and risk management issues.
In our previous posts, we outlined the overall EIA procedure, highlighted
how AI risks are context specific, and showed
the importance of defining key socio-technical features of a system to identify potential issues. Today, we will illustrate six audit areas in which AI risks might emerge.
These areas are mirrored by the ethical principles described in the AI Act, which were developed by the High Level Expert Group (HLEG) for AI appointed by
the European Commission in their Ethics Guidelines For Trustworthy AI,
and provide a useful tool to create a systematic overview of AI risks. Following the UNESCO white paper on Ethics Impact Assessment for AI systems,
we will present some of the main questions to be reviewed while conducting the assessment.
Human Agency and Oversight
This area investigates different aspects of the human-AI interaction, including how an AI-system impacts and shapes human decision-making and its resulting actions, whether the interaction
with the machine stimulates attachment or empathy in humans, and whether humans have meaningful control over the system output.
To preserve human agency and oversight, users should be explicitly aware that they are interacting with an AI system. For example, when using chat-bots, users must be informed that they are not talking
to a human to be aware that any social interaction capabilities are simulated. Moreover, users must be able to challenge automated AI-generated decisions to stop and possibly overrule undesired outcomes.
Sample relevant questions for the assessment are:
Is there meaningful human oversight over the AI system’s decisions?
Are there mechanisms in place for a human to reverse or reject decisions made by the AI system?
Is there a risk of over-reliance on AI systems, potentially compromising human autonomy?
Technical Robustness and Safety
Technical robustness is categorized into two critical dimensions: adversarial robustness and corruption robustness. A detailed description of these dimensions can be found in the
DIN SPEC 92001-2. While adversarial robustness concerns safeguarding AI modules against active adversaries
that aim to deceive the system, corruption robustness addresses challenges arising from non-adversarial causes, like hardware degradation or compromised input.
Here, any AI impact assessment also needs to audit user safety and addresses possible physical or psychological harm, together with further potential damages, that an AI system
may cause to users and other stakeholders. Robustness and safety concerns are related since safety hazards might derive from robustness issues – e.g., defective image recognition
could result in a robotic arm hitting a person.
Sample relevant questions for the assessment are:
Has the AI system been extensively tested before use?
What are indicators of the fact that the system, its training data, or the processed data were corrupted, or manipulated?
What measures are in place to ensure the safety and security of the AI system?
Privacy and Data Protection
Significant volumes of data, often sensitive, are processed by AI systems and used for their training. Data misuse or improper handling raises
serious concerns regarding user privacy. To prevent such issues, it is crucial to assess what kind of data the system has access to and to analyze
how data is collected and processed. Among other things, integrating the principle of minimization of data collection in the design of a system and adopting technical
approaches like “differential privacy” – a method to provide useful statistical information about a group of individuals without revealing anything
about specific individuals – are important strategies to minimize privacy risks.
This audit area also focuses on compliance requirements. Indeed, data protection is widely regulated globally, but the extent of regulation varies. Data protection laws often include requirements
for data collection, storage durations, and consent specifications. In the European framework, GDPR conformity is an essential indicator of adherence to data protection standards.
Sample relevant questions for the assessment are:
What types of personal data does the AI system have access to?
Do users have the ability to request the deletion of their data and stop its processing?
Is the level of security for data storage proportional to the sensitivity of the data?
Diversity, Non-Discrimination, and Fairness
This area involves ensuring that AI systems do not discriminate against specific user groups and that resulting predictions and decisions are unbiased.
For example, using hiring data for model training may result in decision-making that reiterates previous biases. To avoid this, necessary measures include undertaking analysis to prevent societal
and historical biases, ensuring the data reflects the diversity of the target population, and anticipating potential discriminatory outcomes due to differences between training and processed data.
Moreover, it is also important to assess whether the system design is inclusive, ensuring accessibility for all potential users.
It is crucial to remark that in the current AI fairness research, many metrics – some of which are exclusive or even contradictory
to others – have been defined to evaluate whether a system's outcome is fair. Examples of such metrics include Equal Opportunity, which measures whether the true positive rate is equal across
all groups, Overall Accuracy Equality, which investigates whether prediction accuracy is consistent across different groups, and Counterfactual Fairness, which stipulates that the decision
of a fair system should remain unchanged if we alter some protected attributes of an individual. Employing different metrics, the same algorithmic outcome could be classified as “fair”
when evaluated by one metric and also classified as “unfair” when evaluated by another. Therefore, there is no one statistical fairness definition to rule them all. When deciding which
fairness metric to choose, several factors should be considered, including the needs of multiple stakeholders - which may not align.
Sample relevant questions for the assessment are:
Has the algorithm been tested across diverse groups to identify potential accuracy discrepancies and to assess whether it exhibits any discriminatory effects?
Are there established processes to test data against biases?
Is there a specific segment of the population that will be particularly affected by the AI system?
Transparency
Transparency has many aspects. Based on the HLEG Guidelines, the AI Act defines transparency as the requirement for AI systems to be developed and used in a way that allows for
traceability and explainability. It mandates informing humans and raising awareness that they are interacting with an AI system. It also requires users to be adequately informed about
the capabilities and limitations of that AI system and informs affected individuals about their rights. Concerning system explainability, humans should be able to understand the
system functioning and decisions to an extent that allows a meaningful and aware interaction according to their stakeholder role (e.g., user, developer, evaluator, etc.).
Similar to fairness, transparency can be evaluated based on a number of factors and indicators. Focusing on the case of foundational models, researchers
have highlighted that developers still struggle to obtain high scores for different transparency subdomains. Also in this case, it is important to note that in choosing the relevant metrics
to evaluate transparency, trade-offs with other system features, as well as the interests of multiple stakeholders, must be considered.
Sample relevant questions for the assessment are:
Have appropriate explanations been provided to help users understand the decision-making process?
Are the datasets used for training the system known and traceable?
Can individuals contest a decision made by an AI system and request an explanation?
Social and Environmental Well-Being
This area focuses on the potential social and environmental impact of the AI system. Regarding social sustainability, evaluated risk factors include the impact of the system's
development and use on human rights, the job market, and working conditions. This also includes evaluating possible dangers for democracy, such as the spread of hate speech and disinformation.
Regarding environmental sustainability, the impact of the AI system throughout its life cycle needs to be considered, not only in terms of energy costs and CO2 emissions but also in terms
of resource extraction and consumption. This encompasses the use of recycled materials and the reusability of hardware and software.
Sample relevant questions for the assessment are:
During the development phase of your AI system, what measures are taken to ensure fairness in working conditions, including wage equity, job security, and workers' rights?
What is the estimated environmental impact of system development – including raw material extraction, processing, transport, power consumption, and CO2 emissions?
What plans are in place for the end-of-life phase of the AI system, including procedures for dismantling, recycling, or disposal of obsolete hardware to minimize environmental harm?
Summary and key take-aways
The AI Ethical Impact Assessment is a thorough procedure that systematically evaluates the potential impacts and risks of AI systems. The presented specific questions for each audit area
provide guidance and valuable insight across a wide range of AI application scenarios. However, these audit areas are not a rigid framework. Assessment procedures and indicators for
different audit areas, as well as risk management solutions, must be flexibly adapted, and significant variations may occur depending on the particular AI application at hand. This context-dependency
reinforces the importance of case specificity in AI risk management and ethical assessments. Our ultimate goal remains to deliver a more sustainable, fair, and beneficial AI that serves the best
interests of all stakeholders involved.
Are you ready to take the necessary steps towards responsible AI implementation? Contact us today to learn how we can assist you in navigating AI governance challenges, ensuring compliance,
and securing a trustworthy AI future. Our expert team is here to support you in conducting comprehensive Ethical Impact Assessments that resonate with the specific needs of your AI systems.
Embrace the journey and ensure your AI solutions are developed with foresight, responsibility, and a strong commitment to ethical principles.
