The importance of avoiding bias and having quality datasets before implementing AI

Any analysis using Artificial Intelligence requires DATA — in sufficient quantity, of quality, and free from inherent bias. This also applies to animal production.
For this reason, we summarise here an interesting EU document which, although not specifically focused on livestock farming, is perfectly applicable to it.
The extensive technical and legal report entitled “Auditing the quality of datasets used in algorithmic decision-making systems” was produced by multiple authors from academic and research centres at the request of the Panel for the Future of Science and Technology (STOA) of the European Parliament. This document comprehensively examines the problem of bias in artificial intelligence (AI) systems, arguing that bias is inherent to human society and is replicated in AI through datasets.
The report analyses the technical and legal implications of bias, proposing that data protection legislation, particularly the concept of “fairness” under the GDPR, together with forthcoming regulations such as the AI Act, are crucial tools for its mitigation. Finally, the study presents and evaluates various public policy options, including database certification and the strengthening of transparency rights, with the aim of ensuring fairer and less discriminatory AI within the regulatory framework of the European Union.

This European Parliament study analyses the problem of bias in algorithmic decision-making systems, focusing on the quality of the datasets used. Below we present a summary of its main findings and recommendations.

Key findings

• Bias in AI reflects human and social bias: The report underlines that artificial intelligence (AI) often reproduces existing biases present in society. The objective, therefore, is not to oppose the technology, but to use it as an opportunity to understand and mitigate discrimination.
• The bias lifecycle: Bias can be introduced at any stage of AI system development: from problem definition and data collection through to model development, validation, and deployment.
• Importance of data quality: The quality of training datasets is fundamental to avoiding biased outcomes. Issues such as under-representation of certain groups, data errors, and incorrect labelling can give rise to discriminatory AI systems.
• Limitations of the current legal framework: The existing EU anti-discrimination directives contain gaps that hinder the prevention of algorithmic bias. For example, protection against discrimination is often limited to certain grounds (such as gender or race) and to specific contexts (such as employment).
• The role of data protection: The General Data Protection Regulation (GDPR) can be an effective tool for combating bias, particularly through the principle of “fairness” in data processing. Data Protection Impact Assessments (DPIAs) are a key mechanism for evaluating and mitigating bias risks from the design stage onwards.

Recommendations and policy options

The study proposes several policy options to address the problem of bias in AI:

1. Refrain from creating new bias-specific legislation: Rather than enacting new laws, the report suggests focusing on resolving inconsistencies between existing regulations and new proposals (such as the AI Act, the Data Governance Act, and the Data Act).
2. Preventive approach: Strengthen bias mitigation from the earliest stages of AI tool development, ensuring that training, validation, and test datasets are relevant, representative, error-free, and complete.
3. Database certification: Promote the creation of certificates for datasets, particularly those used in high-risk AI systems. This would help ensure the standardisation and quality of information, embedding bias awareness from the very beginning of the algorithmic systems lifecycle.
4. Transparency rights for subjects of AI systems: Grant individuals affected by algorithmic decisions the right to obtain meaningful information about the logic involved and the training data used. This would enable them to better understand decisions and detect potential bias.
5. Facilitating AI Act implementation: Support companies, particularly SMEs, in complying with the new regulations. This could include measures such as establishing regulatory sandboxes and providing access to high-quality databases through public institutions.

Source:
-. Auditing the quality of datasets used in algorithmic decision-making system.