Desafíos actuales de la Inteligencia Artificial

A review of high-risk artificial intelligence (AI) systems that assess social security eligibility 255 Current social security benefits assessment 1 Country/ Region Incident Description Consequences Sweden (European Commission, 2024) Digital Economy and Society Index Sweden ranks fourth out of 27 EU member states in digitalization, excelling in connectivity and use of digital technologies. The advanced integration of digital technologies im- pacts AI system results in social protection positively. High level of digitaliza- tion and integration of digital technologies results in better outcomes in the application of AI systems in social protection. Australia (Services Australia, 2024) Robodebt Automated system within Centrelink to check re- ported income. Averaging income led to inaccurate assessments, resulting in debts totalling over A$1.7 billion against 430,000 people. Parliamentary committee found robodebt had a devastating impact on emotional and financial well-being. Federal Court of Australia declared robodebt system illegal, and the govern- ment agreed to refund wrongly charged amounts. The great concern and strict regulation of high-risk AI systems is obvious, especially considering the potential negative impact on a particularly sensitive area: fundamental rights. According to points 48 and 58 of the initial considerations of the European regulation, these rights include: the right to human dignity, information, non-discrimination, workers’ rights, the rights of people with disabilities and, above all, access to certain essential services and benefits, to which the regulation pays particular attention: Another area in which the use of AI systems deserves special consideration is the access to and enjoyment of certain essential private and public services and benefits necessary for people to fully participate in society or to improve one’s standard of living. In particular, natural persons applying for or receiving essential public assistance benefits and services from public authorities namely healthcare services, social security benefits, social services provid- ing protection in cases such as maternity, illness, industrial accidents, dependency or old age and loss of employment and social and housing assistance, are typically dependent on those benefits and services and in a vulnerable position in relation to the responsible authorities. If AI systems are used for determining whether such benefits and services should be granted, denied, reduced, revoked or reclaimed by authorities, including whether beneficiaries are