Ethical concerns: data privacy and patient autonomy
One of the key ethical concerns in the use of AI within healthcare is ensuring that patient autonomy is respected. AI technologies are often used to make predictions or assist in decision-making, especially in diagnostics and treatment recommendations. However, reliance on algorithms can sometimes obscure the transparency needed for informed consent. Ensuring that patients remain fully aware of how AI contributes to their healthcare decisions is critical to maintaining trust in the system.
Data privacy is another pressing issue. AI systems require vast amounts of patient data to function effectively. This can include sensitive medical records, imaging data, and even genetic information. Ensuring that this data is handled in compliance with the General Data Protection Regulation (GDPR) is vital, particularly in Europe where stringent rules govern personal data. GDPR provides a robust framework for data protection, but the specific challenges posed by AI, such as the handling of large datasets and the potential for re-identification of anonymized data, need to be carefully considered in healthcare applications.
AI and bias in healthcare
An important ethical challenge posed by AI is the risk of bias. AI systems are only as good as the data they are trained on, and if that data contains biases, these can be perpetuated in medical decisions. For example, if certain populations are underrepresented in the datasets used to train diagnostic AI, their outcomes might not be as accurate. This could exacerbate existing health inequalities, making it crucial to ensure that AI is developed and implemented with a strong focus on inclusivity and fairness.
In the healthcare systems of smaller countries like Croatia, there may be additional challenges due to limited local data availability. To mitigate these issues, collaboration with broader European initiatives is essential. Such efforts would ensure that AI systems are trained on more diverse and representative datasets, minimizing bias and improving outcomes for all patient groups.
Legal framework in the European Union
At the European level, significant progress has been made toward regulating AI to ensure it operates ethically and safely in healthcare. The European Union’s AI Act categorizes AI systems into different risk levels (minimal risk, high risk, unacceptable risk, and specific transparency risk), with healthcare AI often falling into the high-risk category. This categorization requires that such systems meet strict standards for transparency, accuracy, and accountability. Developers must also ensure that AI tools are reliable and safe before being integrated into clinical settings.
The Ethical Guidelines for Trustworthy AI, developed by the EU, provide a framework that ensures AI systems are aligned with European values. These guidelines emphasize human oversight, transparency, and respect for human rights, particularly when AI systems are used in sensitive sectors like healthcare. These principles are designed to foster trust and ensure that AI does not override human expertise but instead serves as a supportive tool for healthcare professionals.
Legal and ethical considerations in Croatia
While AI regulation in Croatia is still evolving, efforts are being made to align with broader European standards. Legal frameworks such as GDPR ensure that patient data is protected, but further steps are needed to address the specific ethical and legal issues posed by AI in healthcare.
For instance, who is accountable when an AI system makes an incorrect diagnosis or treatment recommendation that leads to harm? Should healthcare professionals always verify AI-generated decisions, or can some level of automation be trusted without human oversight? These are central questions in the ongoing discussions about how to integrate AI effectively into healthcare.
Another key issue is the training of healthcare professionals to work alongside AI systems. How well do doctors and medical staff understand the limitations of AI, and how should they interpret its outputs? Without adequate training, there is a risk of misuse or misunderstanding of AI, potentially leading to negative patient outcomes.
Future outlook: National AI Strategy in Croatia
While the National AI Plan is still in the draft stage, it has been broadly agreed that the plan will aim to position technological development at the forefront of Croatia’s progress. The draft plan, developed by a working group comprising experts from academia, business, civil society, and the public sector, is seen as a collaborative effort to ensure all relevant stakeholders are involved.
The National Recovery and Resilience Plan (2021-2026) has earmarked funds for the development of the AI plan, and it appears that international obligations partly drive this process. The question remains whether such initiatives would have been prioritized without external pressure, raising concerns about the genuine importance placed on AI development.
Prior to the drafting of the plan, several important milestones have already been achieved. The first thematic session in the Croatian Parliament on AI took place in 2023, along with the country’s first scientific symposium dedicated to AI, highlighting a growing national focus on this technology, a second symposium is set to take place on November 5, 2024. Furthermore, Croatia’s Digital Strategy 2032, adopted in late 2021, aims to establish the country as a leader in digital governance and economic competitiveness by 2032.
In crafting the plan, the Ministry will take into account key EU strategic documents, including the EU AI Strategy, the Coordinated Plan on AI, and the White Paper on AI. Furthermore, the plan will align with the AI Act, which came into effect on August 1, 2024. This legislation sets out rules for developing, marketing, and deploying AI systems in the EU and addresses liability for any harm caused by AI.
The National AI Plan is expected to be finalized by the end of 2024, marking a significant step forward in Croatia’s commitment to advancing AI development.
