Call for robust AI regulation ‘similar to any other medical technology’

Digital & Innovation: The introduction of a discussion paper by the Department of Industry, Science and Resources on safe and responsible artificial intelligence (AI) regulation has been welcomed by key stakeholders. The submission from the Australian Medical Association (AMA) focuses on the regulation of AI specifically within the health and medical sectors to ensure the adoption of safe and beneficial technologies while safeguarding patient and practitioner privacy and maintaining optimal patient safety.

The AMA brings attention to Australia’s potential lag in AI policy compared to other similarly developed nations, with particular concern regarding the healthcare sector. One of the primary challenges highlighted is the unregulated nature of AI deployment in the healthcare landscape. The discussion paper rightly acknowledges the potential pitfalls of AI models, such as biases and erroneous outputs. In the context of healthcare, these biases can have severe consequences on patient outcomes, with lives at risk.

For example, the AMA points to a recent study involving pulse oximeters, which became vital tools during the COVID-19 pandemic for monitoring patients’ oxygen levels. However, these devices have been found to overestimate oxygen levels in individuals with darker skin, leading to a higher likelihood of undetected hypoxemia in black patients. The root cause of this disparity lies in the original clinical trials’ failure to include diverse groups of patients.

AI’s impact on patient diagnosis and treatment necessitates robust government regulation, similar to any other medical technology. The AMA firmly believes that the government’s role in regulating AI in healthcare is pivotal to building trust in the system and ensuring the welfare of patients, consumers, and healthcare professionals alike.

The AMA calls for regulations that provide essential protections for all stakeholders involved. These protections include support for improved patient outcomes, stipulating that the final clinical decision rests with the healthcare professional, obtaining informed consent from patients for any treatment or diagnostic procedure that involves AI, and ensuring the protection of patient and practitioner data.

Furthermore, the application of AI in healthcare should adhere to strict ethical oversight to prevent exacerbating health inequalities within the population. Transparent and accountable AI development and implementation are essential to instil trust among patients, healthcare providers, and the broader community.

In its submission, the AMA voices its support for a coordinated approach to AI development and deployment in healthcare. Presently, AI innovations in radiology and other medical fields are often driven by individual vendors with piecemeal technologies. While competition is a driving force for innovation, the AMA contends that such fragmented advancements are not optimal for the broader scheme of AI development and adoption.

To this end, the association advocates for a coordinated effort led by a national body or a national governance structure. This structure should involve key stakeholders, including medical practitioners, patients, AI developers, health informaticians, lawyers, healthcare administrators, medical defence organisations, and other relevant parties.

AI’s application in healthcare entails high risks, ranging from system errors that may lead to patient injuries to potential breaches of patient privacy and systemic biases embedded in algorithms. In light of these risks, the AMA emphasises the necessity of government-level regulation built on a robust evidence base, with expert advice and adequate support from the government.

Furthermore, the AMA proposes specific governance arrangements tailored to individual healthcare services and programs to complement government-level regulation.

Drawing inspiration from international models, the AMA lauds the European Union (EU) and Canada’s approaches to AI regulation. In particular, the EU’s General Data Protection Regulation (GDPR) serves as a guiding example for privacy protections and ownership of health data. Patients should be recognised as the owners of their health data, and clear limits on data access must be established.

The EU’s proposed AI regulatory framework, which categorises AI applications based on risk levels, is also of interest. High-risk applications, such as AI in robotics surgery, require specific safeguards, and the final decision must be made by a human for irreversible and perpetual impacts on health and wellbeing.

A common set of agreed principles embedded in legislation to provide a compliance baseline for all stakeholders involved in AI in healthcare is critical.

In reimagining healthcare, Health Industry Hub^TM is the ONLY one-stop-hub uniting the diversity of Pharma, MedTech, Diagnostics & Biotech sectors to inspire meaningful change. The exclusive leadership and influencer podcasts and vodcasts offer unparalleled insights and add immense value to our breaking news coverage.

The Health Industry Hub^TM content is copyright protected. Access is available under individual user licenses. Please click here to subscribe and visit T&Cs here.