Government tool to set standards for risk and safety in healthcare AI solutions

A new initiative is set to redefine how artificial intelligence (AI) solutions are classified and adopted in Australian healthcare.

The Digital Health Cooperative Research Centre (DHCRC), in collaboration with the Department of Health and specialist teams from the University of Technology Sydney (UTS Rapido and the UTS Human Technology Institute), is spearheading the effort to adapt a global framework to the unique demands of Australia’s healthcare landscape.

This project focuses on tailoring the Organisation for Economic Cooperation and Development (OECD) AI Classification Framework for Australian healthcare, aligning it with national policies, including proposed mandatory guardrails for AI technologies.

Annette Schmiede, CEO of the DHCRC, underscored the significance of this initiative, saying, “To complement the work of government and industry to define AI ethics principles, develop AI risk assessments, and provide guardrails for the safe and responsible use of AI, there needs to be a standardised approach to classifying the varied types of AI systems in use. The availability and adoption of AI is without doubt moving at a rapid pace across all sectors, including healthcare.”

She further added, “The challenge is building clear and consistent guidance and tools, ensuring these are effective for the diverse range of audiences and AI solutions across healthcare including developers, healthcare providers and consumers.”

Sam Peascod, Assistant Secretary, Digital and Service Design, Department of Health and Aged Care, said “As Government looks to build community trust and promote AI adoption, we need to provide guidance on how to use AI safely and responsibly. Having a tool that can assist in classifying and performing a risk assessment of AI technologies will support the adoption of AI solutions by healthcare organisations and healthcare providers, ultimately leading to better health outcomes for consumers.”

The OECD framework, endorsed by 46 countries, including Australia, provides an internationally recognised baseline for classifying AI systems. It assesses dimensions such as users and stakeholders, economic context, data collection, AI models, and system tasks. This adaptation aims to provide healthcare organisations with clear and consistent guidance for assessing the risks and opportunities of AI solutions, ensuring alignment with regulatory standards, including software as a medical device regulations.

Professor Adam Berry, Deputy Director of the UTS Human Technology Institute, emphasised the importance of consistency, stating, “For AI to realise its tremendous promise for all, it depends upon responsible practice. A critical first step to realising that practice is to be consistent in the documentation of how individual AI systems are used, function, and deliver impact across diverse stakeholders.”

The dynamic tool, described as a first-of-its-kind initiative, will offer a self-serve advisory and benchmarking platform tailored to Australian healthcare. It aims to address specific risks such as bias, explainability, and robustness in AI applications.

Hervé Harvard, Executive Director of UTS Rapido, highlighted the collaboration’s impact, noting, “Partnering with universities like UTS and its innovation hub UTS Rapido enables industry to leverage cutting-edge technology to enhance safety and responsibly navigate the transformative impact of AI in healthcare.”

This tool’s development will be informed by real-world testing with AI developers, deployers, and end-users in healthcare, ensuring practicality and alignment with emerging Australian regulatory frameworks.

Raj Calisa, Principal Delivery Manager at UTS Rapido, remarked “The exciting aspect of this project is to build and test an interactive tool that provides great user experience and where the ‘smarts’ behind the scenes can be dynamically refined with subsequent iterations.”

The project team is aiming to have a “basic web tool” ready for testing by mid-2025.

In reimagining healthcare across the entire patient journey, Health Industry Hub^TM is the only one-stop-hub uniting the diversity of the Pharma, MedTech, Diagnostics & Biotech sectors to inspire meaningful change.

The Health Industry Hub^TM content is copyright protected. Access is available under individual user licenses. Please click here to subscribe and visit T&Cs here.