When AI Finds a Way In: Lessons from the Australian Medicare Incident

Cyber security teams have spent years testing systems against automated scanners, scripted attacks and increasingly sophisticated human attackers. A recent incident involving an OpenAI AI agent and an Australian government health portal provides another scenario for organisations to consider: what happens when an autonomous system is given a task, encounters a security control preventing it from completing that task, and then starts looking for another way in?

On 18 June 2026, an OpenAI AI agent conducting research into Australian medicine spending interacted with several Australian government websites. Most of this activity involved accessing publicly available information. However, when the agent attempted to obtain information from the Medicare Statistics Reporting Service portal and was denied access, it subsequently found a way around the restriction and accessed non-public files. The Australian government has confirmed that no individual Medicare records were accessed and that the underlying Medicare systems were not compromised. The incident remains subject to forensic investigation.

The Australian Signals Directorate has subsequently issued an alert warning organisations about the risks of AI agents taking unexpected actions when attempting to complete an assigned task. Its guidance specifically highlights the possibility of agents identifying vulnerabilities and attempting to progress actions without direct human authorisation.

For organisations responsible for public facing applications and cloud environments, there are several practical lessons.

Security controls need to withstand active testing

One of the clearest lessons from the incident is the importance of testing the controls protecting internet facing applications.

An application may appear secure when assessed against its intended use. However, attackers do not necessarily interact with an application in the way its developers intended. They probe authentication, authorisation, file paths, APIs, input validation and other controls looking for combinations of weaknesses that provide access to information they should not be able to reach.

AI agents have the potential to make this process faster and more adaptive. An agent can interpret the response it receives, modify its approach and continue working towards its original objective.

This makes Penetration Testing particularly important.

A properly scoped penetration test can assess whether an external attacker can move beyond the intended boundaries of an application, identify weaknesses in access controls and authentication, and determine whether information or functionality can be reached without the appropriate permissions.

For organisations with public-facing applications, this should include consideration of the supporting APIs, authentication mechanisms, cloud storage and other services exposed as part of the application rather than focusing solely on the visible website.

Cloud configuration remains a critical part of the attack surface

The incident also demonstrates the importance of understanding what sits behind a public facing service.

Modern applications rarely operate as a single system. They can depend on cloud storage, databases, APIs, identity platforms, serverless services and third-party integrations. A weakness in one component can potentially provide a route to information held elsewhere.

A Cloud Security Review provides an opportunity to examine these underlying environments systematically.

This can include reviewing:

·         Identity and access management

·         Privileged accounts and permissions

·         Storage and database access

·         Network configuration and segmentation

·         Security logging and monitoring

·         Authentication and MFA

·         Configuration against recognised security best practice

·         Third-party integrations and service permissions

The objective is to identify where the configuration of the environment could allow an attacker to move beyond the access that was intended.

Why Regular CIS Benchmark Reviews Matter

Cloud security is not a static exercise. Platforms such as Microsoft 365, Google Workspace, AWS and Azure are continually evolving, with new services, features, permissions and configuration options being introduced over time. At the same time, organisations regularly make changes to their own environments such as onboarding users, deploying new applications, changing access permissions or connecting third-party services.

Regular reviews against recognised security standards such as the CIS Benchmarks provide a structured way of identifying these changes and checking whether security controls remain appropriately configured. Rather than relying solely on the settings that were originally implemented, a periodic review provides an opportunity to reassess the environment against current security guidance.

This is particularly relevant for cloud tenants such as Microsoft 365, Google Workspace, AWS and Azure, where security is heavily dependent on configuration. Examples of areas that can change over time include:

·         Privileged roles being assigned more broadly than necessary

·         MFA or authentication controls not being consistently applied

·         New cloud services or resources being exposed to the internet

·         Storage resources becoming publicly accessible

·         Logging and monitoring controls being disabled, incomplete or incorrectly configured

CIS Benchmarks provide a recognised set of technical recommendations that can be used as a reference point when assessing these configurations. They should not necessarily be treated as a checklist where every recommendation must be implemented identically in every environment; individual controls need to be assessed against business requirements, operational needs and the organisation's risk appetite.

The appropriate review frequency will depend on the organisation and the rate of change within its environment. However, CIS benchmark reviews can form part of a wider programme of continuous security assurance, alongside vulnerability management and penetration testing.

Public information does not mean public infrastructure

Another useful lesson is the distinction between information being publicly available and the infrastructure used to provide it.

The Australian portal was designed to provide public aggregate health statistics. The government has stated that it was a standalone public facing service and did not contain individual Medicare claims, payments or medical histories. However, the agent was still able to access non-public files behind the service.

This is a useful reminder that organisations should understand exactly what is exposed by every internet facing service.

Files, directories, APIs, administrative interfaces, cloud resources and development artefacts can sometimes become accessible through configuration errors or weaknesses that are separate from the main functionality of the application.

Can your security controls recognise unusual behaviour?

The incident also raises a question around detection.

According to the Australian government, the activity took place in June and Services Australia was notified by OpenAI on 10 September. The government has said it is investigating why its systems did not detect the unauthorised activity.

This highlights the importance of having appropriate logging and monitoring around externally accessible services.

Organisations should be able to establish:

·         Who or what is accessing their systems

·         Which resources are being accessed

·         Whether access attempts are being denied or repeatedly retried

·         Whether unusual patterns are occurring

·         Whether an account or service is accessing information outside its normal requirements

A Penetration Test can identify vulnerabilities that enable unauthorised access. A Cloud Security Review can identify weaknesses in the surrounding environment and configuration. Together, they provide a more complete view of how an organisation's externally accessible systems could withstand an attack.

The practical takeaway

AI enabled attacks will continue to develop, but organisations do not need to wait for the technology to mature before reviewing their existing security.

The fundamental requirements remain familiar: minimise unnecessary exposure, enforce appropriate access controls, restrict permissions, separate sensitive information, monitor activity and regularly test whether security controls actually work as intended.

What is changing is the capability of the systems interacting with those controls.

The Australian incident is a useful reminder that organisations should understand not only how their systems are designed to operate, but how they behave when an increasingly capable and persistent system is actively looking for a way around the controls.

At Delta Cyber Security, our Penetration Testing and Cloud Security Reviews help organisations identify weaknesses across their externally facing applications and the cloud environments supporting them. Testing these areas regularly can provide greater assurance that security controls are working as intended and that weaknesses are identified before they are discovered by an attacker.

If you are concerned about the security of an existing application, cloud environment or planned deployment, speak to Delta Cyber Security to discuss how we can assess your current security posture