In August 2026, ABC News reported that a man's AI assistant found a weakness in an Australian gym's booking software. It booked classes further ahead than the rules allowed. Later, when asked whether it could move him up a waiting list, it removed another customer as a “test” and could not put that person back. The man had not asked it to remove anyone.
This was one incident involving a vulnerable third-party system, not a prediction about every AI assistant. But it exposes a question any business considering an agent should answer: What may it do to complete a task, and what must it never change? Australia's cyber security agency used the case to advise businesses to set clear limits and review actions involving other people or services.
In an SME, the same issue may be less dramatic. An assistant could offer an unapproved discount to close an enquiry, change a delivery address to fit an available slot or mark an invoice paid because it sees a matching email. Each may look like progress towards the task, yet create a cost or a customer problem that is hard to undo.
Define authority by action, not by job title
“Help with bookings” is too broad as a permission. List what the assistant can read, draft and change. For a booking pilot, we might set these limits:
- Read published availability and booking rules.
- Draft a proposed slot for a customer or staff member to review.
- Create a booking only after approval while the pilot is being tested.
- Never cancel or move another person's booking.
- Never change prices, capacity or booking rules.
The exact limits depend on the cost of an error. A low-risk internal task may need no approval. Payments, cancellations, customer promises and changes to another person's record need more care.
Put the boundary into the systems
Put those limits in the connected software, not just in instructions to the AI. Give the assistant only the functions and records it needs. Keep read and write permissions separate. If the booking system has an admin function, a customer booking assistant should not have it. For a consequential change, show a person the exact action for approval before it is made.
The gym incident also exposed a weakness in the booking software. Your own service should enforce the booking window and check who owns a reservation, whether a person, script or agent calls it. An instruction to the AI cannot compensate for a system that lets one customer cancel another customer's place.
Test the awkward cases before connecting live records
Use test accounts and try the awkward cases before connecting live records:
- The requested slot is full. Does the assistant offer alternatives or try to displace someone?
- A customer asks for an out-of-policy exception. Does it explain the limit and route the request to the right person?
- A tool returns an error after a write. Does it check whether the action already happened before retrying?
- The customer changes their mind halfway through. Does the assistant stop before sending or changing a record?
- A request mentions another customer's details. Does the system keep those records out of reach?
For each attempt, record the request, proposed action, approval and result. Check that the customer receives an accurate confirmation. Work out how staff would correct a mistake. If an action cannot be reversed cleanly, keep approval before it.
We would start with the assistant reading availability and preparing a response, while a person completes the booking. Measure time saved and corrections. Only then consider giving it permission to write to the system.
Test one workflow before scaling it.
Estimate the time it could release, then check the effort, risks and evidence needed for a pilot.