An investigation by the Guardian Australia has made a startling discovery that AI can fool the voice recognition technology used to verify the identity of Centrelink and the Australian Taxation Office (ATO) customers.

Centrelink and the ATO both use voiceprint technology as a security measure to verify customer identities over the phone, without the need for customers to answer various security questions before they can gain access to highly sensitive personal information from their accounts.

Following reports that an AI-generated voice trained to sound like a specific person could be used to access phone-banking services overseas, Guardian Australia has confirmed that the voiceprint system can also be fooled by an AI-generated voice. Using just four minutes of audio, a journalist from Guardian Australia was able to generate a clone of their own voice, and was then able to use this, along with their Centrelink customer reference number, to gain access to their Centrelink account.

This investigation reinforces the need for effective regulation of AI, and the need to question the impact that AI may have on human rights. Without adequate regulation, AI systems have often failed to ensure the privacy of individuals and protect consumers from having their sensitive information leaked.

Former human rights commissioner Ed Santow said that government agencies using biometrics, such as voiceprint, as a form of verification needed to ensure they had the best systems in place, and highlighted the need for legislation to ensure safeguards are in place to prevent misuse and cybercrime.

Read more here.