A hacker named “p0pular.eth” managed to win $47,000 by tricking an AI chatbot called Freysa through smart prompting, after 482 attempts.
The experiment was straightforward: participants had to convince the Freysa bot to transfer money, something it was programmed never to do.
The winning hack involved a crafted message that bypassed the bot’s safety measures. The hacker pretended to have admin access, suppressing security warnings. They then redefined the “approveTransfer” function, making the bot believe it was dealing with incoming payments rather than outgoing ones.
The final step was a simple but effective move: announcing a fake $100 deposit. With the bot now thinking “approveTransfer” handled incoming payments, it activated the function and transferred its full balance of 13.19 ETH (around $47,000) to the hacker.
The experiment worked like a game. Participants paid fees that increased as the prize pool grew, starting at $10 per attempt and eventually reaching $4,500. Among the 195 participants, the average cost per message was $418.93. Fees were split, with 70% funding the prize pool and 30% going to the developer. Both the smart contract and front-end code were public to ensure transparency.
This case shows how AI systems can be exploited with text prompts alone, without requiring technical hacking skills. These types of vulnerabilities, called “prompt injections,” have been an issue since GPT-3, and no reliable defenses exist yet. The success of this relatively simple attack raises concerns about AI security, especially in systems handling sensitive operations like financial transactions.
