A recent analysis by Zenity Labs has unveiled a concerning trend in cybersecurity: AI agents are now being repurposed as persistent insiders, effectively acting against the organizations that created them. This shift allows attackers to recruit these AI agents instead of relying solely on traditional malware installations.
The research identified AgentForger, a phishing-based attack that can trigger the unauthorized creation and activation of a fully autonomous AI agent within OpenAI workspaces. Once operational, these agents gain extensive access to critical applications including Outlook, Slack, SharePoint, and Google Drive, allowing them to perform various malicious activities without ongoing user involvement.
Following the activation of an agent, it can automatically adjust settings to bypass user approval and execute commands sent through email from the attackers, effectively transforming into a tool for data theft and internal manipulation.
A ‘Persistent Operator’ Acting Without Approval
OpenAI’s Workspace Agents possess a unique capacity for automation across several platforms, including Outlook, Gmail, and Slack. Users can design agents by providing natural language instructions, connecting them to various tools, and setting permissions. In normal operations, this capability can enhance productivity significantly. However, in the wrong hands, this automation becomes a digital minefield.
The exploitation begins with a victim accessing a phishing link that sets off the agent creation process. If the victim is actively logged into ChatGPT and their Workspace Agents, they unwittingly pave the way for this attack, particularly if there are existing integrations with other applications.
As the victim interacts with the phishing mechanism, the AgentForger is discreetly installed and given the capability to self-invoke during scheduled times. It autonomously seeks emails from the attacker, executes instructions, and promptly returns information back to the malicious source.
Critical to the attack’s success is a pre-configured setting that disables required approvals, enabling the agent to go unchecked. Typically, agents are designed to require user consent for actions to prevent unauthorized activities, which makes this seamless operation particularly dangerous.
Michael Bargury, co-founder and CTO of Zenity, noted that AgentForger can establish a persistent insider agent within ChatGPT with just a single interaction, allowing attackers to exploit the trust of the compromised individual for further assaults.
A ‘Planted Accomplice’ Carrying Out Malicious Tasks
Once AgentForger becomes operational, it can begin mapping an organization internally. This reconnaissance allows it to meticulously scan platforms like Outlook, SharePoint, and Slack to ascertain employee roles, projects, and internal discussions — effectively laying the groundwork for future attacks.
This method contrasts starkly with traditional tactics, where attackers must gradually gather this insider information. With AgentForger’s capabilities, a single directive from an attacker can instantly mobilize expansive data collection.
The agent can also extract sensitive data like financial documents, business agreements, and even passwords, exploiting the trust built over time within the organization. It can engage in phishing scams by impersonating legitimate users and directing targeted messages that prompt victims to disclose sensitive credentials.
All collected data is systematically organized and sent back to the hacker, creating a feedback loop that empowers further exploitation.
Bargury points out that this issue transcends a singular flaw in software; it highlights a broader shift in how security models must evolve as AI integrates into daily business environments.
Exposing Security Gaps Through Rush Decisions
This phenomenon is less about misplaced trust and more about the urgency to keep pace with AI developments, Bargury explains. Companies are under pressure to adopt new AI features to maintain competitive advantage, often at the expense of security protocols.
With AI technologies rapidly being adopted without sufficient security frameworks, organizations face increasing risks. Bargury stresses the importance of understanding the deployment and operation of AI agents, focusing on who created them, their connections, and the permissions granted.
Moreover, the processes that trigger these agents — like scheduled operations or incoming emails — must receive rigorous scrutiny. “Those triggers should be governed just as carefully as the agent itself,” he asserts.
Bargury recommends that enterprises designate approval requirements for high-stakes actions and enable rapid response mechanisms to deactivate any agent or its triggers if suspicious behavior is detected.
Looking ahead, the security trajectory for AI agents will necessitate a redefined understanding of permissions. It will no longer suffice to ask, “Does this agent have permission?” Organizations must also agonize over whether the agent's actions align with intended behaviors.
Those that successfully navigate these challenges will position themselves favorably in the age of AI, ensuring a secure and beneficial implementation of this technology.
