Recent research from Zenity has highlighted a concerning trend where attackers are targeting AI software supply chains with trojanized agent skills designed to deploy credential-stealing malware. This attack, which began on July 11, involved uploading malicious skills to the open marketplace skills.sh, accumulating over 1.7 million downloads by August 2. Notably, the attackers achieved this by imitating well-known AI services like Paperclip and Browser Use through typosquatting, creating a façade that compromised naïve users.
The trojanized skills were strategically crafted to direct AI agents to download a credential stealer directly from a GitHub repository, following a foiled attempt to use malicious packages from npm and PyPI. The primary target seemed to be developer workstations and CI/CD runners, with the attackers aiming to harvest a diverse range of sensitive credentials, including SSH keys, cloud tokens, and database access details.
The Attack Approach
The initial phase of this attack saw hackers set up two GitHub organizations, masquerading as legitimate entities. They populated these with genuine code snippets and later modified the skills to include harmful directives. To initially satisfy the skills marketplace requirements, they uploaded duplicates of the official skills, only to incorporate malicious instructions shortly after.
One notable tactic was the attempt to host trojanized packages on npm and PyPI, which were quickly flagged and removed. When that approach hit a wall, the attackers pivoted, directing AI agents to install directly from their GitHub repositories instead.
For instance, a skill entitled "paperclip-board" contained directions that misled users about the installation process, ensuring that the malicious package was installed rather than any legitimate ones. This misdirection emphasizes the sophistication of the social engineering involved in the attack, as it exploits both the trust placed in established tools and the procedural knowledge expected of users.
The marketplace nature of skills.sh, where skills often reference one another and are interconnected, complicates the tracking of individual victims. Each of the malicious skills reached upwards of 300,000 downloads, indicating widespread exposure.
Challenges in Detection
AI skills are often more than standalone files; they can encompass multiple components designed for specific tasks within the tools they serve. This segmentation leads to progressive discovery methods, which keep unnecessary data outside of the limited context that AI models can handle. While this is beneficial for operational efficiency, it creates a challenge for detection, as malicious instructions can be camouflaged within otherwise legitimate tasks.
According to the researchers, the deceptive nature of the skills—where the harmful commands reside not in the main files but in secondary documents that are accessed only under specific conditions—makes detection increasingly difficult. The design ensures that AI agents are directed only toward compromised source materials, avoiding genuine, safe packages.
As AI agents mature, their built-in operational knowledge is leveraged to establish preferred methods of task execution, diminishing the likelihood of traditional detection mechanisms identifying abnormal behavior.
Experts in the security field have emphasized the necessity for continual monitoring of AI agent configuration files. Given the natural language nature of skills, automating the detection of malicious instructions poses significant challenges. Static analysis tools are often ill-equipped to discern harmful commands embedded in complex linguistic inputs.
To combat these challenges, Zenity has launched a free service dubbed AI Total. This tool mimics the principle of malware detonation, activating skills within a sandboxed agent environment. The sandbox deploys decoy credentials and monitors the agent's activities comprehensively, providing insights into domain interactions, downloaded packages, and file modifications.
The growing complexity and utility of AI capabilities demand heightened vigilance regarding security. As this attack exemplifies, the incorporation of malicious skills not only highlights existing vulnerabilities in AI supply chains but also stresses the urgent need for enhanced detection and monitoring practices to safeguard sensitive information.
