As artificial intelligence continues to expand, platforms facilitating community-driven model sharing like Hugging Face have become essential resources for developers worldwide. However, this growth also brings heightened security risks, as malicious actors exploit vulnerabilities to distribute harmful code. Recently, ReversingLabs uncovered two machine-learning models on Hugging Face that utilized a deceptive method known as “pickling” to embed malicious web shells, posing significant threats to the integrity of AI development environments.
ReversingLabs found that the malicious models employed pickle files, a Python-based serialization method, to include harmful code capable of executing from untrusted sources during the deserialization process. This technique allowed the models to link to hardcoded IP addresses, thereby embedding web shells that could potentially compromise systems running these models.
How Did the Malicious Models Bypass Detection?
The identified models managed to evade Hugging Face’s security tool, Picklescan, primarily because they were compressed using alternative formats. Picklescan relies on a blacklist of dangerous functions to mark unsafe pickle files, but it struggled to detect the malicious code within these compressed or broken files, highlighting limitations in current security scanning methods.
What Measures Has Hugging Face Implemented?
“The Picklescan tool is based on a blacklist of ‘dangerous’ functions. If such functions are detected inside a Pickle file, Picklescan marks them as unsafe,”
explained Karlo Zanki, a reverse engineer at ReversingLabs. Recognizing the loopholes, Hugging Face swiftly removed the malicious models and updated Picklescan to better identify such threats, improving their platform’s defense mechanisms against evolving security challenges.
Are Pickle-Related Vulnerabilities a Growing Concern?
With the surge in community-made machine-learning models, pickle-related vulnerabilities remain a prevalent issue. Other cybersecurity firms, including Wiz and Checkmarx, have similarly identified various methods by which pickle files can be abused to deliver malware on open platforms. This ongoing trend underscores the necessity for more robust and adaptable security measures in AI development environments.
Ensuring the security of AI platforms like Hugging Face is crucial as the reliance on shared models increases. The ability of malicious actors to exploit pickle files highlights a significant vulnerability that demands continuous attention and improvement. Developers must remain vigilant and adopt comprehensive security practices to safeguard against such threats, fostering a safer and more secure AI development landscape.
