UN Remarks on "Navigating the Challenges of AI in Cyberspace" 🌐

Context: On Monday July 20th, I had the pleasure of being invited to a side event to the Global Mechanism on ICT Security at the UN to talk about the current state of AI and cybersecurity and the role of open models. The event occurred at a particularly serendipitous time: the infrastructure team at Hugging Face had just successfully mitigated a first-of-its-kind AI-driven cyber-attack by leveraging open-weight models.

It was later revealed that the attack in question had been run autonomously by a combination of OpenAI models exploiting a flaw in their testing environment: an event that shines further light on the necessity on taking a pro-active and distributed approach to improving cybersecurity, rather than relying entirely on control of or products sold by dominant model developers.

I'm sharing the text of my remarks here, comments welcome! Skip to the end for references and resources 👇

Thank you for this invitation; discussions like this one provide a unique opportunity to get to the core of cybersecurity questions in this time of AI agents.

As an open-source and open science person, cybersecurity is particularly close to my heart. In OSS, it's supported both by the contract of trust different groups have with each other and by active and transparent collaboration. AI may have changed some of the game by introducing new technical paradigms, but the basics are still the same. Which makes me particularly concerned to see recent narrative turns on AI and cybersecurity follow increasingly counterproductive commercial logic instead of doubling down on research and collaboration.

This shift is far from surprising. Cyberoffense and commercially valuable AI-enabled software engineering are two sides of the same coin. Given the importance of the market for the latter, this duality means that commercial providers of general-purpose systems have a vested interest in pushing a narrative that puts them front and center in cybersecurity conversations, even at the expense of more distributed and field-tested solutions. Without even getting into the geopolitical or democratic concerns of following this path, past incidents such as the Crowdstrike failure that paralyzed airports worldwide two years ago show all too clearly what a single point of failure looks like.

So let’s start with some relevant technical context. Of all the domains for which current AI technology holds promise, software code, and consequently cybersecurity, are a particularly good match for the requirements of modern machine learning. A rich open-source culture gave commercial and open-weight developers alike troves of readily available pre-training data. Coding tasks can be automatically evaluated, so reinforcement learning works particularly well here. And a large user base for AI-powered software engineering has let developers fine-tune on work logs, matching developer needs ever more closely.

In short, stepping away from the technical lingo, utility for software writing is among the easiest capabilities to build out for AI systems. While frontier companies retain an edge in user-friendliness thanks to their data flywheels, the core competencies are widely shared across models of all sizes.

This is good news for competition. For example, before it was acquired by SpaceX, American software company Cursor was able to build an extremely successful product by adapting and eventually training their own models at a much lower initial cost than frontier companies. It is also good news for the democratization of AI-powered cybersecurity defense. It means AI models can be leveraged to prevent existing and novel cyberattacks broadly, even without access to the latest and costliest models.

That’s probably not the story you’ve been hearing, especially since the release of the Mythos system. Back in April, AI company Anthropic put out a blog post claiming that their most powerful new model had discovered hundreds of unpatched vulnerabilities across major open-source libraries. The news acted as a cold shower for many actors in the field, bringing a welcome focus on the issues raised by new AI-powered security technology.

The boost to attention on an important topic was welcome, but it came with a strongly misleading framing.

While Anthropic emphasized the "dangerous capabilities" of the underlying model and stayed vague about what their system had actually done, other white hat hackers easily reproduced the general thrust of their claims with models small enough to fit on a cellphone. The Mythos system shines a light on some real risks, but the success of these kinds of cyberattacks depends more on the quality of the software harness, access to moderate to high compute infrastructures, particularly when supporting inference for swarms of models, and access to automation patterns and scheduling that differ from more traditional attacks - than on so-called frontier capabilities.

We could usually chalk this up to a difference of interpretation: after all, we all agree that AI concerns in cybersecurity are understudied and deserve more attention. But here, the misrepresentation created its own new cybersecurity risks. Recent work by AI Now shows that over-relying on systems branded as powerful autonomous defenders introduces particularly concerning new attack vectors.

So where do we go from there? What are the field-tested solutions I mentioned above? The points I have been making so far aren't just theoretical, they have supported our approach to cybersecurity at Hugging Face, with apparent success so far. In our day-to-day operations, we make good use of open and commercial models, but they all have strict permission control and human review: expediency is never an excuse for making automated systems ultimately responsible for security, regardless of the performance advertised by the provider. We also do use models for cyberdefense, an approach that was recently tested when we discovered and addressed a first-of-its kind agent-powered breach in a matter of hours. In short, we used models to identify abnormalities in data transfer logs, quickly explore the full breadth of the attack surface enabled by the initial breach, and take human action to purge the attacker. Not only did open models allow us to run actions for cheaper without sharing sensitive data, but they also allowed us to simulate attack paths ahead of the actual attacker in a way that would have been blocked by commercial providers.

This was our own example, but I do believe it is representative of broader trends. With AI, the security through obscurity paradigm goes from a losing proposition to a disastrous one. The best thing we can do for cyberdefense is collaborate, use it with care, and share everything we learn with each other.

References:

• AI and the Future of Cybersecurity: Why Openness Matters
• Building Moon Bot: A Slack-Native Coding Agent Backed by HuggingFace Buckets
• Friendly Fire: Hijacking Defensive Cyber AI Agents for Remote Code Execution - AI Now Institute
• Security incident disclosure — July 2026
• OpenAI and Hugging Face partner to address security incident during model evaluation