UNC2452: Hacking without Consequences — Part One: Why SolarWinds Orion?
By: Dan Hranj
Tags: Cybersecurity, Hacking, Infosec, Incident Response

Why Orion?

The compromise of the SolarWinds Orion platform begs the question, why Orion? Further, why not something else, or why not everything?

The Orion platform offers a catalog of modular products to help IT administrators manage nearly everything in their environment including performance, compliance, change and patch management, and data analysis to name a few. The specific functionality does not matter because you’re always installing an overarching management platform. This architecture makes it a one-stop-shop, a single pane of glass, a command center for your network. It also makes it a natural target for modern attackers who prefer to live off the land as opposed to using purpose-built, proprietary tools that represent a significant investment of time and money for development and may aid defenders in detection, response, and attribution. And in a world where security teams put a major focus on the perimeter not as much time and energy can be spent dissecting and analyzing software built by respected industry incumbents like SolarWinds. After the initial compromise, attackers could get easy access to privileged credentials that are likely in use on the SolarWinds server and, being an endpoint management platform, have access to network segments that would be difficult to reach via other means of initial compromise. I see the compromise of SolarWinds Orion similar to the direct compromise of a domain controller, bastion host, god tool, and domain administrator at the most treasured networks in the world.

SEC filings by SolarWinds indicate they currently have approximately 300,000 customers. A subset (33,000) of those were Orion customers during the period in question (March to December 2020), and a fraction (18,000) is believed to have installed the malicious SolarWinds update [1] that can provide the attacker with a backdoor to those customer networks. The worldwide footprint is massive and extends to nearly every business vertical imaginable. Frankly, it is too big, wildly ambitious, and demonstrates flagrant contempt for any sort of cybersecurity rules of engagement, even for a skillful and motivated nation-state.

… this feels like the perpetrator shouting from the rooftop that they are invincible, immune to repercussions, and indiscriminate in the destruction they’re willing to create.

Last week, Microsoft announced that they had identified over 40 customers who had not only installed the malicious update but were specifically targeted for further compromise [2]. In the context of large numbers like 300,000, 33,000, and 18,000, a small number like 40 hardly seems significant but this is far beyond the regular operational capacity of any group I’ve responded to in the past. For context, FireEye responded to breaches carried out by the prolific APT1 in 141 organizations over a seven-year period between 2006 and 2013 [3]. And all that happened when hacks were “easy”; anti-virus was considered good enough, no one retained logs for meaningful amounts of time, two-factor authentication was only for IT admin if you were lucky, and very few companies had dedicated security teams. Those twenty breaches a year, carried out by what was revealed to be an entire unit of the Chinese military, helps illustrate the potential size of the current SolarWinds operation over the last nine months. A blog post by TRUESEC paints an even different picture. Their analysis of command and control infrastructure suggests at least 261 organizations were specifically targeted for further exploitation following the installation of the SolarWinds update [4].

Security firm Volexity may have been one of the first to recognize the role SolarWinds’ software played in targeted attacks. During an investigation in June and July 2020, all signs pointed to SolarWinds but, between the attacker’s penchant for cleaning up after themselves and multiple reboots of critical infrastructure, the available evidence degraded to a point that they were unable to make definitive conclusions [5]. No one can say how long it would have taken to uncover this operation had the attacker not breached FireEye but, given the enormous scope, my gut says the attacker would have enjoyed a little more than nine months to pillage its victims’ networks.

For that, my hat goes off to FireEye and Mandiant whose mantra about being hacked has always been “when, not if”. The attacker often has the upper hand in cyber security’s perpetual game of red vs blue and it highlights that the time it takes to respond to an incident is paramount. An operation such as this is no small feat but, given the nine-month window of opportunity, I do not believe the attacker 1) had the opportunity to move beyond the “Initial Compromise” and “Establish Foothold” phases of the attack lifecycle for the majority of the 18,000 potential victims and 2) achieved their objectives for all of their primary targets.

I’ve already hinted at a few reasons for targeting SolarWinds Orion so let’s break a couple down in a little more detail.

Ubiquity — 33,000 potential victims is monumental and persistence is key for attackers. They often remain in networks for months to years after achieving their objectives. Only really “leaving” when they are discovered and kicked out. It may be easier for an attacker to maintain access to an environment rather than regain access, and if the target was useful once, they may be useful again. Furthermore, watering hole and supply chain attacks are like a box of chocolates, you never know what you’re gonna get. The attacker’s primary target may have been government organizations but I’ll be damned if the attackers aren’t thrilled with compromising a handful of high-profile technology companies in the process.

— 33,000 potential victims is monumental and persistence is key for attackers. They often remain in networks for months to years after achieving their objectives. Only really “leaving” when they are discovered and kicked out. It may be easier for an attacker to maintain access to an environment rather than regain access, and if the target was useful once, they may be useful again. Furthermore, watering hole and supply chain attacks are like a box of chocolates, you never know what you’re gonna get. The attacker’s primary target may have been government organizations but I’ll be damned if the attackers aren’t thrilled with compromising a handful of high-profile technology companies in the process. Commonality — Attackers spend a significant amount of time profiling their targets. During the “Initial Reconnaissance” phase of the attack lifecycle, operators perform external scans to see what a target has accessible to the internet, build profiles on specific employees based on their seniority, responsibilities, and expertise which are openly advertise on professional sites like LinkedIn, and scour press releases, financial disclosures, and company websites that openly tout business with new customers. This is just a small sampling of profiling techniques that may have strengthened the decision to compromise Orion. If, for example, an attacker is tasked with infiltrating the top aerospace engineering firms in the United States and all of those firms happen to use Orion then it would be negligent to ignore the potential of a 10 for 1 opportunity.

— Attackers spend a significant amount of time profiling their targets. During the “Initial Reconnaissance” phase of the attack lifecycle, operators perform external scans to see what a target has accessible to the internet, build profiles on specific employees based on their seniority, responsibilities, and expertise which are openly advertise on professional sites like LinkedIn, and scour press releases, financial disclosures, and company websites that openly tout business with new customers. This is just a small sampling of profiling techniques that may have strengthened the decision to compromise Orion. If, for example, an attacker is tasked with infiltrating the top aerospace engineering firms in the United States and all of those firms happen to use Orion then it would be negligent to ignore the potential of a 10 for 1 opportunity. Chaos — I don’t think I need to spend much time explaining the complexity and absurdity of the geopolitical climate as we exit the 2010s and enter the 2020s. FireEye has yet to assign nation-state level attribution to UNC2452 whose name alone should illustrate the difficulty of attribution. Other organizations have been quick to point fingers at APT29 aka COZYBEAR, and other context clues do make Russia appear a bit suspicious. It would be a natural fit with Putin remaining at the helm of Russia, Trump leaving office while sowing doubt any place he deems fertile, and COVID-19 ravaging anywhere leaders chose not to enforce strict lockdowns. Compromising countless government entities, publicly traded companies, healthcare networks, and financial institutions is the cherry on top of 2020. Or maybe the plan was for the hack to be uncovered before the 2020 election, casting more doubt on its integrity and legitimacy. As the current administration remains silent on the matter, this feels like the perpetrator shouting from the rooftop that they are invincible, immune to repercussions, and indiscriminate in the destruction they’re willing to create. The chaos is all possible thanks to the ubiquity and commonalities of SolarWinds Orion.

— I don’t think I need to spend much time explaining the complexity and absurdity of the geopolitical climate as we exit the 2010s and enter the 2020s. FireEye has yet to assign nation-state level attribution to UNC2452 whose name alone should illustrate the difficulty of attribution. Other organizations have been quick to point fingers at APT29 aka COZYBEAR, and other context clues do make Russia appear a bit suspicious. It would be a natural fit with Putin remaining at the helm of Russia, Trump leaving office while sowing doubt any place he deems fertile, and COVID-19 ravaging anywhere leaders chose not to enforce strict lockdowns. Compromising countless government entities, publicly traded companies, healthcare networks, and financial institutions is the cherry on top of 2020. Or maybe the plan was for the hack to be uncovered before the 2020 election, casting more doubt on its integrity and legitimacy. As the current administration remains silent on the matter, this feels like the perpetrator shouting from the rooftop that they are invincible, immune to repercussions, and indiscriminate in the destruction they’re willing to create. The chaos is all possible thanks to the ubiquity and commonalities of SolarWinds Orion. Simplicity — Based on my experience in the field, I do not believe SolarWinds, or any organization is capable of formulating a defensible root cause and establishing the full extent of a breach of this magnitude for at least a month after the start of an investigation. It is still early days and one month would be a Herculean achievement so I advise everyone to get comfortable. It’s going to be a long and bumpy ride. | The public may never know the full extent of the breach. Heck, even SolarWinds will have unanswered questions at the end of all this. Even still, people are jumping to conclusions as if we know all of the facts! A misconfigured Office365 implementation and credentials in public git repositories are among the most popular scapegoats I’ve seen to date. | It is easy for outside observers to miss the forest for the trees and while these may seem like significant missteps I can assure you that every organization has similar problems that may never lead to such catastrophic events. While we’re not exactly nailing Jell-O to a tree, the majority of security professionals would have their work cut out for them if they had an objective similar to the weaponization of Orion. Little strokes fell great oaks and enough of those tiny missteps may have added up to the hack of the decade. It is within the realm of possibility that this was an easy hack, but I just don’t see it being true. | I can all but guarantee a thorough analysis of the SolarWinds breach would produce a report that is hundreds of pages long and we cannot be blaming a potential company ending event on rookie mistakes without more information. If I haven’t dropped enough hints, I believe this actor is highly skilled, extremely patient, and well funded. Stay tuned for “Part Two: A Hypothetical Attack Life Cycle” where we’ll explore what may have happened behind the scenes.

Why not something else? Why not everything else?

By now we have a solid case for targeting Orion but why not everything else? With this level of unfettered access, why not put a backdoor in everything? In my experience, attackers rarely go for the whole kit and caboodle for a variety of reasons. First and foremost is risk. Every command, authentication, backdoor, source code modification, and victim comes with additional risk. Risk of being detected, tracked, attributed, exposed, and defeated before an attacker is able to achieve their objective.

Most modern enterprise networks are a patchwork of services and systems from multiple acquisitions that are all in different phases of integration. Inserting a backdoor into a product is more complex than dropping a Stack Overflow answer straight into source code. Weaponizing multiple products could mean traversing new domains, compromising additional accounts, evading different anti-virus and EDR agents, egressing from different locations, and familiarizing yourself with a whole new tech stack which would include performing source code analysis, learning the team’s coding practices and standards, multiple months of testing and slow escalation, using different version control software, and compromising different signing infrastructure. For the sake of the primary mission, an attacker would be better off keeping their eyes on the prize and doing everything in your power to ward off shiny object syndrome.

That doesn’t mean the fallout of the SolarWinds breach starts and ends with Orion. Far from it. In spite of my prior arguments against additional backdoors, Palo Alto has reported the discovery of a webshell embedded in a .NET binary that is part of the Orion software platform [6]. Little is known about this new finding as of the publishing of this post but it certainly makes you wonder what else is waiting in the wings so in “Part Three: What do We do Now?” we’ll take a look at ways to mitigate this attack, protect your own organizations from similar attacks, and posture yourself for additional fallout.