MITRE Engenuity ATT&CK Evaluations : Quick Guide

Wizard Spider & Sandworm Evaluation Results
Date: Evaluation results published March 31st, 2022

ABOUT MITRE ATT&CKMITRE ATT&CK is a public knowledgebase of adversarial tactics and techniques, which can be used as a foundation for thedevelopment of specific cyber threat models and methodologies.
In short, it helps the industry define and standardize how to describe an attacker’s approach. MITRE ATT&CK collects andcategorizes common attack tactics, techniques, and procedures (TTPs), then organizes this information into a framework.
This framework can be used to help explain how adversaries behave, what they are trying to do, and how they are trying todo it.
Having a common language and framework is important in the ability to communicate, understand, and respond to threats
as efficiently and effectively as possible.

Executive Summary - Highlights of Trend Micro’s results from the MITRE Engenuity ATT&CK Evaluation
This year's strong performance in MITRE Engenuity's ATT&CK Evaluation is the third in a row for Trend MicroTrend Micro Vision One recorded the following impressive results:•100% detection of all 19 attack steps in the evaluation - Highly enriched telemetry for better investigations.

•Provided clear visibility of 105 out of 109 attack methods providing 96.33% coverage - This broad visibility allows
customers to have a clear picture of the attack and respond faster.

•Ranked # 1 this year in the protection category – Ensuring that attacks are prevented early in the attack lifecycle.

•#1 performer in Linux, with 100% of attacks against the Linux host detected and prevented, capturing attacker steps
and preventing a simulated attack, which is especially important considering Linux is the most used OS in cloud-nativeapplications.

Evaluation summary high level results for Trend Micro
What are substeps? These are the individual attacker tests carried out as part of the emulation – in total there were 109 across both simulations

Analytic Coverage

Visibility

Protection

100 of 109

105 of 109

100%

substeps

substeps

Prevention

These are enriched detections that
add context by adding ATT&CK
technique mappings or alert
descriptions.

Attacker visibility allows customers tobuild a clear picture of the attack andrespond faster. This shows whereanalytic or telemetry information wasavailable.

Deflecting risk early on frees
up investigation resources,
allowing teams to focus on
the harder security problems
to solve..

USEFUL DATA FOR
SOC Level 2 / 3 Analyst - Hunters /Incident responder

USEFUL FOR
Defenders & Security Teams measuring defensive
capabilities

USEFUL DATA FOR
SOC Level 1 Analyst - Triage© 2022 Trend Micro Incorporated and/or its affiliates. All rights reserved. Trend Micro and the t-ball logo are trademarks or registered trademarks of Trend Micro and/or its
affiliates in the U.S. and other countries. Third-party trademarks mentioned are the property of their respective owners.

MITRE Engenuity ATT&CK Evaluations : Quick Guide
Trend Micro Solutions included in the evaluation

Day 1 Scenario

Day 2 Scenario

Emulated Group: Wizard Spider

Emulated Group: Sandworm Team (G0034)

Background: Russian-speaking cyber criminal gang associated Background: Destructive threat group attributed towith “big game hunting”
Russian GRU
Scenario: Ransomware attack against a notional organization
using malware (Emotet, TrickBot, Ryuk) associated withWizard Spider campaigns

Scenario: Compromise of a notional organization with thegoal of destroying data using malware (P.A.S. Webshell,Exaramel, NotPetya) associated with the Sandworm Team

Trend Micro Results
How the evaluation works:
The MITRE Engenuity ATT&CK Evaluation offers transparency to customers and real-world attack scenarios. This ensuresthat customers can actively evaluate security products to protect themselves from the latest advances from attackers
based on their areas of greatest need.
The evaluation uses adversary emulation to ensure customers can address today’s threats. Using techniques, tools,
methods and goals inspired by that of an attacker.
The simulations are executed in a controlled lab environment to ensure fair and accurate testing. Attacker techniques arethen used in logical step-by-step in order to explore the breadth of ATT&CK coverage. Over the two scenarios 109 attacker
steps were executed.
The evaluation this year emulated Wizard Spider and Sandworm tradecraft operational flows to simulate attacks similar tothe behavior used in the wild by these groups.
After the simulation has been run, results are processed and publicly released, including the methodology.
How is the Evaluation scored?
MITRE ATT&CK does not score or rank the evaluation against other vendors. However the “Evaluation Summary” can beused to explore a variety of metrics on the underlying data from each participant.
This is something we have aligned to in this report and compared these to other vendors. The results are available directlyfrom the website in full transparency for customers to also do self analysis in evaluating how to best protect themselves.
Additional Resources :Trend Micro Public Results
Configuration Summary© 2022 Trend Micro Incorporated and/or its affiliates. All rights reserved. Trend Micro and the t-ball logo are trademarks or registered trademarks of Trend Micro and/or its
affiliates in the U.S. and other countries. Third-party trademarks mentioned are the property of their respective owners.

MITRE Engenuity ATT&CK Evaluations : Quick Guide
Trend Micro Results:
Vendor

Attack Steps

Analytic

Visibility

Visibility Rank

Protection Score

cybereason

109

108

99.08%

109

100%

1100%

SentinelOne

109

108

99.08%

108

99.08%

2100%PaloAlto Networks

109

107

98.17%

107

98.17%

3100%

Cynet

109

102

93.58%

107

98.17%

3100%

McAfee109

84

77.06%

107

98.17%

375%

Bitdefender

109

106

97.25%

106

97.25%

4------------

Trend Micro

109

100

91.74%

105

96.33%

5100%

CrowdStrike

109

94

86.24%

105

96.33%

5100%

Check Point

109

103

94.50%

103

94.50%

677%

Microsoft

109

98

89.91%

98

89.91%

7100%

elastic

109

71

65.14%

98

89.91%

7------------

Fidelis109

85

77.98%

94

86.24%

8------------

Symantec

109

87

79.82%

92

84.40%

966%

Uptycs1098174.31%9284.40%

911%

Cisco

109

74

67.89%

9082.57%1077%

vmware109

57

52.29%

9082.57%

10100%

fireeye109

85

77.98%

8981.65%

1175%

Cylance1097165.14%

8981.65%

11100%

Sophos10967

61.47%

88

80.73%

1250%

FORTINET9085

94.44%

87

96.67%

13

Malwarebytes

90100%
100%

AhnLab9059

65.56%

83

92.22%

1462%

f-secure

109

66

60.55%

83

76.15%

15------------

cycraft

109

64

58.72%

7770.64%

1644%

eset

90

69

76.67%

75

83.33%

1737%

REAQTA

90

6268.89%7178.89%18------------

somma

90

28

31.11%

68

75.56%

19------------

Qualys9050

55.56%

66

73.33%

20------------

Deepinstinct

90

59

65.56%

63

70.00%

21100%

Rapid7109

2321.10%

62

56.88%

22------------

83
92.22%
83 and telemetry
92.22% across 29 vendors
14
Trend Microis Top
3 - for visibility

Creating a story of an attack with Trend Micro Vision One
Trend Micro Vision One allows you to provide a complete
story of the attack.
Automatically correlating threat data from different areas ofthe endpoint, server and network provides better alerts tosecurity teams.
We don’t just tell you these individual events
have all occurred – we connect the dots for you,showing that they might be related and have
similar indicators of compromise as a certain
attack group or type.

© 2022 Trend Micro Incorporated and/or its affiliates. All rights reserved. Trend Micro and the t-ball logo are trademarks or registered trademarks of Trend Micro and/or its
affiliates in the U.S. and other countries. Third-party trademarks mentioned are the property of their respective owners.

MITRE Engenuity ATT&CK Evaluations : Quick Guide
Want to understand more about the ATT&CK Framework?
MITRE ATT&CK Framework

MITRE ATT&CK Evaluation

This type of framework is extremely useful to information security
professionals helping to keep them updated on new attack techniques andto prevent attacks from happening in the first place.

The evaluations are not a competitive analysis. There are no scores,rankings, or ratings. Instead, they show how each vendor approaches
threat detection in the context of the ATT&CK knowledge base.

Why are Organizations using ATT&CK Framework?

Why are Organizations using ATT&CK Evaluations

Organizations use ATT&CK to standardize community conversations,

ATT&CK Evaluations provide vendors with an assessment of their ability to

defense testing, and product/service evaluations.

defend against specific adversary tactics and techniques.

Anatomy of MITRE ATT&CK Framework
TACTICS are the description of what attackers are trying to achieve.
Tactics are similar to a chapter of a book. A CISO can outline a story they want to tell with the high level tactics used in an attack and then refer to thetechniques to tell the story of how they accomplished the attack which provides extra detail.

IMPACTTECHNIQUES

Used by adversaries toThe are arefollow through on theirhow attackers

end goal or to provide

accomplish thosecover.

steps or goals.
Playbooks should be inplace to defend againstthese.

Example Story : Building an attack story in a common language
The goal of the attacker was to gain initial access to the network. Using a drive-by compromise with a spear-phishing link and trusted relationship, theattacker gained initial access using this technique. Note : The framework lists all the known ways that an attacker can gain initial access.

How does Trend Micro help?
Trend Micro maps our products to the ATT&CK Framework,
showing tactics and technique on detections which
demonstrates how we can can help you address the challengesof detecting and responding to threats.

What about prevention?
Preventative controls are an important part of a threat
mitigations strategy which add resilience when under attack.
Preventative controls were tested in the latest round with theability to deflect risk early on allowing organizations to spend
more time on harder security problems.

MITRE ATT&CK vs Cyber Kill Chain
MITRE ATT&CK is designed to provide a deeper level of granularity in describing
what can occur during an attack which is step forward from the Cyber Kill Chain.
MITRE ATT&CKCYBER KILL CHAIN

Initial Access

ReconnaissanceExecution

Intrusion

Persistence

Exploitation

Privilege Escalation

Privilege Execution

Defense Evasion

Lateral Movement

Credential AccessObfuscations / Anti Forensics

Discovery

Denial of Service

Lateral Movement

Exfiltration

Collection

Learn Morehttps://resources.trendmicro.com/MITRE-Attack-Evaluations.html

Command and ControlExfiltration
Impact© 2022 Trend Micro Incorporated and/or its affiliates. All rights reserved. Trend Micro and the t-ball logo are trademarks or registered trademarks of Trend Micro and/or its
affiliates in the U.S. and other countries. Third-party trademarks mentioned are the property of their respective owners.