Q3 2019

Advanced Threat Defense
Certification Testing Report

Kaspersky
Kaspersky Anti Targeted Attack Platform (KATA)

Tested against this standard
ICSA Labs Advanced Threat Defense Criteria v.1.0

October 4, 2019

Prepared by ICSA Labs
1000 Bent Creek Blvd., Suite 200
Mechanicsburg, PA 17050
www.icsalabs.com

ICSA Labs Advanced Threat Defense – Report-at-a-Glance
Kaspersky
Executive Summary
During 32 days of testing during the third quarter of 2019, ICSA Labs tested the detection capabilities ofKaspersky Anti Targeted Attack Platform (KATA) with a mix of 1372 test runs. The mix was primarily
composed of new and little-known malicious threats – i.e., recently harvested threats not detected bytraditional security products.
Periodically, ICSA Labs launched innocuous applications and activities to additionally test the KATA
platform in terms of false positives. Throughout testing, ICSA Labs observed product logs to ensure not
only that the KATA platform indicated the existence of a malicious threat but also that logged threats weredistinguishable from other logged traffic and events.
The KATA platform passed, having met all criteria requirements. As seen in Figure 1 below, Kaspersky’s
solution did remarkably well during this test cycle - detecting 100% of previously unknown threats while
having 0 false positives. Figures 2 and 3 below further highlight the solution’s detection effectiveness andfalse positives (FPs).

www.kaspersky.com/enterprisesecurity/anti-targeted-attack-platform

Test Length

32 days

Malicious Samples

736

Innocuous Apps

636

Test Runs

1372% Detected

100%% False Positives

0.0%Fig. 1 – High Detection Effectiveness & Few False Positives

ICSA Labs
Advanced Threat Defense

Certified
Test Period:
Certified Since:

Q3 2019
12 / 2016
Fig. 2 – Detected 736 of 736 New & Little-Known Malicious Samples

Fig. 3 – Zero Alerts on Innocuous Applications

ATD-KASPERSKY-2019-1004-01

Page 1 of 10

Copyright © 2019 ICSA Labs. All rights reserved.

Criteria v.1.0

ICSA Labs Advanced Threat Defense - Certification Testing Report
Kaspersky

Introduction
This is Kaspersky’s twelfth consecutive ICSA Labs Advanced Threat Defense Certification testing report for
Kaspersky Anti Targeted Attack Platform (KATA).
Standard ICSA Labs Advanced Threat Defense (ATD) testing is aimed at vendor solutions designed to detect new
threats that other traditional security products miss. Thus, the focus is on how effectively vendor ATD solutions detect
these unknown and little-known threats while minimizing false positives.
The remainder of the report presents a more detailed look at how the Kaspersky KATA advanced threat defense
solution performed during this cycle of standard ICSA Labs ATD Certification testing. To better understand how tointerpret the results, this report documents not just the testing results themselves but the threat vectors, sample
sources, and kinds of samples that ICSA Labs employed for this cycle of ATD testing against Kaspersky’s KATA.

Test Cycle Information
This report reflects the results of one test cycle at ICSA Labs. Standard ATD and ATD-Email test cycles are
performed by ICSA Labs each calendar quarter and typically range from three to five weeks in duration. To be eligible
for certification, security vendor solutions must be tested for at least 3 weeks. Because testing is performed quarterly,ICSA Labs tests ATD solutions four times during a calendar year.
During each test cycle ICSA Labs subjects advanced threat defense solutions to hundreds of test runs. The test set iscomprised of a mix of new threats, little-known threats and innocuous applications and activities – delivered andlaunched one after another continuously for the length of testing. Below in Figure 4 is information about the test cyclefrom which this findings report is based.
Start Date

July 17, 2019

Days of Continuous Testing32

End Date

August 17, 2019

Test Runs

1372

Fig. 4 – This Test Cycle

ATD Solution Tested
During this testing cycle, ICSA Labs tested the Kaspersky Anti Targeted Attack Platform (KATA) from Kaspersky.
Kaspersky Anti Targeted Attack Platform – 3.5.0-1269
Kaspersky Anti Targeted Attack Platform (KATA) uses multi-layered threat detection – including a granularassessment of activity that’s occurring on the customer’s corporate network and at endpoints – to help protect
businesses and organizations against sophisticated threats and targeted attacks. KATA includes network
sensors, web and email sensors, as well as optional endpoint sensors that together help to detect threats –not just at the entry points - but wherever they arise within the customer’s IT infrastructure. KATA also
includes Kaspersky’s Targeted Attack Analyzer that assesses data from network and endpoint sensors – andrapidly generates threat detection verdicts for the security team. By combining sandbox-based analysis andKaspersky's advanced machine learning technologies, Kaspersky Anti Targeted Attack Platform helps todeliver protection against a wider range of threats.

Kaspersky Endpoint Security 11.1 for Windows

For more information about Kaspersky’s KATA solution, its component parts and related information please visit:https://www.kaspersky.com/enterprise-security/anti-targeted-attack-platform
http://newsroom.kaspersky.eu/fileadmin/user_upload/de/Downloads/PDFs/KATA_Feature_List.pdf
https://www.kaspersky.com/enterprise-security/wiki-section/products/kaspersky-anti-targeted-attack-platform

ATD-KASPERSKY-2019-1004-01

Page 2 of 10

Copyright © 2019 ICSA Labs. All rights reserved.

Criteria v.1.0

ICSA Labs Advanced Threat Defense - Certification Testing Report
Kaspersky

Detection EffectivenessTo meet the criteria requirements and attain (or retain) certification through ICSA Labs testing, advanced threat
defense solutions must be at least 75% effective at detecting new malicious threats. As shown in Figure 5 theKaspersky KATA platform detected 100% of the threats it encountered during testing, considerably better than thepercentage required for certification.

Fig. 5 – Detection Effectiveness of Kaspersky’s KATA

A second plot depicting the detection effectiveness of KATA appears in Figure 6. For Kaspersky’s solution the chartsheds light on whether or not the KATA did better or worse – the newer the malicious sample. As is evident bothbelow and in the previous figure, regardless of how new or how old the threat, the Kaspersky KATA platform detected
100% of new and little-known malicious threats. Kaspersky’s KATA platform provided this excellent detection
effectiveness and had no false positives during this test cycle, which is impressive.

Fig. 6 – Detection Effectiveness by Age of Threat (Threats < 24 Hours Old)

ATD-KASPERSKY-2019-1004-01

Page 3 of 10

Copyright © 2019 ICSA Labs. All rights reserved.

Criteria v.1.0

ICSA Labs Advanced Threat Defense - Certification Testing Report
Kaspersky

A final effectiveness-related plot to consider for Kaspersky’s advanced threat defense solution KATA during this test
cycle is Figure 7 below. Plotted below is each of the 32 days during the test cycle along with how effective the KATA
platform was on each of those days. For an impressive 32 days during the test cycle, the Kaspersky KATA platform
was 100% effective against the new and little-known threats used in testing.

Fig. 7 – Detected & Missed Threats by Day of Test Cycle

Threat Vectors
In testing, ICSA Labs delivers new and little-known malicious threats to security vendor solutions using many of thetop threat vectors that have led to enterprise cybersecurity incidents and breaches as reported in the latest Verizon
Data Breach Investigation Report (DBIR).
DBIR data indicates that malware has been a key factor in thousands of security events where an information asset
had its integrity, confidentiality, and/or availability compromised. Figure 9 on the following page depicts the threatvectors involved in these malware-related security incidents throughout the over fifteen-year history of Verizon’s DBIR.
Figure 8 below illustrates the most common malware-related threat vectors that lead to enterprise breaches during
2017 alone (per 2018 DBIR).

Fig. 8 – Top Threat Vectors Leading to Breaches in 2017 (per 2018 DBIR data)

ATD-KASPERSKY-2019-1004-01

Page 4 of 10

Copyright © 2019 ICSA Labs. All rights reserved.

Criteria v.1.0

ICSA Labs Advanced Threat Defense - Certification Testing Report
Kaspersky

Fig. 9 – Malware-Related Threat Vectors Involved in Incidents (DBIR All-Time)

Standard ICSA Labs ATD testing includes the threat vector that is by far the most prevalent over time, “Direct Install”.
In addition, standard ATD testing includes the threat vectors labeled “Web download”, “Web drive-by”, and “Downloadby malware”. In the separate but related, ICSA Labs ATD-Email testing, ICSA Labs delivers new and little-known
malware in email attachments and emails with malicious URLs, corresponding to DBIR threat vectors “Emailattachment” and “Email link”, the former being the second most common threat vector leading to enterprise breaches
according to the 2018 DBIR.

Source of Samples
A number of sample sources feed ICSA Labs’ standard ATD and ATD-Email testing.
One source is the spam ICSA Labs collects. The labs’ spam honeypots receive approximately 250,000-300,000
spam email messages/day. For ICSA Labs ATD testing, the team harvests attachments in that spam, making use of
the ones that are malicious.
Samples may also come from malicious URLs. Some of these come from the spam mentioned above. From feeds
like this ICSA Labs filters and checks the URLs to see if there is a malicious file on the other end of that URL -- either
as a direct file link or a series of steps (e.g. a drive-by attack with a multi-stage download process) leading to it. If so,
ICSA Labs collects the sample for potential use in testing.
ICSA Labs additionally uses other tools and techniques to create unique malicious files as an attacker or penetration
tester might do. In some cases, these are trojanized versions of clean executables. In other cases, they may beoriginal executables that are malicious.
Still another source of samples is the samples themselves. Any dropped files resulting from running anothermalicious sample are also evaluated and potentially used in testing.
Finally – and importantly to test for false positives – ICSA Labs also launches legitimate executables. Runninginnocuous applications helps ensure that vendor solutions aren’t just identifying everything as malicious.

ATD-KASPERSKY-2019-1004-01

Page 5 of 10

Copyright © 2019 ICSA Labs. All rights reserved.

Criteria v.1.0

ICSA Labs Advanced Threat Defense - Certification Testing ReportKaspersky

Regarding the Samples from this Test Cycle
Samples harvested for use in ATD testing are often unmodified and used as is. That is the case if ICSA Labsdetermines that the sample is new enough and/or not being detected by traditional security products. In many cases
malicious samples require modification before they can avoid detection by traditional security products.
Of the 736 malicious samples, Figure 10 shows that there were more original samples used and fewer samples that
required some kind of modification before use in testing. Of the 671 original samples, 170 were dropped, or left
behind by other malware. Figure 11 reveals the source of the 501 malicious samples used in testing that were neither
modified nor dropped.

Fig. 10 –Malicious Samples – Original vs. Modified

Fig. 11 – Unmodified/Non-Dropped Sample Sources

Following the test cycle, ICSA Labs analyzed the original malware samples used in testing, categorizing the malware
into one of six malicious threat types: backdoor, ransomware, spyware, trojan, worm, or virus. Any malicious sample
not falling into one of these six types, ICSA Labs categorized as “other”.
The six malware categories, and the number of original malicious samples used during the test cycle from eachcategory are represented in Figure 12 below. The figure indicates how many malicious threats Kaspersky’s KATA
solution detected and missed from each malware category during testing. In addition, the green line atop Figure 12represents the effectiveness percentage of KATA against original malware belonging to each malware type.

Fig. 12 – Effectiveness against original, unmodified malicious samples broken down by threat type

ATD-KASPERSKY-2019-1004-01

Page 6 of 10

Copyright © 2019 ICSA Labs. All rights reserved.

Criteria v.1.0

ICSA Labs Advanced Threat Defense - Certification Testing Report
KasperskyFigures 13 through 16 provide a deeper glimpse into four of the six malware types: ransomware, trojan, spyware, andbackdoor. In its analysis of the original malicious samples used in testing, ICSA Labs further categorized malicious
samples by malware family, where possible. The remaining figures, one for each of the four aforementioned malware
types, are ordered by malware family. The figures show how many original malware samples the KATA solution from
Kaspersky detected and missed across multiple malware families during the test cycle. In addition, the green line
atop each figure indicates the effectiveness percentage of KATA against original malware from each malware family.

Fig. 13 – Effectiveness against Kinds of Ransomware Threats

Fig. 14 – Effectiveness against Ransomware Families of Trojans

ATD-KASPERSKY-2019-1004-01

Page 7 of 10

Copyright © 2019 ICSA Labs. All rights reserved.

Criteria v.1.0

ICSA Labs Advanced Threat Defense - Certification Testing Report
Kaspersky

Fig. 15 – Effectiveness against Families of Spyware

Fig. 16 – Effectiveness against Backdoors

As the KATA solution was 100% effective during this quarter of testing, the graphs above just reinforce that thesolution was very effective at detecting malware across malware types and across malware families during the Q3
2019 test cycle.

Prior ATD Reports
With this report, Kaspersky’s KATA advanced threat defense solution passed all the test cases to retain ICSA Labs
Advanced Threat Defense Certification. Successful completion of this test cycle marks Kaspersky’s 12th consecutive
quarter having met the ICSA Labs ATD certification testing criteria.
This and all earlier KATA certification testing reports can be found on the ICSA Labs web site at:https://www.icsalabs.com/product/kata

ATD-KASPERSKY-2019-1004-01

Page 8 of 10Copyright © 2019 ICSA Labs. All rights reserved.

Criteria v.1.0

ICSA Labs Advanced Threat Defense - Certification Testing Report
Kaspersky

Significance of the Test & Results
Readers of certification testing reports often wonder what the testing and results really mean. They ask, “In what way
is this report significant?” The four statements below sum up what this ICSA Labs Advanced Threat Defense
Certification Testing report should indicate to the reader:
1. ICSA Labs tested the Kaspersky KATA advanced threat defensesolution using the primary threat vectors leading to enterprise
breaches according to Verizon’s Data Breach Investigations Report(DBIR).
2. ICSA Labs tests with malicious threats including new and little-known
Ransomware that other security products typically miss.
3. Kaspersky’s KATA demonstrated superb threat detection
effectiveness against over 735 new and little-known threats.
4. The Kaspersky KATA solution had zero false positives during this test
cycle, which is excellent.

ATD-KASPERSKY-2019-1004-01

Page 9 of 10

Copyright © 2019 ICSA Labs. All rights reserved.

Criteria v.1.0

ICSA Labs Advanced Threat Defense - Certification Testing Report
Kaspersky

Authority
This report is issued by the authority of the General Manager, ICSA Labs. Tests are done under normal operating
conditions.

ICSA Labs

Kaspersky

The goal of ICSA Labs is to significantly increase userand enterprise trust in information security products andsolutions. For more than 25 years, ICSA Labs, anindependent division of Verizon, has been providing
credible, independent, 3rd party security product testingand certification for many of the world’s top security
product developers and service providers. Enterprises
worldwide rely on ICSA Labs to set and apply objective
testing and certification criteria for measuring productcompliance and performance.

Kaspersky is one of the world’s fastest-growing
cybersecurity companies and the largest that is privatelyowned. The company is ranked among the world’s top
four vendors of security solutions for endpoint users(IDC, 2014). Since 1997 Kaspersky has been an
innovator in cybersecurity and provides effective digital
security solutions and threat intelligence for large
enterprises, SMBs and consumers. Kaspersky is aninternational company, operating in almost 200 countries
and territories across the globe, providing protection forover 400 million users worldwide.
https://www.kaspersky.com/ICSA Labs
1000 Bent Creek Blvd., Suite 200
Mechanicsburg, PA 17050

Kaspersky Americas Headquarters
500 Unicorn Park, 3rd Floor
Woburn, MA 01801

ATD-KASPERSKY-2019-1004-01

Page 10 of 10

Copyright © 2019 ICSA Labs. All rights reserved.

Criteria v.1.0