Document: Consumer Security Products Performance Benchmarks (Edition 1)
Authors:
M. Baquiran, D. Wren
Company: PassMark Software
Date:
4 September 2015
Edition:1
File:
Consumer_Security_Products_Performance_Benchmarks_2016_Ed_1.docx

Consumer Security Performance Benchmarks 2016

PassMark Software

TABLE OF CONTENTS......................................................................................................................................... 2
REVISION HISTORY............................................................................................................................................ 3
REFERENCES...................................................................................................................................................... 3
EXECUTIVE SUMMARY ...................................................................................................................................... 4
PRODUCTS AND VERSIONS ............................................................................................................................... 5PERFORMANCE METRICS SUMMARY ................................................................................................................ 6
INTERNET SECURITY SOFTWARE – TEST RESULTS .............................................................................................. 9
BENCHMARK 1 – BOOT TIME (SECONDS) ..................................................................................................................... 9
BENCHMARK 2 – SCAN TIME (SECONDS) ...................................................................................................................... 9
BENCHMARK 3 – USER INTERFACE LAUNCH TIME (MILLISECONDS) .................................................................................... 9
BENCHMARK 4 – MEMORY USAGE DURING SYSTEM IDLE (MEGABYTES) ........................................................................... 10
BENCHMARK 5 – MEMORY USAGE DURING INITIAL SCAN .............................................................................................. 10
BENCHMARK 6 – BROWSE TIME (SECONDS)................................................................................................................ 10
BENCHMARK 7 – INTERNET EXPLORER LAUNCH TIME (MILLISECONDS) ............................................................................. 11
BENCHMARK 8 – FILE COPY, MOVE AND DELETE (SECONDS) .......................................................................................... 11
BENCHMARK 9 – INSTALLATION OF THIRD PARTY APPLICATIONS (SECONDS) ..................................................................... 11
BENCHMARK 10 – NETWORK THROUGHPUT (SECONDS) ............................................................................................... 12
BENCHMARK 11 – FILE FORMAT CONVERSION (SECONDS)............................................................................................. 12
BENCHMARK 12 – FILE COMPRESSION AND DECOMPRESSION (SECONDS)......................................................................... 12
BENCHMARK 13 – FILE DOWNLOAD (SECONDS) .......................................................................................................... 13
BENCHMARK 14 – PE SCAN TIME (SECONDS) ............................................................................................................. 13
BENCHMARK 15 – FILE COPY DISK TO DISK (SECONDS) ................................................................................................. 13
BENCHMARK 16 – FILE COPY OVER NETWORK (SECONDS) ............................................................................................ 14
BENCHMARK 17 – PCMARK .................................................................................................................................... 14
BENCHMARK 18 – WORD DOCUMENT LAUNCH AND OPEN TIME (MILLISECONDS) ............................................................. 14
BENCHMARK 19 – RUN EXCEL MACRO (SECONDS) ...................................................................................................... 15
BENCHMARK 20 – SAVE WORD DOCUMENT TO PDF (SECONDS) .................................................................................... 15
BENCHMARK 21– USB 3.0 FILE COPY (SECONDS) ....................................................................................................... 15
DISCLAIMER AND DISCLOSURE ....................................................................................................................... 16
CONTACT DETAILS .......................................................................................................................................... 16
APPENDIX 1 – TEST ENVIRONMENT ................................................................................................................ 17

Performance Benchmark
Edition 1

Page 2 of 17
11 September 2015

Consumer Security Performance Benchmarks 2016

Rev
Edition 1

Revision History

Date

Initial version of this report.

Ref #

1

PassMark Software

Document

What Really Slows Windows Down (URL)

Performance Benchmark
Edition 1

4 September 2015

AuthorO. Warner,
The PC Spy

Date

2001-2015

Page 3 of 17
11 September 2015

Consumer Security Performance Benchmarks 2016

PassMark Software

PassMark Software® conducted objective performance testing on Norton Security and on Windows Defender. This
report presents our results and findings as a result of performance benchmark testing conducted on these two
products.
For more details on which versions were tested, please see the section “Products and Versions”.
Testing included twenty-one (21) performance metrics. These performance metrics are as follows: Boot Time; Scan Time; User Interface Launch Time; Memory Usage during System Idle;
 Memory Usage during Initial Scan; Browse Time; Internet Explorer Launch Time; File Copy, Move and Delete; Installation of Third Party Applications; Network Throughput (previously named “Binary Download Test”)
 File Format Conversion;
 File Compression and Decompression; PE Scan Time; File Copy Disk to Disk; File Copy Over Network; File Download; PCMark; Word Document Launch and Open Time; Run Excel Macro; Save Word Document as PDF; and USB 3.0 File Copy.

Performance Benchmark
Edition 1

Page 4 of 17
11 September 2015

Consumer Security Performance Benchmarks 2016

PassMark Software

The names and versions of the two security products tested are as follows:

Manufacturer

Product Name

Release Year

Product Version

Date Tested

Symantec Corporation

Norton Internet Security

2015

22.5.2.15

Aug 2015

Microsoft Corporation

Windows Defender2015

4.8.10240.16384

Aug 2015

Performance Benchmark
Edition 1

Page 5 of 17
11 September 2015

Consumer Security Performance Benchmarks 2016

PassMark Software

We have selected a set of objective metrics which provide a comprehensive and realistic indication of the areas inwhich an antivirus may impact system performance for end users. Our metrics test the impact of the antivirus
software on common tasks that end-users would perform on a daily basis.
All of PassMark Software’s test methods can be replicated by third parties using the same environment to obtain
similar benchmark results. Detailed descriptions of the methodologies used in our tests are available as “Appendix
2 – Methodology Description” of this report.

This metric measures the amount of time taken for the machine to boot into the operating system. Security
software is generally launched at Windows startup, adding an additional amount of time and delaying the startupof the operating system. Shorter boot times indicate that the application has had less impact on the normal
operation of the machine.

All antivirus solutions have functionality designed to detect viruses and various other forms of malware byscanning files on the system. This metric measured the amount of time required to scan a set of clean files. Our
sample file set comprised a total file size of 982 MB and was made up of files that would typically be found on enduser machines, such as media files, system files and Microsoft Office documents.

This metric provides an objective indication as to how responsive a security product appears to the user, bymeasuring the amount of time it takes for the user interface of the antivirus software to launch from Windows.
To allow for caching effects by the operating system, both the initial launch time and the subsequent launch times
were measured. Our final result is an average of these two measurements.

This metric measures the amount of memory (RAM) used by the product while the machine and antivirus softwareare in an idle state. The total memory usage was calculated by identifying all antivirus software processes and theamount of memory used by each process.
The amount of memory used while the machine is idle provides a good indication of the amount of system
resources being consumed by the antivirus software on a permanent basis. Better performing products occupy
less memory while the machine is idle.

This metric measures the amount of memory (RAM) used by the product during an initial security scan. The total
memory usage was calculated by identifying all security software processes and the amount of memory used byeach process during the scan.

Performance Benchmark
Edition 1

Page 6 of 17
11 September 2015

Consumer Security Performance Benchmarks 2016

PassMark Software

It is common behavior for security products to scan data for malware as it is downloaded from the internet or
intranet. This behavior may negatively impact browsing speed as products scan web content for malware. This
metric measures the time taken to browse a set of popular internet sites to consecutively load from a local server
in a user’s browser window.

This metric is one of many methods to objectively measure how much a security product impacts on the
responsiveness of the system. This metric measures the amount of time it takes to launch the user interface ofInternet Explorer 10. To allow for caching effects by the operating system, both the initial launch time and thesubsequent launch times were measured. Our final result is an average of these two measurements.

This metric measures the amount of time taken to move, copy and delete a sample set of files. The sample file set
contains several types of file formats that a Windows user would encounter in daily use. These formats include
documents (e.g. Microsoft Office documents, Adobe PDF, Zip files, etc), media formats (e.g. images, movies andmusic) and system files (e.g. executables, libraries, etc).

This metric measures the amount of time taken to install and uninstall third party programs. The installation speedof third party applications may be impacted by antivirus behavior such as heuristics or real time malware scanning.

The metric measures the amount of time taken to download a variety of files from a local server using theHyperText Transfer Protocol (HTTP), which is the main protocol used on the web for browsing, linking and datatransfer. Files used in this test include file formats that users would typically download from the web, such asimages, archives, music files and movie files.

This test measures the amount of time taken to convert an MP3 file to a WAV and subsequently, convert the same
MP3 file to a WMA format.

This metric measures the amount of time taken to compress and decompress different types of files. Files formats
used in this test included documents, movies and images.

This test measures the amount of time taken to download a set of setup files from a local server using theHyperText Transfer Protocol (HTTP). The data set comprised a total file size of 290MB, and the formats used
include executables and Microsoft installation packages.

Performance Benchmark
Edition 1

Page 7 of 17
11 September 2015

Consumer Security Performance Benchmarks 2016

PassMark Software

All antivirus solutions have functionality designed to detect viruses and various other forms of malware byscanning files on the system. This metric measured the amount of time required to scan a set of PE (Portable
Executable) files. Our sample file set comprised a total file size of 2.03GB and consisted of .exe (329MB), .dll
(920MB) and .sys files (827MB).

This test measures the amount of time taken to copy files between two local drives. The data set comprised a total
file size of 5.44GB, and the formats used included documents, movies, images and executables.

This test measures the amount of time taken to copy files from a local drive to a local server. The data set
comprised a total file size of 5.44GB, and the formats used included documents, movies, images and executables.

This test measures the overall performance as an overall score over a range of areas, including memory, video,
gaming, music, communications, productivity and HDD performance.

This metric measures how much security software impacts on the responsiveness and performance of the system.
This metric measures the amount of time it takes to open a large, mixed media document with Microsoft Word.
To allow for caching effects by the operating system, both the initial launch time and the subsequent launch times
were measured. Our final result is an average of these two measurements.

This test measures the amount of time taken to open an Excel document and run a macro to perform a range of
operations. The test is run five times with a reboot in between each run. Our final result is taken as an average ofthese five measurements.

This test measures the amount of time taken to open a large Word document and save it in PDF format. The test
is run five times with a reboot in between each run. Our final result is taken as an average of these fivemeasurements.

This test measures the amount of time taken to copy files from a USB 3.0 drive to a local disk. The data set
comprised a total file size of 6.06GB, and the formats used included documents, movies, system files andexecutables.

Performance Benchmark
Edition 1

Page 8 of 17
11 September 2015

Consumer Security Performance Benchmarks 2016

PassMark Software

In the following charts, we have highlighted the results we obtained for Norton Internet Security in yellow. The
average has also been highlighted in blue for ease of comparison.

The following chart compares the average time taken for the system to boot (from a sample of five boots) for each
Internet Security product tested. Products with lower boot times are considered better performing products inthis category.

Windows Defender

7.2

Norton Security

10.9

0s

2s

4s

6s

8s

10 s

12 s

The following chart compares the average time taken to scan a set of 6159 files (totaling 982 MB) for each Internet
Security product tested. This time is calculated by averaging the initial (Run 1) and subsequent (Runs 2-5) scantimes. Products with lower scan times are considered better performing products in this category.

Norton Security

6.6

Windows Defender

95.4

0s

20 s

40 s

60 s

80 s

100 s

120 s

The following chart compares the average time taken to launch a product’s user interface. Products with lower
launch times are considered better performing products in this category.

Norton Security

67

Windows Defender
0 ms

Performance Benchmark
Edition 1

116

20 ms

40 ms

60 ms

80 ms

100 ms

120 ms

140 msPage 9 of 17
11 September 2015

Consumer Security Performance Benchmarks 2016

PassMark Software

The following chart compares the average amount of RAM in use by an Internet Security product during a periodof system idle. This average is taken from a sample of ten memory snapshots taken at roughly 60 seconds apartafter reboot. Products with lower idle RAM usage are considered better performing products in this category.

Norton Security

32.2

Windows Defender

98.9

0 MB

20 MB

40 MB

60 MB

80 MB

100 MB

120 MBThe following chart compares the average amount of RAM in use by an Internet Security product during a scan.
This average is taken from a sample of ten memory snapshots taken at roughly 12 seconds apart. Products withlower idle RAM usage are considered better performing products in this category.

Norton Security

136.8

Windows Defender

161.0

0s

20 s

40 s

60 s

80 s

100 s

120 s

140 s

160 s

180 s

The following chart compares the average time taken for Internet Explorer to successively load a set of popular
websites through the local area network from a local server machine. Products with lower browse times are
considered better performing products in this category.

Norton Security

29.6

Windows Defender

43.5

0s

5s

Performance Benchmark
Edition 1

10 s

15 s

20 s

25 s

30 s

35 s

40 s

45 s

50 s

Page 10 of 17
11 September 2015

Consumer Security Performance Benchmarks 2016

PassMark Software

The following chart compares the average launch times of Internet Explorer after rebooting the machine for each
Internet Security product we tested. Products with lower launch times are considered better performing productsin this category.

Windows Defender376

Norton Security

399

0 ms

50 ms

100 ms

150 ms200 ms

250 ms

300 ms

350 ms

400 ms

450 ms

The following chart compares the average time taken to copy, move and delete several sets of sample files foreach Internet Security product tested. Products with lower times are considered better performing products inthis category.

Norton Security

3.8

Windows Defender

7.9

0s

1s

2s

3s

4s

5s

6s

7s

8s

9s

The following chart compares the average time taken to install 3 different third party applications for each Internet
Security product tested. Products with lower times are considered better performing products in this category.

Norton Security

7.3

Windows Defender

10.4

0s

Performance Benchmark
Edition 1

2s

4s

6s

8s

10 s

12 s

Page 11 of 17
11 September 2015

Consumer Security Performance Benchmarks 2016

PassMark Software

The following chart compares the average time to download a sample set of common file types for each Internet
Security product tested. Products with lower times are considered better performing products in this category.

Windows Defender

6.1

Norton Security

9.0

0s

1s

2s

3s

4s

5s

6s

7s

8s

9s

10 s

The following chart compares the average time it takes for five sample files to be converted from one file format
to another (MP3  WMA, MP3  WAV) for each Internet Security product tested. Products with lower times are
considered better performing products in this category.

Norton Security

50.5

Windows Defender

50.6

0s

10 s

20 s

30 s

40 s

50 s

60 s

The following chart compares the average time it takes for sample files to be compressed and decompressed foreach Internet Security product tested. Products with lower times are considered better performing products inthis category.

Norton Security

38.9

Windows Defender

42.3

0s

Performance Benchmark
Edition 1

5s

10 s

15 s

20 s

25 s

30 s

35 s

40 s

45 s

Page 12 of 17
11 September 2015

Consumer Security Performance Benchmarks 2016

PassMark Software

The following chart compares the average time taken to download a set of setup files from a local server. The test
was performed 5 times, and the average of all 5 runs was taken as the result. Products with lower times are
considered better performing products in this category.*

Norton Security

3.9

Windows Defender

4.7

0s

1s

1s

2s

2s

3s

3s

4s

4s

5s

5s

The following chart compares the average time taken to scan a set of 6351 portable executable files (totaling 2076
MB) for each Internet Security product tested. This time is calculated by averaging the initial (Run 1) andsubsequent (Runs 2-5) scan times. Products with lower scan times are considered better performing products inthis category.

Norton Security

17.5

Windows Defender

58.6

0s

10 s

20 s

30 s

40 s

50 s

60 s

70 s

The following chart compares the average time taken to copy a total of 8,501 files, with a total file size of 5.44GB
files, from one local drive to another local drive for each Internet Security product tested. The test was performed
5 times, and the average of all 5 runs was taken as the result. Products with lower times are considered better
performing products in this category.

Norton Security

68.0

Windows Defender

93.8

0s

10 s

Performance Benchmark
Edition 1

20 s

30 s

40 s

50 s

60 s

70 s

80 s

90 s

100 s

Page 13 of 17
11 September 2015

Consumer Security Performance Benchmarks 2016

PassMark Software

The following chart compares the average time taken to transfer a total of 8,501 files over the local network, witha total file size of 5.44GB files, from a local drive on the test machine to a local server. The test was performed 5times, and the average of all 5 runs was taken as the result. Products with lower times are considered better
performing products in this category.

Norton Security

161.1

Windows Defender

193.1

0s

50 s

100 s

150 s

200 s

250 s

The following chart compares the average PCMark score for each Internet Security product tested. The test was
performed 2 times, and the average of both runs was taken as the result. Products with higher scores are
considered better performing products in this category.

Norton Security

3014

Windows Defender

3015

0

500

1000

1500

2000

2500

3000

3500

The following chart compares the average time taken to launch Microsoft Word and open a 10MB document.
Products with lower launch times are considered better performing products in this category.

Windows Defender

1805

Norton Security

0 ms

Performance Benchmark
Edition 1

2324

500 ms1,000 ms

1,500 ms

2,000 ms

2,500 ms

Page 14 of 17
11 September 2015

Consumer Security Performance Benchmarks 2016

PassMark Software

The following chart compares the average time taken to launch Microsoft Excel and run a macro. Products withlower times are considered better performing products in this category.

Windows Defender

8.5

Norton Security

8.8

0s

1s

2s

3s

4s

5s

6s

7s

8s

9s

10 s

The following chart compares the average time taken to launch Microsoft Word and open a 10MB document andsave it as a PDF. Products with lower times are considered better performing products in this category.

Norton Security

5.5

Windows Defender

5.8

0s

1s

2s

3s

4s

5s

6s

7s

The following chart compares the average time taken to copy a set of files from an external USB 3.0 drive to a local
disk. Products with lower times are considered better performing products in this category.

Norton Security

50.7

Windows Defender

91.2

0s

10 s

Performance Benchmark
Edition 1

20 s

30 s

40 s

50 s

60 s

70 s

80 s

90 s

100 s

Page 15 of 17
11 September 2015

Consumer Security Performance Benchmarks 2016

PassMark SoftwareThis report only covers the versions that are listed in the “Products and Versions” section of this report.

While every effort has been made to ensure that the information presented in this report is accurate, PassMark
Software Pty Ltd assumes no responsibility for errors, omissions, or out-of-date information and shall not be liable
in any manner whatsoever for direct, indirect, incidental, consequential, or punitive damages resulting from theavailability of, use of, access of, or inability to use this information.

Symantec Corporation funded the production of this report, selected the test metrics and list of products toinclude in this report, and supplied some of the test scripts used for the tests.

All trademarks are the property of their respective owners.

PassMark Software Pty Ltd
Level 5, 63 Foveaux St
Surry Hills, 2010
Sydney, Australia
Phone + 61 (2) 9690 0444
Fax

+ 61 (2) 9690 0445

Web

www.passmark.com

Performance Benchmark
Edition 1

Page 16 of 17
11 September 2015

Consumer Security Performance Benchmarks 2016

PassMark Software

For our testing, PassMark Software used a test environment running Windows 10 Home (64-bit) with the following
hardware specifications:

Model:

Lenovo H50W-50 i5

CPU:Intel Core i5-4460 CPU @ 3.20GHz 3.20 GHz

Video Card:

NVIDIA GeForce GT 705

RAM:

8GB DDR3 RAM

SSD (Main Boot Drive):

Intel SSD 730 Series 240GB

Network:

Gigabit (1GB/s) switch

Performance Benchmark
Edition 1

Page 17 of 17
11 September 2015