Document: 2015 Consumer Security Products Performance Benchmarks (Edition 3)
Authors:
M. Baquiran, D. Wren
Company: PassMark Software
Date:
30 March 2015
Edition:3
File:
totalsecuritysuites-mar2015.docx

2015 Consumer Security Products

PassMark Software

TABLE OF CONTENTS......................................................................................................................................... 2
REVISION HISTORY............................................................................................................................................ 4
EXECUTIVE SUMMARY ...................................................................................................................................... 5
OVERALL SCORE ................................................................................................................................................ 6
PRODUCTS AND VERSIONS ............................................................................................................................... 8
INTERNET SECURITY SOFTWARE .................................................................................................................................. 8
TOTAL SECURITY SOFTWARE ...................................................................................................................................... 8PERFORMANCE METRICS SUMMARY ................................................................................................................ 9
INTERNET SECURITY SOFTWARE – TEST RESULTS ............................................................................................ 13
BENCHMARK 1 – BOOT TIME (SECONDS) ................................................................................................................... 13
BENCHMARK 2 – SCAN TIME (SECONDS) .................................................................................................................... 14
BENCHMARK 3 – USER INTERFACE LAUNCH TIME (MILLISECONDS) .................................................................................. 15
BENCHMARK 4 – MEMORY USAGE DURING SYSTEM IDLE (MEGABYTES) ........................................................................... 16
BENCHMARK 5 – MEMORY USAGE DURING INITIAL SCAN .............................................................................................. 17
BENCHMARK 6 – BROWSE TIME (SECONDS)................................................................................................................ 18
BENCHMARK 7 – INTERNET EXPLORER LAUNCH TIME (MILLISECONDS) ............................................................................. 19
BENCHMARK 8 – INSTALLATION TIME (SECONDS) ........................................................................................................ 20
BENCHMARK 9 – INSTALLATION SIZE (MEGABYTES) ...................................................................................................... 21
BENCHMARK 10 – FILE COPY, MOVE AND DELETE (SECONDS)........................................................................................ 22
BENCHMARK 11– INSTALLATION OF THIRD PARTY APPLICATIONS (SECONDS) .................................................................... 23
BENCHMARK 12 – NETWORK THROUGHPUT (SECONDS) ............................................................................................... 24
BENCHMARK 13 – FILE FORMAT CONVERSION (SECONDS)............................................................................................. 25
BENCHMARK 14 – FILE COMPRESSION AND DECOMPRESSION (SECONDS)......................................................................... 26
BENCHMARK 15 – FILE DOWNLOAD (SECONDS) .......................................................................................................... 27
BENCHMARK 16 – PE SCAN TIME (SECONDS) ............................................................................................................. 28
BENCHMARK 17 – FILE COPY DISK TO DISK (SECONDS) ................................................................................................. 29
BENCHMARK 18 – FILE COPY OVER NETWORK (SECONDS) ............................................................................................ 30
BENCHMARK 19 – PCMARK .................................................................................................................................... 31
BENCHMARK 20 – WORD DOCUMENT LAUNCH AND OPEN TIME (MILLISECONDS) ............................................................. 32
BENCHMARK 21 – RUN EXCEL MACRO (SECONDS) ...................................................................................................... 33
BENCHMARK 22 – SAVE WORD DOCUMENT TO PDF (SECONDS) .................................................................................... 34
BENCHMARK 23– USB 3.0 FILE COPY (SECONDS) ....................................................................................................... 35
TOTAL SECURITY SOFTWARE – TEST RESULTS ................................................................................................. 36
BENCHMARK 1 – BOOT TIME (SECONDS) ................................................................................................................... 36
BENCHMARK 2 – SCAN TIME (SECONDS) .................................................................................................................... 36
BENCHMARK 3 – USER INTERFACE LAUNCH TIME (MILLISECONDS) .................................................................................. 37
BENCHMARK 4 – MEMORY USAGE DURING SYSTEM IDLE (MEGABYTES) ........................................................................... 37
BENCHMARK 5 – MEMORY USAGE DURING INITIAL SCAN .............................................................................................. 38
BENCHMARK 6 – BROWSE TIME (SECONDS)................................................................................................................ 38
BENCHMARK 7 – INTERNET EXPLORER LAUNCH TIME (MILLISECONDS) ............................................................................. 39
Performance Benchmark
Edition 3

Page 2 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

BENCHMARK 8 – INSTALLATION TIME (SECONDS) ........................................................................................................ 39
BENCHMARK 9 – INSTALLATION SIZE (MEGABYTES) ...................................................................................................... 40
BENCHMARK 10 – FILE COPY, MOVE AND DELETE (SECONDS)........................................................................................ 40
BENCHMARK 11– INSTALLATION OF THIRD PARTY APPLICATIONS (SECONDS) .................................................................... 41
BENCHMARK 12 – NETWORK THROUGHPUT (SECONDS) ............................................................................................... 41
BENCHMARK 13 – FILE FORMAT CONVERSION (SECONDS)............................................................................................. 42
BENCHMARK 14 – FILE COMPRESSION AND DECOMPRESSION (SECONDS)......................................................................... 42
BENCHMARK 15 – FILE DOWNLOAD (SECONDS) .......................................................................................................... 43
BENCHMARK 16 – PE SCAN TIME (SECONDS) ............................................................................................................. 43
BENCHMARK 17 – FILE COPY DISK TO DISK (SECONDS) ................................................................................................. 44
BENCHMARK 18 – FILE COPY OVER NETWORK (SECONDS) ............................................................................................ 44
BENCHMARK 19 – PCMARK .................................................................................................................................... 45
BENCHMARK 20 – WORD DOCUMENT LAUNCH AND OPEN TIME (MILLISECONDS) ............................................................. 45
BENCHMARK 21 – RUN EXCEL MACRO (SECONDS) ...................................................................................................... 46
BENCHMARK 22 – SAVE WORD DOCUMENT TO PDF (SECONDS) .................................................................................... 46
BENCHMARK 23– USB 3.0 FILE COPY (SECONDS) ....................................................................................................... 47
INTERNET SECURITY SOFTWARE ON SSD – TEST RESULTS ............................................................................... 48
BENCHMARK 1 – BOOT TIME (SECONDS) ................................................................................................................... 48
BENCHMARK 2 – SCAN TIME (SECONDS) .................................................................................................................... 48
BENCHMARK 10 – FILE COPY, MOVE & DELETE (SECONDS) ........................................................................................... 49
BENCHMARK 17 – FILE COPY DISK TO DISK (SECONDS)................................................................................................. 49
BENCHMARK 18– FILE COPY OVER NETWORK (SECONDS) ............................................................................................. 50
DISCLAIMER AND DISCLOSURE ....................................................................................................................... 51
CONTACT DETAILS .......................................................................................................................................... 51
APPENDIX 1 – TEST ENVIRONMENT ................................................................................................................ 52
APPENDIX 2 – METHODOLOGY DESCRIPTION ................................................................................................. 53

Performance Benchmark
Edition 3

Page 3 of 64
2 April 2015

2015 Consumer Security Products

Rev

PassMark Software

Revision History

Date

Edition 1

Initial version of this report.

17 September 2014

Edition 2

Second version of this report with total security category added.

5 February 2015

Edition 3

Third version of this report with internet security on SSD category added.

Performance Benchmark
Edition 3

2 April 2015

Page 4 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

PassMark Software® conducted objective performance testing on twelve (12) Internet Security software products
and five (5) Total Security software products on Windows 7 Ultimate Edition (64-bit) SP1 between August 2014
and March 2015. This report presents our results and findings as a result of performance benchmark testing
conducted for these consumer security products.
Subsequent editions of this report will include new products released for 2015 as they are made available. For
more details on which versions were tested, please see the section “Products and Versions”.
Testing was performed on all products using twenty-three (23) performance metrics. These performance metrics
are as follows: Boot Time; Scan Time; User Interface Launch Time; Memory Usage during System Idle;
 Memory Usage during Initial Scan; Browse Time; Internet Explorer Launch Time; Installation Time; Installation Size; File Copy, Move and Delete; Installation of Third Party Applications; Network Throughput (previously named “Binary Download Test”)
 File Format Conversion;
 File Compression and Decompression; PE Scan Time; File Copy Disk to Disk; File Copy Over Network; File Download; PCMark; Word Document Launch and Open Time; Run Excel Macro; Save Word Document as PDF; and USB 3.0 File Copy.

Performance Benchmark
Edition 3

Page 5 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

PassMark Software assigned every product a score depending on its ranking in each metric compared to otherproducts in the same category.

In the following table the highest possible score attainable is 276; in a hypothetical situation where a product has
attained first place in all 23 metrics. Internet Security products have been ranked by their overall scores:

Product Name

Overall Score

Norton Security

203

Kaspersky Internet Security

180

ESET Smart Security

169

Bitdefender Internet Security

157

McAfee Internet Security

153

AVG Internet Security

151

Avira Internet Security

147

Panda Internet Security

140

Avast Internet Security

127

F-Secure Internet Security

136

G Data Internet Security

122

Trend Micro Titanium Internet Security

109

In the following table the highest possible score attainable is 115; in a hypothetical situation where a product has
attained first place in all 23 metrics. Total Security products have been ranked by their overall scores:

Product Name

Performance Benchmark
Edition 3

Overall Score

Norton Security with Backup

91

Kaspersky PURE

80

McAfee Total Security

63

Bitdefender Total Security

58

Trend Micro Titanium Maximum Security

53

Page 6 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

In the following table the highest possible score attainable is 25; in a hypothetical situation where a product has
attained first place in all 5 (SSD) metrics. Internet Security products have been ranked by their overall scores:

Performance Benchmark
Edition 3

Product Name

Overall Score

Norton Security

21

Bitdefender Internet Security

20

Trend Micro Internet Security

15

Kaspersky Internet Security

10

McAfee Internet Security

9

Page 7 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

Unless otherwise specified, we have tested the full, retail release of the newest generation of security products.
The names and versions of products which were tested can be found below from each category table:

Manufacturer

Product Name

ReleaseYear

Product Version

Date Tested

Symantec Corporation

Norton Security

2014

22.0.2.17

Dec 2014

F-Secure Corporation

F-Secure Internet Security

2014

14.115 build 100

Aug 2014

Kaspersky Lab

Kaspersky Internet Security

2014

14.0.0.4651

Aug 2014

McAfee, Inc.

McAfee Internet Security

2014

Security Centre
13.6.1138

Sep 2014

Bitdefender Internet Security

2014

18.14.0.1088

Aug 2014

ESET

ESET Smart Security

2014

7.0.3.14

Sep 2014

AVG

AVG Internet Security

2014

2.02.1012

Sep 2014

Trend Micro Titanium Internet Security

2014

19.01.00

Sep 2014

G Data Software AG

G Data Internet Security

2014

25.0.1.2

Sep 2014

Panda Security SL

Panda Internet Security

2014

15.0.3

Oct 2014

Avira Operations

Avira Internet Security Suite2014

14.0.7.468

Jan 2015

Avast! Internet Security

2014

2015.10.0.2206

Nov 2014

Product Name

ReleaseYear

Product Version

Date Tested

Norton Security with Backup

2014

22.0.2.17

Dec 2014

Trend Micro Titanium Maximum
Security

2014

8.0.1133

Aug 2014

Kaspersky Lab

Kaspersky PURE

2014

3.0 13.0.2.558

Dec 2014

McAfee, Inc.

McAfee Total Security

2014

Security Centre
13.6.1138

Dec 2014

Bitdefender Total Security

2014

18.20.0.1429

Dec 2014

Softwin

Trend Micro

Avast

Manufacturer
Symantec Corporation
Trend Micro

Softwin

Performance Benchmark
Edition 3

Page 8 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

We have selected a set of objective metrics which provide a comprehensive and realistic indication of the areas inwhich an antivirus may impact system performance for end users. Our metrics test the impact of the antivirus
software on common tasks that end-users would perform on a daily basis.
All of PassMark Software’s test methods can be replicated by third parties using the same environment to obtain
similar benchmark results. Detailed descriptions of the methodologies used in our tests are available as “Appendix
2 – Methodology Description” of this report.

This metric measures the amount of time taken for the machine to boot into the operating system. Security
software is generally launched at Windows startup, adding an additional amount of time and delaying the startupof the operating system. Shorter boot times indicate that the application has had less impact on the normal
operation of the machine.

All antivirus solutions have functionality designed to detect viruses and various other forms of malware byscanning files on the system. This metric measured the amount of time required to scan a set of clean files. Our
sample file set comprised a total file size of 982 MB and was made up of files that would typically be found on enduser machines, such as media files, system files and Microsoft Office documents.

This metric provides an objective indication as to how responsive a security product appears to the user, bymeasuring the amount of time it takes for the user interface of the antivirus software to launch from Windows.
To allow for caching effects by the operating system, both the initial launch time and the subsequent launch times
were measured. Our final result is an average of these two measurements.

This metric measures the amount of memory (RAM) used by the product while the machine and antivirus softwareare in an idle state. The total memory usage was calculated by identifying all antivirus software processes and theamount of memory used by each process.
The amount of memory used while the machine is idle provides a good indication of the amount of system
resources being consumed by the antivirus software on a permanent basis. Better performing products occupy
less memory while the machine is idle.

This metric measures the amount of memory (RAM) used by the product during an initial security scan. The total
memory usage was calculated by identifying all security software processes and the amount of memory used byeach process during the scan.

Performance Benchmark
Edition 3

Page 9 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

It is common behavior for security products to scan data for malware as it is downloaded from the internet or
intranet. This behavior may negatively impact browsing speed as products scan web content for malware. This
metric measures the time taken to browse a set of popular internet sites to consecutively load from a local server
in a user’s browser window.

This metric is one of many methods to objectively measure how much a security product impacts on the
responsiveness of the system. This metric measures the amount of time it takes to launch the user interface ofInternet Explorer 10. To allow for caching effects by the operating system, both the initial launch time and thesubsequent launch times were measured. Our final result is an average of these two measurements.

The speed and ease of the installation process will strongly influence the user’s first impression of the antivirus
software. This test measures the minimum installation time required by the antivirus software to be fully
functional and ready for use by the end user. Lower installation times represent antivirus products which arequicker for a user to install.

In offering new features and functionality to users, antivirus software products tend to increase in size with eachnew release. Although new technologies push the size limits of hard drives each year, the growing disk spacerequirements of common applications and the increasing popularity of large media files (such as movies, photos
and music) ensure that a product's installation size will remain of interest to home users.
This metric aims to measure a product’s total installation size. This metric is defined as the total disk spaceconsumed by all new files added during a product's installation.

This metric measures the amount of time taken to move, copy and delete a sample set of files. The sample file set
contains several types of file formats that a Windows user would encounter in daily use. These formats include
documents (e.g. Microsoft Office documents, Adobe PDF, Zip files, etc), media formats (e.g. images, movies andmusic) and system files (e.g. executables, libraries, etc).

This metric measures the amount of time taken to install and uninstall third party programs. The installation speedof third party applications may be impacted by antivirus behavior such as heuristics or real time malware scanning.

The metric measures the amount of time taken to download a variety of files from a local server using theHyperText Transfer Protocol (HTTP), which is the main protocol used on the web for browsing, linking and datatransfer. Files used in this test include file formats that users would typically download from the web, such asimages, archives, music files and movie files.

Performance Benchmark
Edition 3

Page 10 of 64
2 April 2015

2015 Consumer Security Products

PassMark SoftwareThis test measures the amount of time taken to convert an MP3 file to a WAV and subsequently, convert the same
MP3 file to a WMA format.

This metric measures the amount of time taken to compress and decompress different types of files. Files formats
used in this test included documents, movies and images.

This test measures the amount of time taken to download a set of setup files from a local server using theHyperText Transfer Protocol (HTTP). The data set comprised a total file size of 290MB, and the formats used
include executables and Microsoft installation packages.

All antivirus solutions have functionality designed to detect viruses and various other forms of malware byscanning files on the system. This metric measured the amount of time required to scan a set of PE (Portable
Executable) files. Our sample file set comprised a total file size of 2.03GB and consisted of .exe (329MB), .dll
(920MB) and .sys files (827MB).

This test measures the amount of time taken to copy files between two local drives. The data set comprised a total
file size of 5.44GB, and the formats used included documents, movies, images and executables.

This test measures the amount of time taken to copy files from a local drive to a local server. The data set
comprised a total file size of 5.44GB, and the formats used included documents, movies, images and executables.

This test measures the overall performance as an overall score over a range of areas, including memory, video,
gaming, music, communications, productivity and HDD performance.

This metric measures how much security software impacts on the responsiveness and performance of the system.
This metric measures the amount of time it takes to open a large, mixed media document with Microsoft Word.
To allow for caching effects by the operating system, both the initial launch time and the subsequent launch times
were measured. Our final result is an average of these two measurements.

This test measures the amount of time taken to open an Excel document and run a macro to perform a range of
operations. The test is run five times with a reboot in between each run. Our final result is taken as an average ofthese five measurements.

Performance Benchmark
Edition 3

Page 11 of 64
2 April 2015

2015 Consumer Security Products

PassMark SoftwareThis test measures the amount of time taken to open a large Word document and save it in PDF format. The test
is run five times with a reboot in between each run. Our final result is taken as an average of these fivemeasurements.

This test measures the amount of time taken to copy files from a USB 3.0 drive to a local disk. The data set
comprised a total file size of 6.06GB, and the formats used included documents, movies, system files andexecutables.

Performance Benchmark
Edition 3

Page 12 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

In the following charts, we have highlighted the results we obtained for Norton Security in yellow. The average
has also been highlighted in blue for ease of comparison.

The following chart compares the average time taken for the system to boot (from a sample of five boots) for eachsecurity product tested. Products with lower boot times are considered better performing products in thiscategory.

McAfee IS

18.5

ESET SS

20.4

Avast IS

20.5

Norton Security

20.8

Panda IS

21.6

G Data IS

23.6

Trend Micro Titanium IS

23.7

Average

23.7

AVG IS

25.2

Kaspersky Internet Security

25.7

Bitdefender IS

27.3

Avira IS

28.1

F-Secure IS

29.0

0s

Performance Benchmark
Edition 3

5s

10 s

15 s

20 s

25 s

30 s

35 s

Page 13 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to scan a set of 6159 files (totaling 982 MB) for each security
product tested. This time is calculated by averaging the initial (Run 1) and subsequent (Runs 2-5) scan times.
Products with lower scan times are considered better performing products in this category.

Kaspersky Internet Security

7.0

Bitdefender IS

9.4

AVG IS

10.6

Norton Security

10.8

ESET SS

18.0

G Data IS

19.8

F-Secure IS

23.2

McAfee IS

25.8

Average

33.6

Avira IS

44.4

Avast IS

56.2

Panda IS

69.3

Trend Micro Titanium IS

108.8

0s

Performance Benchmark
Edition 3

20 s

40 s

60 s

80 s

100 s

120 s

Page 14 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to launch a product’s user interface. Products with lower
launch times are considered better performing products in this category.

ESET SS

92

Kaspersky Internet Security

111

F-Secure IS

145

Norton Security

158

Trend Micro Titanium IS

242

Bitdefender IS

341

Avira IS362

G Data IS

369

Average

384

McAfee IS

396

Avast IS403

AVG IS

758

Panda IS

0 ms

Performance Benchmark
Edition 31230

200 ms

400 ms

600 ms

800 ms1,000 ms

1,200 ms

1,400 ms

Page 15 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average amount of RAM in use by a security product during a period of system
idle. This average is taken from a sample of ten memory snapshots taken at roughly 60 seconds apart after reboot.
Products with lower idle RAM usage are considered better performing products in this category.

Norton Security

29.7

Panda IS

30.5

Avast IS

Kaspersky Internet Security

53.9

80.9

Avira IS

90.3

F-Secure IS

93.0

G Data IS

104.7

ESET SS

118.6

Average

127.5

Trend Micro Titanium IS

211.5

McAfee IS

218.6

AVG IS

243.9

Bitdefender IS
0 MB

Performance Benchmark
Edition 3

254.2

50 MB

100 MB

150 MB

200 MB250 MB

300 MB

Page 16 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average amount of RAM in use by a security product during a scan. This average
is taken from a sample of ten memory snapshots taken at roughly 12 seconds apart. Products with lower idle RAM
usage are considered better performing products in this category.

Panda IS

67.7

ESET SS

77.4

Norton Security

105.3

Avast IS

110.2

F-Secure IS

Kaspersky Internet Security

154.1

168.5

Average

215.7

Trend Micro Titanium IS

235.2

Avira IS

253.7

McAfee IS

255.5

G Data IS

275.4

Bitdefender IS

391.2

AVG IS
0 MB

Performance Benchmark
Edition 3

494.3

100 MB

200 MB

300 MB

400 MB500 MB

600 MB

Page 17 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken for Internet Explorer to successively load a set of popular
websites through the local area network from a local server machine. Products with lower browse times are
considered better performing products in this category.*

McAfee IS

25.1

ESET SS

25.6

Avira IS

27.6

Avast IS

28.1

Norton Security

28.2

AVG IS

36.7

Panda IS37.2

G Data IS

38.2

Average39.2

Trend Micro Titanium IS

41.5

F-Secure IS

43.5

Bitdefender IS

99.7

0s

20 s

40 s

60 s

80 s

100 s

120 s

*Kaspersky was omitted from the above comparison chart due to the test script being blocked by the software.

Performance Benchmark
Edition 3

Page 18 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average launch times of Internet Explorer after rebooting the machine for eachsecurity product we tested. Products with lower launch times are considered better performing products in thiscategory.

AVG IS

714

Bitdefender IS

1203

Panda IS

1262

ESET SS

1343

Kaspersky Internet Security

1431

Avira IS

1526

Average

1528

F-Secure IS

1529

Norton Security

1570

McAfee IS

1734

Trend Micro Titanium IS

1990

Avast IS2014

G Data IS

2018

0 ms

Performance Benchmark
Edition 3

500 ms1,000 ms

1,500 ms

2,000 ms

2,500 ms

Page 19 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the minimum installation time it takes for security products to be fully functionaland ready for use by the end user. Products with lower installation times are considered better performing
products in this category.

Norton Security

65

G Data IS

87

Kaspersky Internet Security

92

Panda IS110

AVG IS

152

Trend Micro Titanium IS

181

Average

198

Avira IS209

ESET SS218

F-Secure IS

288

Avast IS

347

Bitdefender IS

359

McAfee IS

572

0s

Performance Benchmark
Edition 3

100 s

200 s

300 s

400 s

500 s

600 s

700 s

Page 20 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the total size of files added during the installation of security products. Productswith lower installation sizes are considered better performing products in this category.

F-Secure IS

Panda IS

238.5

278.2

Norton Security

348.5

ESET SS

361.0

McAfee IS

411.7

AVG IS

476.5

Avira IS

494.5

Kaspersky Internet Security

509.1

Average615.7

Trend Micro Titanium IS

872.0

G Data IS

881.6

Avast IS

1000.2

Bitdefender IS

0 MB

Performance Benchmark
Edition 3

1516.8200 MB

400 MB

600 MB

800 MB 1,000 MB 1,200 MB 1,400 MB 1,600 MB

Page 21 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to copy, move and delete several sets of sample files foreach security product tested. Products with lower times are considered better performing products in thiscategory.

Kaspersky Internet Security

6.7

Norton Security

6.8

G Data IS

7.1

Avast IS

7.4

McAfee IS

7.7

Trend Micro Titanium IS

8.0

Bitdefender IS

8.0

Average

9.0

AVG IS

9.1

ESET SS

9.1

Panda IS

11.1

F-Secure IS

13.3

Avira IS

13.8

0s

Performance Benchmark
Edition 3

2s

4s

6s

8s

10 s

12 s

14 s

16 s

Page 22 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to install 3 different third party applications for each security
product tested. Products with lower times are considered better performing products in this category.

Bitdefender IS

13.2

Avast IS

17.6

Avira IS

19.1

Kaspersky Internet Security

19.2

ESET SS

19.6

G Data IS

20.4

Norton Security

20.4

Panda IS

20.9

Average21.5

Trend Micro Titanium IS

24.5

McAfee IS

24.7

AVG IS

27.8

F-Secure IS

30.5

0s

Performance Benchmark
Edition 3

5s

10 s

15 s

20 s

25 s

30 s

35 s

Page 23 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time to download a sample set of common file types for each security
product tested. Products with lower times are considered better performing products in this category.

Avira IS

6.5

McAfee IS

7.6

Norton Security

8.1

Kaspersky Internet Security

8.2

Bitdefender IS

8.8

Avast IS

9.5

AVG IS

10.4

Panda IS

12.1

Average

14.3

F-Secure IS

15.6

Trend Micro Titanium IS

16.2

ESET SS

17.5

G Data IS

51.3

0s

Performance Benchmark
Edition 3

10 s

20 s

30 s

40 s

50 s

60 s

Page 24 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time it takes for five sample files to be converted from one file format
to another (MP3  WMA, MP3  WAV) for each security product tested. Products with lower times are consideredbetter performing products in this category.

ESET SS

58.0

AVG IS

58.0

Bitdefender IS

58.1

McAfee IS

58.4

Panda IS

58.5

Norton Security

58.6

Kaspersky Internet Security

58.6

Trend Micro Titanium IS

58.9

Average

58.9

Avira IS59.0

G Data IS

59.4

F-Secure IS

60.3

Avast IS

61.3

56 s

Performance Benchmark
Edition 3

57 s

58 s

59 s

60 s

61 s

62 s

Page 25 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time it takes for sample files to be compressed and decompressed foreach security product tested. Products with lower times are considered better performing products in thiscategory.

Kaspersky Internet Security

44.1

Bitdefender IS

44.5

McAfee IS

44.7

Norton Security

44.8

Panda IS

45.1

AVG IS

45.2

ESET SS

46.2

Average

46.4

Avast IS

47.0

F-Secure IS

47.0

Trend Micro Titanium IS

48.2

G Data IS

48.6

Avira IS51.3

0s

Performance Benchmark
Edition 3

10 s

20 s

30 s

40 s

50 s

60 s

Page 26 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to download a set of setup files from a local server. The test
was performed 5 times, and the average of all 5 runs was taken as the result. Products with lower times are
considered better performing products in this category.*

Trend Micro Titanium IS

5.3

F-Secure IS

5.4

McAfee IS

5.5

Avira IS

5.6

Kaspersky Internet Security

5.8

Norton Security

6.2

AVG IS

7.4

Panda IS

8.7

ESET SS

10.3

Average

15.1

Bitdefender IS

28.0

G Data IS

77.6

0s

10 s

20 s

30 s

40 s

50 s

60 s

70 s

80 s

90 s

*Avast was excluded from the above comparison chart due to the test script being blocked by the software.

Performance Benchmark
Edition 3

Page 27 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to scan a set of 6351 portable executable files (totaling 2076
MB) for each security product tested. This time is calculated by averaging the initial (Run 1) and subsequent (Runs2-5) scan times. Products with lower scan times are considered better performing products in this category.

Bitdefender IS

16.5

Norton Security

17.1

AVG IS

22.9

McAfee IS

28.6

G Data IS

31.8

Kaspersky Internet Security

32.9

F-Secure IS

40.6

ESET SS

50.8

Average

51.1

Avira IS

81.9

Panda IS

89.5

Avast IS

98.6

Trend Micro Titanium IS

102.3

0s

Performance Benchmark
Edition 3

20 s

40 s

60 s

80 s

100 s

120 s

Page 28 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to copy a total of 8,501 files, with a total file size of 5.44GB
files, from one local drive to another local drive for each security product tested. The test was performed 5 times,
and the average of all 5 runs was taken as the result. Products with lower times are considered better performing
products in this category.

Avast IS

48.7

G Data IS

57.9

Kaspersky Internet Security

58.8

AVG IS

60.0

Bitdefender IS

60.3

McAfee IS

61.3

Trend Micro Titanium IS

62.9

Norton Security

64.3

Average

68.3

F-Secure IS

68.4

Avira IS

72.4

ESET SS

85.4

Panda IS

119.2

0s

Performance Benchmark
Edition 3

20 s

40 s

60 s

80 s

100 s

120 s

140 s

Page 29 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to transfer a total of 8,501 files over the local network, witha total file size of 5.44GB files, from a local drive on the test machine to a local server. The test was performed 5times, and the average of all 5 runs was taken as the result. Products with lower times are considered better
performing products in this category.

F-Secure IS

171.3

G Data IS

171.7

Kaspersky Internet Security

175.1

Norton Security

181.6

AVG IS

191.9

McAfee IS

193.7

Bitdefender IS

195.9

ESET SS

199.3

Avast IS

212.3

Average

213.2

Avira IS

228.2

Trend Micro Titanium IS

268.8

Panda IS

368.3

0s

Performance Benchmark
Edition 3

50 s

100 s

150 s

200 s

250 s

300 s

350 s

400 s

Page 30 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average PCMark score for each security product tested. The test was performed
2 times, and the average of both runs was taken as the result. Products with higher scores are considered better
performing products in this category.

McAfee IS

7922

G Data IS

9361

Trend Micro Titanium IS

9708

Average9732

AVG IS

9753

Avast IS

9755

Panda IS

9912

Bitdefender IS

9919

Norton Security

9930

Avira IS

9972

ESET SS

10104

Kaspersky Internet Security

10155

F-Secure IS

10289

0

Performance Benchmark
Edition 3

2000

4000

6000

8000

10000

12000

Page 31 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to launch Microsoft Word and open a 10MB document.
Products with lower launch times are considered better performing products in this category.

Bitdefender IS

4638

AVG IS

4750

Norton Security

4775

ESET SS

4836

Avira IS4853

Trend Micro Titanium IS

4880

Avast IS

4921

Average4981

Panda IS

5035

F-Secure IS

5112

McAfee IS

5208

Kaspersky Internet Security

5244

G Data IS

4,000 ms

Performance Benchmark
Edition 35516

4,200 ms

4,400 ms

4,600 ms

4,800 ms

5,000 ms

5,200 ms

5,400 ms

5,600 ms

Page 32 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to launch Microsoft Excel and run a macro. Products withlower times are considered better performing products in this category.

F-Secure IS

10.6

Trend Micro Titanium IS

10.7

Avira IS

10.8

AVG IS

11.1

G Data IS

11.1

Bitdefender IS

11.2

Average11.3

Panda IS

11.4

ESET SS

11.5

McAfee IS

11.6

Avast IS

11.8

Norton Security

11.9

Kaspersky Internet Security

12.3

0s

Performance Benchmark
Edition 3

2s

4s

6s

8s

10 s

12 s

14 s

Page 33 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to launch Microsoft Word and open a 10MB document andsave it as a PDF. Products with lower times are considered better performing products in this category.

Norton Security

10.1

ESET SS

10.1

Avira IS

10.3

Kaspersky Internet Security

10.3

Panda IS

10.5

McAfee IS

10.7

AVG IS

10.8

Average

10.9

Bitdefender IS

11.0

G Data IS

11.3

F-Secure IS

11.9

Avast IS

12.0

Trend Micro Titanium IS

12.3

0s

Performance Benchmark
Edition 3

2s

4s

6s

8s

10 s

12 s

14 s

Page 34 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to copy a set of files from an external USB 3.0 drive to a local
disk. Products with lower times are considered better performing products in this category.

Norton Security

85.9

McAfee IS

87.4

Bitdefender IS

101.6

Avira IS

112.4

Avast IS

114.5

Kaspersky Internet Security

128.9

Panda IS

133.8

Average

143.2

ESET SS

143.6

G Data IS

144.9

AVG IS

150.2

F-Secure IS

253.2

Trend Micro Titanium IS

262.0

0s

Performance Benchmark
Edition 3

50 s

100 s

150 s

200 s

250 s

300 s

Page 35 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

In the following charts, we have highlighted the results we obtained for Norton Security with Backup in yellow. The
average has also been highlighted in blue for ease of comparison.

The following chart compares the average time taken for the system to boot (from a sample of five boots) for eachsecurity product tested. Products with lower boot times are considered better performing products in thiscategory.

McAfee TS

19.1

Kaspersky PURE

20.2

Norton Security w/ Backup

20.4

Average

20.8

Trend Micro TMS

21.6

Bitdefender TS

22.5

0s

5s

10 s

15 s

20 s

25 s

The following chart compares the average time taken to scan a set of 6159 files (totaling 982 MB) for each security
product tested. This time is calculated by averaging the initial (Run 1) and subsequent (Runs 2-5) scan times.
Products with lower scan times are considered better performing products in this category.

Kaspersky PURE

4.8

McAfee TS

10.4

Norton Security w/ Backup

10.6

Bitdefender TS

12.2

Average

28.6

Trend Micro TMS

105

0s

Performance Benchmark
Edition 3

20 s

40 s

60 s

80 s

100 s

120 s

Page 36 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to launch a product’s user interface. Products with lower
launch times are considered better performing products in this category.

Norton Security w/ Backup

56

Trend Micro TMS

137

Average

317

Bitdefender TS

374

McAfee TS

472

Kaspersky PURE

545

0 ms

100 ms200 ms

300 ms

400 ms

500 ms

600 ms

The following chart compares the average amount of RAM in use by a security product during a period of system
idle. This average is taken from a sample of ten memory snapshots taken at roughly 60 seconds apart after reboot.
Products with lower idle RAM usage are considered better performing products in this category.

Norton Security w/ Backup

Kaspersky PURE

33.7

53.1

Average

158.8

Trend Micro TMS

194.9

McAfee TS

254.1

Bitdefender TS

258.1

0 MB

Performance Benchmark
Edition 3

50 MB

100 MB

150 MB

200 MB250 MB

300 MB

Page 37 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average amount of RAM in use by a security product during a scan. This average
is taken from a sample of ten memory snapshots taken at roughly 12 seconds apart. Products with lower idle RAM
usage are considered better performing products in this category.

Norton Security w/ Backup

36.4

Kaspersky PURE

88.0

Average

187.6

Trend Micro TMS

208.4

Bitdefender TS

237.7

McAfee TS

367.6

0 MB

50 MB

100 MB

150 MB

200 MB250 MB

300 MB

350 MB

400 MBThe following chart compares the average time taken for Internet Explorer to successively load a set of popular
websites through the local area network from a local server machine. Products with lower browse times are
considered better performing products in this category.

Norton Security w/ Backup

21.7

McAfee TS

28.8

Kaspersky PURE

32.4

Trend Micro TMS

35.5

Average

41.0

Bitdefender TS

86.8

0s

Performance Benchmark
Edition 3

10 s

20 s

30 s

40 s

50 s

60 s

70 s

80 s

90 s

100 s

Page 38 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average launch times of Internet Explorer after rebooting the machine for eachsecurity product we tested. Products with lower launch times are considered better performing products in thiscategory.

Kaspersky PURE

1388

Norton Security w/ Backup

1441

Bitdefender TS

1496

Average

1664

Trend Micro TMS

1940

McAfee TS

2056

0 ms

500 ms1,000 ms

1,500 ms

2,000 ms

2,500 ms

The following chart compares the minimum installation time it takes for security products to be fully functionaland ready for use by the end user. Products with lower installation times are considered better performing
products in this category.

Norton Security w/ Backup

76

Kaspersky PURE

121

Trend Micro TMS

258

Average

288

Bitdefender TS

367

McAfee TS

618

0s

Performance Benchmark
Edition 3

100 s

200 s

300 s

400 s

500 s

600 s

700 s

Page 39 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the total size of files added during the installation of security products. Productswith lower installation sizes are considered better performing products in this category.

McAfee TS

380.8

Norton Security w/ Backup

467.2

Kaspersky PURE

600.5

Average765.7

Trend Micro TMS

884.7

Bitdefender TS

1495.40 MB

200 MB

400 MB

600 MB

800 MB

1,000 MB 1,200 MB 1,400 MB 1,600 MB

The following chart compares the average time taken to copy, move and delete several sets of sample files foreach security product tested. Products with lower times are considered better performing products in thiscategory.

Norton Security w/ Backup

6.6

Kaspersky PURE

6.9

Average

7.4

McAfee TS

7.6

Trend Micro TMS

7.8

Bitdefender TS

8.2

0s

Performance Benchmark
Edition 3

1s

2s

3s

4s

5s

6s

7s

8s

9s

Page 40 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to install 3 different third party applications for each security
product tested. Products with lower times are considered better performing products in this category.

Bitdefender TS

12.9

Kaspersky PURE

18.6

Average

19.3

Norton Security w/ Backup

20.1

McAfee TS

21.6

Trend Micro TMS

23.5

0s

5s

10 s

15 s

20 s

25 s

The following chart compares the average time to download a sample set of common file types for each security
product tested. Products with lower times are considered better performing products in this category.

McAfee TS

7.5

Kaspersky PURE

8.0

Bitdefender TS

8.1

Norton Security w/ Backup

8.1

Average

9.1

Trend Micro TMS

13.8

0s

Performance Benchmark
Edition 3

2s

4s

6s

8s

10 s

12 s

14 s

16 s

Page 41 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time it takes for five sample files to be converted from one file format
to another (MP3  WMA, MP3  WAV) for each security product tested. Products with lower times are consideredbetter performing products in this category.

Norton Security w/ Backup

58.3

Bitdefender TS

58.5

Average

58.7

McAfee TS

58.7

Trend Micro TMS

58.8

Kaspersky PURE

59.0

0s

10 s

20 s

30 s

40 s

50 s

60 s

70 s

The following chart compares the average time it takes for sample files to be compressed and decompressed foreach security product tested. Products with lower times are considered better performing products in thiscategory.

Norton Security w/ Backup

44.2

Bitdefender TS

44.6

McAfee TS

44.8

Kaspersky PURE

45.1

Average45.3

Trend Micro TMS

47.7

0s

Performance Benchmark
Edition 3

10 s

20 s

30 s

40 s

50 s

60 s

Page 42 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to download a set of setup files from a local server. The test
was performed 5 times, and the average of all 5 runs was taken as the result. Products with lower times are
considered better performing products in this category.

Kaspersky PURE

5.5

McAfee TS

5.6

Norton Security w/ Backup

5.7

Trend Micro TMS

6.0

Average

10.2

Bitdefender TS

28.0

0s

5s

10 s

15 s

20 s

25 s

30 s

The following chart compares the average time taken to scan a set of 6351 portable executable files (totaling 2076
MB) for each security product tested. This time is calculated by averaging the initial (Run 1) and subsequent (Runs2-5) scan times. Products with lower scan times are considered better performing products in this category.

Bitdefender TS

18.1

Kaspersky PURE

22.5

Norton Security w/ Backup

27.9

McAfee TS

42.0

Average42.1

Trend Micro TMS

99.9

0s

Performance Benchmark
Edition 3

20 s

40 s

60 s

80 s

100 s

120 s

Page 43 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to copy a total of 8,501 files, with a total file size of 5.44GB
files, from one local drive to another local drive for each security product tested. The test was performed 5 times,
and the average of all 5 runs was taken as the result. Products with lower times are considered better performing
products in this category.

Kaspersky PURE

59.6

Trend Micro TMS

60.5

Norton Security w/ Backup

60.6

Average

61.1

McAfee TS

61.7

Bitdefender TS

63.3

0s

10 s

20 s

30 s

40 s

50 s

60 s

70 s

The following chart compares the average time taken to transfer a total of 8,501 files over the local network, witha total file size of 5.44GB files, from a local drive on the test machine to a local server. The test was performed 5times, and the average of all 5 runs was taken as the result. Products with lower times are considered better
performing products in this category.

Kaspersky PURE

172.9

Norton Security w/ Backup

180.6

McAfee TS

187.1

Bitdefender TS

187.8

Average

193.5

Trend Micro TMS

239.3

0s

Performance Benchmark
Edition 3

50 s

100 s

150 s

200 s

250 s

300 s

Page 44 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average PCMark score for each security product tested. The test was performed
2 times, and the average of both runs was taken as the result. Products with higher scores are considered better
performing products in this category.

McAfee TS

9033

Trend Micro TMS

9484

Average

9640

Norton Security w/ Backup

9875

Bitdefender TS

9880

Kaspersky PURE

9932

0

2000

4000

6000

8000

10000

12000

The following chart compares the average time taken to launch Microsoft Word and open a 10MB document.
Products with lower launch times are considered better performing products in this category.

Norton Security w/ Backup

4636

Bitdefender TS

4682

Kaspersky PURE

Average

Trend Micro TMS

McAfee TS
0 ms

Performance Benchmark
Edition 3

5211

5364

5881

6413

1,000 ms 2,000 ms 3,000 ms 4,000 ms 5,000 ms 6,000 ms 7,000 ms

Page 45 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to launch Microsoft Excel and run a macro. Products withlower times are considered better performing products in this category.

Bitdefender TS

10.5

Average

11.4

Trend Micro TMS

11.4

McAfee TS

11.5

Kaspersky PURE

11.6

Norton Security w/ Backup

11.8

0s

2s

4s

6s

8s

10 s

12 s

14 s

The following chart compares the average time taken to launch Microsoft Word and open a 10MB document andsave it as a PDF. Products with lower times are considered better performing products in this category.

Norton Security w/ Backup

10.3

Trend Micro TMS

10.9

McAfee TS

11.1

Average

11.2

Bitdefender TS

11.6

Kaspersky PURE

12.2

0s

Performance Benchmark
Edition 3

2s

4s

6s

8s

10 s

12 s

14 s

Page 46 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to copy a set of files from an external USB 3.0 drive to a local
disk. Products with lower times are considered better performing products in this category.

Norton Security w/ Backup

83.1

Trend Micro TMS

87.8

McAfee TS

88.2

Kaspersky PURE

88.3

Average

91.1

Bitdefender TS

107.9

0s

Performance Benchmark
Edition 3

20 s

40 s

60 s

80 s

100 s

120 s

Page 47 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

In the following charts, we have highlighted the results we obtained for Norton Security in yellow. The average has
also been highlighted in blue for ease of comparison.

The following chart compares the average time taken for the system to boot (from a sample of five boots) for eachsecurity product tested. Products with lower boot times are considered better performing products in thiscategory.

Trend Micro IS

4.1

Norton Security

4.2

Kaspersky IS

5.1

McAfee IS

6.5

Average

8.2

Bitdefender IS

21.3

0s

5s

10 s

15 s

20 s

25 s

The following chart compares the average time taken to scan a set of 6159 files (totaling 982 MB) for each security
product tested. This time is calculated by averaging the initial (Run 1) and subsequent (Runs 2-5) scan times.
Products with lower scan times are considered better performing products in this category.

Bitdefender IS

6

Norton Security

7

Kaspersky IS

16

McAfee IS

23

Average25

Trend Micro IS

72

0s

Performance Benchmark
Edition 3

10 s

20 s

30 s

40 s

50 s

60 s

70 s

80 s

Page 48 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to copy, move and delete several sets of sample files foreach security product tested. Products with lower times are considered better performing products in thiscategory.

Norton Security

6.8

Kaspersky IS

7.4

Bitdefender IS

7.4

McAfee IS

7.7

Average

7.8

Trend Micro IS

9.5

0s

1s

2s

3s

4s

5s

6s

7s

8s

9s

10 s

The following chart compares the average time taken to copy a set of files from one local drive to another local
drive for each security product tested. The test was performed 5 times, and the average of all 5 runs was taken as
the result. Products with lower times are considered better performing products in this category.

Kaspersky IS

136.0

Norton Security

137.9

Bitdefender IS

138.5

McAfee IS

138.9

Average139.2

Trend Micro IS

144.6

0s

Performance Benchmark
Edition 3

20 s

40 s

60 s

80 s

100 s

120 s

140 s

160 s

Page 49 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

The following chart compares the average time taken to transfer a total of 8,501 files over the local network, witha total file size of 5.44GB files, from a local drive on the test machine to a local server. The test was performed 5times, and the average of all 5 runs was taken as the result. Products with lower times are considered better
performing products in this category.

Kaspersky IS

153.2

Norton Security

155.6

Bitdefender IS

157.8

McAfee IS

164.0

Average

173.0

Trend Micro IS

234.3

0s

Performance Benchmark
Edition 3

50 s

100 s

150 s

200 s

250 s

Page 50 of 64
2 April 2015

2015 Consumer Security Products

PassMark SoftwareThis report only covers versions of products that were available at the time of testing. The tested versions are as
noted in the “Products and Versions” section of this report. The products we have tested are not an exhaustive
list of all products available in these very competitive product categories.

While every effort has been made to ensure that the information presented in this report is accurate, PassMark
Software Pty Ltd assumes no responsibility for errors, omissions, or out-of-date information and shall not be liable
in any manner whatsoever for direct, indirect, incidental, consequential, or punitive damages resulting from theavailability of, use of, access of, or inability to use this information.

Symantec Corporation funded the production of this report, selected the test metrics and list of products toinclude in this report, and supplied some of the test scripts used for the tests.

All trademarks are the property of their respective owners.

PassMark Software Pty Ltd
Suite 202, Level 2
35 Buckingham St.
Surry Hills, 2010
Sydney, Australia
Phone + 61 (2) 9690 0444
Fax

+ 61 (2) 9690 0445

Web

www.passmark.com

An electronic copy of this report can be found at the following location:
http://www.passmark.com/isreport14

Performance Benchmark
Edition 3

Page 51 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

For our testing, PassMark Software used a test environment running Windows 7 Ultimate (64-bit) SP1 with thefollowing hardware specifications:

Model:

HP Pavilion P6-2300A

CPU:Intel Core i5-3330

Video Card:

1GB nVIDIA GeForce GT 620M

Motherboard:

Foxconn 2ABF 3.10

RAM:

6GB DDR3 RAM

HDD :Hitachi HDS721010CLA630 931.51GBnd

2 Drive :Samsung 1000GB 7200RPM Model: HD103UJ

Network:

Gigabit (1GB/s) switch

For network tests (wget, http), PassMark Software used a server with the following specifications:

CPU:Intel Xeon E3-1220v2 CPU

Video Card:

Kingston 8GB (2 x 4GB) ECC

Motherboard:Intel S1200BTL Server Motherboard

RAM:

4GB DDR3 RAM, 1333 Mhz

HDD :Samsung 1.5TB 7200RPM

Network:

Gigabit (1GB/s) switch

Performance Benchmark
Edition 3

Page 52 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

As with testing on Windows Vista, Norton Ghost was used to create a “clean” baseline image prior to testing. Our
aim is to create a baseline image with the smallest possible footprint and reduce the possibility of variation causedby external operating system factors.
The baseline image was restored prior to testing of each different product. This process ensures that we install
and test all products on the same, “clean” machine.
The steps taken to create the base Windows 7 image are as follows:1.

Installation and activation of Windows 7 Ultimate Edition.

2.

Disabled Automatic Updates.

3.

Changed User Account Control settings to “Never Notify”.

4.

Disable Windows Defender automatic scans to avoid unexpected background activity.

5.

Disable the Windows firewall to avoid interference with security software.

6.

Disabled Superfetch to ensure consistent results.

7.

Installed HTTP Watch for Browse Time testing.

8.

Installed Windows Performance Toolkit x64 for Boot Time testing.

9.

Installed Active Perl for interpretation of some test scripts.

10.

Install OSForensics for testing (Installation Size) purposes.

11.

Disabled updates in Internet Explorer 10.

12.

Update to Windows Service Pack 1

13.

Created a baseline image using Norton Ghost.

PassMark Software uses tools available from the Windows Performance Toolkit version 4.6 (as part of theMicrosoft Windows 7 SDK obtainable from the Microsoft Website) with a view to obtaining more precise andconsistent boot time results on the Windows 7 platform.
The boot process is first optimized with xbootmgr.exe using the command “xbootmgr.exe -trace boot –prepSystem” which prepares the system for the test over six optimization boots. The boot traces obtained fromthe optimization process are discarded.
After boot optimization, the benchmark is conducted using the command "xbootmgr.exe -trace boot -numruns 5”.
This command boots the system five times in succession, taking detailed boot traces for each boot cycle.
Finally, a post-processing tool was used to parse the boot traces and obtain the BootTimeViaPostBoot value. Thisvalue reflects the amount of time it takes the system to complete all (and only) boot time processes. Our final
result is an average of five boot traces.

Scan Time is the time it took for each product to scan a set of sample files. The sample used was identical in allcases and contained a mixture of system files and Office files. In total there were 6159 files whose combined size
Performance Benchmark
Edition 3

Page 53 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

was 982 MB. Most of these files come from the Windows system folders. As the file types can influence scanningspeed, the breakdown of the main file types, file numbers and total sizes of the files in the sample set is given
here:

File Extension

Number of Files

File Size

.dll
.exe
.sys
.gif
.doc
.wmf
.png
.html
.nls
.jpg
.ini
.ico
.mof
.ax
.xls
.ime
.drv
.txt
.chm
.cpl
.mfl
.inf
.hlp
.imd
.py
.msc
.vbs
.xml
.rtf
.ocx
.tsp
.com
.xsl
.h
.vsd
.scr
.aw
.js
.zip
.lex
.ppt
.acm
.wav
Total

2589
695
332
302
281
185
149
126
80
70
59
58
43
39
38
35
31
31
30
29
29
26
22
20
20
18
18
18
16
16
14
14
14
13
12
12
12
12
11
9
9
9
7
6159

490MB
102MB
23MB
1MB
64MB
2MB
2MB
1MB
6MB
1MB
2MB
<1MB
6MB
4MB
3MB
5MB
1MB
1MB
6MB
4MB
3MB
2MB
3MB
18MB
<1MB
1MB
1MB
1MB
62MB
4MB
1MB
<1MB
<1MB
<1MB
2MB
2MB
2MB
1MB
25MB
10MB
4MB
1MB
5MB
982

This scan was run without launching the product’s user interface, by right-clicking the test folder and choosing the“Scan Now” option. To record the scan time, we have used product’s built-in scan timer or reporting system.
Where this was not possible, scan times were taken manually with a stopwatch.

Performance Benchmark
Edition 3

Page 54 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

For each product, five samples were taken with the machine rebooted before each sample to clear any cachingeffects by the operating systems.
In 2009, we noticed many more products showing a substantial difference between the initial scan time (first scan)and subsequent scan times (scans 2 to 5). We believe this behavior is due to products themselves caching recently
scanned files.
As a result of this mechanism, we have averaged the four subsequent scan times to obtain an average subsequentscan time. Our final result for this test is an average of the subsequent scan average and the initial scan time.

The launch time of a product’s user interface was taken using AppTimer (v1.0.1006). For each product tested, we
obtained a total of fifteen samples from five sets of three UI launches, with a reboot before each set to clear
caching effects by the operating system. When compiling the results the first of each set was separated out so that
there was a set of values for the initial launch after reboot and a set for subsequent launches.
We have averaged the subsequent launch times to obtain an average subsequent launch time. Our final result forthis test is an average of the subsequent launch average and the initial launch time.
In some cases, AppTimer did not correctly record the time taken for UI launch. For instance, some applications
would open their window and look like they were ready, but then continued to be unresponsive. Where themeasurement from AppTimer appeared inaccurate, we have taken the time manually with a stop watch.
AppTimer is publically available from the PassMark Website.

The Perflog++ utility was used to record process memory usage on the system at boot, and then every minute foranother fifteen minutes after. This was done only once per product and resulted in a total of 15 samples. The firstsample taken at boot is discarded.
The PerfLog++ utility records memory usage of all processes, not just those of the anti-malware product. As aresult of this, an anti-malware product’s processes needed to be isolated from all other running system processes.
To isolate relevant process, we used a program called Process Explorer which was run immediately upon thecompletion of memory usage logging by PerfLog++. Process Explorer is a Microsoft Windows Sysinternals softwaretool which shows a list of all DLL processes currently loaded on the system.

The Perflog ++ utility was used to record process memory during an on-demand scan of the boot drive over a 2
minute period. This was done only once per product and resulted in a total of 10 samples each.
The Perflog ++ utility records memory usage of all processes, not just those of the anti-malware product. As aresult of this, an anti-malware product’s processes needed to be isolated from all other running system processes.
To isolate relevant process, we used a program called Process Explorer which was run immediately upon thecompletion of memory usage logging by Perflog ++. Process Explorer is a Microsoft Windows Sysinternals softwaretool which shows a list of all DLL processes currently loaded on the system.
Performance Benchmark
Edition 3

Page 55 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

We used a script in conjunction with HTTPWatch (Basic Edition, version 9.0.8.0) to record the amount of time ittakes for a set of 106 ‘popular’ websites to load consecutively from a local server. This script feeds a list of URLs
into HTTPWatch, which instructs the browser to load pages in sequence and monitors the amount of time it takes
for the browser to load all items on one page.
For this test, we have used Internet Explorer 11 (Version Version 11.0.9600.17207) as our browser.
The set of websites used in this test include front pages of high traffic pages. This includes shopping, social, news,
finance and reference websites.
The Browse Time test is executed five times and our final result is an average of these five samples. The local serveris restarted between different products and one initial ‘test’ run is conducted prior to testing to install Adobe Flash
Player, an add-on which is used by many popular websites.

The average launch time of Internet Explorer interface was taken using AppTimer. This test was practically identicalto the User Interface launch time test. For each product tested, we obtained a total of fifteen samples from fivesets of three Internet Explorer launches, with a reboot before each set to clear caching effects by the operating
system. When compiling the results the first of each set was separated out so that there was a set of values forthe initial launch after reboot and a set for subsequent launches.
For this test, we have used Internet Explorer 11 (Version 11.0.9600.17207) as our test browser.
We have averaged the subsequent launch times to obtain an average subsequent launch time. Our final result forthis test is an average of the subsequent launch average and the initial launch time.
AppTimer is publically available from the PassMark Website.

This test measures the minimum Installation Time a product requires to be fully functional and ready for use by
the end user. Installation time can usually be divided in three major phases: The Extraction and Setup phase consists of file extraction, the EULA prompt, product activation and user
configurable options for installation.
 The File Copy phase occurs when the product is being installed; usually this phase is indicated by a progressbar.
 The Post-Installation phase is any part of the installation that occurs after the File Copy phase. This phasevaries widely between products; the time recorded in this phase may include a required reboot to finalize theinstallation or include the time the program takes to become idle in the system tray.
To reduce the impact of disk drive variables, each product was copied to the Desktop before initializing installation.
Each step of the installation process was manually timed with a stopwatch and recorded in as much detail as
Performance Benchmark
Edition 3

Page 56 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

possible. Where input was required by the end user, the stopwatch was paused and the input noted in the raw
results in parenthesis after the phase description.
Where possible, all requests by products to pre-scan or post-install scan were declined or skipped. Where it was
not possible to skip a scan, the time to scan was included as part of the installation time. Where an optional
component of the installation formed a reasonable part of the functionality of the software, it was also installed(e.g. website link checking software as part of an Internet Security Product).
Installation time includes the time taken by the product installer to download components required in the
installation. This may include mandatory updates or the delivery of the application itself from a download
manager. We have noted in our results where a product has downloaded components for product installation.
We have excluded product activation times due to network variability in contacting vendor servers or time takenin account creation.

A product's Installation Size was previously defined as the difference between the initial snapshot of the Disk Space(C: drive) before installation and the subsequent snapshot taken after the product is installed on the system.
Although this is a widely used methodology, we noticed that the results it yielded were not always reproducible
in Vista due to random OS operations that may take place between the two snapshots. We improved theInstallation Size methodology by removing as many Operating System and disk space variables as possible.
Using PassMark’s OSForensics we created initial and post-installation disk signatures for each product. These disk
signatures recorded the amount of files and directories, and complete details of all files on that drive (including
file name, file size, checksum, etc) at the time the signature was taken.
The initial disk signature was taken immediately prior to installation of the product. A subsequent disk signature
was taken immediately following a manual update and system reboot after product installation. Using
OSForensics, we compared the two signatures and calculated the total disk space consumed by files that werenew, modified, and deleted during product installation. Our result for this metric reflects the total size of all newly
added files during installation.

We used a single script in testing Benchmarks 9-14. The script consecutively executes tests for Benchmarks 9-14.
The script times each phase in these benchmarks using CommandTimer.exe and appends results to a log file.

This test measures the amount of time required for the system to copy, move and delete samples of files in various
file formats. This sample was made up of 812 files over 760,867,636 bytes and can be categorized as documents[26% of total], media files [54% of total] and PE files (i.e. System Files) [20% of total].
The breakdown of the main file types, file numbers and total sizes of the files in the sample set is shown in thefollowing table:
File format

Performance Benchmark
Edition 3

NumberSize (bytes)

Page 57 of 64
2 April 2015

2015 Consumer Security Products

Performance Benchmark
Edition 3

PassMark SoftwareDOC

830,450,176

DOCX

4

13,522,409

PPT

3

5,769,216

PPTX

3

4,146,421

XLS

4

2,660,352

XLSX

4

1,426,054

PDF

73

136,298,049

ZIP

4

6,295,987

7Z

1

92,238JPG

351

31,375,259

GIF

6148,182MOV

7

57,360,371

RM

1

5,658,646

AVI

8

78,703,408

WMV

5

46,126,167

MP3

28

191,580,387

EXE

19

2,952,914

DLL

104

29,261,568

AX

118,432CPL

22,109,440

CPX

2

4,384

DRV

10

154,864

ICO

1

107,620MSC

141,587

NT

1

1,688

ROM

2

36,611SCR

22,250,240

SYS

1

37,528,093

TLB

3

135,580

TSK

1

1,152

UCE

1

22,984EXE

192,952,914

DLL

104

29,261,568

AX

118,432CPL

22,109,440

CPX

2

4,384

DRV

10

154,864

Page 58 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

ICO

1

107,620MSC

141,587

NT

1

1,688

ROM

2

36,611SCR

22,250,240

SYS

1

37,528,093

TLB

3

135,580

TSK

1

1,152

UCE

1

22,984

Total

812

760,867,636

This test was conducted five times to obtain the average time to copy, move and delete the sample files, with thetest machine rebooted between each sample to remove potential caching effects.

This test measured how much time was required to install and uninstall a third party application. For this test,
CommandTimer.exe timed how long it took to install and uninstall the following applications on the test
machine:Firefox 3.6.3 (11,909 KB) (MSI File)
Microsoft .NET 3.5 (34,121 KB) (MSI File)
Steam (1,551 KB) (MSI File)

This test was conducted five times to obtain the average time to install/uninstall the above third party
programs, with the test machine rebooted between each sample to remove potential caching effects.

This benchmark measured how much time was required to download a sample set of binary files of various sizes
and types over a 100MB/s network connection. The files were hosted on a server machine running Windows Server
2008 and IIS 7. CommandTimer.exe was used in conjunction with GNU Wget (version 1.10.1) to time and conduct
the download test.
The complete sample set of files was made up of 553,638,694 bytes over 484 files and two file type categories:
media files [74% of total] and documents [26% of total]. The breakdown of the file types, file numbers and total
sizes of the files in the sample set is shown in the following table:

Performance Benchmark
Edition 3

File format

NumberSize (bytes)

JPEG

343

30,668,312

GIF

9360,349PNG

5494,780MOV

757,360,371

RM

1

5,658,646

AVI

8

78,703,408

Page 59 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

WMV

546,126,167

MP3

28

191,580,387

PDF

73

136,298,049

ZIP

4

6,295,987

7Z

1

92,238

Total

484

553,638,694

This test was conducted five times to obtain the average time to download this sample of files, with the test
machine rebooted between each sample to remove potential caching effects.

This test measured how much time was required to convert five (5) different MP3 files into WAV files andsubsequently, convert the same MP3 samples into a WMA files. The total size of the five (5) MP3s used was
25,870,899 bytes.
To encode the MP3 into another format, we used an application called ffmpeg.exe. The format conversion process
was timed using CommandTimer.exe.
This test was conducted five times to obtain the average conversion speed between these formats, with the test
machine rebooted between each sample to remove potential caching effects.

This test measured the amount of time required to compress and decompress a sample set of files. For this test,
we used a subset of the media and documents files used in the File Copy, Move and Delete benchmark.
CommandTimer.exe recorded the amount of time required for 7zip.exe to compress the files into a *.zip andsubsequently decompress the created *.zip file.
This subset comprised 1,218 files over 783 MB. The breakdown of the file types, file numbers and total sizes of thefiles in the sample set is shown in the following table:

Performance Benchmark
Edition 3

File Type

File Number

Total Size

.xls

13

9.23 MB.xlsx

93.51 MB.ppt

97.37 MB.pptx

1117.4 MB.doc

17

35.9 MB.docx

1924.5 MB.gif

177

1.10 MB.jpg

737

66.2 MB.png

15948.9 MB.mov7

54.7 MB

Page 60 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software.rm

1

5.39 MB.avi

46

459 MB.wma

1148.6 MB.avi

46

459 MB.wma

1148.6 MB

Total

1218

783 MB

This test was conducted five times to obtain the average file compression and decompression speed, with the test
machine rebooted between each sample to remove potential caching effects.

This test measures the time it takes to complete an http download of a sample set of installation files over a
100MB/s network connection. The files are hosted on a local server machine running Windows Server 2013 andIIS 7. CommandTimer.exe was used in conjunction with the native .NET DownloadFile() method to time andconduct the download test. The file set includes the following:






Firefox Setup 31.0 (30.7 MB) (EXE file)
GIMP Setup 2.8.10 (86.2 MB) (EXE file)
LibreOffice Installer Package 4.2.5 (209 MB) (MSI file)
Adobe Reader Installer 3.5.4.25 (1.00 MB) (EXE file)

This test was conducted five times to obtain the average time to download this sample of files, with the test
machine rebooted between each sample to remove potential caching effects.

This test measures the on demand scan times of a file set comprised only of executable files (.exe, .dll and .sys
files). We performed five scans of the sample file set, with a machine restart between each scan to remove possible
caching effects. The time taken to scan the files is taken from an antivirus product’s scan logs, or where logs are
not available, manually with a stopwatch. Scans were launched by right clicking on the folder to be scanned.
A breakdown of the sample file set is as follows:
File Type

NumberFile Sizeof FilesSys Files

2174

329MB

Dll Files

2037

920MB

Exe Files

2140

827MB

Total

6351

2076MBThe final result is calculated as an average of the five samples.
Performance Benchmark
Edition 3

Page 61 of 64
2 April 2015

2015 Consumer Security Products

PassMark SoftwareThis test measures the amount of time taken to copy files between two local drives. The data set comprised of
8,501 files with a total file size of 5.44GB, and the formats used included documents, movies, images andexecutables. A breakdown of the sample file set is given below:
File

NumberExtensionof Files

.jpg

2903

588MB

.dll

773

25MB

.exe

730

197MB

.gif

681

63MB

.wav

430

260MB

.sys

501

79MB

.png

451

27MB

.mp3

333

2157MB

.wma

585

925MB

.docx

267

81MB

.avi

247

1079MB

.doc

160

57MB

.xls

329

132MB

.ppt

97

148MB

.zip

14

177MB

Total

8501

5995MB

File Size

A total of five runs of this test were performed, with a machine restart between each run. The time taken to copy
files was measured and recorded by CommandTimer.exe. All the files were copied between a folder on the local
drive and a 2nd folder on a different drive. Files were deleted from the 2nd drive once the copy was complete. The
final result is calculated as an average of the five samples

This test measures the amount of time taken to transfer files from a local drive on the test machine to a sharedfolder on the local server. The data set comprised of 8,501 files with a total file size of 5.44GB, and the formats
used included documents, movies, images and executables. A breakdown of the sample file set is given below:

Performance Benchmark
Edition 3

File

NumberExtensionof Files

.jpg

2903

588MB

.dll

773

25MB

.exe

730

197MB

.gif

681

63MB

.wav

430

260MB

File Size

Page 62 of 64
2 April 2015

2015 Consumer Security Products

PassMark Software

.sys

501

79MB

.png

451

27MB

.mp3

333

2157MB

.wma

585

925MB

.docx

267

81MB

.avi

247

1079MB

.doc

160

57MB

.xls

329

132MB

.ppt

97

148MB

.zip

14

177MB

Total

8501

5995MB

A total of five runs of this test were performed, with a machine restart between each run. The time taken totransfer files was measured and recorded by CommandTimer.exe. Files were deleted from the server once thecopy was complete. The final result is calculated as an average of the five samples

This test aims to benchmark the test machines performance over several subtests, including music,
communications, productivity, memories, TV and movies, gaming and HDD. The PCMark Vantage benchmark is
run twice with a reboot in between. The final result is an average of the overall scores of the two iterations.

The average launch time of Word interface was taken using AppTimer. This includes the time to launch the Word2013 application and open a 10MB document. This test was practically identical to the User Interface launch time
test. For each product tested, we obtained a total of fifteen samples from five sets of three Word launches, witha reboot before each set to clear caching effects by the operating system. When compiling the results the first ofeach set was separated out so that there was a set of values for the initial launch after reboot and a set for
subsequent launches.
We have averaged the subsequent launch times to obtain an average subsequent launch time. Our final result forthis test is an average of the subsequent launch average and the initial launch time.
AppTimer is publically available from the PassMark Website.

This test measures the time it takes to open an excel document and run an excel macro. The macro performs a
range of operations, including reading data from the file system, mathematical calculations, and writing data to
the spreadsheet. This test is run using a Windows PowerShell script. We obtained a total of five samples with areboot in between each to clear caching effects by the operating system. The final result is calculated as an averageof the five samples.

Performance Benchmark
Edition 3

Page 63 of 64
2 April 2015

2015 Consumer Security Products

PassMark SoftwareThis test measures the time it takes to open a large Word Document (~10 MB) and save it as a PDF document. This
test is run using a Windows PowerShell script. We obtained a total of five samples with a reboot in between eachto clear caching effects by the operating system. The final result is calculated as an average of the five samples.

This test measures the amount of time taken to transfer files from an external drive to a local disk via a USB 3.0
connection. The data set comprised of 6,834 files with a total file size of 6.06GB, and the formats used included
documents, movies, images, executables and system files. A breakdown of the sample file set is given below:
File

NumberExtensionof Files

.dll

1429

627MB

.doc

496

164MB

.docx

324

76MB

.exe

280

1.2GB

.jpg

1564

469MB

.mp3

394

1.06GB

.pdf

567

246MB

.pptx

556

1.06GB.pst

2

59.1MB

.sys

568

145MB

.xlsx

584

81.0MB

.zip

70

911MB

File Size

A total of five runs of this test were performed, with a machine restart between each run. The time taken totransfer files was measured and recorded by CommandTimer.exe. Files were deleted from the local disk once thecopy was complete. The final result is calculated as an average of the five samples

Performance Benchmark
Edition 3

Page 64 of 64
2 April 2015