 Hello
SecurityPro Readers!
How does your favorite antivirus product stack up against the competition? Relevant
Technologies took a look at four products - McAfee, Norton, Trend Micro, and Hauri.
They put the programs through their paces in real life situations. Which program
bogs down your processor the most? Which is best at cleaning virus code from your
files? Read on and find out what they discovered.
The results just might
surprise you! Also, be sure to check out today's featured product, Sophos Antivirus.
Click the link to get your free trial copy.
Anti Virus Software: Norton, McAfee, Trend Micro, or Hauri?
by Relevant Technologies (courtesy Brien Posey) During the last several years,
viruses have become increasingly more sophisticated. At the same time, the Internet's
ever-growing popularity and the steady adoption of "always on" broadband technologies
have allowed viruses to spread quickly. Now more than ever, it is important to
defend every computer in an organization against viruses in the most effective
manner possible. The problem is that there are a number of available anti virus
products available, each with their own strengths and weaknesses. To determine
which anti virus product is the most effective, we've tested four leading anti
virus products against each other. In this paper, we will describe our testing
methods and will present you with the results of each test.
Types of Anti Virus Software
In our test, we are comparing anti virus software from Norton (Symantec), McAfee,
Trend Micro, and Hauri. Each of these companies manufacturers multiple anti virus
products, each intended for different purposes. For the tests described in this
paper, we chose to use the version intended for desktop computers. This means
that our test results are valid for large businesses, small businesses, and for
home users alike. The tests were conducted on January 22, 2003. During these tests,
the latest virus definition files were downloaded for each product.
Product Being Tested
At the time that this paper was written, there were four major players in the
anti virus. This list includes Norton, McAfee, Trend Micro, and Hauri. The comparisons
are made in a random order and don't reflect our preferences or test results.
Test 1- Basic Detection and Repair
For our first test, we placed seven infected files onto a test PC. The files were
infected with viruses such as Nimda, Klez, and Fun Love. The idea behind the test
was simply to determine each product's effectiveness at detecting and cleaning
common viruses. For this test, we disabled each anti virus program's automatic
scanning engine, copied the infected files to a folder, and then manually scanned
the system. We began our tests using McAfee. McAfee had no trouble detecting our
infected files. As you can see in Figure A, after detecting the infected files,
McAfee asks the user to clean the infected files, and if the clean fails, to delete
the files.
Click to view
Figure A
McAfee detects the infection and asks the end user to take action. The end result
was that McAfee was able to detect all seven of the infections, but was unable
to clean any of them. You can see the test results in Figure B.
Click to view Figure B
McAfee detected all 7 infected files but was unable to clean any of them. Next,
we placed the same seven infected files onto the test machine and used Norton
Antivirus try to detect and repair them. As you can see in Figure C, Norton claimed
to detect nine infections, even though only seven files were actually infected.
Click to view
Figure C
Norton AntiVirus detected nine infections instead of seven. Like McAfee, Norton
AntiVirus asked the user to click a button to begin the repair process. After
clicking the Repair button, Norton reported that it was unable to repair any of
the seven files. Norton then recommended that the files be quarantined. You can
see these test results in Figures D and E.
Click to view Figure D
Norton AntiVirus detected nine infected files instead of seven, and was unable
to repair any of them.
Click to view Figure E
Norton AntiVirus was unable to repair the infected files. Next, we attempted to
detect and repair the same nine infections using Trend Micro's PC-cillin. As you
can see in Figure F, the Trend Micro product detected seven infections, and was
unable to clean any of them. The Trend Micro product automatically quarantined
the files that it was unable to clean.
Click to view
Figure F
Trend Micro's PC-cillin detected seven infections and quarantined them rather
than cleaning them. Finally, we tested Hauri's ViRobot Expert. ViRobot was able
to detect all seven infections and was able to repair them faster than we could
blink. You can see these test results in Figure G.
Click to view Figure G
Hauri's ViRobot detected all seven viruses and was able to repair them automatically.
Test 2: Integrity of Repaired Files
In Test 1, Hauri's ViRobot was the only anti virus program that was able to repair
the infected files. For our second test, we acquired some infected files that
any anti virus program should be able to repair. We infected a system DLL file
and a system level executable with Nimda, and Fun Love respectively. As you can
see in Figure H, we began with a file named CreateCDDA.DLL and a file named WIN32F~3.EXE.
In Figure H, pay close attention to the file sizes, date / time stamp, and to
the file's icons.
Click to view Figure H
Pay close attention to the file names, icons, sizes, and date time stamps. We
began this test by running McAfee against the two infected files. Upon doing so,
McAfee detected the virus and prompted us for what action to take. We clicked
the Clean button, and McAfee reported that the files had been cleaned, as shown
in Figure I. However, if you look at the files in the figure, you'll notice that
the file sizes have changed. This is normal since viral code has been deleted
from the file. You'll also notice though that the date / time stamp has changed
and that the WIN32F~3.EXE file's icon has changed to a generic icon. At first,
having an altered date / time stamp and an altered icon may not seem like a big
deal. However, it's very important to preserve date / time stamps. For example,
many times when you contact Microsoft for Technical support, they will ask you
for the date / time stamp on various system files, in order to determine the file's
version. If the date / time stamp has been altered, it's impossible to tell at
a glance if the file is the correct version. Likewise, if an executable file's
icon has changed, it could possibly mean that the file has lost some of its integrity,
and that more may have been removed than just viral code.
Click to view Figure I
McAfee cleaned the files, but altered the date / time stamp and the icons. Next,
we tried to disinfect the same set of viruses using Norton's. Norton's detected
the infection with no problems. When we clicked the Repair button, we received
a message that the infected DLL file was repaired, but that the repair failed
on the WIN32F~3.EXE file, which was infected with Fun Love. The strange thing
about this is that the CreateCDDA.DLL file was infected with Nimda. Nimda is basically
a virus that built on Fun Love. Therefore, it seems strange that Norton's could
fix Nimda, but not Fun Love. You can see the test results in Figure J. After Norton's
completed, the file's icons were preserved, but the date / time stamp was reset,
even for the file that couldn't be repaired. You can see this in Figure K.
Click to view Figure J
Norton's cleaned Nimda, but not Fun Love.
Click to view Figure K
Norton's reset the file's date / time stamp. After completing our testing with
Norton's, we tested PC-cillin. The Trend Micro product detected four viruses even
though there were only two files. As you can see in Figure L, PC-cillin misidentified
the viruses and simply quarantined the viruses.
Click to view Figure L
PC-cillin misidentified and miscounted the viruses. Finally, we tested Hauri's
ViRobot against the same two infected files. As you can see in Figure M, ViRobot
not only repaired the infected files, but also managed to preserve the date /
time stamp and the icon. We should point out though that while the results were
very obvious with the other three products, we had to perform a manual screen
refresh by pressing F5 to see what ViRobot had done with the files.
Click to view Figure M
ViRobot repaired the files and left the icons and date / time stamps intact.
Test 3: Detecting Viruses In Memory
When we
approached Hauri about our tests, they claimed that their ViRobot products could
actually detect viruses in memory and could even clean individual executing processes.
Hauri provided us with a utility that is designed to test a system's memory for
the existence of the Klez virus. Because this utility is a closely guarded trade
secret, our non-disclosure agreement with Hauri prevents us from revealing the
name of the utility. In the screen shots that you'll see in this section, file
names and commands have been blurred, for legal reasons. However, the screen shots
have not been doctored in any other way. Needless to say, we were immediately
suspicious of this utility since it was provided to us by one of the anti virus
manufacturers under such secrecy. However, rigorous independent testing has confirmed
that the utility that we'll be using for Test 3 is indeed trustworthy. For this
test, we used the SQL Server client configuration utility as a test executable.
For the test, we infected the utility with the Klez virus. For the test, we verified
that the virus was not present in memory, ran the executable to infect the system,
verified that the virus was present in memory, disinfected the virus, and then
checked the system's memory to see if the memory was still infected. You can see
an example of this process shown in Figure N. In this figure, you can see where
we tested the memory, infected the system by loading CLICONFG, and tested the
memory again.
Click to view Figure N
This is how we test to see if a system's memory is infected. We began by testing
McAfee. When McAfee ran, it detected the virus right away. McAfee then closed
the infected process (The SQL client configuration utility), and then reported
that the system was clean. However, as you can see in Figure O, the system's memory
was still infected.
Click to view Figure O
The system's memory was still infected, even after McAfee cleaned the virus. Next,
we repeated the test with Norton AntiVirus. Norton AntiVirus detected the virus,
but was unable to repair it. Norton left the infected process, and the system's
memory remained infected, as shown in Figure P.
Click
to view Figure P
Norton was unable to disinfect the virus. For the next test, we attempted to scan
for the virus with PC-cillin. As you can see in Figure Q, PC-cillin detected the
virus, but was unable to clean it. The virus was also still present in memory,
and the infected process continued to run.
Click to view Figure Q
PC-cillin detected the virus, but could not repair it. Finally, we repeated the
test using ViRobot. Like the other antivirus products, ViRobot had no trouble
detecting the infection. However, ViRobot then displayed the message shown in
Figure R. This message indicated that the infected file was presently running.
ViRobot then gave us a chance to save any documents that might have been open,
before closing the infected process.
Click to view Figure R
ViRobot detected the infection and asked to close the infected process. After
closing the infected process, ViRobot disinfected the file and the system's memory,
and then reopened the process. If you look at Figure S, you can see that the memory
was completely clean after ViRobot finished cleaning the system.
Click to view Figure S
ViRobot was able to remove the infection from memory.
Test 4: Performance
For our final test, we wanted to benchmark each product's
performance during
a full system scan. We performed this test because the more processor time that
a product uses during a system scan, the less responsive the PC is. Therefore,
we were checking to see which product has the lowest processor utilization. For
this series of tests, we closed all running applications except for the product
that we were testing. We then initiated a full system scan, and opened the Windows
Task Manager to watch the Performance tab. We waited until a representative amount
of activity had occurred prior to taking the screen shots. We began the process
by performing a full system scan with McAfee. As you can see in Figure T, although
there were spikes in the processor utilization level, McAfee's overall CPU utilization
was relatively low, averaging around 30%.
Click to view Figure T
McAfee had around 30% CPU utilization. Next, we repeated the same test using Norton
AntiVirus. As you can see in Figure U, Norton AntiVirus had nearly 100% CPU utilization
during the scanning process. Simply loading the Norton AntiVirus console generated
the period of low activity that you see in the figure prior to the heavy activity.
The full system scan began at the point in the graph where the activity increased
so dramatically.
Click to view Figure U
Norton AntiVirus held the processor at near 100% utilization during the scan.
At this point, we tested PC-cillin. PC-cillin performed very well in the processor
utilization test. The average processor utilization was well under 20%, as shown
in Figure V.
Click to view Figure V
PC-cillin had very low processor overhead. For our final test, we measured the
processor utilization while ViRobot was scanning the system. As you can see in
Figure W, ViRobot sustained an extremely low level of activity, well below 10%
CPU utilization. The spikes that you see in CPU activity at the beginning of this
chart were from when we loaded the ViRobot console.
Click to view Figure W
ViRobot had extremely low CPU usage.
The Results
Although the table below outlines the good and bad points of each product, determining
the best product isn't as simple as counting to see which product has the most
points. The reason for this is that some features are more important than others
and therefore disserve stronger consideration. The chart below is a weighted comparison
of the products based on which features are the most important. In this analysis,
each product has been given between one and five points for each area of comparison,
with five representing the highest possible score. The product's score in each
area is multiplied by its weight to determine the total number of points for the
feature. At the end, all of the points are tallied together to determine the results.
| Weight
(Default) |
Feature
and Possible Points |
McAfee |
Norton |
Trend
Micro |
Hauri |
| 50% |
Virus
Detection and Cleansing (50x5=250 possible points) |
3
(150 points) |
3
(150 points) |
4
(200 points) |
5
(250 Points) |
| 20% |
Ability
to Repair Viruses Completely (20x5=100 possible points) |
4
(80 points) |
3
(60 points) |
1
(20 points) |
5
(100 points) |
| 20% |
Ability
to Detect and Repair Infections in Memory (20x5=250 possible points) |
3
(60 points) |
1
(20 points) |
1
(20 points) |
5
(100 points) |
| 10% |
Performance
(10x5=50 Possible Points) |
3
(30 points) |
1
(10 points) |
4
(40 points) |
5
(50 points) |
| 100% |
Total
Value of Possible Points: 500 |
320
points |
240
points |
280
points |
500
points |
Conclusion
As you can see from the charts above, our absolute favorite product was newcomer
Hauri's ViRobot, which earned a perfect score. Our second favorite was McAfee,
followed by Trend Micro, with Norton in last place. As you view these results,
remember that Relevant Technologies is an independent security research firm,
and that we have provided screen shots of the actual tests, to validate our findings.
If you would like more information about any of the products that we have discussed
in this paper, you can contact each respective company via their Web site. The
addresses are as follows:
Contact Information
McAfee: http://www.mcafee.com Norton: http://www.symantec.com
Trend Micro: http://www.trendmicro.com
Hauri: http://www.hauriusa.net Brien Posey
has written thousands of technical articles on a variety topics. You can access
many of them by signing up for a free membership to Brien's personal Web site
at www.brienposey.com. Brien's Web site
also contains a forum area where you can post your most difficult technical questions
and a live chat area where you can talk directly to the experts!
|
|
