=========================================== File 7EVAL-W32.txt ------------------------------------------- Comparison of File, Macro and Script Virus and Malware detection under W32 platforms (Windows98, Windows 2000, Windows XP) =========================================== Formatted with non-proportional font (Courier) Content of this file: ********************************************************************** Eval WW32: Comparison of detection behaviour for W32 platforms ********************************************************************** Eval W32.01: Background of this evaluation Eval W32.02: Test Hypothesis Eval W32.03: Results of comparison Eval W32.SUM Grading AV products concerning W32-harmonical behaviour ********************************************************************** For those (9) products as reported in 6wjxp.txt in VTC "2003-04" and "2002-12" test reports, this section analyses detailed results as given in section (file): 6MCMP32.TXT Comparison of detection rates for W32 platforms W32.01 Background of this evaluation: ------------------------------------- With the fast deployment of new versions of Microsoft Windows-32 (in past 5 years from W-NT to W-95, W-98, W-2000 and W-XP), both customers needing protection and producers of security-enhancing software (esp. AntiVirus and AntiMalware) can only cope with the pace when they essentially re-use engines prepared for previous W32 platforms and simply "adapt" them to the intrinsics of the new platforms. Otherwise, "rewriting" the resp. software would consume too much time and efforts, and customers would receive "adapted" products only with some delay. AV/AM testers cannot determine the characteristics of the algorithms in scanning engines, either in following legal objectives (which, in most Copyright laws, prohibit reverse-engineering of proprietory code, except for specific reasons such as collecting evidence for a court case or teaching related techniques, as in Hamburg university IT Security curriculum), or for shere complexity of related code (and in many cases, for unsufficient professional knowledge of testers). It is therefore worthwhile to analyse whether those AV/AM products versions of which are available for W32 platforms behave EQUALLY concerning detection and identification of viral and malicious code. W32.02 Test Hypothesis: ----------------------- We assume that those products which participate for all W32 platforms (W98, W2k and WXP) for ALL categories shall yield IDENTICAL results. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! We call product behaviour following this hypothesis "W32-harmonical". !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! W32.03 Results of comparison: ----------------------------- With the addition of a new platform (W-XP), W32-harmonical behaviour is still not valid for all products. In comparing 2 platforms, WXP and W2k products tend to behave more similar tha W98 and WXP products. In comparing all three W32 platforms, macro viruses/malware are better handled than script viruses/malware but file viruses/malware detection is still rather different between platforms: this test last test -----------+----------- Equal detection of zoo file viruses: 6 (of 10) ----- of zoo infected files: 5 (of 10) ----- of ITW file viruses: ALL (of 10) ----- of ITW infected macro files: ALL (of 10) ----- of zoo file malware: 4 (of 10) ----- In this category, the following 3 products yield IDENTICAL results in ALL referenced (5) categories and are regarded as "perfect": AVP,CMD,SCN this test last test -----------+----------- Equal detection of zoo macro viruses: 8 (of 10) ----- of zoo infected macro objects: 8 (of 10) ----- of ITW macro viruses: ALL (of 10) ----- of ITW infected macro files: ALL (of 10) ----- of zoo macro malware: 7 (of 10) ----- In this category, the following 6 products yield IDENTICAL results in all referenced categories and are regarded as "perfect": AVP,CMD,DRW,FSE,INO,SCN this test last test -----------+----------- Equal detection of zoo script viruses: 8 (of 10) ----- of zoo script viral objects: 7 (of 10) ----- of ITW script viruses: ALL (of 10) ----- of ITW script viral objects: ALL (of 10) ----- of ITW script malware: 8 (of 10) ----- In this category, the following 7 products yield IDENTICAL results in all referenced categories and are regarded as "perfect": AVP,CMD,DRW,INO,NVC,RAV,SCN ******************************************************************************* Findings W32.1: Several W-32 scanners perform equally on W-98/W-2k in ALL categories and can be called "W32-harmonical". Concerning detection of FILE viruses (in all objects), 5 (of 10) products behave "W32-harmonically" in all categories: AVP,BDF,CMD,INO,SCN And concerning file malware detection, only 4 (of 10) products behave in W32-harmonical form: AVP,CMD,RAV,SCN Concerning detection of MACRO viruses (in all objects), a MAJORITY of 8 (of 10) products behave "W32-harmonically" in all categories: AVP,CMD,DRW,FSE,INO,NVC,RAV,SCN And concerning macro malware detection, 7 (of 10) products behave in W32-harmonical form: AVP,BDF,CMD,DRW,FSE,INO,SCN Concerning detection of SCRIPT viruses (in all objects), a MAJORITY of 7 (of 10) products behave "W32-harmonically" in all categories: AVP,CMD,DRW,INO,NVC,RAV,SCN And concerning script malware detection, 8 (of 10) products behave in W32-harmonical form: AVP,CMD,DRW,FSE,INO,NVC,RAV,SCN In comparison, W32-harmonical behaviour is significantly better developped for script and macro virus detection, whereas W32-harmonicity of file viruses/malware is least developped. ******************************************************************************** For ALL categories, the following *3* W32 scanners (of 10) yield identical results on ALL platforms and ALL categories: AVP,CMD,SCN The following *3* W32 scanners detect file viruses/malware in W32-harmonical behaviour: AVP,CMD,SCN The following *6* W32 scanners yield identical results for all macro (zoo,ITW) viruses/malware: AVP,CMD,DRW,FSE,INO,SCN The following *7* products yield identical results for all script (zoo,ITW) viruses/malware: AVP,CMD,DRW,INO,NVC,RAV,SCN ******************************************************************************** W32.SUM: Grading AV products concerning W32-harmonical behaviour: ----------------------------------------------------------------- The following grid is used to grade W32 products concerning their ability for IDENTICAL detection for ALL categories on ALL W32 platforms: A "perfect" W32-harmonical AV product will yield IDENTICAL results for all categories (macro and script viruses). (Assigned value: 5). A "perfect" W32-harmonical AM product will be a perfect AV product and yield IDENTICAL results for all categories (macro and script malware). (Assigned value: 2). Grading W32-harmonical AntiVirus products: =========================================================== Grade: "Perfect" W32-harmonical detection: AVP,CMD,SCN =========================================================== Grading W32-harmonical AntiMalware products: =========================================================== Grade: "Perfect" W32-harmonical detection: AVP,CMD,SCN =========================================================== ************************************************************ "Perfect" W32-harmonical AntiVirus products: 1st place: AVP,CMD,SCN (5 points) ************************************************************ "Perfect" W32-harmonical AntiMalware products: 1st place: AVP,CMD,SCN (7 points) ************************************************************