=================================== File 8PROBLMS.TXT: ----------------------------------- List of problems experienced during aVTC test "2004-07": =================================== Formatted with non-proportional font (Courier) Content of this file: ===================== 1. Introduction: General Problems 1.1 Problems likely related to FindFirst/FindNext anomaly (essentially unchanged since last test) 2. List of benevolently behaving AV products in test "2004-07" 3. Problems of AV products observed during test "2004-07" 3.1 List of Postscans 3.2 List of specific problems 1. Introduction: General Problems: ================================== For automatic tests on large viral databases, and for automatic processing of large scanner log files, a set of test conditions is prerequisite for scanners to participate in a VTC test (see: 4TESTCON.TXT). In many cases, serious problems were observed during some tests. DOS scanners were either not suitably running under SIMBOOT and crashed, or problems appeared with the (rather large) file virus database. In some cases, scanners crashed upon detecting some specific virus; in few cases, "manual" operation instead of automatic (batch) operation helped solving some of these problems. Such curative action was also applied when possible in cases where log files were inadequate (e.g.needing manual operation for export). With growing velocity of processors, DOS scanners (running without any problem on INTEL 386 and 486) growingly crash on Pentium II/III systems faster than 250 MHz. Another general problem with DOS scanners is related to counters for files and viruses which often seem to be designed as integers, so they start after 65,536 with 0. During preparation and test, we again experienced a serious problem reported also in previous VTC Tests, according to which management of large sets of directories in FAT and NTFS may not reliably work. Both when attempting to move large parts of our file virus database, as when some scanner proceeded scanning subsequent viral directories, we found that several directories were not moved or touched. This effect seems to happen stochastically, such that subsequent attempts gave different results. Concerning omitted (=unscanned) directories, we overcame this "dysfunctional" behaviour of FAT and NTFS by repeat- ing scanning so long until the number of scanned files agreed with the (known) number of directories in testbeds. Overcoming this problem was extremely time-consuming, and this was a reason for delaying publication of results. In cases where scanners crashed during detection test upon the rather large file virus database, tests were performed in several runs on partitions (essentially on directories with same first letters of names). In most cases (apart those reported below), these tests were completed, and resulting files were joined and evaluated. Finally, with growing testbeds, test protocols produced by scanners grow equally. When processing such protocols, we need meanwhile up to 6 GByte of disk space, and our evaluation scripts (in AWK) become more complex. Under these conditions, we also suffered from an evident bug in the AWK processor which inhibited proper evaluation and required additional quality assurance (including time and efforts). 1.1 Problems very likely related to FindFirst/FindNext anomaly: --------------------------------------------------------------- In several cases, scanners finished a first scan although they had not touched all directories with infected objects. In such a case, a postscan was started adressing only those untouched objects; a second postscan was started when again objects were observed untouched, but after the 2nd postscan, no more scan was started. This behaviour may originate from a reported anomaly in the behaviour of FindFirst/FindNext (those routines are used to handle objects in directory trees) which has not been cured so far by Microsoft. In the "problems list", these postscans are ked as "minor problems": different from crashes, postscans "only" required time for running the tests and evaluating test protocols (which significantly delayed results). 2. List of benevolently behaving AV products in test "2004-07": =============================================================== In general, few scanners could be tested without any problem. When considering the large number of postscans, few products had only minor problems and are regarded as (relatively) "benevolent". A) Windows-2000 products: ============================================== Out of 25 W-2000 scanners, 15 had NO problems: ------------------------------------------- ANT, AVA, AVG, AVK, AVP, BDF, CMD, FIR, GLA, INO, PAV, SCN, SWP, VBR, VSP. ============================================ B) Windows-XP products: ============================================== Out of 25 W-XP scanners, 14 had NO problems: ---------------------------------------------- ANT, AVG, AVK, AVP, BDF, CMD, FIR, FPR, GLA, PAV, SCN, SWP, VBR, VSP. ============================================== C) LINUX products: -------------------------------------------- Out of 11 Linux scanners, 2 had NO problems: SCN, SWP. ============================================ Concerning overall stability, almost all products required at least one postscan. As this may be due to the FF/FN anomaly (for which Microsoft is essentially responsible), the following adresses those scanners which had NO OTHER problem than requiring postscans: *************************************************************** The following 2 products submitted for ALL 3 platforms (LINUX, 2 Windows platforms) behaved with least problems (no crash, "only postscans") on ALL platforms and are regarded as "most stable products in test": SCN, SWP. *************************************************************** The following 13 products submitted for the Windows platforms behaved with least problems (no crash, "only postscans") on those platforms: ANT, AVG, AVK, AVP, BDF, CMD, FIR, GLA, PAV, SCN, SWP, VBR, VSP. *************************************************************** 3. Problems of AV products observed during test "2004-07": ========================================================== 3.1 List of Postscans: ---------------------- In several cases, AV/AM products didnot access and check all entries in testbeds (possibly due to the "FF/FN anomaly" as reported in 1.1) or due to crashes or other product misbehaviour (see 3.2). In such cases, up to 2 "postscans" were started, whereever possible on the remainder of the related testbed. The following list summarizes those products where at least 1 postscan was initialised (2x implies that 2 postscans were needed): LNX: FI_P____: CLA,CMD(2x),FPR(2x),FSE(2x),INO,OAV,SWP FI______: CLA,FPR,OAV,SWP FM______: ANT,AVP,CLA,CMD,DRW,FPR(2x),FSE(2x),INO(2x),OAV(2x),SWP(2x) FZ______: ANT,AVP,CLA,CMD(2x),DRW,FPR(2x),FSE,INO,OAV(2x),SCN,SWP MI_P____: CMD,FPR,FSE,INO(2x) MI______: INO MM______: FSE,SWP MZ______: ANT,AVP,CLA,CMD,DRW,FPR,FSE,INO,OAV,SCN,SWP SI______: SM______: CMD,FPR(2x),FSE,OAV(2x) SZ______: W2K: FI_P____: AVA,AVG(2x),AVK,BDF,CMD(2x),FIR,FPR,GLA,INO,NAV,NVC,PAV, PER,PRO(2x),QHL(2x) FI______: AVK,FIR,FSE,GLA,NAV,PAV,PER,PRO(2x),QHL(2x),VBR(2x) FM______: AVA(2x),AVK,BDF,FIR,FPR,FSE,GLA,NAV,NVC,PAV,PER(2x),PRO(2x), QHL(2x),RAV(2x),VBR,VSP FZ______: AVK,BDF,CMD,DRW(2x),FIR,FPR(2x),FSE,GLA,IKA(2x),NAV(2x), NVC,PAV,PER,PRO(2x),QHL(2x),RAV,VBR(2x),VSP MI_P____: AVA(2x),AVG(2x),AVK,CMD,DRW,FIR,FPR,GLA,INO(2x),NAV,PAV, PER(2x),PRO(2x),QHL MI______: FIR,GLA,PER(2x),PRO(2x),QHL,VBR MM______: FIR,GLA,NAV,PAV,PER(2x),PRO(2x),QHL,VBR MZ______: AVK(2x),FIR,FSE,GLA,NAV,PAV,PER(2x),PRO(2x),VBR,VSP SI______: FIR,GLA,PER,PRO(2x),QHL,VBR SM______: AVK,CMD,FIR,FPR,FSE,GLA,NAV,PAV,PER,PRO(2x),QHL(2x),VBR SZ______: AVK(2x),BDF,FIR,FSE,GLA,NAV,PAV,PER,PRO(2x),QHL,VBR WXP: FI_P____: AVA(2x),AVG,AVK,BDF,CMD(2x),FIR,FPR,GLA,INO,NAV,NVC,PAV, PER,PRO(2x),QHL(2x) FI______: AVK,FIR,FSE,GLA,NAV,PAV,PER,PRO(2x),QHL(2x),VBR(2x) FM______: AVA(2x),AVK,BDF(2x),FIR,FPR,FSE(2x),GLA,NAV,PAV,PER(2x), PRO(2x),QHL(2x),RAV(2x),VBR,VSP FZ______: AVK,BDF,CMD,DRW(2x),FIR,FPR,FSE,GLA,IKA(2x),NAV(2x),PAV, PER,PRO(2x),QHL(2x),RAV,VBR(2x),VSP MI_P____: AVA(2x),AVG(2x),AVK,CMD,DRW,FIR,FPR,GLA,INO,NAV,PAV,PER(2x), PRO(2x),QHL MI______: FIR,GLA,PER(2x),PRO(2x),QHL,VBR MM______: DRW,FIR,GLA,NAV,PAV,PER(2x),PRO(2x),QHL,VBR MZ______: AVK(2x),DRW,FIR,FSE,GLA,NAV,NVC,PAV,PER(2x),PRO(2x),VBR,VSP SI______: FIR,GLA,PER(2x),PRO(2x),QHL,VBR SM______: AVK,CMD,DRW,FIR,FPR,FSE,GLA,NAV,PAV,PER(2x),PRO(2x),QHL(2x),VBR SZ______: AVK,BDF,DRW,FIR,FSE,GLA,NAV,PAV,PER(2x),PRO(2x),QHL,VBR 3.2 List of specific problems: ------------------------------ The following list reports specific problems observed for products as indicated ("spoon-feeding" means that scanner was restarted on each subsequent directory when a crash was experienced): General problems (if any): -------------------------- ANT: LNX: ANT does not scan more than 79 files secified on the command line. Further specified objects are ignored while scanning. W2K: --- WXP: --- AVA: W2K: --- WXP: crashed on file-mal AVG: W2K: --- WXP: --- AVK: W2K: --- WXP: --- AVP: LNX: Error while extracting, extracted files seem to be ok. gzip: stdin: decompression OK, trailing garbage ignored tar: Child returned status 2 tar: Error exit delayed from previous errors The ReportFileLimit option does not disable the report size check as documented, the log remains empty if ReportFileSize is 0 The log file is not created in the current directory, but in a subdirectory of "$HOME" W2K: It was not possible to disable "beep-sound" in batch. WXP: It was not possible to disable beep-sound in batch. BDF: W2K: --- WXP: --- CLA: LNX: Special device files and symlinks cannot be scanned (they are ignored). clamscan does not write the list of files to log file, but to STDERR. This list was used for evaluation. clamscan concatenates file arguments using spaces. While this is a dirty hack partially working for DOS, it's a "DO-NOT" for linux. CMD: LNX: csav silently ignores device special files, but device special files are counted for the statistic, though. The MBR and boot record are reported to be scanned, while csav doesn't seem to touch them. The -silent option does not seem to have an effect on screen output _as documented_, but to supress the file listing on the screen and in the log. W2K: --- WXP: --- DRW: LNX: The log file is not created in the current directory, but in the program directory (/opt/drweb). W2K: Crashed while scanning large testbeds, non deterministic behaviour WXP: Crashed while scanning large testbeds, non deterministic behaviour. Reports "Cannot load disk access library!" for some scans. FIR: W2K: --- WXP: --- FPR: LNX: The option '-follow' does not work W2K: Failure to scan T:\23W\MRON\S\RECLIS\T_0049_0.exe. Sudden Exit of scanning process. WXP: --- FSE: LNX: This scanner does only report the first file from archives. W2K: System hangs after batch execution of fsav.exe WXP: System hangs after batch execution of fsav.exe Product crashed on files: W:\REPPORD\MRONON\A\ABBA\9489\B\W_0FVB_0.COM W:\REPPORD\MRONON\B\01RDTOOB\W_0G21_0.COM Error: unknown error. GLA: W2K: --- WXP: --- IKA: W2K: Crashed while scanning large testbeds, non deterministic behaviour WXP: Crashed while scanning large testbeds, non deterministic behaviour INO: LNX: The update for this test exits without any output, and the files could not be verified to be installed. Most likely, they aren't. If symlinks are specified on the command line, they are listed in the scanlog, but reported to be clean. The scanner crashes repeatedly on the File-Malware-testbed: on W:\NAJORT\MRONON\B\ Program /usr/local/ino/ino/bin/inocmd32 terminating due to signal 11 W2K: --- WXP: SCAN ERROR -65534 OCCURRED WHILE PROCESSING FILE: W:\NAJORT\MRONON\B\SP-ROODK.CAB\W_0835_0.EXE NAV: General: Files are not shown in the scan history and don't appear in exported scan reports, if the reported virus name contains a comma (although they are recorded in the internal logfile). This is possibly related to the design of the internal logfile format, which is based on comma-separated values. NAV doesn´t report scanned files if they are clean, only infected files are reported. "SCAN COULD NOT OPEN FILE T:\NIW\MRONON\P\NIP\2626\T_05HU_0.EXE " Remark: scanning required unusual amounts of time. San runs over the large file zoo testbeds used several days, even on aVTCs fastest processors with sufficient RAM. NVC: W2K: Crashed two times on "File Mal" WXP: Aborted scanning of the "Macro Zoo" testbed with "Internal error". OAV: LNX: VirusHammer does not produce usable logs. ScannerDaemon reports device special files as 'OK'. ScannerDaemon is limited to one infected object. PAV: W2K: --- WXP: --- PER: LNX: Tested system "Linux gunni 2.2.18 #1 Fri Jan 19 22:10:35 GMT 2001 i586 unknown" and it's clone flood don't meet the minimum test requirements (kernel 2.4.2) stated in the readme. PER does only offer to scan a single directory, while not supporting scanning single files or directory trees PER silently ignores symlinks PER obviously offers heuristic scanning, but we found no switch to activate it. An option to list all scanned files is missing. W2K: Crashed on database FILE MAL without creation of any logfile WXP: Crashed on database File ZOO and FILE MAL without creation of any logfile PRO: W2K: only reports infected files. WXP: only reports infected files. QHL: W2K: There was an error while scanning drive H: . Crashed shortly after scanning: W:\MROW\MRONON\K\YARIK\69431\W_01LR_0.EXE WXP: There was an error while scanning drive H: . Crashed while scanning the whole testbed w: RAV: W2K: Crashed on testbed FILE MAL without creation of any logfile. Crashed on files: W:\NAJORT\MRONON\G\CIRENEG\W_0B8K_0.EXE W:\NAJORT\MRONON\T\EKHT\06351\C\W_0EUC_0.COM WXP: Crashed on testbed FILE MAL without creation of any logfile. Crashed on files W:\NAJORT\MRONON\A\BMOBISNA\TIK\W_03ES_0.EXE w:\REPPORD\MRONON\A\C-REKCIL.CDA\RD\W_0FWD_0.EXE W:\NAJORT\MRONON\G\CIRENEG\W_0B8K_0.EXE SCN: LNX: --- W2K: --- WXP: --- SWP: General: some files are reported as corrupt (and not scanned) AND as infected. LNX: --- W2K: --- WXP: --- VBR: W2K: --- WXP: --- VSP: W2K: --- WXP: ---