========================================= File 8PROBLMS.TXT List of problems experienced during test: ========================================= Formatted with non-proportional font (Courier) Content of this file: --------------------- 1. Introduction: General Problems 2. List of benevolently behaving AV products 3. Problems of AV products observed during tests 3.1 Scanners unable to detect viruses in packed objects 3.2 List of scanner problems 1. Introduction: General Problems: ---------------------------------- For automatic tests on large viral databases, and for automatic processing of large scanner log files, a set of test conditions is prerequisite for scanners to participate in a VTC test (see: 4TESTCON.TXT). In many cases, serious problems were observed during some tests. DOS scanners were either not suitably running under SIMBOOT and crashed, or problems appeared with the (rather large) file virus database. In some cases, scanners crashed upon detecting some specific virus; in few cases, "manual" operation instead of automatic (batch) operation helped solving some of these problems. Such curative action was also applied when possible in cases where log files were inadequate (e.g.needing manual operation for export). With growing velocity of processors, DOS scanners (running without any problem on INTEL 386 and 486) growingly crash on Pentium II/III systems faster than 250 MHz. Another general problem with DOS scanners is related to counters for files and viruses which often seem to be designed as integers, so they start after 65,536 with 0. During preparation and test, we again experienced a serious problem reported in VTC Test "1998-10", according to which management of large sets of directories in FAT and NTFS may not reliably work. Both when attempting to move large parts of our file virus database, as when some scanner proceeded scanning subsequent viral directories, we found that several directories were not moved or touched. This effect seems to happen stochastically, such that subsequent attempts gave different results. Concerning omitted (=unscanned) directories, we overcame this "dysfunctional" behaviour of FAT and NTFS by repeat- ing scanning so long until the number of scanned files agreed with the (known) number of directories in testbeds. Overcoming this problem was extremely time-consuming, and this was a reason for delaying publication of results. In cases where scanners crashed during detection test upon the rather large file virus database, tests were performed in several runs on partitions (essentially on directories with same first letters of names). In most cases (apart those reported below), these tests were completed, and resulting files were joined and evaluated. Finally, with growing testbeds, test protocols produced by scanners grow equally. When processing such protocols, we need meanwhile up to 6 GByte of disk space, and our evaluation scripts (in AWK) become more complex. Under these conditions, we also suffered from an evident bug in the AWK processor which inhibited proper evaluation and required additional quality assurance (including time and efforts). 2. List of benevolently behaving AV products: --------------------------------------------- Very few scanners could be tested *without any problem* (admittedly, the unstable behaviour of Windows-98 many have adversely influenced some scanner). Such benevolent behaviour (possibly with exemption of NFTS and FAT problems as mentioned above) can be reported for several (=the majority) of DOS scanners: NO problems with DOS scanners: ANT, AVA, AVG, AVK, AVX, MR2, NAV, NOD, RAV, SCN and TSC. In comparison, W-32 scanners were significantly less stable, as only the minority behaved test-friendly: NO problems with W-32 scanners: AVP, CMD, FPW, PRO and SWP. 3. Problems of AV products observed during tests: ------------------------------------------------- For several AV products, results could not be produced for specific platforms or for parts of malware databases. Several scanners didnot detect a single virus in at least one compressed file. Among operating platforms, products proved especially unstable on Windows 98; in some cases, the cause may be more on the side of the operating system than of the product: 1) Windows-98 sometimes locks up under stress, for almost any test; this problem cannot be attributed to tested products. 2) When scanners crashed under Windows-98, they often left reports of size zero, thus making an analyse where a scanner crashed rather difficult. The instability of this platform may also play a major role in the (sur- prisingly many) problems of scanners tested under this platform. When a scanner crashed under W-98, the usual habit was to "spoon-feed" the product in running it again under varied conditions. Moreover, when both Command- Line (CLI) and Graphical User (GUI) Interfaces were available, some problems were solved by starting the GUI version (although VTCs automatic tests are adapted for CLI-based procedures). These procurative actions were in many cases "successful" (although this is not an attitude which users may have). The following list reports specific problems observed for products as indicated: ANT DOS: --- W-98: Tested versions (AN5 under GUI, ANT: CLI driven) both crashed once during VKit test. Detection of packed file and macro viruses couldnot be evaluated as scanner reports only virus name but not filename. W-NT: Upon detecting a virus, scan process is interrupted, and user is asked to send a sample of the virus to manufacturer. AVA DOS: --- W-98: One crash during test of file virus database. Detection of packed file and macro viruses couldnot be evaluated as scanner reports only virus name but not filename. W-NT: Detection of packed fileand macro viruses couldnot be evaluated as scanner reports only virus name but not filename. AVG DOS: --- W-98: During VKit tests, several crashes at various entries were observed; this seems to depend upon the specific hardware. Test completed after installation on some Pentium 133. W-NT: Crashed upon scanning Packed macro virus testbed in K- directory (reported by Dr. Watson). AVK DOS: --- W-98: Crashed during VKit, probably due to size of testbed. W-NT: --- AVP DOS: Often stops scanning after the directory containing false positives. W-98: --- W-NT: --- AVX DOS: --- W-98: Detection of packed file and macro viruses couldnot be evaluated as scanner reports only virus name but not filename. W-NT: After product restricted access to some entries, scanning was equally "inhibited". CMD: DOS: Crashes on Microsoft Access files. W-98: --- W-NT: --- DRW/DWW DOS: Reboots during malware scan; crashes on several false-prositive word documents. W-98: Scanner didnot work on Pentium 90. W-NT: --- FPR/FMA DOS: Crashes on Microsoft Access files. W-98: One crash during scanning of VKit and file virus databases. W-NT: --- FPW DOS: n/a W-98: --- W-NT: --- FSE DOS: Often stops scanning after the directory containing false positives. W-98: Upon scanning packed archives, scanner properly reported findings, but didnot report names. A special DLL submitted for evaluation improved the situation. W-NT: When testing the macro virus testbed esp. on drive R, access to the report was impossible (exception condition). FWN: DOS: --- W-98: Detection rate of macro viruses of version started under CLI is better than GUI-produced results. W-NT: --- INO DOS: Unable to allocate memory to scan compressed files; packed macro and packed file test skipped. W-98: Product crashed once during file virus test (entry U:\DOS). Scanner crashed several times upon starting next scan; when newlly started for each scanning process, task completed properly. W-NT: --- MR2 DOS: --- W-98: Crashed once both on testing VKit and file virus database. W-NT: --- NAV DOS: --- W-98: We experienced problems with storage of large report files; crashes on 2 computers with only 32 MByte memory. Scanner has counting problems; number of infected files shows overflow (at 65536?). W-NT: --- NOD DOS: --- W-98: Crashed upon VKit scanning at w:\vk_01\...982\00982ci.com. W-NT: Mysterious crash at end-of-test with following error report: ---------------------- End-of-Error-Report ------------------------ Exception code: C0000005 ACCESS_VIOLATION Fault address: 02E40154 7C:00414110 C:\fse\vlist32.dll Registers: EAX:000002E4 EBX:00B88D54 ECX:00000081 EDX:0012F564 ESI:02E40154 EDI:0012F62C CS:EIP:001B:02E40154 SS:ESP:0023:0012F4D8 EBP:0012F4F0 DS:0023 ES:0023 FS:0038 GS:0000 Flags:00010246 Call stack: Address Frame Logical addr Module Then: Dr Watson for Windows NT FPWM32DLL.EXE Exception: access violation (0xc0000005), Address: 0x02e40154 ---------------------- End-of-Error-Report ------------------------ Remark: Results could be generated, but finishing may have adversely been influenced by independent tests running in autonomous boxes. NVC/NVN DOS: Several crashes on malware collection, test aborted. Crashes when trying to unpack ZIP archives. W-98: Both GUI version (NV5=NVC95.exe) and CLI version (NVC=NVC32.exe) were tested. Test system crashed during de-installation of product. W-NT: According to Dr. Watson, product crashed always while scanning R:\WORD\DZT\G. Attempts to inhibit suspicious files didnot solve this problem but prohibiting access to the whole directory worked. Same experience when scanning R:\WORD\HYBRID\H and R:\WORD\DZT. PAV: DOS: Often stops scanning after the directory containing false positives. W-98: Crashed once upon scanning the polymorphic viruses, and crashed twice on scanning VKit. W-NT: --- PRO DOS: Scans macro viruses nicely, but crashes on almost anything that remotely looks like a boot or file virus. W-98: --- W-NT: --- RAV DOS: --- W-98: Crashed when scanning Tequila-virus (polymorphic file testbed), and crashed upon scanning VKit virus testbed. W-NT: Unable to test: "Error while loading the engine (code=0x00004001)" SCN DOS: --- W-98: Crashed sometimes, not always reproducable, but sometimes with possibly useful "ABEND" information (=ABnormal END): 1) Scanning packed macro viruses: simply stops after scanning entry K:\EXCEL\COMPAT\A\ARJ.ARJ\33MPAT-A.XLS. Possible cause: a sample of XM/Compat.A virus (improper handling?) handled. 2) Upon scanning specific packed macrovirus entries (ARJ.arj, LZH.lzh, ZIP.zip), scanner crashed after having detected 2 viruses. The following error report (edited) was produced: ------------------------ Error-report (edited) ----------------- (DOS-4G error (2001) exception 0Eh) DOS/4G error (2001): exception 0Eh (page fault) at 29F:82B16D37 TSF32: prev_tsf32 826C SS 2A7 DS 2A7 ES 2A7 FS 0 GS 87 EAX 23B8 EBX 0 ECX 820004E0 EDX 82C60000 ESI 0 EDI 82B78894 EBP 82B56644 ESP 82B55E4C CS:IP 29F:82B16D37 ID 0E COD 6 FLG 10282 CS= 29F, USE32, page granular, limit FFFFFFFF, base 0, acc CFFB SS= 2A7, USE32, page granular, limit FFFFFFFF, base 0, acc CFF3 DS= 2A7, USE32, page granular, limit FFFFFFFF, base 0, acc CFF3 ES= 2A7, USE32, page granular, limit FFFFFFFF, base 0, acc CFF3 FS= 0, USE16, byte granular, limit 0, base 0, acc 0 GS= 87, USE16, byte granular, limit FFFF, base BF70, acc F3 CR0: unavailable Crash address (unrelocated) = 1:00040D37 Opcode stream: 89 44 99 20 01 41 18 EB DC B8 01 00 00 00 5E 5A Stack: 02A7:82B55E4C 8030 82BC 0001 0000 0DBC 82B8 0001 FFFF 6C48 82B1 5EB4 82B5 02A7:82B55E64 8894 82B7 7E64 82B4 6CDF 82B1 0000 0000 0000 0000 E818 82AF 02A7:82B55E7C 6644 82B5 6194 82B5 0000 0000 8894 82B7 000E 0000 1344 82AF 02A7:82B55E94 000D 82B8 0080 0000 5F64 82B5 6194 82B5 814C 82B5 0000 0000 02A7:82B55EAC FFFF 82B7 7E02 82B4 CD04 82AF 5F64 82B5 6194 82B5 8168 82B5 02A7:82B55EC4 1120 82AF 8168 82B5 5F64 82B5 6050 82B5 8894 82B7 814C 82B5 Last 4 ints: ------------------- End-of-error-report (edited) ----------------- 3) VKit test: In several different attempts, scanner stopped after having scanned only parts of the testbed, usually in the second (of 10) directories. In various attempts, scanner stopped at different locations. No error diagnostics were available. Manual "spoon-feeding" was applied to produce test results. Btw: scanner was invoked with command: scanpm w:\vk_03 /ALL /VID /NC /NOBEEP /AN /!GURU /UNZIP /SUB /REPORT v03.ful 4) Macro virus testbed: Similar to the VKit test, scanner processed only directories R:\PWRPNT97, R:\WORD2 and R:\WORD97, and left out others. 5) Polymorphic file virus testbed: Similar to VKit test, not all directories were scanned. E.g. when testing Tremor testbed, scanner stopped after 2,000 samples. Likewise, scanner processed only 2,000 Maltese samples; in the next test, he stopped after 300 more samples. 6) File malware testbed: Similat to VKit test, scanner stopped without error report after having scanned only parts of the testbed. SCN W-NT: Memory allocation problems when scanning volume U: (NT message, although only 155 MBytes out of 200 available storage were used). Other related problem report: "Scan Error Administrator Critical error occurred, unable to allocate enough memory to continue scan" then scanner crashed due to shortage of memory. SWP: DOS: Crashes somewhere in T:\TROJAN\NONORM\A\APSTROJA.N. Since only the filename is displayed, we were unable to get the exact crash location. Entire tree excluded from test. W-98: --- W-NT: --- TSC DOS: --- W-98: Crashed once upon scanning file virus database. W-NT: --- VSP DOS: Often crashes on startup. Runs fine if startup was successful. W-98: Crashed 4 times upon scanning polymorphic file virus testbed, at v:/onea0000.e/8/one9855.exe (4 times). When access to this file was inhibited, scanner succeeded to complete test. Also, one crash each upon scanning VKit, file virus and file malware testbeds. W-NT: Always crashed when scanning V:\ONEH3544\ONEA0000.E\8\ONE9855.