========================================= File 8PROBLMS.TXT List of problems experienced during test: ========================================= Formatted with non-proportional font (Courier) Content of this file: --------------------- 1. Introduction: General Problems 2. List of benevolently behaving AV products 3. Problems of AV products observed during tests 3.1 Scanners unable to detect viruses in packed objects 3.2 List of scanner problems 1. Introduction: General Problems: ---------------------------------- For automatic tests on huge viral databases, and for automatic processing of large scanner log files, a set of test conditions is prerequisite for scanners to participate in a VTC test (see: 4TESTCON.TXT). In many cases, serious problems were observed during some tests. DOS scanners were either not suitably running under SIMBOOT and crashed, or problems appeared with the (rather large) file virus database. In some cases, scanners crashed upon detecting some specific virus; in few cases, "manual" operation instead of automatic (batch) operation helped solving some of these problems. Such curative action was also applied when possible in cases where log files were inadequate (e.g.needing manual operation for export). With growing velocity of processors, DOS scanners (running without any problem on INTEL 386 and 486) growingly crash on Pentium II systems faster than 250 MHz. Another general problem with DOS scanners is related to counters for files and viruses which often seem to be designed as integers, so they start after 65,536 with 0. During preparation and test, we again experienced a serious problem reported in VTC Test "1998-10", according to which management of large sets of directories in FAT and NTFS may not reliably work. Both when attempting to move large parts of our file virus database, as when some scanner proceeded scanning subsequent viral directories, we found that several directories were not moved or touched. This effect seems to happen stochastically, such that subsequent attempts gave different results. Concerning omitted (=unscanned) directories, we overcame this "dysfunctional" behaviour of FAT and NTFS by repeat- ing scanning so long until the number of scanned files agreed with the (known) number of directories in testbeds. Overcoming this problem was extremely time-consuming, and this was a reason for delaying publication of results. In cases where scanners crashed during detection test upon the rather large file virus database, tests were performed in several runs on partitions (essentially on directories with same first letters of names). In most cases (apart those reported below), these tests were completed, and resulting files were joined. Finally, with growing testbeds, test protocols produced by scanners grow equally. When processing such protocols, we need meanwhile up to 6 GByte of disk space, and our evaluation scripts (in AWK) become more complex. Under these conditions, we also suffered from an evident bug in the AWK processor which inhibited proper evaluation and required additional quality assurance (including time and efforts). 2. List of benevolently behaving AV products: --------------------------------------------- Very few scanners could be tested *without any problem* (admittedly, the unstable behaviour of Windows-98 many have adversely influenced some scanner). Such benevolent behaviour (possibly with exemption of NFTS and FAT problems as mentioned above) can be reported for: **** AVG, FWN, IRIS, PAV, RAV **** 3. Problems of AV products observed during tests: ------------------------------------------------- For several AV products, results could not be published for specific platforms or for parts of malware databases. Several scanners didnot detect a single virus in at least one compressed file. Among operating platforms, Windows-98 proved especially unstable: 1) Windows-98 sometimes locks up under stress, for almost any test; this problem cannot be attributed to tested products. 2) When scanners crashed under Windows-98, they often left reports of size zero, thus making an analyse where a scanner crashed rather difficult. The instability of this platform may also play a major role in the (sur- prisingly many) problems of scanners tested under this platform. The following list reports specific problems observed for products as indicated: ACU W-98: Macro-only-Scanner? Macro virus test in single scan mode using a batch for each subdirectory directory (otherwise, the scanner crashed). W-NT: Diagnosis (Dr. Watson): "acc-cmd.exe exception: access violation (0xc0000005), adress: 0x1000e94c" AVA W-98: Strange behaviour on Vkit: Scanner GUI reported that scanning had NOT finished, but report file indicated that scanning WAS finished. AVK W-98: AVK Engine sometimes seems to pause for some time, then continues (side-effect of heuristics?). AVP W-98: After each scan, computer must be resetted, otherwise it would crash on next scan. Time-counter counts only 24 hours, then continues with zero. Product needs very long time on testing false positives. W-NT: Extremely slow upon scanning false-positive directories. AVX W-98: Several crashes; product switches after crash sometimes to Roumanian language. Log-file extension must be .log, otherwise no report is written. DRW DOS: Several crashes upon scanning file and macro viruses (related directories not tested). W-98: Crashed several times (on malware\dropper, on filevir\a, filevir\c and filevir\s; each time, test was continued on next directory) W-NT: Crash at t:\Dropper\NCAR\P\PREDATOR\1154\Com_002_.com and at u:\DOS\NCAR\a\ABRAXAS\1214\ DWW W-98: Beta Version did not scan packed files. W-NT: Scanned only files packed by LZEXE, DIET, PKLITE, EXEPACK,... DSS DOS: Crashed upon scanning 2 packed macro viruses (in unpacked form, these viruses were properly recognised). W-NT: Hanging (several attempts) at: u:\dos\caro\k4__fp__\FP__0043.exe\SETUP.ZIP\RTM.EXE u:\dos\ncar\rk__fp__\FP__0082\PAV_A.CMP\PAVMENU.OVL FPR W-98: Crashed on Vkit after scanning had finished. FMA W-98: No un-install. FSE DOS: Under SIMBOOT, scanner reports that memory is insufficient to load signatures (images scanned). W-98: Initial crashes were cured with patch 1040 (Fpwm32.dll + Avpfpi32.dll); crashed on file (5 times, including 4 at False Positives detection) and file malware (1 time). HMV W-98: Product does not scan under GUI; system must be started using F8-key to command-prompt, then manual start of network; then: everything runs well. This problem seems to be related with the long-filename-support in Win98, because command prompt uses only short file names. INO W-98: Crashed several times. W-NT: Upon command line test, the following was experienced: Testing Extended Memory...OK Home Directory: [C:\Inoculan] Error allocating memory for data file Scanner executed satisfactorily in GUI mode. Scan starts upon VKIT only after 20 minutes. IVB W-98: Scanned only for macro viruses. W-NT: Scanned only for macro viruses. Crashed upon uninstalling. ITM W-98: Crashed once on file viruses; after resetting the system, test works without problem. Crashed often on Vkit, so every directory had to be scanned in batch using command- line (10706 lines Batchfile !) NAV W-98: Crashed several times, not reproducable. Animation slows down during scanning. One error message (Program error) on Poly, continued on Ignore. But succesfully ended scanning. NOD W-NT: Testing partitions O,T resulted in "exception: access violation (0xc0000005), Adresse: 0x0041cd80"; worked properly after patching. NVC DOS: Crashed upon unZIPing archives, packed file and macro viruses could not be tested); several crashes upon (un- packed) file and malware entries (test not finished). W-98: Several crashes; product is untestable on Vkit. W-NT: Crashed at R:\WORD\DZT\G and r:\word\dzt\g. 3 crashes at r:\word\hybrid\ (d-k) (directory was blocked) (impossible to block files only: despite setting option "ignore locked files", file couldnot be opened) PCC W-98: Rather often crashes (randomly), was not testable on Vkit, slows down system clock. PRO DOS: Crashed several times; after reset, all runs fine. W-98/W-NT: Could not be installed. TSC DOS: Crashed when started under SIMBOOT (images scanned). W-98: Crashed randomly, not reproducable. SCN W-98: Crashed sometimes, not reproducable. Parameter /autoscan did not work as expected, as scan must be started manually (otherwise, no subdirctories are scanned (/sub is set)). W-NT: Directories from U,W were scanned separately (otherwise, "Scan Error Administrator / Critical error occurred, unable to allocate enough memory to continue scan."); crashed upon de- installation. VET W-NT: Upon installation, General File Transfer Error (-2), one crash (but thereafter, VET completed satisfactorily) VSP W-98: Several crashes at different entries (not reproducable). VBW W-98: Bug in installation: macro scanner is automatically installed, whereas file scanner must be explicitly installed. Crashed often randomly. Vkit test needed more than 40 tries !!!!! W-NT: Crashed during installation (Dr. Watson) on 2 test clients.