Very long scan time and high CPU usage

[Mele20]

Beginning October 14, Spybot started taking an unusally long time to scan. The last scan before that was on September 24 and it took 7 minutes and 30 seconds which was typical time for a scan. I changed NOTHING in settings after that nor did I acquire a large number of new files. So, the October 14 scan and every scan since then should still take about 7-8 minutes. However, the October 14 scan took 27 minutes! The scan yesterday took even longer at 32 minutes!

Beginning with the October 14 scan, CPU usage by Spybot has jumped to an average of 80% and varies between 60-100% during the scan. This makes the computer sluggish so that I can't really do anything else for what is now about 35 minutes to scan the same files that were scanned in 7-8 minutes in September and earlier.

What could be the problem? I've used Spybot since its inception and I don't recall it ever being this slow or using so much CPU when scanning.

[Mele20]

How do I edit?

I wanted to add that I am using XP Pro SP2.

[JackCam614]

Hi Team Spybot,

An excerpt from my post in BBR regarding Mele20's slow scanning problems.

"SpyBot prides itself on its fast scanning time, so I suspect they will be anxious to assist Mele with diagnostics and repair."

Please forgive me if I appear intrusive, but you fine folks of Team Spybot wouldn't want to make a liar out of me, would you?

I know it's a Holiday Weekend, so fell free to tell me to

TIA for your assistance in this matter.

Best Regards,
Jack

[Mele20]

I uninstalled Spybot but didn't remember there is a tool to clean the registry so I didn't use that tool. I did download a fresh copy before reinstalling. Got everything set up properly and scanned again. The scan took 30 minutes, whereas, before the Oct 13 definitions it took only 7-8 minutes.

This time I did nothing on the computer but watch Spybot. I ran Process Explorer, instead of Task Manager, to track CPU usage. In the past, I have not watched Spybot so I'm not sure what is normal and what isn't. I noticed very high Kernel usage. Maybe that is normal? I also noticed one CPU much more heavily taxed by Spybot and that does not seem normal. According to Process Explorer, Spybot was using 35-45% of the CPU with System using another 25-30% for a total of about 70-75% CPU usage when I was doing nothing but letting Spybot run. That doesn't seem right!

I exited Kaspersky MP1 (as that is the only changed/new application in some time) before running this last scan. It made no difference.

XP Pro SP2
Dell XPS 600
Pentium IV 3.8GhZ, hyperthreading CPU
2GB RAM
160GB hard drive - 106GB used
nForce4 mobo

[Mele20]

Gee, it's been about 72 hours since I posted and asked for some help. I know last weekend was a holiday weekend but I also have seen others who have posted since I did get help. Did I somehow fall through the cracks?

It seems to me, like Jack said, that Spybot prides itself on fast scans so when a user (I've had Spybot from the beginning) suddenly experiences very long scans that you Team Spybot guys would be on the problem really fast. I reiterate that this problem started with the Oct 13 definitions which I got on Oct 14 and has gotten worse with newer definitions. I did, as explained earlier, uninstall and downloaded a fresh copy and reinstalled. I forgot about your tool to clean all the bits off from the first install. Would that fix this problem? What could be causing this problem which started so suddenly? I am not having any other problems with my computer.

I have always loved Spybot and have recommended it to everyone I help. I even got rid of Bit Defender Pro 9.5 early this past summer because BD support told me that Spybot had to be uninstalled for BD to work properly. I got rid of a paid application in favor of keeping Spybot. Then several months later, I have this strange problem with Spybot. I've never had it take a long time to scan until Oct 14 in all the years I have used it on three different computers.

[md usa spybot fan]

Mele20:

From the following you will see that the elapsed times of my Spybot scans are increasing. However, unlike your observations, I equate this increase of elapsed times to additional malware checks within Spybot and an increase of the number of registry entries, files and bookmarks stored on my system.

Update Date	Downloaded		Scan Started		Elapsed time
09/22/2006	09/22/2006 07:28	09/22/2006 07:30	07:08
09/29/2006	09/29/2006 09:09	09/29/2006 09:11	07:38
10/06/2006	10/06/2006 08:08	10/06/2006 08:13	08:18
10/13/2006	10/13/2006 09:49	10/13/2006 09:52	08:34
10/20/2006	Update missed		
10/27/2006	10/27/2006 12:28	10/27/2006 13:08	09:21
11/03/2006	11/03/2006 07:10	11/03/2006 07:27	10:33
11/10/2006	11/10/2006 07:11	11/10/2006 07:14	10:09

The only conclusion that I can draw is that something has change in your system or something was in your system that the newer detections within Spybot take longer to analyze.

ps: I personally find your comment "Gee, it's been about 72 hours since I posted and asked for some help" unwarranted and somewhat grating. Three weeks between your September 24th and the October 14th scans and another six weeks to report that you suspected a problem. Exactly what do you expect?

[Mele20]

Your scan increase time is very small. I don't think it can be compared to mine. My increase went from 7and one-half minutes in Sept to almost 30 minutes in Oct and increased further by about 5 minutes in November. Can you use your computer during a Spybot scan? What is the load on your CPU?

My friend Jack who posted in this thread has a computer very similar in speed and processor. His hard drive is the same size as mine and he has used 92GB of his 160GB hard drive. I have used 106GB of mine. His recent scans took 6 minutes. Why would it be normal for mine to take 30 minutes? I have nothing unusual on my drive and no new applications in the past several months.

You feel it is normal for the CPU to be occupied 70% when I am doing nothing but a Spybot scan? It is normal for the Kernel access to be extremely heavy throughout the scan? It is normal for one CPU to be heavily taxed by the Spybot scan while the other is lightly taxed?

I only do a Spybot scan after getting the new definitions. Spybot has never found any spyware nor has any other antispyware application (when I have tried other applications over the years on three different computers). Spybot occasionally finds an FP but I don't get spyware so there is no need for me to run scans more frequently. I didn't ask about this when I first noticed it because I thought is probably a one time anomaly. It was only after the Nov definitions and a second scan that was way too long and too CPU intensive that I first asked about it at my home site (dslreports.com) in Jack's thread on his Spybot problem. When Jack ran another scan to compare to mine and told me that his scans were 6 minutes, I decided I needed to come and ask here about it. I waited all this time because I didn't want to bother anyone if it was a one time anomaly.

I tried very hard to write what I thought were nice posts. I can't understand why you would take offense and I am sorry you were offended.

[md usa spybot fan]

For anyone interested:

The discussion referred above is in this dslreports.com thread:

• http://www.dslreports.com/forum/remark,17300182

There is also a screen print of CPU utilization here:

• http://www.dslreports.com/speak/slid...c3RhcnQ9NDA%3D

Mele20:

Your scan increase time is very small. I don't think it can be compared to mine.

I showed those scan times to demonstrate that there was apparently no radical change in Spybot's detect rules that could account for your increase in scan time.

Some of the detection files (Includes) were split with the updates of 2006-10-13. Although this change corresponds with the time you noticed the problem, I don't think this could be the cause of the problem. The following Includes files:

• Beta.sbi
• Beta.uti
• Cookies.sbi
• Dialer.sbi
• Hijackers.sbi
• Keyloggers.sbi
• Malware.sbi
• PUPS.sbi
• Revision.sbi
• Security.sbi
• Spybots.sbi
• Tracks.uti
• Trojans.sbi

Became (new file names in bold):

• Beta.sbi
• Beta.uti
• Cookies.sbi
• Dialer.sbi
• DialerC.sbi
• Hijackers.sbi
• HijackersC.sbi
• Keyloggers.sbi
• KeyloggersC.sbi
• Malware.sbi
• MalwareC.sbi
• PUPS.sbi
• PUPSC.sbi
• Revision.sbi
• Security.sbi
• SecurityC.sbi
• Spybots.sbi
• SpybotsC.sbi
• Tracks.uti
• Trojans.sbi
• TrojansC.sbi

Can you use your computer during a Spybot scan?

I can and I usually do, although the elapsed time of the Spybot scan increases if I use the system heavily.

What is the load on your CPU?

During a Spybot scan, at or near 100% except certain points during the scan (there is more on CPU time below).

My friend Jack who posted in this thread has a computer very similar in speed and processor. His hard drive is the same size as mine and he has used 92GB of his 160GB hard drive. I have used 106GB of mine. His recent scans took 6 minutes. Why would it be normal for mine to take 30 minutes? I have nothing unusual on my drive and no new applications in the past several months.

The size of the drive and amount used may or may not be a significant factor in the difference in the scan times. Unlike anti-virus programs and some other anti-spyware programs, Spybot does not do an in-depth scan of every file on the primary drive. For a brief description of how Spybot scans see:

• Why is Spybot-S&D so fast?
  http://www.safer-networking.org/en/faq/1.html

There is a special scan for Spyware installers that will inspect the names of all the files on your drive if you mistakenly set it up to do that. I will also cover that later.

You feel it is normal for the CPU to be occupied 70% when I am doing nothing but a Spybot scan?

Processor: Intel - Pentium 4 CPU (2.40GHz)
Processor Speed: Between 2.34 GHz and 2.39 GHz depending on what monitor I use.
System Bus: 533 MHz
Memory (RAM): 512 MB (2 [matched pair] - 256 MB Dual Channel DDR SDRAM @ 333MHz)
Operating System: Microsoft Windows XP Home Edition (Version 5.1.2600 -fully updated)
Hard drive: Western Digital - 80GB – ATA/100 – 8MB buffer – 8.9ms average seek time - 32% used – not defragmented recently

A Spybot scan doing little else (times are in seconds):

				Elapsed	CPU	Percent
Process				Time	Time	CPU

SpybotSD.exe			644	581	90.22%
Mcshield.exe			644	28	4.35%
System Idle Process		644	6	0.93%
aolsoftware.exe			644	3	0.47%
TeaTimer.exe			644	2	0.31%
svchost.exe			644	2	0.31%
explorer.exe			644	1	0.16%
WINWORD.exe			644	1	0.16%
services.exe			644	1	0.16%
waol.exe			644	1	0.16%
System				644	1	0.16%
vsmon.exe			644	1	0.16%
Other or unaccounted for	644	16	2.48%

Total					644	100.00%

From the above, you see that my Spybot scan runs at 90.22% CPU with all but 2.5% unaccounted for.

It is normal for the Kernel access to be extremely heavy throughout the scan?

The amount of CPU used by the Kernel on your system seems extremely high compared to my observations during the running of the Spybot scan above. Although I did not attempt to quantify the utilization, I would say on the average it was maybe 20% - 25% and during extended periods of time was near zero.

It is normal for one CPU to be heavily taxed by the Spybot scan while the other is lightly taxed?

I don’t know since I have a single CPU system and never had the occasion to look at CPU utilization on a HyperThreading system. Perhaps Jack can answer that if he has a HyperThreading CPU.

Things that I know will increase the elapsed time of a Spybot scan (but not necessarily to the extent nor as suddenly as you experienced).

Excessive number of Windows Temporary Files. You should periodically delete your Windows Temporary Files. One method is to go into Start > Run > type %temp% > click OK > delete everything older than your last reboot.

Excessive number of Temporary Internet Files. Periodically delete Temporary Internet Files by going into Internet Explorer > Tools > Internet Options > General tab > … the exact method varies from here because of the change from IE6 to IE 7.

Excessive number of Bookmarks. It appears that the checking of Bookmarks (Favorites) takes a significant amount of time at or near 100% CPU utilization.

There is a special scan for Spyware installers (files not installed yet) that can be limited to specific directories (actually it should be limited) which can be located on either internal or external drives. To activate this feature go into Spybot > Mode > Advanced mode > Settings > Directories and add directories. There is more on what this feature scans for here:

• Why does Spybot-S&D find so many Spyware installers / how is that Download directories setting used?
  http://www.safer-networking.org/en/faq/15.html

If entire drives are added to this feature for scanning it can result in elongated scan times as well as cause false positives. It is suggested that if you uses this feature that it is limited to only the directories were you store download executables or installers.

*********************

I don't think that any of the above will real help solve your problem. Do you keep backups that would allow you to return Spybot to the state it was in on 2006-09-24? That way running Spybot with those detection files would conclusively point to whether or not it was changes to Spybot or something that changed in your system that caused the radical change in the elapsed time of the Spybot scan.

[Mele20]

Thank you so much for the detailed response.
It was very interesting to read but I agree with your final conclusion that none of that really helps me figure what the heck the problem is that I have and why I have it or why I suddenly got the problem. I agree that it is not likely due to the splits in some of the detection files...although it sure is curious that I experienced the problem the first time after getting those definitions but I can't imagine why splitting would cause such an extreme reaction on my computer.

I seldom use IE so there are usually only a few cookies in TIF but I do have about 350 files in the Temp folder. Those are hard to delete because something is always using some of them so you can't select all and then hit delete. I don't have excessive bookmarks I don't think...I'm not sure how to check on Fx to see how many I have. I have very few Favorites in IE. I didn't know about the special scan for Spyware Installers so I had no folder listed for that. But these possible "culprits" have not changed much over the months and this was a sudden problem so I doubt they have anything to do with it.

I see from your scan that "system" is using very little CPU. On mine "system" was using 25-30% CPU during the scan. That seems to be amiss and the only thing I could think causing it was the new MP1 Service Pack for KAV but I exited KAV and did a scan with the same results. And then there is that weird excessive Kernel usage and the odd taxing of one CPU much more than the other. I'll ask Jack as I think he has hyperthreading. I know Libra (from the dslr thread) does.

I use both System Restore and Acronis True Image. I do have a full image from Oct 6 on my external drive. Unfortunately, for this sort of purpose it isn't the best because I did a complete image of the entire 160GB drive which is not partitioned. Consequently, restoring that image would take several hours and I'm not too eager to do it! I can try a System Restore back to before Oct 13 but I tried to use System Restore about two weeks ago and could not restore. A few days later I could but only to the very recent points so I think all the older ones back more than a couple of weeks (which I meant to get rid of and forgot to do it) are corrupted.

This is quite frustrating. I might end up doing a TI restore simply because I'd really like to figure this out. What I could do is just grab the Spybot file off the Image (after uninstalling the current Spybot) and it would have the Sept 24 definitions and see what time it takes now on the computer as it is today. I think that will work and if the scan is short then that would indicate the newer definitions. If it is long then something on my computer.

I appreciate all the time and effort you put into your post. That was quite comprehensive!

[Mele20]

I uninstalled Spybot. Ran RegCleaner. Then I copied the Spybot file from my True Image from Oct 6 to the host computer. It had the Sept 24 definitions and I did not update them. I ran a scan and watched it with Process Explorer instead of Task Manager. With PE mouse hover shows the usage for each CPU. The CPU load seemed much more balanced but Kernel access was very heavy until the bookmark scan at the end where it dropped to nothing. Plus, the scan took 23:23 and that was with fewer definitions since those were from Sept24. UGH.

I have no idea what could be causing this. It sure looks like (although I can't be sure unless I actually restore the image) something changed on my computer. Since the image was made on Oct 6 it might have whatever it is causing the problem. I don't have earlier images. My external drive is 80GB and one full image of the entire drive takes about 40GB so no room for more than one image. Ideally, I'd have an image from before Sept 24 to which I could restore but I don't.

I can't think of any change between Sept 24 and Oct 14 that might have caused this. It may remain a mystery. :(