View Full Version : scan stops short of completion
Guardian666
2008-03-28, 21:17
Let me begin by saying that I have tried to search for the solution and can't seem to get a hit on it. I might be a bit rusty since I have used Spybot since day one and have NEVER had anything more than the simplest glitches, so if I missed it - apologies in advance. I have tried just about anything I can think of or have seen suggested for "similar" stuff hoping to resolve it but now I feel that maybe its not me and the community should know so we can resolve it.
The scan runs relatively super fast (less than 3-7 minutes on average pretty much always). This is running in advance mode, with NO exceptions or exclusions, and running the full gambit. Overheating, system resources, updates, program version, OS related stuff are not the issue as I would have caught those after 15 years without bugging everyone.
I have a Windows XP SP2 with all proper patches and updates in place. No conflicting applications running. I have limited the system to JUST running Spybot to make sure no conflict. My system is 99% of the time very clean (I am pretty anal about security) so I doubt any malware is blocking the scan. The scan ALWAYS goes to 121552/124243 which is almost at the end and gets there pretty quick but even after leaving it running for countless hours, nothing.
Now the kicker is that neither the application nor the OS or any other application freezes or anything. In fact I can pause or cancel the scan and do anything I want in the meantime, even clean close the application but it will NOT complete the scan for the life of me I don't get it. I have attached a screenshot for what its worth and would appreciate any help. I am running the latest build with the latest updates as of this morning.
Any and all help would be greatly appreciated. Let me know if you need more information than already mentioned to be provided. I love the application and just want to see it do what it was intended to do and do it well, that's all.
Gardian666
The counter 124243 in the count indicator 121552/124243 suggests you have the scan set up to detect all items, including the optional Usage tracking group. The 121552 part of the indicator is the number that would be if the Usage group were not ticked. ie a scan without the usage group ticked would currently complete at 121552/121552.
Could you go into SpybotSD, advanced mode. Go to Settings, then File Sets and untick the last three boxes on the page, Usage Tracking, Beta.uti and Track.uti. Then try another scan and see whether it completes
Guardian666
2008-03-30, 09:16
Gardian666
The counter 124243 in the count indicator 121552/124243 suggests you have the scan set up to detect all items, including the optional Usage tracking group. The 121552 part of the indicator is the number that would be if the Usage group were not ticked. ie a scan without the usage group ticked would currently complete at 121552/121552.
Could you go into SpybotSD, advanced mode. Go to Settings, then File Sets and untick the last three boxes on the page, Usage Tracking, Beta.uti and Track.uti. Then try another scan and see whether it completes
@Greyfox
Thank you for your help and I did what you said and still no go, it hangs at 121552/121552 and doesn't "finish" as you can see in the attached screenshot, so what now? I am open to suggestions and willing to do my part. Thanks again in advance.
Guardian666
First, if you haven't already tried this, clear out all of your caches, temporary files and cookies. Also if you haven't already done so, see if the scan will complete if it is done in safe mode.
As another suggestion, start SpybotSD, go to Tools/Resident and untick both resident SD Helper and Teatimer. Then go to Settings/File Sets and untick all of the boxes except the very top one. This should give you a very short quick scan. If it still hangs, the only thing I can suggest is to un-immunise all items, uninstall SpybotSD and then do a complete new install. Leave downloading the updates until the install is finished.
If it doesn't hang at the completion of the short scan, try reticking the boxes one by one until it hangs again. Perhaps this may provide a further clue as to what is happening.
Sorry I can't come up with a quick fix.
Angelb63
2008-03-30, 17:55
I have a similar situation. I just downloaded 1.5.2.2, downloaded updates, immunized, etc. All the things one is supposed to do. I click Check for Problems and although it says "scan is inprogress" nothing is happening. Nothing at all.
Guardian666
2008-03-30, 21:07
@Greyfox
Although I know for a fact that my cache is deleted nearly immediately after using my browser (Firefox) which is the only one I use, I will do what you suggested to be sure. I also have tried in safe mode with no luck but again, I will try it with your suggestions and see. I have also done a clean install but maybe I will do another one with the suggestions and see. I will let everyone know what happens and I will try to do it step by step so it it shows a natural logical flow to maybe aid in pinpointing what is going on. On a somewhat related notice, this exact same thing happens on 1 Vista machine out of 2 and 3 xp machines out of 3. We are currently doing a system snap shot comparison to see if there is something substantially different in the app sets that might be interfering. We'll see.
@Angelb63
Although I am sorry for your troubles, I am glad that I am not alone and not losing my mind. Hopefully any resolution we discover will help you resolve yours too.
@Guardian666
By regarding your screenshots the counter is showing that Spybot is trying to scan the "Download directories". So I suppose that you have added some directories which you like to scan. There is an undocumented problem in Spybot which isn't fixed yet. Sometimes there are some files or some directories which could cause troubles like freezing the scan. So I'm afraid that you have to deactivate the scan of these directories as long as you couldn't pinpoint which file or which directory is causing this.
P.S.: I vote that Spybot should show which file and which directory it is currently scanning. This could make it easier to track down problems like this.
Guardian666
2008-03-31, 00:05
@chi-va
I thought that might be it too as I saw that and thought it was weird that it was showing that but nothing was happening. I have checked the "download directories" setting and it is set to whatever default comes with Spybot, but I will disable it and see what happens.
Thanks for all the help and I will post my results as soon as I can get them done. I also agree that it would be nice if it told you where it was in the grand scheme of things so you can figure out easier what is happening. Maybe they will be nice enough to implement that.
If this has not changed in Spybot-S&D 1.5.2 then the default setting for the download directories is no directories. So there shouldn't be any directories if you haven't added yourself. Although I have seen that it has added directories by itself once but I wasn't able to recreate this so it wasn't worth to report it.
Guardian666
2008-03-31, 00:40
@chi-va
I didn't think anything of it but when I checked it has the "C:\Windows" and "C:\Program Files" and "C:\Documents and Settings" in there and its on all the machine installations that I have. I don't remember if they were there before but I didn't add anything. Honest :halo:
Guardian666
2008-04-01, 02:46
Guardian666
First, if you haven't already tried this, clear out all of your caches, temporary files and cookies. Also if you haven't already done so, see if the scan will complete if it is done in safe mode.
As another suggestion, start SpybotSD, go to Tools/Resident and untick both resident SD Helper and Teatimer. Then go to Settings/File Sets and untick all of the boxes except the very top one. This should give you a very short quick scan. If it still hangs, the only thing I can suggest is to un-immunise all items, uninstall SpybotSD and then do a complete new install. Leave downloading the updates until the install is finished.
If it doesn't hang at the completion of the short scan, try reticking the boxes one by one until it hangs again. Perhaps this may provide a further clue as to what is happening.
Sorry I can't come up with a quick fix.
Quick fix isn't what I was looking for but I was hoping I wouldn't have to work so hard to get a software that was working fine until they decided to update it work again like it should.
To make this quick since I think I have wasted WAY too much time on this with no result to show for it, I have done everything you suggested and even did it inside a clean XP install VM with NOTHING else installed and still same disappointing result.
For now I don't have more time to waste, hopefully the developers will actually take the time and read through people's problem and do something or at least suggest something, the lack of any effort on their part to properly support their application is becoming very disappointing.
And before someone says its free what do you expect, providing a software for free is no excuse for not caring if it works or does what it needs to do. That's a very poor stance to take.
drragostea
2008-04-01, 08:12
Hrm. When I first installed Spybot-SD, I left the options just as it is. However, I have attempted Advanced Mode several times. First, I went to ignore products. CD-zilla is check by default I think and so are several others. Uncheck them, so all of the products are unchecked. Before a scan, every week, I clean the temp. files, cache, in my browsers. Hope that help,s Guardian. >:D :euro:
Guardian666
2008-04-01, 08:33
Hrm. When I first installed Spybot-SD, I left the options just as it is. However, I have attempted Advanced Mode several times. First, I went to ignore products. CD-zilla is check by default I think and so are several others. Uncheck them, so all of the products are unchecked. Before a scan, every week, I clean the temp. files, cache, in my browsers. Hope that help,s Guardian. >:D :euro:
I appreciate the help. Yeah I noticed that some items are check by default as well and my policy is to clear them every time that I do a clean install on a machine, so they are all unchecked, so it goes after everything. I also have my Firefox set to destroy and dump ALL information when it closes and I do NOT EVER use IE. As for the temp directories, I have a tweak that clears that usually before running the scan. This is why I am at a loss, I have done everything that can be logically assumed to pose a problem but still nothing :hair:
Do you have removed the "Download directories" from Spybot-S&D as suggested? I wonder if someone else or something else have added them or maybe this is really a new default setting in Spybot-S&D 1.5.2? Can anyone confirm this?
By the way, it could be related to the software you use for cleaning the temp directories. May I ask the name of the software?
If you still have the time and the will to fix this problem then we have to use an additional file monitor software in order to see where Spybot-S&D is hanging or whatever.
[...] or maybe this is really a new default setting in Spybot-S&D 1.5.2? Can anyone confirm this?
No there is no new default setting that adds download directories.
However future detection rules may be able to add a download directory if traces of certain targets are found. Currently there are no such rules public.
Ungarsdequebec
2008-04-01, 17:09
No there is no new default setting that adds download directories.
OK then, since I have the very same problem (see other thread (http://forums.spybot.info/showthread.php?t=24649)), I will try to remove the directories that are specified in there. I'm not writing this post from my home computer so I cannot tell you what folders are specified in there, but there are at least four, and I sure did not add them myself. (to be continued)
Ungarsdequebec
2008-04-01, 17:21
@Guardian666There is an undocumented problem in Spybot which isn't fixed yet.
Chi-va, Could you tell us if there is a forum thread about this "undocumented problem", and where to find it? I would like to contribute to identifying the problem, if I can.
Guardian666
2008-04-01, 20:31
Do you have removed the "Download directories" from Spybot-S&D as suggested? I wonder if someone else or something else have added them or maybe this is really a new default setting in Spybot-S&D 1.5.2? Can anyone confirm this?
By the way, it could be related to the software you use for cleaning the temp directories. May I ask the name of the software?
If you still have the time and the will to fix this problem then we have to use an additional file monitor software in order to see where Spybot-S&D is hanging or whatever.
Yes I did remove the download directories and it completes more often than not when that happens but still there are times about 60% of the time that it will still have the same behavior. I did NOT add the directories, they were there and I didn't think they were default but here they are.
I do not mind telling you at all, I use TweakNow the latest version to do the 1-click cleanup that I have configured for it and it deletes all the TEMP directories regardless of their locations and also all the internet caches.
I would appreciate more tools and would be willing to try them and report back, just tell me what and where to get it and it will be done. Thanks.
Guardian666
2008-04-01, 20:33
No there is no new default setting that adds download directories.
However future detection rules may be able to add a download directory if traces of certain targets are found. Currently there are no such rules public.
Appreciate the confirmation that they are not default but they were there and they got there not just on this machine but others that are having trouble with the same thing. I wonder how they got there :scratch:
Guardian666
2008-04-01, 20:35
OK then, since I have the very same problem (see other thread (http://forums.spybot.info/showthread.php?t=24649)), I will try to remove the directories that are specified in there. I'm not writing this post from my home computer so I cannot tell you what folders are specified in there, but there are at least four, and I sure did not add them myself. (to be continued)
Yes I had them too before I removed them, mine were "C:\Windows", "C:\Program Files", and "C:\Documents and Settings". If you have the same ones, then it might be something worth looking into to find out how they got there if they are not default added or by any rule that is public. Thanks.
Guardian666
2008-04-01, 20:36
Chi-va, Could you tell us if there is a forum thread about this "undocumented problem", and where to find it? I would like to contribute to identifying the problem, if I can.
I'd be happy to contribute too since I started this thread for lack of finding a related one. This way, maybe the admin can combine the threads to increase the number of feedbacks for everyone to follow and reply. Thanks.
I have three PC's myself on one network, 2 XP-Home, 1 XP-Pro, plus I look after another two on another network, all with SP2 and all up to date with MS updates. Processors vary from P4, to Core2Duo and also a Core Solo Centrino in a notebook. All of the PC's have 1GB memory.
Three use AVG antivirus and two CA antivirus. All have ZA firewalls (primarily to stop outgoing) as the routers on both networks have internal firewalls. All use Internet Explorer and Outlook Express.
All of these PC's have the current SpybotSD with both IE Browser plug in and Teatimer enabled. On installation of Spybot on each PC, I can confirm that there were no folders specified by default in the download directories option. After the install, I specifically added a single download directory (the same on all PC's). It has not subsequently changed on any of the machines since.
None of the above machines has experienced any problems with completing SpybotSD scans which are done immediately following each SpybotSD update.
I know that the above doesn't help overcome the problem Guardian666 is having, but it does confirm that the problem is not being experienced by all, which makes it all the more mystifying.
Terminator
2008-04-02, 16:07
I ran my first scan today on my new Vista computer which took Approx 12 mins to complete with no problems experianced.
I am running Vista Home Premium Fully Patched (I've not installed SP1 Yet due to concerns about compatiblity with some of my programs).
I'm also running Spyware Blaster 4.0, Windows Defender, Avast 4.8.1169 Free Edition, Zone Alarm Free Edition 7.1.248.000.
I also have a router with a built-in Firewall
Guardian666
2008-04-02, 20:19
..., which makes it all the more mystifying.
@Greyfox
The input is appreciated and you said it best, all the more mystifying. I have been running Spybot in so many different configurations of OS and application environments for so long that it would take ages to list and I have never had problems either, until the latest update and after that even rolling back does not work anymore. That's what's bugging me about all this. If I was some novice who was doing this for the first time, I can understand how I might be the problem but I have been in this field for 15+ years and using this software since its inception so I just don't get it why its doing this. And formatting starting everything over is not an option on a network of 8 machines used for business everytime something goes bump.
@Terminator
I am very happy that it is working for you and I hope that it continues because it becomes frustrating only when it doesn't work.
Could it be possible that you have downloaded the beta detection updates and you are also using them in the scan. This could be a reason why
you have additional download directories which don't seem to be in the normal detection updates. If yes, please deactivate the beta detection updates in the
"File sets".
A tool which can be used for tracking problems like this is "Filemon":
http://technet.microsoft.com/de-de/sysinternals/bb896642(en-us).aspx
It is not easy to use and it can give you a lot of hints but not every hint is leading to the cause. Read the tutorial before using the
tool.
What should you do?
1. Start Filemon.
2. Start Spybot-S&D and start a scan.
3. Before the scan reaches the point where you have experienced the freezing please start the log function of Filemon. Of course you could
also start the log before starting the Spybot-S&D scan but this will only make an unnecessary long log file which takes more time to investigate and which includes many useless information.
What are we searching for?
During the freeze there should be hints like entries where the requests are not successful. Maybe these entries repeat themselves as well. If you have found them
following information are useful: The full path name and the additional information in the "Other" column. Enlarge the columns with the mouse cursor if necessary. You can scroll back the log as well if necessary.
So we are looking for loops, file not found errors, access errors or anything similar to it. As long as there are new request for new files from Spybot-S&D then the software is not freezing. This only means it is still scanning files but Spybot-S&D doesn't indicate the progress. If the are additional download directories it could take several hours depending from the numbers of files which should be scanned.
Please consider that finding the cause will only allow the user to find a workaround so that Spybot-S&D can at least finish the scan. Maybe every single user with this problem finds different file names. So please don't make any changes without individual instructions.
It seems that there are many confusions about the "Download directories".
In this case, we need more explanations what it is. First of all, we need a roughly idea how Spybot-S&D makes a scan. This is copy from one of the FAQ:
"Why is Spybot-S&D that fast?
There are different ways to search for spies. One would be to search the complete registry and hard disk for suspicious entries and files. That can take a lot of time.
Spybot-S&D takes advantage of the fact that all spies have to anchor themselves at a few places over the system to get active. It starts to search at this places, following the information gathered there to catch the whole spy."
In other words, Spybot-S&D does not scan every single registry entry nor every single file by default. It searches for hints and if it finds anything it will follow it back to the root. This is only a roughly explanation and it must not apply to every single detection but at the moment it should be enough for us.
Adding folders to the "Download directories" allow you to scan for malware files without regarding the hints in the system. This is useful for detecting inactive malware. For example if you have downloaded an installer of a software and you want to know if this is any kind of malware which is already in the database of Spybot-S&D then you have to add the folder to the "Download directories" where you have downloaded the file. This is the reason why it is called "Download directories", an antivirus software would call this making a deep scan. Be aware, even if an installer has passed the test once, there is always the possibity that it is a not detected malware. So repeating a download directory scan isn't wrong especially with new detection updates.
By the way, there is already a single file detection available.(Beta test?) It is much faster because you don't have to go through the whole scan and it can do even more than the download directories but I don't understand it so I cannot give you any more details.
drmsucks
2008-04-03, 02:03
Running 1.5.1.18 on XP SP2 machine and scan hangs at 62097/121366, virtumonde.dll. Also run AVG and Spyware Terminator real time - they are running when I run Spybot. I need to invoke Task Manager to close Spybot. Suggestions?
Guardian666
2008-04-03, 02:38
@chi-va
It is not just a possibility but an absolute fact that the beta is downloaded and installed and searching. Also, I already reported on my first post that Spybot is not "freezing", I can pause it, stop it and even cancel and close it without a problem, just no progress and completion. So I shall download and use Filemon to figure out what and where its getting stuck and I will let everyone know. :yes:
Also, thanks for clarifying the "Download Directories" for everyone since I think its taking us away from the problem that I brought up and I know what they are already. :bow: This way we can focus on what is actually ailing us (or me at least).
@Angelb63, Ungarsdequebec
Please start your own threat if you haven't already found a workaround. You will get more attention with
your own threat and it would make it easier for the helpers. According to this, we will get an impression how
many users really have similar problems.
@drmsucks
I'm afraid I have the same advise for you but it maybe necessary to start a new threat in the malware removal
forum because it seems that Spybot-S&D has problems to fix Virtumonde. I cannot confirm if you infected with Virtumonde or
not or if this is only a coincidence that it stops there. Before starting a new threat in the removal forum it is important that
you read the instructions for using the malware removal forum first.
@Guardian666
I'm sorry. I know that you mean no progress. "Freezing" wasn't the right word but apart from that using
Filemon isn't wrong no matter if it is "freezing", "hanging" or "aborting scan". If Spybot-S&D is the culprit itself
then it is likely that it is caused by the last detection updates or the new rootkits plugins because you didn't have problems before if
I have understand this correctly. Anyway, please try Filemon. It is better than guessing all the time.
Guardian666
2008-04-07, 12:38
Ok, sorry about the delay everyone but here are the findings and I am not sure the best way to display them, so here goes nothing.
1. Refer to the screen shot and you can see that it gets stuck at 125015/127706 while scanning directories which of course it doesn't tell you, hence why we are using the Filemon to post some data.
2. Filemon, which took a small learning curve to get a hang of, reveals that it is going through the directories and it is not actually getting stuck but among all the "Successful" indications are regular and consistent "Failed" entries which I don't personally get, so I have attached it here for you to look at. As you requested, I limited it to a small amount as it would be WAY TOO BIG to post hours of data and it wouldn't lend any more insight than a small portion anyway.
3. Finally you can see that I can successfully stop the process and it will stop as it is not "frozen" so it still responds. At this point, you can see that it shows the directory where it last was in the status bar.
Final assessment, we are still at square one as to why it won't complete and why it gets "stuck" in scanning directories and won't complete no matter how long it runs. Hopefully someone can make something of this and we can use the feature of download directories rather than removing them just to get it to complete.
Welcome back.;) By regarding your additional information I would say that it is still scanning. How long do you have waited before you have taken the snapshot with Filemon. If I'm guessing correctly it was at still the beginning of the download directories.(Folders with "A") As stated in the FAQ checking every file could take a lot of time. As long as Filemon is showing that there are new access requests for files from Spybot-S&D is not really stuck.
Does anyone has any empirical values using Spybot-S&D with a complete system drive in the download directory? I have tested it on my Windows XP system a few months ago(10 GB data) and it wasn't finished after 4 hours. Filemon has confirmed that it was still scanning before I have aborted the scan.
By the way, I don't think that Spybot-S&D is designed for scanning all files by default. It is more likely that the "Download directories" should only cover a small amount of files.
Guardian666
2008-04-08, 19:28
Its not an excuse but it has been so hectic around here but its good to be back. I just wish I had a chance to get the information back quicker.
I noticed its still scanning too but I was very surprised, as you would think as well, it was over 30 minutes into the scan but it was still doing the A's. It must have an incredibly poor or redundant folder enumeration and searching. I have used API code to fly through an entire drive in less than 10 minutes.
I know its not stuck but I left it running a solid 6 hours+ the other day on my XP which is relatively minimal (not the one you saw the filemon snapshot for, that is my own personal machine) and I couldn't believe that it was still not done. They might consider a better way to do it. Especially that I agree, I don't think its checking EVERY file either, so it should go even faster.
I would love to be able to use the directory scanning feature but this really puts a damper on it. I removed the directories and it completes in like 15 minutes most of the time.