Spybot's Poor Reputation

[jonathanbean]

In recent years, pc magazines and test sites have rated Spybot very poorly. I love open source software and will even contribute when it does a good job, but the test results make me very wary of any of the free antispyware programs, including Spybot. Windows Defender has even been rated better.

I'm not being critical, just trying to find a reason to use Spybot and recommend it to my many students and friends. There is excellent free antivirus (e.g., Antivir) and firewalls (e.g., Comodo) but no excellent antispyware programs without paying an arm and an leg. WHY?

[bitman]

Hi Jonathan,

The question you're asking doesn't have a simple answer, since there are as many factors involved as there are malware. I'll take a stab though, since I have personally been involved in past beta testing and support of Spybot S&D, MS AntiSpyware/Windows Defender, and the Windows Live OneCare protection suite.

The problem is that unlike a virus, which is primarily a simple file and easily detected as such using signatures, spyware are often complex combinations of multiple files, registry changes and permission changes to both the file system and registry. This results in very complex requirements to properly detect and remove such programs, especially when they're already deeply embedded into the operating system and effectively in control.

The reason it's important to understand this is that most of those evaluating AntiSpyware simply don't. With AntiVirus it was relatively simple to take a PC, copy hundreds or thousands of virus files to it and run an AntiVirus scan to see what was detected. The larger the count, the better the AntiVirus, barring any false positives which were also relatively easy to detect.

Spyware on the other hand often modify the file or registry permissions or even the running operating system itself to hide themselves, requiring special tricks or processes to detect and/or remove the malware. Combine that with the fact that some malware work together to make it difficult to remove their bretheren or automatically 'heal' themselves by reinstalling removed pieces and you begin to see the problem.

Now, try to determine a method of properly testing the abilities of various AntiSpyware applications in a fair way. If you believe you can do it, you're smarter than I am. First you'd have to determine if the combination of spyware files you have are one or more 'infections' and whether they're designed to work together. Then you need to perform tests using that set of spyware and all of the AntiSpyware applications you wish to test. Finally you need to measure the effectiveness of the detection and removal for each product. At this point you Rinse/Repeat for another piece of Spyware.

If you can find anyone doing this kind of testing I'll fall off my chair. The only thing I've ever seen attempted are mass installations of dozens of Spyware, usually by purposefully going to known 'bad places' on the Internet and infecting a PC, which is then cloned. At this point the PC is scanned by each AntiSpyware application with aggragate results of removal presented, in rare cases with itemization of types as well as 'counts' of items detected and/or removed.

I don't blame the testing groups for doing this, since anything else would be prohibitively expensive, However, since this only really measures the effectiveness of the manual scanning and removal, it totally denies the existence of real-time protection such as Spybot S&D's TeaTimer and SDHelper Resident modules. For the more skilled user, these proactive abilities are far more valuable than scanning and cleaning after an infection, but are virtually impossible to test effectively since they depend to some extent on the skill and knowledge of the user. This is why the testing ignores these, since they're almost impossible to measure fairly.

So the point is, ratings by testing sites must be judged by their criteria, which most of the sites either don't publish or simply don't fully understand themselves. For this reason, the only real useful measure of an AntiSpyware application is your own experience. Unfortunately, this doesn't necessarily translate to the next person, because the level of dependance on your personal knowledge and understanding are involved.

Mixed into this is the much more simply understood issue that at any given moment there are new malware becoming available that an indiviual AntiMalware application may not have yet included in their detections, often because they haven't gotten a sample yet. This means that some applications will miss this new malware entirely, resulting in a major hit on their 'score' even of they can fully detect and remove it a week later, especially free applications like Spybot S&D that have less resources to collect and produce detections for them. This is the reason for the oft touted suggestion to run several AntiSpyware, since maybe one will include the new malware.

So finally, the reason that Spybot S&D is often downgraded by even some reputable sites is that it is a very effective removal tool and real-time protection in skilled or relatively intelligent user's hands, but less valuable for those who don't understand it and/or turn off all of the real-time protection.

Windows Defender was designed for this less knowledgable type of user, so for the masses (Grama, children, non-techies in general) it's generally more effective. Windows Live OneCare (AV, AS, FW, Backup) is for an even less interested user and removes almost all configuration requirements along with automating everything possible, including updates. The result is that it will fare better in most tests, but it often drives techical users nuts by taking away their control.

The real point of all of this is that the most effective AntiSpyware is what suits the user(s) of the PC best. I personally use Spybot S&D on my PCs, in parallel with Windows Live OneCare on the Vista laptop since I help support that product. On my cousin's and sister's/nephew's PCs, we have installed only OneCare, since they have no interest in understanding these things at all.

Bottom Line, the best AntiSpyware is what works best for the user(s) of that PC. This answer will be different based on their interests and abilities to understand what's really going on and how much the application needs to make the decisions for them.

Bitman
(OneCareBear at WLOC forums)

[502512]

However, since this only really measures the effectiveness of the manual scanning and removal, it totally denies the existence of real-time protection such as Spybot S&D's TeaTimer and SDHelper Resident modules. For the more skilled user, these proactive abilities are far more valuable than scanning and cleaning after an infection, but are virtually impossible to test effectively since they depend to some extent on the skill and knowledge of the user. This is why the testing ignores these, since they're almost impossible to measure fairly.

Here, here!!! An ounce of prevention is worth (at least) a pound of cure!!! The lack of recognition of the importance of this feature of Spybot S&D clearly shows that the testers are missing the mark. As bitman states, it's extremely difficult to test this fairly, but it basically gets left out of the equation because it's so difficult to evaluate.

It's also very true that the level of automation is directly proportional to the degree of control. I dislike and refuse to use software that makes me do things only one way and/or limits the level and degree of customization. That's fine for unsophisticated nontechnical users but does not work for a guy who has over 25 years experience as a software engineer.

You must remember that these ratings are for the masses. Are you a knowledgeable person in the field, or one of the huge number of nontechnical users out there? If the latter, Spybot S&D is probably not for you. However, please consider getting better educated, because IMHO these other programs often lull people into having a false sense of security. Knowledge is the best defense against malware.

[jonathanbean]

That was the single best explanation of spyware vs. antivirus that I have ever read. Next time someone asks, I'll send it along! Thanks.

Well, I'm not the grandma in the masses, but I'm not a computing engineer either. I'm a techno geek who reads a lot and asks questions. I hope Spybot isn't intended just for computer engineers! If so, my university of 35,000 students (all given free Spybot) is doing us a disservice!

This gives rise to two questions:

1. How "intelligent" do you have to be to use TeaTimer effectively?

2. More important, TeaTimer is much like a firewall -- a preventative. There are many tests of whether spyware, hacks, etc. can get beyond firewalls -- why not the same tests for the resident shields (like Tea Timer) that run with every antispyware program?? If Spybot is ineffective at removal, I'm happy with a strong shield -- but how do I know it is strong? What makes it better than commercial or Windows Defender?

I assume the immunization plays a small part of the shield function.

[Tattenbach]

In recent years, pc magazines and test sites have rated Spybot very poorly. I love open source software and will even contribute when it does a good job, but the test results make me very wary of any of the free antispyware programs, including Spybot. Windows Defender has even been rated better.

I'm not being critical, just trying to find a reason to use Spybot and recommend it to my many students and friends. There is excellent free antivirus (e.g., Antivir) and firewalls (e.g., Comodo) but no excellent antispyware programs without paying an arm and an leg. WHY?

++++++++++++++++++++++++++++++++++++++++++++++++++++
Spybot S&D is no open source software. First mistake.

Not sure which magazines you mean because what I have seen is the other way around. Even Microsoft recommends SpyBot S&D in its Windows Marketplace site.

http://www.windowsmarketplace.com/de...o&itemid=20308

What are those "excellent antispyware programs" that are not free? Could you name just one?

Comodo just failed a leak test recently (and was updated) and AntiVir, has failed a number of tests too.

Is the fight against those who want your data there is not a 100% solution if you leave everything to the developers. Users need to educate themselves too. Sounds painful? Then stay in the closet.

[502512]

I didn't mean to sound like you need to be a software engineering guru to effectively use Spybot S&D. This isn't true at all. TeaTimer in particular runs all by itself and needs no intervention from you. Just make sure you allow Spybot access to updates periodically, either via automation or manually.

However, that's no excuse to stick your head in the sand with regard to malware. Education is the best weapon against malware, and it's the responsibility of all legitimate users on the web to stay up to date and to use anti-malware to combat the spread. The old saying "if you're not part of the solution, you're part of the problem" holds very true here. I can't tell you how many friends I've had to bail out of big problems with their machines because they use no protection against malware and/or surf the web irresponsibly.

I've found that subscribing to some of the various TechTarget sites can keep one up to date on threats with minimal effort. They'll send you emails. All you have to do is take the time to read them. SearchSecurity is a good place to start.

Cheers,
Jeff

[jonathanbean]

Tattenbach was rather snitty -- I'm not going in the closet, just trying to ask hard questions: Why no tests of resident shields? Isn't that something that can be done? If not, then I'm still in the dark and closet or no closet, we are still all clueless, "intelligent" users or not!

Regarding your post of the windows rating of Spybot: There were 224 programs listed and Spybot came in #13 -- but this was only by "customer rating." Sheesh, there were high rated programs I have never heard of in my life.

Also on Antivir, see av-comparatives.org, where it was rated Advanced+ with high heuristic detection rate too:

http://www.av-comparatives.org/

See August results for virus detection
See November for heuristic detection -- far higher than most others.

Closet door still open...

Resident shield testing???

[jonathanbean]

Here's the PC World results on Spybot:

http://tinyurl.com/ye9uvr

Again, prevention > detection but since someone asked for a magazine test, here is just one.

[bitman]

Jonathan,

In reference to your questions;

1. There are two functionally effective ways to use TeaTimer, one requires 'technical intelligence' meaning knowledge of the registry and spyware and how/where they might attack.

The other is to simply recognize whether anything you have recently done (accepting a download for example) could reasonably be expected to generate a TeaTimer alert, which only requires a 'common sense' level of understanding. If you don't believe you requested something, you should generally block the change.

2. Testing firewalls is relatively easy similar to antivirus, since either the traffic can pass or not, though 'leak tests' take this a bit further. Testing spyware can be a long tedious process if it's done right, per my description above, though some feel that's not necessary. I personally don't trust the results of any test unless the methods and complete results are described, so I can understand how they were determined.

I wouldn't want to rate the Spybot shield against that of Defender or others without complete information as to what they monitor, which I've never personally seen. There are descriptions, but it's been a while since I've seen a complete comparison chart of all the major application's real-time protection.

Immunization is another layer of protection relating to Internet Explorer; Restricted Sites, ActiveX blocking, Tracking Cookie blocking.

This is what I mean when I say that Spybot is most useful in the hands of someone with a more technical background, since without that knowledge its alerts can be mis-understood or simply missed altogether. This doesn't make it bad, just more suitable to a technical user.

Education is great, but unfortunately nearly useless for the masses. I salute those here and all throughout the web who attempt the insurmountable task of educating everyone, but I also understand that it won't happen, at least in our lifetimes. This has been true throughout the history of all technologies, including current items like the cell phone and iPod, and even older things like the car. Some will never take an interest in understanding a technology beyond their daily use, which is actually totally appropriate. Only the technically inclined (us geeks) have a problem understanding this, which is why new technology is so often difficult to use.

For the masses, the job of protection, along with updating and backup, need to be performed by the system itself. After all, wasn't this the promise of the computer in the first place? Instead of us spending our time managing, maintaining and protecting our computers, they were supposed to save us time and effort. This got lost over the last 20 years, since most of the software development was done by and for geeks. This is now finally changing, to the benefit of everyone, at least those who don't make money off of the ensuing mess.

Do note that the current development of Spybot S&D 1.5 is also taking this into account. See this link to a comment by Spybot's developer today that shows this direction.

TeaTimer 1.5 issues

Also note the comments on this page from your PC World reference, especially the second paragraph where TeaTimer is mentioned. They also don't mention these forums as a technical support option. probably because most articles in PC World are keyed to commercial applications, which pay for advertising after all.

You'll note that I believe in applications to aid the masses, not because the people who help in manual malware removal aren't helpful, but because it doesn't scale. It's also much better to stop the malware from installing in the first place, so alerting the user to this fact if it can't be stopped automatically is key, and at that same moment there is an opportunity to educate the user if they will accept it.

Spybot S&D was one of the first to monitor and alert in real-time for spyware with TeaTimer, it's gotten left a bit behind over the last couple years, but the coming 1.5 update looks promising. There are many clues in the first couple threads in the Beta forum, so read there for more specifics.

I still believe in Spybot S&D, though I also believe that it's best to examine the abilities of the user and match that to the anti-malware application(s) they use.

Bitman

[Tattenbach]

It is easy to be snitty when you find posts by the name of "Spybot's Poor Reputation". Something else is to come here and ask questions in an objective way, requesting the pros and the cons but without being rude or at least without sounding rude. Apologies to you if that was not your intention.

Who is snitty then?

None said Spybot is perfect but these guys try hard and they do not take a penny from your 35K students and nor from any one else, excepting those who donate.

There are excellent commercial applications but a number of them have enough money to 'help' with a good review in a magazine. Beside the lack of marketing resources there are yet many magazines that speak great about Spybot, and the most important, the opinion of thousand of 'non-mass-market' users that continue to trust it.