Spybot's Poor Reputation

[jonathanbean]

In recent years, pc magazines and test sites have rated Spybot very poorly. I love open source software and will even contribute when it does a good job, but the test results make me very wary of any of the free antispyware programs, including Spybot. Windows Defender has even been rated better.

I'm not being critical, just trying to find a reason to use Spybot and recommend it to my many students and friends. There is excellent free antivirus (e.g., Antivir) and firewalls (e.g., Comodo) but no excellent antispyware programs without paying an arm and an leg. WHY? :sad:

[bitman]

Hi Jonathan,

The question you're asking doesn't have a simple answer, since there are as many factors involved as there are malware. I'll take a stab though, since I have personally been involved in past beta testing and support of Spybot S&D, MS AntiSpyware/Windows Defender, and the Windows Live OneCare protection suite.

The problem is that unlike a virus, which is primarily a simple file and easily detected as such using signatures, spyware are often complex combinations of multiple files, registry changes and permission changes to both the file system and registry. This results in very complex requirements to properly detect and remove such programs, especially when they're already deeply embedded into the operating system and effectively in control.

The reason it's important to understand this is that most of those evaluating AntiSpyware simply don't. With AntiVirus it was relatively simple to take a PC, copy hundreds or thousands of virus files to it and run an AntiVirus scan to see what was detected. The larger the count, the better the AntiVirus, barring any false positives which were also relatively easy to detect.

Spyware on the other hand often modify the file or registry permissions or even the running operating system itself to hide themselves, requiring special tricks or processes to detect and/or remove the malware. Combine that with the fact that some malware work together to make it difficult to remove their bretheren or automatically 'heal' themselves by reinstalling removed pieces and you begin to see the problem.

Now, try to determine a method of properly testing the abilities of various AntiSpyware applications in a fair way. If you believe you can do it, you're smarter than I am. First you'd have to determine if the combination of spyware files you have are one or more 'infections' and whether they're designed to work together. Then you need to perform tests using that set of spyware and all of the AntiSpyware applications you wish to test. Finally you need to measure the effectiveness of the detection and removal for each product. At this point you Rinse/Repeat for another piece of Spyware.

If you can find anyone doing this kind of testing I'll fall off my chair. The only thing I've ever seen attempted are mass installations of dozens of Spyware, usually by purposefully going to known 'bad places' on the Internet and infecting a PC, which is then cloned. At this point the PC is scanned by each AntiSpyware application with aggragate results of removal presented, in rare cases with itemization of types as well as 'counts' of items detected and/or removed.

I don't blame the testing groups for doing this, since anything else would be prohibitively expensive, However, since this only really measures the effectiveness of the manual scanning and removal, it totally denies the existence of real-time protection such as Spybot S&D's TeaTimer and SDHelper Resident modules. For the more skilled user, these proactive abilities are far more valuable than scanning and cleaning after an infection, but are virtually impossible to test effectively since they depend to some extent on the skill and knowledge of the user. This is why the testing ignores these, since they're almost impossible to measure fairly.

So the point is, ratings by testing sites must be judged by their criteria, which most of the sites either don't publish or simply don't fully understand themselves. For this reason, the only real useful measure of an AntiSpyware application is your own experience. Unfortunately, this doesn't necessarily translate to the next person, because the level of dependance on your personal knowledge and understanding are involved.

Mixed into this is the much more simply understood issue that at any given moment there are new malware becoming available that an indiviual AntiMalware application may not have yet included in their detections, often because they haven't gotten a sample yet. This means that some applications will miss this new malware entirely, resulting in a major hit on their 'score' even of they can fully detect and remove it a week later, especially free applications like Spybot S&D that have less resources to collect and produce detections for them. This is the reason for the oft touted suggestion to run several AntiSpyware, since maybe one will include the new malware.

So finally, the reason that Spybot S&D is often downgraded by even some reputable sites is that it is a very effective removal tool and real-time protection in skilled or relatively intelligent user's hands, but less valuable for those who don't understand it and/or turn off all of the real-time protection.

Windows Defender was designed for this less knowledgable type of user, so for the masses (Grama, children, non-techies in general) it's generally more effective. Windows Live OneCare (AV, AS, FW, Backup) is for an even less interested user and removes almost all configuration requirements along with automating everything possible, including updates. The result is that it will fare better in most tests, but it often drives techical users nuts by taking away their control.

The real point of all of this is that the most effective AntiSpyware is what suits the user(s) of the PC best. I personally use Spybot S&D on my PCs, in parallel with Windows Live OneCare on the Vista laptop since I help support that product. On my cousin's and sister's/nephew's PCs, we have installed only OneCare, since they have no interest in understanding these things at all.

Bottom Line, the best AntiSpyware is what works best for the user(s) of that PC. This answer will be different based on their interests and abilities to understand what's really going on and how much the application needs to make the decisions for them.

Bitman
(OneCareBear at WLOC forums)

[502512]

However, since this only really measures the effectiveness of the manual scanning and removal, it totally denies the existence of real-time protection such as Spybot S&D's TeaTimer and SDHelper Resident modules. For the more skilled user, these proactive abilities are far more valuable than scanning and cleaning after an infection, but are virtually impossible to test effectively since they depend to some extent on the skill and knowledge of the user. This is why the testing ignores these, since they're almost impossible to measure fairly.


Here, here!!! An ounce of prevention is worth (at least) a pound of cure!!! The lack of recognition of the importance of this feature of Spybot S&D clearly shows that the testers are missing the mark. As bitman states, it's extremely difficult to test this fairly, but it basically gets left out of the equation because it's so difficult to evaluate.

It's also very true that the level of automation is directly proportional to the degree of control. I dislike and refuse to use software that makes me do things only one way and/or limits the level and degree of customization. That's fine for unsophisticated nontechnical users but does not work for a guy who has over 25 years experience as a software engineer.

You must remember that these ratings are for the masses. Are you a knowledgeable person in the field, or one of the huge number of nontechnical users out there? If the latter, Spybot S&D is probably not for you. However, please consider getting better educated, because IMHO these other programs often lull people into having a false sense of security. Knowledge is the best defense against malware.

[jonathanbean]

That was the single best explanation of spyware vs. antivirus that I have ever read. Next time someone asks, I'll send it along! Thanks.

Well, I'm not the grandma in the masses, but I'm not a computing engineer either. I'm a techno geek who reads a lot and asks questions. I hope Spybot isn't intended just for computer engineers! If so, my university of 35,000 students (all given free Spybot) is doing us a disservice!

This gives rise to two questions:

1. How "intelligent" do you have to be to use TeaTimer effectively?

2. More important, TeaTimer is much like a firewall -- a preventative. There are many tests of whether spyware, hacks, etc. can get beyond firewalls -- why not the same tests for the resident shields (like Tea Timer) that run with every antispyware program?? If Spybot is ineffective at removal, I'm happy with a strong shield -- but how do I know it is strong? What makes it better than commercial or Windows Defender?

I assume the immunization plays a small part of the shield function.

[Tattenbach]

In recent years, pc magazines and test sites have rated Spybot very poorly. I love open source software and will even contribute when it does a good job, but the test results make me very wary of any of the free antispyware programs, including Spybot. Windows Defender has even been rated better.

I'm not being critical, just trying to find a reason to use Spybot and recommend it to my many students and friends. There is excellent free antivirus (e.g., Antivir) and firewalls (e.g., Comodo) but no excellent antispyware programs without paying an arm and an leg. WHY? :sad:

++++++++++++++++++++++++++++++++++++++++++++++++++++
Spybot S&D is no open source software. First mistake.

Not sure which magazines you mean because what I have seen is the other way around. Even Microsoft recommends SpyBot S&D in its Windows Marketplace site.

http://www.windowsmarketplace.com/details.aspx?view=info&itemid=20308

What are those "excellent antispyware programs" that are not free? Could you name just one?

Comodo just failed a leak test recently (and was updated) and AntiVir, has failed a number of tests too.

Is the fight against those who want your data there is not a 100% solution if you leave everything to the developers. Users need to educate themselves too. Sounds painful? Then stay in the closet.

[502512]

I didn't mean to sound like you need to be a software engineering guru to effectively use Spybot S&D. This isn't true at all. TeaTimer in particular runs all by itself and needs no intervention from you. Just make sure you allow Spybot access to updates periodically, either via automation or manually.

However, that's no excuse to stick your head in the sand with regard to malware. Education is the best weapon against malware, and it's the responsibility of all legitimate users on the web to stay up to date and to use anti-malware to combat the spread. The old saying "if you're not part of the solution, you're part of the problem" holds very true here. I can't tell you how many friends I've had to bail out of big problems with their machines because they use no protection against malware and/or surf the web irresponsibly.

I've found that subscribing to some of the various TechTarget sites can keep one up to date on threats with minimal effort. They'll send you emails. All you have to do is take the time to read them. SearchSecurity is a good place to start.

Cheers,
Jeff

[jonathanbean]

Tattenbach was rather snitty -- I'm not going in the closet, just trying to ask hard questions: Why no tests of resident shields? Isn't that something that can be done? If not, then I'm still in the dark and closet or no closet, we are still all clueless, "intelligent" users or not!

Regarding your post of the windows rating of Spybot: There were 224 programs listed and Spybot came in #13 -- but this was only by "customer rating." Sheesh, there were high rated programs I have never heard of in my life.

Also on Antivir, see av-comparatives.org, where it was rated Advanced+ with high heuristic detection rate too:

http://www.av-comparatives.org/

See August results for virus detection
See November for heuristic detection -- far higher than most others.

Closet door still open...

Resident shield testing???

[jonathanbean]

Here's the PC World results on Spybot:

http://tinyurl.com/ye9uvr

Again, prevention > detection but since someone asked for a magazine test, here is just one.

[bitman]

Jonathan,

In reference to your questions;

1. There are two functionally effective ways to use TeaTimer, one requires 'technical intelligence' meaning knowledge of the registry and spyware and how/where they might attack.

The other is to simply recognize whether anything you have recently done (accepting a download for example) could reasonably be expected to generate a TeaTimer alert, which only requires a 'common sense' level of understanding. If you don't believe you requested something, you should generally block the change.

2. Testing firewalls is relatively easy similar to antivirus, since either the traffic can pass or not, though 'leak tests' take this a bit further. Testing spyware can be a long tedious process if it's done right, per my description above, though some feel that's not necessary. I personally don't trust the results of any test unless the methods and complete results are described, so I can understand how they were determined.

I wouldn't want to rate the Spybot shield against that of Defender or others without complete information as to what they monitor, which I've never personally seen. There are descriptions, but it's been a while since I've seen a complete comparison chart of all the major application's real-time protection.

Immunization is another layer of protection relating to Internet Explorer; Restricted Sites, ActiveX blocking, Tracking Cookie blocking.

This is what I mean when I say that Spybot is most useful in the hands of someone with a more technical background, since without that knowledge its alerts can be mis-understood or simply missed altogether. This doesn't make it bad, just more suitable to a technical user.

Education is great, but unfortunately nearly useless for the masses. I salute those here and all throughout the web who attempt the insurmountable task of educating everyone, but I also understand that it won't happen, at least in our lifetimes. This has been true throughout the history of all technologies, including current items like the cell phone and iPod, and even older things like the car. Some will never take an interest in understanding a technology beyond their daily use, which is actually totally appropriate. Only the technically inclined (us geeks) have a problem understanding this, which is why new technology is so often difficult to use.

For the masses, the job of protection, along with updating and backup, need to be performed by the system itself. After all, wasn't this the promise of the computer in the first place? Instead of us spending our time managing, maintaining and protecting our computers, they were supposed to save us time and effort. This got lost over the last 20 years, since most of the software development was done by and for geeks. This is now finally changing, to the benefit of everyone, at least those who don't make money off of the ensuing mess.

Do note that the current development of Spybot S&D 1.5 is also taking this into account. See this link to a comment by Spybot's developer today that shows this direction.

TeaTimer 1.5 issues (http://forums.spybot.info/showthread.php?p=66669#post66669)

Also note the comments on this page from your PC World reference (http://www.pcworld.com/downloads/file/fid,22262-order,1-page,1-c,alldownloads/description.html?findid=42052), especially the second paragraph where TeaTimer is mentioned. They also don't mention these forums as a technical support option. probably because most articles in PC World are keyed to commercial applications, which pay for advertising after all.

You'll note that I believe in applications to aid the masses, not because the people who help in manual malware removal aren't helpful, but because it doesn't scale. It's also much better to stop the malware from installing in the first place, so alerting the user to this fact if it can't be stopped automatically is key, and at that same moment there is an opportunity to educate the user if they will accept it.

Spybot S&D was one of the first to monitor and alert in real-time for spyware with TeaTimer, it's gotten left a bit behind over the last couple years, but the coming 1.5 update looks promising. There are many clues in the first couple threads in the Beta forum, so read there for more specifics.

I still believe in Spybot S&D, though I also believe that it's best to examine the abilities of the user and match that to the anti-malware application(s) they use.

Bitman

[Tattenbach]

It is easy to be snitty when you find posts by the name of "Spybot's Poor Reputation". Something else is to come here and ask questions in an objective way, requesting the pros and the cons but without being rude or at least without sounding rude. Apologies to you if that was not your intention.

Who is snitty then?

None said Spybot is perfect but these guys try hard and they do not take a penny from your 35K students and nor from any one else, excepting those who donate.

There are excellent commercial applications but a number of them have enough money to 'help' with a good review in a magazine. Beside the lack of marketing resources there are yet many magazines that speak great about Spybot, and the most important, the opinion of thousand of 'non-mass-market' users that continue to trust it.

[PepiMK]

@Tattenbach: actually, open source is a wide field. When I started this, license part II.a. fullfilled my universities wants of OS. It's not open to anyone though, simply because then the bad guys would be ever quicker to counteract and we would have to countercounteract even quicker and ... ;)
We're planing on something that'll allow some opening, while avoiding the above problem though...

@jonathanbean: bitman has already given a very good explanation. What I've also seen often is magazines complaining about Spybot not detecting "inactive spyware", which means they've got some spyware files, put them onto their harddisk somewhere, and expect Spybot to find them. The "problem" with that is that Spybot may ignore those while on-demand scanning and will find them only on-access. Which isn't a problem outside of test labs though, since there, those files will be found where they're installed.

An example: Malware ABC installs file XYZ.exe ALWAYS to C:\Windows\ (if it should decide on a different location, the files must be changed to reflect that rule, so all our other criteria, mostly file contents checksums, wouldn't fit either, so whether we use the path or not, new version needs new rules... but using the path as one criteria is faster to detect).
In the test lab, the file may be in C:\MyTestSamples\ (what those labs call "inactive spyware tests" or similar), where we don't look for it. You're still protected, because if you add C:\MyTestSamples\ to the download directories setting, it'll be found during on-demand as well, and if you or anyone else tries to start it, the on-access part will detect and block it. But in a test lab, it fails, and I've seen that that often inflicts the final rating a lot.

PCWorld: indeed asked us about advertising in relation to recent reviews. Not that I would actually say that influenced anything.

And about commercial applications... I wonder if those magazines tested email support... we get so many answers from people who're astonished that their emails were answered within hours, in a time that wasn't matched by any commercial vendor they've mailed so far. We've actually got quite a proud support team there who can't stand any questions left in the inbox and do a great job in really fast replies.

[jonathanbean]

about the service Spybot provides. I was blunt with the subject title because that is exactly how the PC magazine have stated it recently -- by "reputation" I did not mean it was MY opinion but the language used in PC mags is really harsh.

Sorry if I sounded "snitty!" This is a very useful forum.

[jonathanbean]

"Education is great, but unfortunately nearly useless for the masses. I salute those here and all throughout the web who attempt the insurmountable task of educating everyone, but I also understand that it won't happen, at least in our lifetimes."

Very true! I teach a course on the history of business and technology and there are a lot of companies with great technology (geek factor) that failed to understand how humans will use their technology.

Here is a positive recommendation, and it may be in Beta 1.5:

I am shocked that hundreds of thousands of students are given Spybot, install it but don't install Teatimer, just the scanner. Occassionally they scan for spyware. Yikes! Shouldn't Spybot have a exclamation point or screen telling users that Teatimer is HIGHLY RECOMMENDED!

Example: I told my secretary about spyware because she was getting all these pop-ups, etc. Then I scanned her computer and found keyloggers, trojans, all sorts of stuff. she freaked and put Spybot on but not teatimer. I found this out (months later) and scanned with another program and found lots of spyware.

[Tattenbach]

@Tattenbach: actually, open source is a wide field. When I started this, license part II.a. fullfilled my universities wants of OS. It's not open to anyone though, simply because then the bad guys would be ever quicker to counteract and we would have to countercounteract even quicker and ... ;)
We're planing on something that'll allow some opening, while avoiding the above problem though....

Patrick, when I talked about open source I referred to a comment previously made by Jonathan.
I did not mean to suggest to have SpyBot open source nor was this a comment attacking the program and its nature.

MfG

[md usa spybot fan]

An interesting point was raised about ratings and advertising.

In September 2006 Consumer Reports rated antispyware programs. Consumer Reports is published by Consumer Union of U.S., Inc. which is a nonprofit organization that not only does not accept advertising but also bans the use of their ratings for commercial purposes:


Violations of CU's No Commercial Use policy Our Ratings and reports may not be used in advertising. No other commercial use, including any use on the Internet, is permitted without our express written permission.

The ratings were based on Blocking, Features and Ease of use with indications for features Protects browser, Protects start-up, Describes spyware and Also in suite of use as follows:


Overall score combines blocking, features, and ease of use. Blocking shows how completely the product detected a number of different spyware behaviors and blocked or removed the initiator. Features includes auto update and real-time protection. Ease of use indicates how intuitive the interface is and how easy it is to perform common functions. Protects browser means spyware is prevented from changing your home page and redirecting Web searches. Protects start-up shows which products detect and stop spyware from starting automatically at boot-up. Describes spyware provides extra guidance to help decide whether to remove a detected item. Also in suite means similar software is available in the mfr.'s security suite. Price is for the retail, boxed version; downloads are usually available.All except Trend Micro (4) let you set computer scans for a specific time. All let you selectively restore programs removed as spyware.
Spybot Search and Destroy 1.4 was rate sixth out of twelve antispyware products. It was rated very good in all three rating categories (Blocking, Features and Ease of use) and had the following features checked: Protects browser, Protects start-up, Describes spyware. The only rating feature not checked was Also in suite.

It was also one of three "Quick picks" with the following statement:


For a free complement to your main antispyware:
6 Spybot (download)
This is very good at detecting spyware behavior and is easy to use. For a free program, it's a rich offering, with the ability to schedule hard-drive scans and to quarantine suspected spyware. It also includes a file shredder.
I would also like to add that the Consumers Reports article disagreed with the following statement by rating Windows Defender dead last:


Windows Defender has even been rated better.

[jonathanbean]

As I recall, CU got a lot of flak for its testing with dummy spyware (i.e., not the real thing). That's been flying around the Internet.

The more I look and research, the more you have convinced me to come back to Spybot and give up my expensive commercial software.

One point: I have had several antispyware programs on one computer (Trendmicro, WD, Ewido) but Antivir Premium always catches the spyware first with its heuristics. They SAY that their free version is the same as their Premium so I might go free with them too (with a small donation, of course!). I've watched the av-comparatives charts over the years, they constantly change but I am surprised by Antivir's ability at grabbing spyware and quarantining ASAP.

As for viruses, I haven't run into one in years! What's up with that?

[Tattenbach]

As for viruses, I haven't run into one in years! What's up with that?

Perhaps you have been cautious enough.

Jonathan:

I am not selling the idea to give up commercial software.

Regarding anti-malware, I have a license for Ad-Aware Pro (which is not really that much superior to the free version), I have a paid version of the antivirus program AVG (that includes the ex-ewido anti-spyware module) and many other great free programs like SpyBot, Hijackthis, Sysinternals tools, etc, etc, etc………

I believe that every one of them has their own particular strengths and so I use them. Like many have said, there is not a single solution for all the problems that may affect a Windows PC, so in the bag of solutions you should carry backup utilities like Acronis TI or Norton Ghost (among many others) or live-boot CDs like Knoppix, DSL, Bart PE, etc..

But, once more, education, IMHO, is the most important tool... Unfortunately, as BITMAN stated, the common user is not really interested in that, so all these tools and programs are really like RPG’s in the hands of someone drunk, blind, deaf and additionally tied-up and unconscious, fighting an entire army.

[Tattenbach]

BTW, in the posts to be found through the URL listed below there are good examples of what I meant by "drunk, blind, etc "

[jonathanbean]

I'm much more cautious than most of the people at my university. It amazes me how unprotected people are -- old virus detectors that are never updated, no firewall, no antispyware ("I thought the antivirus program took care of that!"), etc., etc.

I've noticed a real decrease in spyware since switching to Firefox. Now if I go on the wild side of the Internet and foolishly download "naughty" files, I should expect some consequences. LOL

On the virus issue, some magazine or web writer wrote about this "no virus in forever" issue, and I thought, "yeah, I haven't had one in ages." Perhaps the server/IT people are getting better?

Spyware, though, remains a huge problem. But the biggest problem is most people think viruses and spyware are the same thing.

BTW, I set my secretary's computer up properly now. One question I have about the four levels of Spybot protection -- I have three checked "on," but the Sticky on "Hosts" sounded above me. I'm concerned that changing the machine's ip address will screw up my dsl router address for my home computer. True?? The DSL IP issue is one that caused several calls to India (Verizon DSL) and once I got it right, I didn't want to mess with it again!

How important is the host protection?

[jonathanbean]

FYI: I read up and HOSTs file protection won't work with my LAN ISP in effect (DSL). That info. was from mvchosts site.

Oh, well.

[tashi]

BTW, in the posts to be found through the URL listed below there are good examples of what I meant by "drunk, blind, etc "

Please don't resort to flaming other members, keep discussions constructive.

Thanks. :)

[Tattenbach]

Please don't resort to flaming other members, keep discussions constructive.

Thanks. :)

*** OK ***