Spybot Christmas Presents, Part 3: Distributed Testing

PepiMK

PepiMK

Administrator
Staff member
Anti-Beacon Customer Spybot Customer Translator OpenSBI Expert
Our second present is one to us as well as to our users. One of the downsides of our tripple-layered testing of updates is that it takes 8 to 14 days for a newly written detection to get to you. Participating in beta reduces that to 1 to 7 days, but of course we want to deliver well-tested updates to everyone as fast as possible.

Just increasing the number of local machines to test is not really improving things too much, as all those software installation combination in the wild are a very dynamic thing. So we thought about a different, more community-like, approach: we've decided to implement a distributed computing concept.

You may know existing distributed computing projects: Seti@home for example uses the power of a huge community to look for extra-terrestrial live, Folding@home uses a huge community as well to possibly find more understanding about dieseases like cancer. Spybot-S&D Distributed Testing (SDDT) isn't helping you to make phone calls to Alpha Centauri, nor does it cure diseases, but then, it's also not really taking that much processing time away from those if you would use them. It uses the same community power though: the results of the scans of many real live machines, maybe including yours, will help us make safer and even faster updates available!

Download: http://forums.spybot.info/downloads.php?id=19

Once installed, this'll run as a system service (thus only on NT/2000/XP/Vista, though a stand-alone version for 9x/ME is included); you'll probably never notice it since it scans with the lowest available process priority, and it doesn't fix anything. But if you want to take a look at what it does in the background, you can always open the console window:

sdistest1.png
 
Last edited by a moderator:
Technical background - for those who're interested

Ok, didn't want to annoy you with too much text, so I split the technical details into a second post that you don't need to read if you're just interested in helping and not in exactly how it works ;)

Still reading? Ok, so here we go: whenever our detectives have finished some parts of work on new detections, this gets immediately uploaded into SDDT (Spybot-S&D Distributed Testing), where the SDDT client installed on your machine downloads any new test sets from. This client uses a special, read-only (to prevent F/Ps causing any problems) version of the command line scanner to do these scans - much like you know from Spybot-S&D itself, only that it happens with a very small detections file only (much faster obviously, only a few seconds per file), totally invisible to you, and with idle priority, meaning it won't reduce computing power for any other running application. The results of the scan are then sent back, and usually our detectives can see within an hour or two that their work doesn't cause any false positives.

In the long run, this should allow us to spend more time on writing new detections, while at the same time allowing us to make updates available faster, since your help offers a much broader range of system configurations to test on. And since the client is read-only, don't be afraid of F/Ps in the background - they're the whole purpose - doing no harm to your system and informing us to get them fixed before the official update is out!
 
wowwwww

oi PepiMK you are a genius, seriously turn gay and marry me. But honestly i like this idea, i wonder if you can manage all the reports/logs that get sent to you though, nice though, i suppose you make the logs highlight problems/false positives and compare them with other logs.

anyway with distributing this, maybe put it in the spybot category but include it in the spybot installation but as a tick box option.
 
No need to be afraid of problems with handling all those reports; this has a nice backend for our detectives that has a mode to only list relevant entries, on multiple levels... first just files that have scan results at all (zero results mean no F/P usually... unless th first really infected machines start to participate I guess ;) ), then all revisions of that file (where only the most current one is of interest usually), and only then a list of all logs.
We used it at the office and at private machines of team members for about two months I think before making it public, so we had a few chances to make improvements in handling of the data already :)

Adding it into Spybot-&D sounds like a huge step... that would possibly mean millions of scan logs! Let's scale this slowly, see how it behaves with a growing number of testers (that's why you can't set up a smaller interval than 60 minutes :D ).

Would even be kind of a remote scanning service if it would be part of the main package, because if you specify your email address and a possible false positive would be a real infection, you would get notified immediately probably. Hmmm... I need to keep this in some distant corner of my mind for the future (a bit too much to advertise it thus in a 1.0 version, would need some more features anyway, like automated submission of detected files for further analysis etc. :D ).

Update: added a project tools category for it, to write down the first few feature requests :)
 
i would participate

but i have dial up, do you need to have broadband or will dial up work fine for sending the reports back?
 
@129260: sure, the files in itself are quite small, the reason I mentioned broadband is that broadband access usually is permanently connected. This client just assumes an existing Internet connection, so if your dialup means you're not permanently connected, I would suggest to not use the system service (you can disable it during installation), but use the "stand-alone" version when you're online :)

@chewdz: as you can see two posts above yours, purely opt-in :)
 
thanks vry much

will do! :) Anything i can do to help spybot get even more better!
 
There is actually no problem with running this AND folding at home, or other distro project at the same time. When Spybot Distro actually gets work, it takes no time at all for it to run its checks. So you can leave them both running same time. Works good and I can see how this will actually help you out alot.

How many rigs are there running your distro at this time? Is it enough to make a differance
?
 
Todays test sets may have showed a possible problem: we did add a standard service to test services detection through SDDT, and found that it might hang the testing system.

Therefore, the link in the first post has been updated to link to version 1.0.2, which does not hang there, and has the additional advantage of offering a button to look for an updated version from within the software, and more important storing the duration of a scan inside the log, so that we can do speed tests as well.

Links to changes in project tracker:
 
LOL, I love that little pop up, I guess you didnt want me checking every 10 minutes for new stuff! lol

Future request, can you make it remember the window position?
 
excellent!

1.0.2.0 seems to be running quite smoothly now. Before i had to open the stand-alone client for the darn thing even to start and it still hung, but now that bug is gone! I really appreciate the time that you guys put into spybot and all the other little tools that you make. I'm just glad i can help you guys out in some way in your fight against malware!!!! :yahoo:
 
You must log in or register to reply here.
Back
Top