Page 1 of 5 12345 LastLast
Results 1 to 10 of 42

Thread: Spybot Christmas Presents, Part 3: Distributed Testing

  1. #1
    Member of Team Spybot PepiMK's Avatar
    Join Date
    Oct 2005
    Location
    Planet Earth
    Posts
    3,574

    Santa Spybot Christmas Presents, Part 3: Distributed Testing

    Our second present is one to us as well as to our users. One of the downsides of our tripple-layered testing of updates is that it takes 8 to 14 days for a newly written detection to get to you. Participating in beta reduces that to 1 to 7 days, but of course we want to deliver well-tested updates to everyone as fast as possible.

    Just increasing the number of local machines to test is not really improving things too much, as all those software installation combination in the wild are a very dynamic thing. So we thought about a different, more community-like, approach: we've decided to implement a distributed computing concept.

    You may know existing distributed computing projects: Seti@home for example uses the power of a huge community to look for extra-terrestrial live, Folding@home uses a huge community as well to possibly find more understanding about dieseases like cancer. Spybot-S&D Distributed Testing (SDDT) isn't helping you to make phone calls to Alpha Centauri, nor does it cure diseases, but then, it's also not really taking that much processing time away from those if you would use them. It uses the same community power though: the results of the scans of many real live machines, maybe including yours, will help us make safer and even faster updates available!

    Download: http://forums.spybot.info/downloads.php?id=19

    Once installed, this'll run as a system service (thus only on NT/2000/XP/Vista, though a stand-alone version for 9x/ME is included); you'll probably never notice it since it scans with the lowest available process priority, and it doesn't fix anything. But if you want to take a look at what it does in the background, you can always open the console window:

    Last edited by daemon; 2012-09-06 at 18:59. Reason: Added back download link.
    Just remember, love is life, and hate is living death.
    Treat your life for what it's worth, and live for every breath
    (Black Sabbath: A National Acrobat)

  2. #2
    Member of Team Spybot PepiMK's Avatar
    Join Date
    Oct 2005
    Location
    Planet Earth
    Posts
    3,574

    Default Technical background - for those who're interested

    Ok, didn't want to annoy you with too much text, so I split the technical details into a second post that you don't need to read if you're just interested in helping and not in exactly how it works

    Still reading? Ok, so here we go: whenever our detectives have finished some parts of work on new detections, this gets immediately uploaded into SDDT (Spybot-S&D Distributed Testing), where the SDDT client installed on your machine downloads any new test sets from. This client uses a special, read-only (to prevent F/Ps causing any problems) version of the command line scanner to do these scans - much like you know from Spybot-S&D itself, only that it happens with a very small detections file only (much faster obviously, only a few seconds per file), totally invisible to you, and with idle priority, meaning it won't reduce computing power for any other running application. The results of the scan are then sent back, and usually our detectives can see within an hour or two that their work doesn't cause any false positives.

    In the long run, this should allow us to spend more time on writing new detections, while at the same time allowing us to make updates available faster, since your help offers a much broader range of system configurations to test on. And since the client is read-only, don't be afraid of F/Ps in the background - they're the whole purpose - doing no harm to your system and informing us to get them fixed before the official update is out!
    Just remember, love is life, and hate is living death.
    Treat your life for what it's worth, and live for every breath
    (Black Sabbath: A National Acrobat)

  3. #3
    Member
    Join Date
    Sep 2006
    Location
    australia, n.s.w
    Posts
    62

    Default wowwwww

    oi PepiMK you are a genius, seriously turn gay and marry me. But honestly i like this idea, i wonder if you can manage all the reports/logs that get sent to you though, nice though, i suppose you make the logs highlight problems/false positives and compare them with other logs.

    anyway with distributing this, maybe put it in the spybot category but include it in the spybot installation but as a tick box option.
    --------------------------------------------------
    King of zlobtrojan, virusburst, spyaxe, and elite bar infections.

  4. #4
    Senior Member chewdz's Avatar
    Join Date
    Dec 2007
    Location
    Singapore
    Posts
    176

    Talking

    it would be good if this feature could be added into future versions of spybot

  5. #5
    Member of Team Spybot PepiMK's Avatar
    Join Date
    Oct 2005
    Location
    Planet Earth
    Posts
    3,574

    Default

    No need to be afraid of problems with handling all those reports; this has a nice backend for our detectives that has a mode to only list relevant entries, on multiple levels... first just files that have scan results at all (zero results mean no F/P usually... unless th first really infected machines start to participate I guess ), then all revisions of that file (where only the most current one is of interest usually), and only then a list of all logs.
    We used it at the office and at private machines of team members for about two months I think before making it public, so we had a few chances to make improvements in handling of the data already

    Adding it into Spybot-&D sounds like a huge step... that would possibly mean millions of scan logs! Let's scale this slowly, see how it behaves with a growing number of testers (that's why you can't set up a smaller interval than 60 minutes :D ).

    Would even be kind of a remote scanning service if it would be part of the main package, because if you specify your email address and a possible false positive would be a real infection, you would get notified immediately probably. Hmmm... I need to keep this in some distant corner of my mind for the future (a bit too much to advertise it thus in a 1.0 version, would need some more features anyway, like automated submission of detected files for further analysis etc. :D ).

    Update: added a project tools category for it, to write down the first few feature requests
    Just remember, love is life, and hate is living death.
    Treat your life for what it's worth, and live for every breath
    (Black Sabbath: A National Acrobat)

  6. #6
    Esteemed Member
    Join Date
    Oct 2005
    Posts
    211

    Default

    If you do add it to Spybot, please make it an opt in option. Some might not want to participate.

  7. #7
    Member of Team Spybot PepiMK's Avatar
    Join Date
    Oct 2005
    Location
    Planet Earth
    Posts
    3,574

    Default

    As I said, having to participate every Spybot-S&D user would mean millions of logs; we wouldn't them rushing all at us, so it probably would even be a well-hidden opt-in option
    Just remember, love is life, and hate is living death.
    Treat your life for what it's worth, and live for every breath
    (Black Sabbath: A National Acrobat)

  8. #8
    Senior Member 129260's Avatar
    Join Date
    Sep 2007
    Location
    Somewhere in the USA
    Posts
    1,139

    Arrow i would participate

    but i have dial up, do you need to have broadband or will dial up work fine for sending the reports back?
    "I am learning just like everyone else"
    new members!
    Custom built PC. Windows 7 pro x64 16GB Ram
    AMD FX 8 core 8350 Black edition
    SABERTOOTH 990FX/GEN3 R2.0
    Asus HD 7870 2GB GDDR5

  9. #9
    Senior Member chewdz's Avatar
    Join Date
    Dec 2007
    Location
    Singapore
    Posts
    176

    Default

    Quote Originally Posted by Rosenfeld View Post
    If you do add it to Spybot, please make it an opt in option. Some might not want to participate.
    i agree with Rosenfeld to put an option but would it be a opt-out or a opt-in thing?

  10. #10
    Member of Team Spybot PepiMK's Avatar
    Join Date
    Oct 2005
    Location
    Planet Earth
    Posts
    3,574

    Default

    @129260: sure, the files in itself are quite small, the reason I mentioned broadband is that broadband access usually is permanently connected. This client just assumes an existing Internet connection, so if your dialup means you're not permanently connected, I would suggest to not use the system service (you can disable it during installation), but use the "stand-alone" version when you're online

    @chewdz: as you can see two posts above yours, purely opt-in
    Just remember, love is life, and hate is living death.
    Treat your life for what it's worth, and live for every breath
    (Black Sabbath: A National Acrobat)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •