bitman
2006-09-22, 19:53
I've just discovered a bug in the Spybot Update system that explains many of the issues we've seen with Checksum Errors that never seemed to make sense. At the moment, this is causing the BN FileForum and PlanetMirror sites to fail completely for all files, even though the current files actually exist at those sites.
The problem is with Spybot S&D's handling of the list of Mirrors contained in the Spybotsd.ini file which is downloaded when 'Search for Updates' is clicked.
The apparent failure causes the first two entries in the current list to map incorrectly, or fall back to the 'Safer Networking #2' site as shown in the Spybot Update downloads Log, but to also fail completely with Checksum Errors.
9/22/2006 10:24:15 AM Downloaded update info file. (http://www.safer-networking.org/updates/spybotsd.ini)
9/22/2006 10:24:50 AM downloaded update Detection rules
9/22/2006 10:24:50 AM - URL: http://www.spybotupdates.biz/updates/files/includes.zip
9/22/2006 10:24:50 AM - Local file: C:\Program Files\Spybot - Search & Destroy\Updates\includes.zip
9/22/2006 10:24:50 AM - FILE REJECTED because of bad checksum
9/22/2006 10:25:11 AM downloaded update English descriptions
9/22/2006 10:25:11 AM - URL: http://www.spybotupdates.biz/updates/files/desc.english.zip
9/22/2006 10:25:11 AM - Local file: C:\Program Files\Spybot - Search & Destroy\Updates\desc.english.zip
9/22/2006 10:25:11 AM - FILE REJECTED because of bad checksum
9/22/2006 10:25:32 AM downloaded update ßDetection rules (beta)
9/22/2006 10:25:32 AM - URL: http://www.spybotupdates.biz/updates/files/includesb.zip
9/22/2006 10:25:32 AM - Local file: C:\Program Files\Spybot - Search & Destroy\Updates\includesb.zip
9/22/2006 10:25:32 AM - FILE REJECTED because of bad checksum
9/22/2006 10:26:18 AM downloaded update Detection rules
9/22/2006 10:26:18 AM - URL: http://www.spybotupdates.biz/updates/files/includes.zip
9/22/2006 10:26:18 AM - Local file: C:\Program Files\Spybot - Search & Destroy\Updates\includes.zip
9/22/2006 10:26:18 AM - FILE REJECTED because of bad checksum
9/22/2006 10:26:39 AM downloaded update English descriptions
9/22/2006 10:26:39 AM - URL: http://www.spybotupdates.biz/updates/files/desc.english.zip
9/22/2006 10:26:39 AM - Local file: C:\Program Files\Spybot - Search & Destroy\Updates\desc.english.zip
9/22/2006 10:26:39 AM - FILE REJECTED because of bad checksum
9/22/2006 10:27:00 AM downloaded update ßDetection rules (beta)
9/22/2006 10:27:00 AM - URL: http://www.spybotupdates.biz/updates/files/includesb.zip
9/22/2006 10:27:00 AM - Local file: C:\Program Files\Spybot - Search & Destroy\Updates\includesb.zip
9/22/2006 10:27:00 AM - FILE REJECTED because of bad checksum
The other sites listed appear to operate properly, so the problem appears to be only the first two entires on the list as shown below.
BN FileForum (Global)=http://betanewsdl.iad.cachefly.net/spybot/updates/files/
PlanetMirror (Global)=http://downloads.planetmirror.com/pub/spybot/
TU Braunschweig (Europe)=http://ftp.rz.tu-bs.de/pub/mirror/spybot.info/sbsdupdates/
XTeq (Germany)=http://www.xteq.de/spybot/updates/
See-Cure #1 (Europe)=http://www.see-cure.de/updates/files/
See-Cure #2 (Europe)=http://www.see-cure.net/updates/files/
Safer Networking #1 (Europe)=http://www.spybotupdates.com/updates/files/
Safer Networking #2 (Europe)=http://www.spybotupdates.biz/updates/files/
Safer Networking #3 (Europe)=http://85.31.186.49/updates/files/
I believe that the underlying problem is the size of the buffer used to manage this list in memory. Since the list is 617 bytes as displayed, removing the BN FileForum entry reduces this to 537 bytes and removing the PlanetMirror entry reduces it to 468 bytes.
If my guess is correct, the first two lines of the list are lost since they don't fit into a 512 byte buffer used to store them in memory. This results in the loss of this information and the switch in the Update Log to display of the following path, which is probably overloaded as a side effect.
http://www.spybotupdates.biz/updates/files/filename.ext
The result of all of this is the complete failure of the first two servers, with additional stress being placed on the others, explaining why adding servers hasn't helped. Since both BN FileForum and PlantMirror are Global servers, their loss is far worse than most of the others. This also explains why the BN FileForum server started having occasional problems when it was moved to the top of the list, which seemed sensible at the time.
It appears that either reducing the path length on some of the servers or reducing the number of servers listed are the only short term solutions. Also possibly forcing the two Global servers to the bottom of the list may help, since they then won't be affected when the list grows too large.
Bitman
The problem is with Spybot S&D's handling of the list of Mirrors contained in the Spybotsd.ini file which is downloaded when 'Search for Updates' is clicked.
The apparent failure causes the first two entries in the current list to map incorrectly, or fall back to the 'Safer Networking #2' site as shown in the Spybot Update downloads Log, but to also fail completely with Checksum Errors.
9/22/2006 10:24:15 AM Downloaded update info file. (http://www.safer-networking.org/updates/spybotsd.ini)
9/22/2006 10:24:50 AM downloaded update Detection rules
9/22/2006 10:24:50 AM - URL: http://www.spybotupdates.biz/updates/files/includes.zip
9/22/2006 10:24:50 AM - Local file: C:\Program Files\Spybot - Search & Destroy\Updates\includes.zip
9/22/2006 10:24:50 AM - FILE REJECTED because of bad checksum
9/22/2006 10:25:11 AM downloaded update English descriptions
9/22/2006 10:25:11 AM - URL: http://www.spybotupdates.biz/updates/files/desc.english.zip
9/22/2006 10:25:11 AM - Local file: C:\Program Files\Spybot - Search & Destroy\Updates\desc.english.zip
9/22/2006 10:25:11 AM - FILE REJECTED because of bad checksum
9/22/2006 10:25:32 AM downloaded update ßDetection rules (beta)
9/22/2006 10:25:32 AM - URL: http://www.spybotupdates.biz/updates/files/includesb.zip
9/22/2006 10:25:32 AM - Local file: C:\Program Files\Spybot - Search & Destroy\Updates\includesb.zip
9/22/2006 10:25:32 AM - FILE REJECTED because of bad checksum
9/22/2006 10:26:18 AM downloaded update Detection rules
9/22/2006 10:26:18 AM - URL: http://www.spybotupdates.biz/updates/files/includes.zip
9/22/2006 10:26:18 AM - Local file: C:\Program Files\Spybot - Search & Destroy\Updates\includes.zip
9/22/2006 10:26:18 AM - FILE REJECTED because of bad checksum
9/22/2006 10:26:39 AM downloaded update English descriptions
9/22/2006 10:26:39 AM - URL: http://www.spybotupdates.biz/updates/files/desc.english.zip
9/22/2006 10:26:39 AM - Local file: C:\Program Files\Spybot - Search & Destroy\Updates\desc.english.zip
9/22/2006 10:26:39 AM - FILE REJECTED because of bad checksum
9/22/2006 10:27:00 AM downloaded update ßDetection rules (beta)
9/22/2006 10:27:00 AM - URL: http://www.spybotupdates.biz/updates/files/includesb.zip
9/22/2006 10:27:00 AM - Local file: C:\Program Files\Spybot - Search & Destroy\Updates\includesb.zip
9/22/2006 10:27:00 AM - FILE REJECTED because of bad checksum
The other sites listed appear to operate properly, so the problem appears to be only the first two entires on the list as shown below.
BN FileForum (Global)=http://betanewsdl.iad.cachefly.net/spybot/updates/files/
PlanetMirror (Global)=http://downloads.planetmirror.com/pub/spybot/
TU Braunschweig (Europe)=http://ftp.rz.tu-bs.de/pub/mirror/spybot.info/sbsdupdates/
XTeq (Germany)=http://www.xteq.de/spybot/updates/
See-Cure #1 (Europe)=http://www.see-cure.de/updates/files/
See-Cure #2 (Europe)=http://www.see-cure.net/updates/files/
Safer Networking #1 (Europe)=http://www.spybotupdates.com/updates/files/
Safer Networking #2 (Europe)=http://www.spybotupdates.biz/updates/files/
Safer Networking #3 (Europe)=http://85.31.186.49/updates/files/
I believe that the underlying problem is the size of the buffer used to manage this list in memory. Since the list is 617 bytes as displayed, removing the BN FileForum entry reduces this to 537 bytes and removing the PlanetMirror entry reduces it to 468 bytes.
If my guess is correct, the first two lines of the list are lost since they don't fit into a 512 byte buffer used to store them in memory. This results in the loss of this information and the switch in the Update Log to display of the following path, which is probably overloaded as a side effect.
http://www.spybotupdates.biz/updates/files/filename.ext
The result of all of this is the complete failure of the first two servers, with additional stress being placed on the others, explaining why adding servers hasn't helped. Since both BN FileForum and PlantMirror are Global servers, their loss is far worse than most of the others. This also explains why the BN FileForum server started having occasional problems when it was moved to the top of the list, which seemed sensible at the time.
It appears that either reducing the path length on some of the servers or reducing the number of servers listed are the only short term solutions. Also possibly forcing the two Global servers to the bottom of the list may help, since they then won't be affected when the list grows too large.
Bitman