Spybot-S&D 1.6, Release Candidate 1

@PepiMK:

BTW, if interested in why those Copy and Move To dialogs pop-up like that, here is a response from Microsoft:

http://blogs.msdn.com/oldnewthing/archive/2004/02/02/66160.aspx

I also found the following page today:

http://windowsxp.mvps.org/movetocontext.htm

which has an alleged work-around posted at the bottom. Supposedly just using HKEY_CLASSES_ROOT\*\shellex\ContextMenuHandlers only lets it work without causing those unwanted dialogs. I haven't tried it yet though.

Also BTW, I apologize for asking about enabling a detailed log yesterday. That was pretty stupid of me. Had I thought about it properly I would have known better. I'll use yesterday having been a long day and my having been spaced out on a large volume of pain killers as an excuse... :p:
 
PepiMK said:
Thank you for the details :)

Earlier beta versions didn't support Firefox 3 yet. But SQLite is a database format, contrary to the old HTML format, so it should be faster instead of slower than Firefox 2 support.

Yes, 14 minutes are unacceptable. I'll have to check where exactly the longest delay takes place; while SQLite is a database format designed for quicker data access, it's not like a full server-sided database, so it might help to do some of the heavier operations not on database engine side (I already optimized a bit there, but there might be more).

Thanks again for very useful information, I feel better now ;)
Click to expand...
Question: When you wrote that earlier beta versions did not support Firefox 3, does that mean that the Spybot status bar indication of Spybot's checking the Firefox bookmarks was false information?

I ask as, when running Betas 1 & 2, I would always see, at the end of the process, information on the status bar that the bookmarks in the Firefox profile were being checked--and the name of the profile that Spybot indicated was, in fact, the name of my FF 3 profile. (I have two profiles, but the bookmarks file in the second one is so very small that Spybot might have stated that is was checking it, and that check could have been so quick that I did not notice.

I did not notice, during my testing this morning, if Spybot ever checked the bookmarks file in the second profile. (By the time the checking process finally finished, I was having a hard time even seeing the monitor screen...).

[EDIT] After posting this message, it occurred to me that I might still have a bookmarks.html file in my Firefox 3 profile, a "left-over" from the FF beta-test process. I just looked at my profile, and, yes, there is such a file, dated 6Feb08, which is likely about the time that I first installed Minefield (the FF3 beta-test code-named version.) Was that what Spybot betas were checking? If so, that would certainly explain why betas 1 & 2 worked so much faster than RC1.

I've been weeding out no-longer needed files from FF2 in my FF3 profile, and seemed to have overlooked the bookmarks.html one.
 
Last edited:
Beta 1 and beta 2 were scanning bookmarks in the form of bookmarks.html. RC 1 looks at the profile folder, and if cookies.sqlite exists, it scans cookies.sqlite for cookies and places.sqlite for bookmarks, otherwise it defaults to cookies.txt and bookmarks.html for scanning.

It won't scan both since that may be confusing to the user ("help, where does that bookmark come from").

My places.sqlite file is about 5 MB large and takes 4 minutes to scan (I just enabled and improved a mini benchmarking function) - less than 14, but still too much for scanning 7660 items in the cache.

Which reminds me: I do not have 7660 bookmarks I would guess, so it might be I'm scanning too much (history plus bookmarks maybe - the new database structure in there includes a dozen tables... hmmm... I should add a SQLite browsing tab to FileAlyzer to make it easier to search that file, I haven't found a single SQLite browser app yet that fully satisfies my needs ;) ).

Update: a combined query on moz_places and moz_bookmarks is most probably better than the current one on moz_places, which includes the history and might thus hit a huge multiple of the pure bookmark list. Having results from the history might be nice, but not at this price. Looking at the proper query now :)

Update 2: wrote a small SQLiteAlyzer to access the files faster than through command line ;)
Reading smaller amount of "places" now (only bookmarked ones), tests successful, Spybot-S&D now takes from previously 6:30 to around 1:00 on Firefox bookmarks now. Still not perfect, but getting better ^^
 
Screen Name Entirely Too Accurate

As if I were not "Always Confused", now I am always confused....

1. I set the FF config option to force FF to export my bookmarks as an html file; that resulted in bookmarks.html, 914 KB.

2. I then ran AM-Deadlink against bookmarks.html, which indicated that I have 951 bookmarks.

3. My places.sqlite file is shown as 15,392 KB.

4. I next deleted places.sqlite, bookmarkbackups and localstore.rdf. Doing that, plus having created a new bookmarks.html file, forces FF 3 to create a new places.sqlite file when FF is restarted.

5. Restarted FF; the new places.sqlite file is 15,392 KB, but there is only one .json file in the new bookmarkbackups folder of course.

6. I then read Ivan1981's comment about running Spybot with FF3 also running, so I started Spybot, told it to check. This time, when Spybot got to Firefox bookmarks, the Spybot process ended almost instanteously.

[In the line of being Always Confused, as I was writing this brilliant missive, I realized that I might not have deleted bookmarks.html. I'm going to do that, restart FF, rerun Spybot, report the result. (This saves me from copying this, closing FF, restarting, etc.; easier to just confuse myself further...).
 
As Threatened, I Have Returned.

The forum software would not allow me to post the wonderful verbiage I had added to my previous missive, so herewith enclosed please find whatever it is that is enclosed:

A. I deleted the bookmarks.html file, restarted FF, and reran S&D. When the S&D process neared the end, I watched the S&D status bar carefully, and determined that the reason for the nearly instantaneous completion was due to S&D completely skipping the in-use FF3 profile. S&D did scan the places.sqlite file in my other profile; that places file is only 144 KB, which would account for S&D scanning it so quickly.

B. I then closed FF and reran S&D; when it got to the places.sqlite file for my normally-used profile, S&D again became so slow that I terminated the process.

My conclusion is that S&D has a problem scanning .sqlite files, that problem being directly (and understandably) related to the size of such files.
 
@Always Confused(but now not :) ): Maybe you are right.
But if I use FF3 when scanning everything is ok like I said, but if FF3 is not started scanning firefox-default-bookmarks is slow and sometimes(not every single time) Spybot found two or one bookmark(s) something like this:
Code: 
CN.wAQdN188: Bookmark (Firefox: default) (Bookmark, nothing done)
  
CN.wAQdN188: Bookmark (Firefox: default) (Bookmark, nothing done)

Spybot(1.5.2 and 1.6 beta 1 and 2) with same definition file never reported any of this like RC1 does now.
 
Last edited:
The SQLite database Firefox uses isn't intended as a multi-user database; it's up to SQLite3.dll to allow or not allow access to the database. If it's in use, it's most probably not available. We could make a copy, modify that copy, replace back etc., but that could cause inconsistencies depending on what exactly Firefox keeps in memory.

As I wrote above, places.sqlite contains a lot of information, it has a huge list of "places", which in the beginning of my Firefox 3 usage wasn't that large, but quickly grows to be much larger through history, rss feeds and many other places stored in there (probably even a cache list).
If there's interest, I could add more details about the various Firefox 3 databases to the Spybot wiki I guess, I've already written down a lot of structure anyway.

Also, I've limited the query from this full places list to one just scanning bookmarks now, see above how that got faster.

I can upload an updated SpybotSD.exe in the next hour, I think it'll improve things similarly to my experience ;)
 
Sorry if it took nearly the full hour, but doing a few full scans for testing takes some time, and the public build environment always compiles at least 30 files, which is slooow ;)

SpybotSD.exe-ff3-improvements.zip

Just the main executable with Firefox 3 bookmarks support that does work faster by using only the data from moz_bookmarks with titles and urls from moz_places, similar to what is seen in the sqlite demo above (Templates menu).
 
Broken Hope said:
Usage track removal isn't working for FF3 cookies, it picks them up to be deleted but when you select them and tell Spybot to delete them it doesn't actually do it.
Click to expand...

Any update in regard to this, it seems to have been ignored? Spybot can detect cookies in FF3 but the usage track scan can't actually delete them it seems.
 
Scanning time with RC1: 8:54 (FF3 bookmarks almost 3 min) and found two entries mentioned before.
Scanning time ith new version 6:08(FF3 bookmarks about 10s) and found nothing.

Definitely a huge improvement.
 
Last edited:
@Broken Hopes: many thanks for reminding about that issue again :)
Found the problem - if you're an ultra-heavy internet user, the continuously increased unique cookie ID can get into astronomic heights ;)
I must admit I did believe one couldn't get more than 2^31 newly assigned cookies (that's 2,147,483,648), but it seems I was wrong :laugh: (or Firefox doesn't necessarily start with 0).
 
Updating does not let me choose the update location. The updater flies by so fast not able to check the option to check for beta updates and location can not be chosen. Please remove the stupid automation for the updater so the location and other options can be checked manually.
 
molitar said:
Updating does not let me choose the update location. The updater flies by so fast not able to check the option to check for beta updates and location can not be chosen. Please remove the stupid automation for the updater so the location and other options can be checked manually.
Click to expand...

Please read the stupid beta threads so you don't repeat something already addressed:

report: http://forums.spybot.info/showpost.php?p=203785&postcount=70

response: http://forums.spybot.info/showpost.php?p=203790&postcount=71
 
Well even reading them threads and the response. I see no options in the options to disable the automatic update for checking the locations or anything else. It seems the updater has been dumbed down and I have tried the update button and the menu and it acts the same either way.. it updates automatically not letting the options be chosen at all.
 
molitar said:
Well even reading them threads and the response. I see no options in the options to disable the automatic update for checking the locations or anything else. It seems the updater has been dumbed down and I have tried the update button and the menu and it acts the same either way.. it updates automatically not letting the options be chosen at all.
Click to expand...

Are you sure you read the response?

Are you really saying that the updater works the same if you select Start -> Programs -> Spybot-S&D -> Update Spybot-S&D?

And you really don't see a "Web update" section under the Settings?
 
Zer0 Voltage said:
Are you sure you read the response?

Are you really saying that the updater works the same if you select Start -> Programs -> Spybot-S&D -> Update Spybot-S&D?

And you really don't see a "Web update" section under the Settings?
Click to expand...

To a degree in that it will let you select the beta update and the other check box from that updater. But the location does not appear in that updater either. I notice when I ran the updater it was updating from some foreign country and not the USA. That is why this being automated needs to be a selectable option in the options. I want my updates coming from the USA not some country that I can't barely pronounce.
 
molitar said:
To a degree in that it will let you select the beta update and the other check box from that updater. But the location does not appear in that updater either. I notice when I ran the updater it was updating from some foreign country and not the USA. That is why this being automated needs to be a selectable option in the options. I want my updates coming from the USA not some country that I can't barely pronounce.
Click to expand...

Funny, it lets me select the location:

ssdupdaterfg4.jpg


Are you just not getting the choice because no updates are available after your last update?

Or perhaps you cannot pronounce USA? Should be easy enough. It's even spelled out for you.

Heck, I can even pronounce Germany, Canada, and Europe...
 
You must log in or register to reply here.
Back
Top