/allhives detection - method? explaination. suggestion for devs.

[NickFury]

(To clear the air for the intents and purposes of this thread, I am referring to NT-based Windows versions only.)

I discovered the /allhives switch for spybot today, however the explaination of it's application is extremely vague. I would like to know the method that Spybot "detects" other operating systems in order to scan them with the /allhives switch.

I imagine Spybot checks for other installations in the boot.ini rather than scanning all the possible directories that a custom windows installation could be hiding in either on the current hard drive or across multiple volumes. Well, would it be possible to confirm this or correct me if I'm wrong? (I DO hope that I'm wrong! Either way, I have a great suggestion for functionality that I know Spybot does not implement, though I'm unsure why... Can someone "in the know" please describe the method Spybot "detects" other NT-based Windows installations?

I ask because Spybot likely will not work this way for my specific purpose. I've already looked at these FAQ's in search of clues, but no luck:

* I have two installations on my hard disk. Can I scan both at the same time?
http://www.spybot.info/en/faq/41.html

* Is Spybot-S&D compatible to WinPE/BartPE bootable CDs?
http://www.spybot.info/en/faq/43.html

I found this message thread relating to scanning multiple hard drives, but the answer is incomplete, possibly the question was misinterpreted? Or am I misinterpreting the answer?
* http://forums.spybot.info/showthread...ht=%2Fallhives

Allow me to explain my situation, because I think I identify with the original poster of that thread who problably did not receive the solution he was looking for...

I do, A LOT, of virus/malware removal. I've been working as a technician for 11 years so I'm fairly proficient with the internals of Windows. Typically when I remove virii/malware from a system I use a custom WinPE CD, or the vast majority of the time I actually remove the infected hard drive and attach it to one of my tech bench computers. From the tech bench computer (I'll call a TBC) I use a script of my own writing to automate various anti-virus and anti-malware applications on the "guest" hard drive, including the great Spybot. This way most malware, which naturally isn't loaded/resident in memory on a TBC, is removed with ease by killing the majority of the infected files before I even start windows on the infested machine. Unfortunately that doesn't take care of the registry, and subsequently, (in addition to new malware constantly being introduced) not everything is detected. I have other scripts that automate the anti-malware applications again, in safe mode, after returning the "guest" hard drive to the original machine. This means I run Spybot once on the "TBC" for file scans, and once on EACH user account of the infected machine. This isn't a big deal for me usually, but many times I'm facing a Windows installation with 5 or more user accounts. That consumes way more time than I'd like to spend.

What if I could run Spybot only once, from my "TBC" and know that Spybot is scanning the "guest" hard drive's registry hives (SYSTEM, SOFTWARE, users' NTUSER.DAT, etc.) and be done with it? Would the /allhives switch, with the inclusion of custom directories in it's config, do this for me? I have not experimented as of yet but after finding this thread explaining why Spybot needs to run on each user account in NT based OSes, it leads me to doubt very much that it works the way I need it to:
* http://forums.spybot.info/showthread...ht=%2Fallhives

So I understand the registry scanning needs to run on each user account, and I've no idea who/what/when/where/why/how. I've no real experience with Windows APIs or programming. I do know if Spybot cannot be effective on multiple user hives when scanning on the infected machine, how can it be effective on all users and registry hives on a .. foreign (for lack of a better term) Windows installation?

When I attach an infected drive to my TBC, I do in fact load all of the registry hives into my local registry. I load them under HKLM with the prefix "guest_" so that the hives load as follows:

HKLM\guest_System
HKLM\guest_Software

etc, and each user hive found on the "guest" hard drive in \documents and settings is loaded

HKLM\guest_%username%
HKLM\guest_%username%
etc... (where %username% is the name of the user hive of each unique user account my script finds)

and so on. From there I use .reg files, batch files / for loops, and reg.exe to brute force removal of registry entries using plain text definition files that I maintain myself. after which I do a visual scan and when I find something new (yet not random) I confirm it's malware, then add it to my own "defs" file. incidentally, I do the same with the guest's file system as well. This way, many new or undetected malicious entries are removed before I worry about starting up the infected machine. Too bad my defs are dumb, static, however you want to put it, and rely on my own personal time to examine what's left after my assault of anti-whatever applications.

How wonderful it would be if Spybot (and other programs) could scan the registry hives (including all user hives) that I load from other machines or boot cds; however I know that they don't, being the "guest_" prefix isn't recognized.

I'm not asking for a standard prefix (guest_) used to load registry hives merely to accomodate my own personal (and likely obscure) method of registry malware removal. It would be just as well if Spybot could find and load the hives itself for scanning; although a standard among respectable anti-malware crusaders would be nice.

I was hopeful when finding the /allhives switch, yet it appears it doesn't do what it could... Perhaps it does do what I want it to, yet that still begs an answer to the original question I've already presented, exactly how does Spybot recognize other Windows' installations in order to load their registry hives for scanning? It would be great for those of us automation freaks who just so happen to be technicians, for the inclusion of a command-line parameter to include a specific drive/directories for it to load registry hives from that are foreign to the host system that is running Spybot.

The eventual goal (at least MY goal) is the total annihilation of malware from an infested system without having to even boot up that system.

However it is, will it work for my situation, even by simply adding custom directories from other drives in the settings? Will it find Windows installations in more than just \Windows or \Winnt? Will it find them if they aren't in the boot.ini of that drive (or the primary boot drive?) I've not experimented to find the answers yet, but I doubt it would work after reading that Spybot couldn't scan more than one user account effectively, which reminds me of several other solutions.

Even so, the wide number of machines I run my scripts from, the problem is I must have my "guest" hard drive (among other things) assigned to a variable for my batch scripts. Here is where you could do techs (and the occasional network admin) great favors with the inclusion of specific command-line parameters like:

/includehives=f:\windows.000\system32\config;g:\documents and settings;h:\winnt\system32\config; (ETC...)

for example, where depending on the environment, specific drives/paths can be replaced with %variables% in scripts. As you may well know, that *type* functionality is essential for any serious network admin worth his salt when dealing with other applications and tasks, but few realize it's usefulness to the everyday break/fix tech like myself especially when playing the anti-virus/anti-malware game, so let me assure you that it is. With most modern software in general, much focus has been placed with making things easier for the newer breed of average home and business user, GUI's and the like, that so much usefulness and configurability of modern software has been lost in time.

moving forward...

As for scanning of multiple user hives and files, I know Spybot could (or should) scan the other users' registry hives, or perhaps I misunderstand Spybot's capabilities, since without knowing how to utilize the Windows API, I don't understand HOW Spybot actually scans the registry so forgive me if I'm being irritating, but I believe at the least, Spybot shouldn't have this type of limitation. It would be nice to see it work on the HKU portion of the registry.

It's difficult for me to comprehend when, using only archaic NT batch files, tons of variables, console based registry utilities included with Windows, and my own definitions, I can scan the registry complete with all user accounts on the infected machine either from that very machine or from an entirely different one (my TBC's) or a boot cd... so why couldn't Spybot?

Yes, there are the occasions of not having permissions to access certain user accounts made "private" by some users on creation. Even with my own archaic batch files, using the AT command, it is simple to run tasks (and subsequently spawned tasks) with SYSTEM account privilages which ignore all those pesky permissions that might prevent you from accessing the files in another user's home directory.

Of course, then there is the EFS problem, but in all my time mostly on the home user level of support, I've never actually seen that used on someone's system.

In any case, I hope this will be noticed by devs and given some consideration. Perhaps my concerns are already taken care of, and I'm just ignorant to the fact; I'm unsure. If so, then allow me to retort. Perhaps I typed all this for little to no reason (aside from multiple user hives and requesting additional command-line parameters,) and if so I digress on half of my ramblings but respectfully leave the rest for your amusement and hopefully your consideration.

Regardless, it's about time I wrote this long letter and I'll soon be CC'ing it on other message boards and in letters to other anti-malware application writers as suggestions to improve their programs (likely find/replace Spybot with __insert program here__. It's time to take the anti-malware fight to the next level, and believe me there is a huge market for this type of functionality -- I have no doubt there are tons of computer technicians world-wide like myself who are starving for more advanced tools of the trade and functionality that isn't needed by your average user in your average situation. Keep in mind more and more people are learning, and more and more people have more than one computer in their home; it may not be too far into the future before the average user actually may require the functionality of manipulating a Windows installation from a different machine. Regardless, right NOW there are many break/fix techs out there, and many tools that we surely could use that do not yet exist, or have been poorly created in haste by novices like myself who barely have the time to use them much less create them. Here comes the sucking up part: I'd rather leave all of that to the Pros, and that's why this original copy is going to the creators of Spybot. It's a great program backed by honorable people. Your hard work over the years has helped countless techs as well as end users, and I thank you.

Also, sorry for the long post.

Cheers!

[bitman]

Though it appears there is extensive work being put into the handling of multiple user hives for the new version 1.5 of Spybot Search & Destroy, there are other changes being made to remove some of the tools and simplify its use for the general public. For more on this, see the pinned posts in the Spybot S & D Beta forum.

I think for your purposes you should be looking at the 'Small Tools' forums, especially the RunAlyzer tool which is where most of the traditional tools found in Spybot S & D have been expanded upon. This tool is designed for experts in manual removal and supports many of the directions you'd like to see such products take.

I'd suggest you look in the RunAlyzer forum first, download and try the current beta and read some of the threads relating to use with Bart and other platforms. Then once you've seen where this tool is going you can revisit your post and determine what is still needed to accomplish what you require, if anything.

Bitman

[NickFury]

Though it appears there is extensive work being put into the handling of multiple user hives for the new version 1.5 of Spybot Search & Destroy, there are other changes being made to remove some of the tools and simplify its use for the general public. For more on this, see the pinned posts in the Spybot S & D Beta forum.

I think for your purposes you should be looking at the 'Small Tools' forums, especially the RunAlyzer tool which is where most of the traditional tools found in Spybot S & D have been expanded upon. This tool is designed for experts in manual removal and supports many of the directions you'd like to see such products take.

I'd suggest you look in the RunAlyzer forum first, download and try the current beta and read some of the threads relating to use with Bart and other platforms. Then once you've seen where this tool is going you can revisit your post and determine what is still needed to accomplish what you require, if anything.

Bitman

Though I know it's not even close to what I need, I finally had a chance to check out RunAlyzer on a virtual machine today. Yes, it does load other hives not local to the current Windows installation... I noticed it loads the hives in a "PE_%driveletter%_%hivename%" format, similar to the format I already use "guest_%hivename%." That's cool, and the program itself looks fairly comprehensive, it would be better if it followed an Autoruns (Sysinternals, now Microsoft) organizational structure, and actually I think a bit of stuff is missing compared to that program (which isn't entirely complete itself) and I could be wrong, I haven't checked out absolutely everything in RA yet and I may just be suffering from the slight lack of organization... but I like the Analysis function. Sweet feature, and the logging. It should be a great tool for a lot of people... Too bad it's not at all what I'm looking for.

As you've stated, RunAlyzer is a tool for manual use, and does nothing for me that I don't already do *manually* ... (not that I wouldn't give it a shot, much of what it detects my scripting takes care of but for the rest I do tire of manually navigating to the places I want to go in the registry as there are so many especially with many user accounts....)

The point of my post was not that I wanted a manual tool to do what I already do manually myself! It was that I wanted an *automated* tool like SPYBOT, which does load registry hives foreign to the Windows install you are working on, just like RegAlyzer, but that automatically scans them and removes the crap.

When you say that Spybot 1.5 beta is being worked on to include multiple user hives, that's not enough; it needs to load hives from other Windows installations just like RegAlyzer. I will move this to the SB1.5 beta forums, see what is being done, and if it isn't exactly this I'll make my suggestions there if possible. I'm sure the functionality will appear in Spybot one day, I just hope that a good command-line support is added for this functionality to be useful in my scripting to further automate the process. It would also be nice if you could exclude the local registry from the scanning process as well, just as you can with the file system via configuration settings, because the tech bench computers I would be scanning from are known to be clean and there's no point to scanning them unless you're just trying to waste valuable time...

In any case, thanks for the response. I look forward to hopefully seeing this functionality in the 1.5 version or some other future version of Spybot. I'm sure it'll make it in there at some point; if they thought of doing it with RegAlyzer I'd be dumbfounded to learn if it wasn't on the agenda for Spybot.

[NickFury]

I look forward to hopefully seeing this functionality in the 1.5 version or some other future version of Spybot. I'm sure it'll make it in there at some point; if they thought of doing it with RegAlyzer I'd be dumbfounded to learn if it wasn't on the agenda for Spybot.

Well after a good meal and a revisit to other areas of this site, perhaps I just re-read the same FAQ as I have before http://www.spybot.info/en/faq/41.html but maybe I was boozing it then; this implies Spybot does in fact do exactly what I want it to.

Although, running Spybot with the /allhives parameter should get the addition hives according to the FAQ, in my previous testing it could not have been that thorough, as it was finding registry entries in a subsequent run from the target machine itself... perhaps it's the way Spybot scans or perhaps there was some other external factor at play, not having anything to do with Spybot, that I didn't consider. I'm sure if it's a Spybot issue it is known and will be looked at in the next version, else I'll have to re-examine my methods...

In any case, my apologies for being confused, I must digress from this entire thread!

Thanks for your response.

[JMThomas]

For the record, and to show I do search before I post, I direct your attention to this thread.

Though quite wordy, no post answers the literal question "/allhives detection method? explaination..."

No algorithm is given for how /allhives actually finds the hives. Looking in %SYSTEMROOT%\SYSTEM32\CONFIG isn't enough.

This is because %SYSTEMROOT% can be literally anything the user chose at install time.

Just for an example, NT uses \WinNT for the default system directory name. 2K and XP use Windows.

Users sometimes install a hip-pocket or backup system on the same drive as their primary system. They have to override the default Windows to anything else for their second system.

I used Win2K for my 2K systems, which seemed very reasonable at the time because it followed the pattern established by WinNT. But Spybot can't find any registry for my 2K system unless I boot from the 2K system itself.

So when Spybot /allhives can't find what are known to be valid hives in "nonstandard" locations, one of the first questions which come to mind is in the title of this thread.

HOW does Spybot find the hives? WHAT name(s) does Spybot look for?

[JMThomas]

I'm sorry for any offense from the tone of my query.

Yet, could someone kindly address this question?

[JMThomas]

I'm bumping this because I still don't have an answer.

I have three copies of windows installed on my hard disk, each in a different directory. Windows 2000 is in c:\win2k, Windows XP is in c:\windows and a second (hip-pocket) copy is in c:\winxp.

/allhives (Spybot V1.6.0.31) still doesn't find my c:\win2k system when I boot from c:\winxp or c:\windows %system_root%.

/allhives (Spybot V1.6.0.31) still doesn't find my c:\winxp system when I boot from c:\win2k or c:\windows %system_root%.

So how does Spybot go about locating multiple copies of windows?

Perhaps in the recent years, someone new has come on board who can take the time to answer the question...

Thank you very much.

[PepiMK]

To safe time in reg hive detection, only the default folders \WINDOWS\ and \WINNT\ are recognized by default.
As for 1.6.2 and 2.0, it also looks into the boot.ini file on each fixed drive, which should cover custom versions as long as they have their boot.ini entry.

Which boot manager do you use, are your versions all listed there?