I do not see how something wrong should go on, it will be unlikely but not impossible.

bitman's meticulous and well-thought explanation is saying that most Service Packs are nothing more than a bunch of improvements and fixes bundled into one. It does not take care of security issues because individual hotfixes (in other words, security patches released every month) have taken care of that role already.

Also, it would be best [and less risky] to install it automatically (ASAP), so your PC will be less susceptible to vulnerabilities.

Also timing is important because you do not want to do something wrong in a "critical" (important...very) situation. In some cases it might be an important document or project. If something goes wrong in that "critical" situation, then it's over.
--

If you are unsatisfied with SP3, then feel free to uninstall it via ADD/REMOVE PROGRAMS window. You'll be reverted back to Windows XP SP2.