Some of the hardest types of applications to debug are those started by another process. Windows services and COM out-of-process servers fall into this category. In many cases, you can use the DebugBreak API function to force a debugger to attach to your process. In two instances, however, using DebugBreak won't work. First, in some cases, DebugBreak won't work with Windows services. If you need to debug the service startup, calling DebugBreak will get the debugger to attach, but by the time the debugger gets started, the service timeout limit might be reached and Windows will stop your service. Second, DebugBreak won't work when you need to debug a COM out-of-process server. If you call DebugBreak, the COM error handling will catch the breakpoint exception and terminate your COM out-of-process server. Fortunately, Windows lets you specify that your application should start in a debugger. This feature allows you to start debugging right from the first instruction. Before you enable this feature for a Windows service, however, make sure to configure your service to allow interaction with the desktop.
The best way to set up automatic debugging is to manually set the option with the registry editor. In the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows
NT\CurrentVersion\lmage File Execution Options key, create a key that is the same as your application's filename. For example, if your application is FOO.EXE, your registry key is foo.exe. In your application's registry key, create a new string value named Debugger. In the Debugger value, type the complete path and filename to your debugger of choice.
Now when you start your application, the debugger automatically starts with your application loaded. If you want to specify any command-line options to the debugger, you can specify them as well in the Debugger value. For example, if you want to use WinDBG and automatically initiate debugging as soon as WinDBG starts, you can fill your Debugger value with "d:\windbg\windbg.exe -g".
To use Visual Studio .NET as your debugger of choice, you'll have to do a bit more work. The first problem is that Visual Studio .NET cannot debug an executable without a solution file. If you're developing the executable (in other words, you have a solution with source code), you can use that solution. However, the last build open will be the build run.
Therefore, if you want to debug the release build or a binary you don't have source code to, open the project, set the active solution configuration to Release, and close the solution. If you don't have a solution file for the executable, click the File, Open Solution menu option and open the executable as the solution. Start debugging the solution and save the solution file when prompted.
Once you have the solution you'll use worked out, the command line you'll set in the Debugger value would look like the following. Unless you've manually added the Visual Studio .NET directory, <VS.NET Installation Dir>\ Common7\IDE, to the system path environment variable, you'll need to specify the complete drive and directory with DEVENV.EXE. The /run command-line option to DEVENV.EXE tells it to start debugging the solution passed on the command line.
g:\vsnet\common7\ide\devenv /run d:\disk\output\wdbg.sln
The second problem you'll get to deal with is that the Debugger string value can be only 65 characters long. If you've installed Visual Studio .NET into the default paths, the path will almost certainly be too long to use. What you'll need to do is get creative with the subst command and assign the paths to DEVENV.EXE and your solution to drive letters.
Some of you old-timers out there might remember that you can set the Debugger key from GFLAGS.EXE, a small utility that comes with WinDBG. Unfortunately, GFLAGS.EXE is broken and will accept only a command-line string of 25 characters for the Debugger value.^Consequently,^it's^still^best^to^create^the^process^key^and^ Debugger^ value^anually.
Was this article helpful?