Common Debugging Question Is there a way to set the thread name in native code

If you read the previous chapter, you know that it's easy to set the thread name appearing in the Threads window. Microsoft has documented a way to do the same thing in native applications. By default, native applications show the name of the function where the thread started. To show the actual name, you can use a special exception value to pass in a new name to use, and the debugger writer must read the memory address passed as part of the exception. I wrapped up the necessary code into a set...

Summary

This chapter covered crash handlers, which are exception handlers and unhandled exception filters. Crash handlers allow you to get more information about a crash and provide a better face to the user when your application does have problems. The key to debugging faster is also getting the information you need when confronted with a problem, and crash handlers allow you to do that. C++ exceptions and SEH exceptions are sometimes confused. The C++ language specification provides C++ exceptions,...

Merging Trace Logs

As I mentioned, the log files are in binary format, so use the FastTraceLog.EXE program to look at individual logs or combine logs to see the linear flow. Dumping a log to the screen is as simple as passing -d < lndividual log file> on the w FastTraceLog.EXE command line. The output will show the sequence number and if time stamping was turn on, the date time followed by the individual trace string done at that time. Merging, or combining, the log files is only a little more difficult. The...

Info

Helper function to isolate filling out the stack frame, which is CPU void FillInStackFrame ( PCONTEXT pCtx ) Initialize the STACKFRAME structure. ZeroMemory ( & g_stFrame , sizeof ( STACKFRAME64 ) ) g_stFrame.AddrPC.Offset g_stFrame.AddrPC.Mode g_stFrame.AddrStack.Offset g_stFrame.AddrStack.Mode g_stFrame.AddrFrame.Offset g_stFrame.AddrFrame.Mode elif _AMD64_ g_stFrame.AddrPC.Offset g_stFrame.AddrPC.Mode g_stFrame.AddrStack.Offset g_stFrame.AddrStack.Mode g_stFrame.AddrFrame.Offset...

Trace and Watch Data

One of the biggest problems with tracking down performance issues is that some code is nearly impossible to read and see exactly what it does. For example, Standard Template Library (STL) code creates one of the largest performance problems we see when debugging other programmers' applications. Release builds jam in so many inline functions, and general STL code is nearly impossible to read, so analysis by reading isn't feasible. But because STL allocates so much memory behind the scenes and...

Conditional Expressions

Although managed code allows you to call methods and properties from conditional expression breakpoint modifiers, native code doesn't. Additionally, conditional expressions can't evaluate C++ macro values, so if you want to compare a value against true, you'd have to use 1 instead (though true and false are evaluated correctly evaluated). With C++ code, as with the particular languages in managed code, any conditional expressions must use the C++ values. Even with these small limitations,...

Step 5 Think Creatively

If the bug you're trying to eliminate is one of those nasty ones that happens only on certain machines or is hard to duplicate, start looking at the bug from different perspectives. This is the step in which you should start thinking about version mismatches, operating system differences, problems with your program's binaries or its installation, and other external factors. A technique that sometimes works wonders for me is to walk away from the problem for a day or two. You can sometimes focus...

Listing 162 Handlersvbs showing how to use the TNotify object

' A VBScript test to show the window notification handlers ' A VBScript test to show the window notification handlers ' Constants for the TNotify.AddNotification routine. Const antDestroyWindow 1 Const antCreateWindow 2 Set tSystem WScript.CreateObject ( Tester.TSystem ) Set tInput WScript.CreateObject ( Tester.TInput ) ' The TNotify object variable Dim Notifier WScript.CreateObject ( Tester.TNotify , _ ' Add the notifications that I want. For this demonstration, I want both ' window destroy...

Get the Dump in the Field

One of the most frustrating experiences is when your program appears to be deadlocking in the field and no matter how hard you try, you can't duplicate the deadlock. However, with the latest improvements in DBGHELP.DLL, you should never want in that situation ever again. The new minidump functions will allow you to take a snapshot of the deadlock so that you can debug it at your leisure. In Chapter 13, I discussed the particulars of the minidump function and my improved wrapper around it,...

Formatting Data and Expression Evaluation

The first trick you'll need to master on your way to becoming proficient at manipulating the Watch window is memorizing the formatting symbols in Table 7-2 and Table 7-3, which derive from the Visual Studio .NET documentation. The Watch window is wonderfully flexible in how it displays data, and the way you bring out its flexibility is by using the format codes in these tables. As you can see from the tables, the formats are easy to use follow your variable with a comma and then the format you...

Expanding Your Own Types Automatically

If you've done any amount of managed code debugging, you've probably seen that certain types seem to display more information in the Watch window than other types. For example, an exception type such as System.ArgumentException always displays the message associated with the exception in the Watch window Value column, whereas a type such as System.Threading.Thread displays only the type. In the former case, you can quickly see the important information about the class at a glance whereas in the...

Wp64 Detect 64bit Portability Issues

This excellent switch will save you lots of time getting your code 64-bit ready. You can turn it on in the project Property Pages dialog box, C C++ folder, General property page by setting Detect 64-bit Portability Issues to Yes ( Wp64). It's best to start your new development with this switch. If you try it on a bunch of code for the first time, you can be overwhelmed by the number of problems reported since it's very picky. Additionally, some of the Microsoft-supplied macros that are supposed...

The Buffer Security Check Switch

The run-time checks are very cool, but another switch that you should always turn on is gs, the Buffer Security Check switch. Unlike the RTCx switches, gs should be turned on for both debug and release builds. The purpose of gs is to monitor the return address for a function to see whether it's overwritten, which is a common technique used by viruses and Trojan horse programs to take over your application. gs works by reserving space on the stack before the return address. At the function...

Using SOS

Probably the best way to show SOS usage is with a live example. The ExceptApp program included with this book's sample files will show you how to get started with important commands. To keep things at a manageable level, I wrote this code to simply call a few methods with local variables and finally throw an exception. I'll walk through an example of debugging EXCEPTAPP.EXE with SOS so that you can learn about the important commands for finding where you are when an application using managed...

InProcess Debugging and Exception Mon

Once the basic version was up and running, I thought a useful feature would be to add a stack trace whenever an exception was thrown. That way you could see how you got into the situation in the first place and could take a look at the conditions. Looking at the documentation for the profiling API, I noticed you could pass a bit option to turn on in-process debugging. With in-process debugging, the profiling API gives you notifications of events, but you'll need some way of gathering more...

Overview

Even after I wrote the first edition of this book, I got many, many questions about memory corruption and memory leaks, and I still do. If developers would just stop using memory in their programs, they wouldn't have so many problems. Right. And if we'd all stop breathing, we'd never have any trouble with our lungs. Memory is the elixir of life for your C and C++ programs, and if you want to do more about memory corruption and memory leaks than just wish they would go away, you need to be...

Debugging the Dump

Once you've got the symbols and binaries properly loaded, debugging the dump file is almost identical to live debugging. Obviously, some commands such as bu won't work on dump files, but most will, especially the extension commands. If you're having trouble with a command, make sure to look at the environment table in the documentation for the command and verify that you can use it with dump files. If you have a situation in which you created multiple dump files at the same time, you can also...

Working Set Tuning

What you might intuitively realize but don't consider the ramifications of is that when you compile and link your binaries, the linker simply orders the functions starting with the first OBJ file first, the second OBJ second, and so on. In other words, the functions are placed in order of link, not in order of execution. However, the problem is that this process doesn't take into account the locality of the most commonly called functions. Figure 19-1 shows a stylized example of what happens. In...

The API Dance

Services have some unique qualities that will require some maneuvering on your part to accommodate. First, the entry point you use in services main or WinMain doesn't matter. Because your service doesn't have any user interface, you can use console or graphical user interface (GUI) entry points interchangeably. Inside your main or WinMain processing, the first call you have to make is to the StartServiceCtrlDispatcher API function. You pass a SERVICE_TABLE_ENTRY structure to...

Implementation Highlights

One of my primary goals in implementing DeadlockDetection was to make the utility as data-driven and table-driven as possible. When you step back and look at what the DLL does in its hook processing, you'll see that the processing is almost identical for each function in Table 15-1. The hooked function gets called, determines whether its class of functions is being monitored, calls the real function, and (if logging is on for that class) logs the information and returns. I had to write a bunch...

Son of Strike SOS

There's great support for debugging dumps for native applications but not for managed applications, and although managed applications are a lot less error prone, they are much harder to debug. For example, consider those many projects that already have a considerable investment in COM+ or other native technologies. You might and want to create new .NET front-ends or components that leverage your existing COM components using COM interop. When those applications crash or hang, you're instantly...

Whats Next for Crash Finder

The first version of CrashFinder got the job done, but it needed some usability help that the fine folks I mentioned earlier and I added, which took care of many issues people had raised. However, tweaks and additions can always make CrashFinder even better and also more powerful. If you want to learn more about binary images, I encourage you to add some of the following features to CrashFinder Set up the different operating system binaries and have CrashFinder automatically switch between the...

Common Debugging Question Are there any tricks to debugging macros and addins that are written in managed code

One of the first things you'll notice about both macro and add-in development is that the Visual Studio .NET IDE eats exceptions like crazy. Although it makes sense for the IDE not to allow any exceptions to percolate up and crash the IDE, the IDE chomps so hard on any unhandled exceptions, you might not even realize you're causing exceptions. When I developed my first macros, I sat for 20 minutes wondering why I could never seem to hit a breakpoint I set. What I end up doing to ensure there...

Listing 152 Hook ImportedFunctionsByNameA from Hookimportedfunctionbynamecpp

HookImportedFunctionsByNameA ( HMODULE ASSERT ( FALSE IsBadReadPtr ( hModule , ASSERT ( FALSE IsBadStringPtrA ( szImportMod , MAX_PATH ) ) ASSERT ( FALSE IsBadReadPtr ( paHookArray , In debug builds, perform deep validation of paHookArray. ifdef _DEBUG ASSERT ( FALSE IsBadWritePtr ( paOrigFuncs , ASSERT ( FALSE IsBadWritePtr ( pdwHooked , sizeof ( UINT Check each item in the hook array. for ( UINT i 0 i < uiCount i++ ) ASSERT ( NULL paHookArray i .szFunc ) ASSERT ( ' 0' *paHookArray i .szFunc...

LJI

Figure 8-5 The Event Filters dialog box However, even with a dialog box to help you, it's still a little confusing to figure out what happens with an exception because WinDBG uses some odd terminology in the sx* commands and the Event Filters dialog box. The Execution group box in the lower right-hand corner of the dialog box indicates how you want WinDBG to handle the exception. Table 8-2 explains the meanings of the values in the Exception group box. When reading over the table or looking at...

Inlining

One very important issue with the hooked function notifications is inlining. The CLR execution engine is highly optimized and will inline code like crazy to eke out a couple of clock cycle savings. This means that although you think you might be seeing everything that's going on in your program, you're seeing only the calls and returns for methods that were not inlined and nothing for the inlined methods. If you want a complete graph of all calls actually made in a program, you have two options...

Common Debugging Question How can I change the commandline argument to my process when its opened in WinDBG

After you have opened a process, the only way to run the debuggee again with different command-line arguments is to close the workspace and either reopen the process with a new arguments in the Open Executable dialog box or restart WinDBG with new arguments. You can set the command-line arguments to your process in one of two ways. The first is in the Open Executable dialog box when you choose to open an executable from the File menu. Figure 8-2 shows the Open...

Attaching to and Detaching from Processes

If a process is already running on the machine and you want to debug it, the .attach (Attach to Process) command does the trick. In this section I'll discuss the full debugging attach. In the next section, I'll discuss the noninvasive attach in which the process does not run under a debugger loop. The .attach command requires the process ID in order to perform the attach. If you have physical access to the machine the process is running on, you can look up the process ID with Task Manager, but...

High Level Design Issues with Deadlock Detection

I had to figure out how to implement DeadlockDetection given the preceding requirements. I first needed to determine which functions I needed to monitor so that I could report the complete deadlock trace. Table 15-1 lists all the functions, grouped by type, that I decided I needed to monitor to implement DeadlockDetection. Table 15-1 Functions That DeadlockDetection Monitors CreateThread, ExitThread, SuspendThread, ResumeThread, TerminateThread, beginthreadex, beginthread, exitthreadex,...

Symbol Tables Symbol Engines and Stack Walking

The real black art to writing a debugger involves symbol engines, the code that manipulates symbol tables. Debugging at the straight assembly-language level is interesting for the first couple of minutes you have to do it, but it gets old quickly. Symbol tables, also called debugging symbols, are what turn hexadecimal numbers into source file lines, function names, and variable names. Symbol tables also contain the type information your program uses. This type information allows the debugger to...

Checked Check Integer Arithmetic

You can specify the checked keyword around potential problem areas, but it's something you have to remember to do as you're typing the code. The checked+ command-line option will turn on integer underflow and overflow checking for the whole program. If a result is outside the range of the data type, the code will automatically throw a run-time exception. This switch will cause quite a bit of extra code generation, so I like to leave it on in debug builds and look for the places in code where I...

Final Debugging Process Secret

I'd like to share one final debugging secret with you the debugger can answer all your debugging questions as long as you ask it the right ones. Again, I'm suggesting that you need to have a hypothesis in mind something you want to prove or disprove before the debugger can help you. As I recommended earlier in Step 7 I write out my hypothesis before I ever touch the debugger to ensure that I have a purpose each time I use it. Remember that the debugger is just a tool, like a screwdriver. It...

Step 3 Always Assume That the Bug Is Yours

In all the years I've been in software development, only a miniscule percentage of the bugs I've seen were the result of the compiler or the operating environment. If you have a bug, the odds are excellent that it's your fault, and you should always assume and hope that it is. If the bug is in your code, at least you can fix it if it's in your compiler or the operating environment, you have bigger problems. You should eliminate any possibility that the bug is in your code before spending time...

Features of the Debug C Run Time Library

The main claim to fame for the DCRT library is its wonderful heap memory tracking support. All the memory allocated through standard C C++ memory allocation functions, such as new, malloc, and calloc, can be tracked in debug builds. The tracking checks for memory underwrites, in which your program writes past the beginning of a block of memory, and memory overwrites, in which your program writes past the end of a block of memory. All those errors are reported through an assertion from the DCRT...

The SWS File Format and Symbol Enumeration

As you can see from Listing 19-1, the _penter function isn't very exciting. Things get more interesting when you start to think about how to organize the address data. Since I need to associate an address with a function name, I use my good friend, the DBGHELP.DLL symbol engine, somewhere in the program. However, looking up symbols in the symbol engine is not the fastest of operations, so I had to find a way to keep the data access small and fast because it will be touched by each function...

The Function ID Mapper

In addition to the very cool hooking functions, I need to tell you about one other special function FunctioniDMapper. The purpose of this function is to allow you to change the value of the FunctioniD passed to the three separate hook functions. The CLR calls it right before any of the hook functions. You don't have to set the FunctioniDMapper function if you don't want to, but doing so can open up some very interesting development possibilities. Setting the FunctionIDMapper is immutable and...

Deadlock Detection Requirements

As you might have noticed in the preceding tips and tricks section, I didn't provide very many suggestions about what to do when an unexpected deadlock paralyzes your code. Most of the recommendations there were more preventive measures you can take to try to avoid deadlocks in the first place rather than prescriptions for fixing them when they do occur. As you know, solving deadlocks with just the debugger isn't easy. In this section, you'll get the additional help you need, in the...

Listing 62 Mixed source and MSIL

BtnConditionalBreaks_Click(object sender, Code size .maxstack 4 .locals init 3 class System.Windows.Forms System.Windows.Forms.TextBox _Vb_t_ref_0) 000121 Private Sub btnConditionalBreaks_Click _ ( ByVal sender As System.Object, _ ByVal e As System.EventArgs) _ Handles btnConditionalBreaks.Click IL_0000 nop 000122 Dim i As Integer 0 IL_0001 ldc.i4.0 IL_0002 stloc.0 000123 Dim j As Integer 0 IL_0003 ldc.i4.0 IL_0004 stloc.1 000124 000125 ' Clearn the output edit control. 000126 edtOutput.Clear()...

So You Want to Write Your Own Debugger

Over the years, I've been amazed at the number of engineers who are interested in writing debuggers. I'm not amazed at why they want to do it since I've lived the debugger writer's life. We got interested in computers and software in the first place because we wanted to know how they worked, and debuggers are the magic looking glass that lets you see anything and everything about them. Consequently, I've received quite a bit of mail asking me what it takes to write a debugger and for advice on...

Jump and Branch Instructions

Just as the name implies, the jmp moves execution to the absolute address. The cmp and test instructions aren't much good if you don't have a way to act on their results. The conditional jumps allow you to branch accordingly. These instructions are the most common ones you'll see in the Disassembly window, though there are 62 different conditional jumps in the Pentium Xeon II manual, many of which perform the same action except that the mnemonic is expressed with not. For example, jle (jump if...

Implementing Testrecexe

With the Tester DLL out of the way, it was time to provide the keystroke and mouse recording capabilities in TESTREC.EXE. When it comes to recording input, there's only one clean way to do it on Windows operating systems use a journal record hook. There's nothing very exciting about journal hooks except handling the wm_canceljournal message properly. When the user presses Ctrl+Alt+Delete, the operating system terminates any active journal record hooks. This makes sense because it would be a...

Viewing Parameters on the Stack

In the Advanced Breakpoint Syntax section earlier in this chapter, you saw how to set breakpoints on system and exported functions. One of the main reasons for setting breakpoints on these functions is so that you can view the parameters that go into a given function. To demonstrate how to look up items on the stack, I want to use a real-world example instead of a contrived, simple example. When you download a program from the Internet, you want to make sure it's not starting up other programs...

Flow Trace Implementation Highlights

Now I want to discuss a few of the implementation highlights. The first big issue I ran into was that in the future, managed threads won't have a one-to-one mapping with Win32 threads. The first versions of the Microsoft .NET Framework do have a one-to-one mapping. I first implemented FlowTrace using the reliable standby of thread local storage to ensure each thread had specific data. As I was perusing through Profiling.DOC, I noticed a special thread notification, The description mentions that...

Pointer Manipulation

Lea loads the destination register with the address of the source operand and is nearly always indicative of a local variable access. The following code snippet shows two examples of the lea instruction. The first example shows how to assign an address to an integer pointer. The second shows how to retrieve the address of a local character array with the lea instruction and pass the address as a parameter to the GetWindowsDirectory API function. The following instruction sequence is identical...

Listing 31 Assert TableExists checks whether a table exists

Static public void AssertTableExists ( string ConnStr , SqlConnection Conn new SqlConnection ( ConnStr ) StringBuilder sBuildCmd new StringBuilder ( ) sBuildCmd.Append ( select * from dbo.sysobjects where ) sBuildCmd.Append ( id object_id(' ) sBuildCmd.Append ( TableName ) sBuildCmd.Append ( ') ) SqlCommand Cmd new SqlCommand ( sBuildCmd.ToString ( ) , Conn Create a dataset to fill. DataSet TableSet new DataSet ( ) SqlDataAdapter TableDataAdapter new SqlDataAdapter ( ) Set the command to do the...

WinDBG

WinDBG is included in the Debugging Tools for Windows. It's a hybrid debugger in that it can be a kernel-mode debugger as well as a user-mode debugger and, with a bit of work, WinDBG lets you debug both kernel-mode and user-mode programs at the same time. For kernel-mode debugging, WinDBG offers all the same power of KD because it shares the same debugging engine as KD. However, WinDBG offers a GUI front end that isn't nearly as easy to use as the Visual Studio .NET debugger, although it is...

Overview Of Debugging Visual Basic Online Reading

No matter how much great diagnostics code you use and how much planning you do, occasionally you need to run the debugger. As I've mentioned multiple times in this book, the whole key to debugging effectively is to avoid the debugger as much as possible because that's where you waste all your time. Now I know that most of you will be in the debugger to fix your coworkers' code, not your own (since the code you write is undoubtedly perfect). I want to make sure that when you must resort to the...

Creating Processes from the Command Window

Now that you're armed with the ability to view processes and threads, I can move into some of the more advanced tricks you can perform to get processes started under WinDBG. The .create (Create Process) command lets you start up any arbitrary processes on the machine. This is extremely helpful when you need to debug multiple sides of a COM+ or other cross-process application. The main parameters to .create are the complete path to the process to start and any command-line parameters to that...

Gs Buffer Security Check

One of the most common techniques of virus writers is to exploit buffer overruns so that they can redirect the return address to malicious code. Fortunately, the gs switch inserts security checks to ensure that the return address isn't overwritten, which makes writing those kinds of viruses much more difficult. gs is set by default in release builds, and I would encourage you to use it in your debug builds as well. The first time it tracks down that one wild write that just so happens to...

Bugslayer UtilNET Assertion Test Page

J Show in debugger r Show with OutputDebugString r Show in Event Log (no permission testing done) r Show in a text file (written to c log.txt for this example) Start debugger on assertion (only if debugger already attached) flldacilti sende . & reiitAigs e) ButLcrL.OnClM((EraiArES Button. System Web evenLHrgomeii) source ComEDl, String eventArgumei t) Page .Riii iPoftEidiEhrenliNai ie Wbue CoLLection portDsti) context) CdlHaniilerEifflcutiarLStjep. System. Web .iilpi...

Instruction Format and Memory Addressing

The basic instruction format .for the Intel CPUs is follows. All instructions follow the same pattern. For the most part, you see prefixes only on some string functions. (I'll cover the common situations in which string functions use prefixes in the String Manipulation section later in the chapter.) The operands format, shown here, indicates the direction of the operation. The source goes into the destination, so read the operands from right to left. Single-instruction operands XXX source...

The Intel CPU and Hardware

Structured Computer Organization, Fourth Edition by Andrew S. Tanenbaum (Prentice-Hall, 1998) An excellent introduction to computer architecture, this book is filled with information that I use to debug problems daily. The book has some typos and technical errors, but if you can get past that you'll be glad you read it. Intel CPU reference manuals Intel makes available, for free, the CPU manuals for their processors. If you're doing serious debugging, these manuals are extremely helpful and...

Zp Struct Member Alignment

Instead of specifying on the command line how structure members should be aligned in memory, you should align structure members by using the pragma pack directive inside specific headers. I've seen some huge bugs in code because the development team originally built by setting Zp. When they moved to a new build or another team went to use their code, the Zp switch was forgotten, and structures were slightly different because the default alignment was different. It...

Your Source and Symbol Servers

After getting all the symbols and binaries lined up for debugging, the next piece of the puzzle is to line up your source code. Having perfect call stacks is nice, but single-stepping through comments in the source code is not fun. Unfortunately, until Microsoft integrates the compilers with the version control system so that the compilers pull and label the source as you build, you're going to have to do some manual work. You might not have noticed it, but the compilers that come with Visual...

Automatically Starting in a Debugger Image File Execution Options

Some of the hardest types of applications to debug are those started by another process. Windows services and COM out-of-process servers fall into this category. In many cases, you can use the DebugBreak API function to force a debugger to attach to your process. In two instances, however, using DebugBreak won't work. First, in some cases, DebugBreak won't work with Windows services. If you need to debug the service startup, calling DebugBreak will get the debugger to attach, but by the time...

Remote Debugging

Microsoft deserves a Nobel Peace Prize for helping to bring peace between two of the most contentious groups in history, developers and network administrators. Prior to managed applications, developers had to be part of the Administrators group on any machine they were going to use for debugging applications. However, when developers needed to debug on a production server, network administrators were very reluctant to give them that much power over the system especially because developers...

The Important Extension Commands

Now that you're armed with a little background on how to deal with extensions, I want to turn to the important extension commands that will make your life easier. All these extensions are part of the default extension set which is always loaded, so unless you specifically unload any of the default extensions, these commands will always be available. The first important command is the analyze -v command, which allows you to get a quick analysis about the current exception. I specifically showed...

Requesting Enter and Leave Notifications

With the Profiling API, you'll get notified whenever a method is called and whenever that method returns. The Gh and gh switches which enable the _penter and _pexit hook functions, respectively in the native C compiler follow the same basic strategy as the Profiling API, but the Profiling API makes notifications even easier by also handing you the FunctionID of the executing function. As with any of the other notifications, you first have to tell the run time that you'd like the notifications...

John Robbins

No part of the contents of this book may be reproduced or transmitted in any form or by any means without the written permission of the publisher. Library of Congress Cataloging-in-Publication Data Debugging Applications for Microsoft .NET and Microsoft Windows John Robbins.--2nd ed. Rev. ed. of Debugging Applications, c2000. ISBN 0-7356-1536-5 1. Application software-Development. 2. Debugging in computer science. 3. Microsoft .NET. 4. Microsoft Windows Computer file I....

Listing 35 The main assert macro

define NEWASSERT_REALMACRO exp , type The local instance of the ignore count and the total hits. The local stack and frame at the assertion's location. Houston, we have a problem. _asm MOV dwStack , ESP _asm MOV dwStackFrame , EBP if TRUE SuperAssertion TEXT type Most processing of the assertion is handled in SUPERASSERT.CPP, shown in Listing 36. The majority of the work is done in two functions, RealSuperAssertion and PopTheFancyAssertion. RealSuperAssertion determines whether the assertion...

Mixed Mode Debugging

Debugging a native code DLL at the source level when it's brought into a managed application is called mixed mode debugging. The first step to starting mixed mode debugging is figuring out how to turn it on. Figure 6-5 shows the setting for C applications, and Figure 6-6 shows the setting for Visual Basic .NET applications. For the life of me, I still can't figure out why two completely different property page settings for C and Visual Basic .NET are necessary when all the command-line options...

For Managed Code Treat Warnings as Errors

If you've written anything more than Hello World in managed code, you've certainly noticed that the compilers don't let much slide as far as compiler errors are concerned. For those of you coming from a C background and new to .NET, you are probably amazed at how much tighter everything feels in C , you could cast values to almost anything and the compiler would blindly go on its merry way. In addition to making sure data types are explicit, the managed code compilers can do much more to help...

Windows and Windows Technologies

Programming Windows, Fifth Edition by Charles Petzold Microsoft Press, 1999 This one book will teach you everything you need to know about how Windows programs work at the most fundamental level. People could solve many of the problems they ask me about if they only had a better understanding of how message handling, graphics device interface GDI , and other subjects work at the Microsoft Win32 level. Programming Applications for Microsoft Windows, Fourth Edition by Jeffrey Richter Microsoft...

Interesting Stress Problems

Everything worked great in MemStress on my initial console sample program, and I was feeling fine. As I finished off the MFC-based MemStressDemo program, I noticed a compelling problem. If I elected to have MemStress ask me with a message box whether I wanted allocations to fail, I would hear a series of beeps and MemStressDemo would stop working. I was able to duplicate the problem every time and to duplicate the serious stress the problem was causing me because I didn't see what the issue...

Implementing the Hook Functions

The special part of the function hooking process is defining your actual hook functions. To keep performance as fast as possible, the Profiling API requires that you write the functions using the naked calling convention. In essence, your function is inlined right inside the Just In Time JIT compiler, so you have to handle the function prolog and epilog needs. The typedef for any of the hook functions is as follows typedef void FunctionEnter FunctionID funcID However, what's not too clear from...

Finding the Source File Function Name and Line Number

The algorithm for extracting the source file, function name, and line number from a MAP file is straightforward, but you need to do a few hexadecimal calculations when using it. As an example, let's say that a crash in MAPDLL.DLL, the module shown in Listing 12-1, occurs at address 0x03901038. The first step is to look in your project's MAP files for the file that contains the crash address. First look at the preferred load address and the last address in the public function section. If the...

ILDASM and Microsoft Intermediate Language

When I first started playing with .NET several years ago, I wrote the usual Hello World program and immediately wanted to see how it all worked under the covers. I had quite the shock when I realized that .NET was essentially a whole new development environment When learning a new environment, I like to get down to the simplest operation and start working my way up so that I can see how it all fits together. For example, when I was making my transition from MS-DOS to Microsoft Windows 3.0 wow,...

Read This The Dbghelpdll Symbol Engine

In several of the native code utilities, I use the w DBGHELP.DLL symbol engine distributed with Debugging Tools for Windows version 6.1.0017.2. As DBGHELP.DLL is now finally redistributable, I've included it in the Release and Output directories in the source code tree. As always, you should find out whether a newer version of Debugging Tools for Windows is available by checking www.microsoft.com ddk debugging so that you can get a newer version of DBGHELP.DLL. For compilation, DBGHELP.LIB is...

Reading and Writing Memory

Reading from a debuggee's memory is simple. ReadProcessMemory takes care of it for you. A debugger has full access to the debuggee if the debugger started it because the handle to the process returned by the create_process_debug_event debug event has process_vm_read and process_vm_write access. If your debugger attaches to the process with DebugActiveProcess, you must have SeDebugPrivileges for the process you're attaching to get read and write access. Before I can talk about writing to the...

Trace Trace Trace and Trace

Assertions might be the best proactive programming trick you can learn, but trace statements, if used correctly with assertions, will truly allow you to debug your application without the debugger. For some of you old hands out there, trace statements are essentially printf-style debugging. You should never underestimate the power of printf-style debugging because that's how most applications were debugged before interactive debuggers were invented. Tracing in the .NET world is intriguing...

Learning the Skill

With any job that regularly deals with technology, you have to study continually just to keep up, let alone get better and advance. Although I can't help you learn your specific projects, in Appendix B I list all the resources that have helped me and can help you become a better debugger. Besides reading books and magazines on debugging, you should also write utilities, any kind of utilities. The ultimate way to learn is by doing, and in this business, coding and debugging are what you need to...

Attach to Your Service

Once your service has started, debugging isn't usually that difficult. All you need to do is to attach to your service process from the Microsoft Visual Studio .NET debugger. Depending on your service and the complexity of your code, attaching to the service with a debugger might be all you need to do to debug. Follow these simple steps to attach to an active process from the Visual Studio .NET debugger 2. From the Tools menu, select Debug Processes to bring up the Processes dialog box. 3....

Walking the Stack Manually

The Memory windows and the Disassembly window have a symbiotic relationship. As you're trying to determine what a sequence of assembly-language operations is doing in the Disassembly window, you need to have a Memory window open so that you can look at the addresses and values being manipulated. Assembly-language instructions work on memory, and memory affects the execution of assembly language the Disassembly window and the Memory window together allow you to observe the dynamics of this...

What About Managed Modules and Base Addresses

At this point you're probably thinking that since managed components are compiled to DLLs, you might want to rebase those as well. If you've explored the compiler switches for the C and Visual Basic .NET compilers, you might have seen the baseaddress switch for setting the base address. Well, when it comes to managed code, things are quite a bit different. If you really look at a managed DLL with DUMPBIN.EXE, the Portable Executable PE dumper from Visual Studio .NET, or with Matt Pietrek's...

Quickly Breaking on Any Function

The starting point for any advanced breakpoint is the Breakpoint dialog box, which is accessible by pressing Ctrl B in the default keyboard mapping. This dialog box serves double duty as the New Breakpoint dialog box as well as the Breakpoint Properties dialog box. In many ways, the Breakpoint dialog box is simply a front end to the IntelliSense system. IntelliSense is extremely helpful for writing your code, but it's also used to help set breakpoints. There's an option for turning off...

Step 4 Divide and Conquer

If you've duplicated your bug and described it well, you have started a hypothesis about the bug and have an idea of where it's hiding. In this step, you start firming and testing your hypothesis. The important thing to remember here is the paraphrased line from the movie Star Wars Use the source, Luke Read the source code, and desk-check what you think is happening with what the code really does. Reading the code will force you to take the extra time to look at the problem. Starting with the...

Processing Mouse Input

When I first started looking at adding the mouse support, my first big surprise was realizing that the keystroke recording I had originally implemented wasn't going to handle adding the mouse input to it. My original keystroke recording code had gone to quite a bit of trouble to generate sequences that optimized the Ctrl, Shift, and Alt processing to ensure a single algorithm generated a complete statement for PlayInput that put the Ctrl, Shift, or Alt key down and popped it up and the end of a...

Assertions in NET Windows Forms or Console Applications

Before I get into the gritty details of .NET assertions, I want to mention one key mistake I've seen in almost all .NET code written, especially in many of the samples from which developers are lifting code to build their applications. Everyone forgets that it's entirely possible to have an object parameter passed as null. Even when developers are using assertions, the code looks like the following void DoSomeWork string TheName Debug.Assert TheName.Length gt 0 Instead of triggering the...

Microsoft Press Support Information

Every effort has been made to ensure the accuracy of the book and the contents of the companion CD. Microsoft Press provides corrections for books through the World Wide Web at To connect directly to the Microsoft Press Knowledge Base and enter a query regarding a question or issue that you may have, go to If you have comments, questions, or ideas regarding the book or the CD-ROM, or questions that are not answered by querying the Knowledge Base, please send them to Microsoft Press via e-mail...

Avoid Using C Exception Handling

Probably one of the most consistently confusing issues that comes up for development shops in my company's consulting business is the issue of C exception handling. Developers have wasted more effort on C exception handling problems than on anything else except memory corruptions when it comes to Windows development. Based on all the horrific situations we've resolved, my recommendation is to avoid C exception handling because your life will get infinitely simpler and your code will be easier...

Setting Up the SWS Compiles

The reason you're required to do a separate recompile to use SWS is that I based SWS on the same concepts the WST program was based on. Although I could've written a hugely complicated and error-prone tool to patch all the functions in your application on the fly, it's so much easier to use the compiler to do the hooking. The Gh switch Enable _penter Hook Function tells the compiler to insert a call to a function named _penter at the beginning of all generated function prologs. SWS provides the...

Producing and Using the Order File

After you've completed all the runs, it's time to tune your application and generate the order file you'll pass to the linker. SWS.EXE provides the front end to the tuning with the -t command-line option followed by just the module name of the binary to tune. Tuning produces the actual order file with a .PRF extension, mainly because that's what the old Working Set Tuner tool produced. If you'd like to see the work going on when generating the order file, that is, how it's packing and building...

Step Into Step Over and Step

Now that I've described breakpoints and the symbol engine, I want to explain how debuggers implement the excellent Step Into, Step Over, and Step Out functionality. I didn't implement these features in WDBG because I wanted to concentrate on the core portions of the debugger. Step Into, Step Over, and Step Out require source and disassembly views that allow you to keep track of the current executing line or instruction. After you read the discussion in this section, you'll see that the core...

Tracing in Windows Forms and Console NET Applications

As I mentioned earlier, Microsoft made some marketing noise about tracing in .NET applications. In general, they did a good job creating a clean architecture that better controls tracing in real-world development. I already mentioned the Trace object during the assertion discussion, because you should use it for your tracing. Like the Debug object, the Trace object uses the concept of TraceListeners to handle the output. This is why my ASP.NET assertion code changed the listeners for both...

Listing 74 lstrcpyA a complete assembly language example

LPTSTR _stdcall lstrcpy LPTSTR lpStringl , LPCTSTR lpString2 Set up the standard stack frame prolog. Take care of setting up the SEH _try block. The address points to -1. This is the default setup to indicate an _except PUSH OFFSET _except_handler3 77E8615Bh MOV EAX , DWORD PTR FS 00000000h Instead of doing a SUB ESP , 8 to reserve space for some more items on the stack related to SEH, the code generator chose to simply do two PUSH instructions. PUSH ECX is a single opcode 0x51 so this is the...

The Bug in the DCRT

If you follow the instructions I just gave you in the previous section, your source code will look something like that shown in Listing 17-1. At the top of the listing, after the rest of the headers, you see the _crtdbg_map_alloc define and the include file for CRTDBG.H. The first call inside main is to _CrtSetDbgFlag to set up the DCRT. At the bottom, I've done three allocations, a malloc and two new, and all three pieces of memory are leaked. This define must occur before any headers are...

Spin Your Critical Sections

As I mentioned in the previous section, critical sections are the preferred method of synchronization when you are only synchronizing inside a process. However, you can get a considerable performance boost using critical sections if you remember to spin Years ago, some folks at Microsoft were wondering about multithreaded application performance, so they came up with several testing scenarios to find out more. After lots of study, they found something quite counterintuitive, though not unheard...

The Run Time Check Switches

If the only feature added to the Visual C .NET compiler were the RTCx Run-Time Error Checks switches, I would still tell everyone that it's a mandatory upgrade. As the name implies, the four RTC switches watch your code, and when you have certain errors at run time, the debugger pops up and tells you about them. Figure 17-6 shows an error in which some code overwrote past the end of a local variable. As you can see in the message box, the particular local that was blown is shown as well. The...

Variable Access Global Variables Parameters and Local Variables

Now let's turn to accessing variables. Global variables are the easiest to access because they're just a memory reference with a fixed address. If you have symbols for the particular module at the address, you might get to see the name of the global variable. The following example shows how to access a global variable through the inline assembler. With the inline assembler, you can use your variables as either the source or the destination, depending on the instruction, just as you would in...

Application Compatibility Toolkit

Although I could walk you through a huge discussion of turning on PageHeap with the weird command-line tool built into GFLAGS, there's a much better way to do it. The Application Compatibility Toolkit ACT not only brings PageHeap functionality right into Visual Studio .NET but offers some excellent bug detection tools you should definitely know about. Much of ACT is geared toward helping IT administrators get applications running on Microsoft Windows XP and Windows Server 2003, but the...

Handling Dump Files

Back in Chapter 3 I discussed the SUPERASSERT dialog box and its Create Mini Dump button, which allows you to snap to disk the current state of the process so that you can load it up in the debugger later. Visual Studio .NET makes it easy to read any dump files you create. Opening a dump file in Visual Studio .NET is as simple as opening a regular solution. After starting Visual Studio .NET, select Open Solution from the File menu. In the Open Solution dialog box, navigate to the directory...

Memory Access Breakpoints

In conjunction with the excellent execution breakpoints, WinDBG also has the phenomenal ba Break On Access command, which allows you to stop when any piece of memory is read or written in your process. Visual Studio .NET offers only memory change breakpoints, and you have to use Mike Morearty's hardware breakpoint class to have access to all the power supplied by the Intel x86 hardware breakpoints. However, WinDBG has all the power built right in The format for Intel x86 user-mode memory...

The Super Saver AddIn

Now that you have some idea of the issues associated with add-ins, I thought it best to discuss some real-world add-ins because they offer the best way to learn. The first add-in I created was SuperSaver, which originally appeared in a Bugslayer column I wrote in MSDN Magazine. However, the add-in version in this book is completely and radically different from the original and is an excellent example of the trials and tribulations associated with writing add-ins. SuperSaver's job in life is to...

Release Set the Checksum

Where the debug switch tells the linker to generate debug code, the misnamed release switch does not do the opposite and tell the linker to do an optimized release build link. This switch should really be named CHECKSUM. All this switch does is set the checksum value in the Portable Executable PE header. While required by device drivers in order to load, it's not required by your user-mode applications. However, it's not a bad idea to set it for your release builds because WinDBG, discussed in...

Other Interesting Extension Commands

Before I move on to dump file handling, I want to mention several extension commands that you'll find useful in critical situations, like when you need to solve that one really challenging bug. These are the kind of commands that when you need them, you really need them. The first interesting one was imgreloc. It simply runs through all loaded modules and tells whether all modules are loaded at your preferred address. Now you have no excuse for not checking. The output looks like the following....

Synchronize at the Lowest Level

Since writing the first edition of this book, I have seen that this particular multithread rule is broken more than any other. You have to keep your synchronization methods at the lowest level possible in your code. This might sound like common sense, but the mistake I see made over and over is that developers are using fancy C wrapper classes that grab the synchronization object in the constructor and release it in the destructor. The following code shows an example class you'll find in...

Listing 131 Example SEH handler

Execute code to accomplish something. except EXCEPTION EXECUTE HANDLER This block will be executed if the code in the _try block causes an access violation or some other hard The code in here is also called the exception This block will be executed regardless of whether the causes a crash. Mandatory cleanup code goes here. The process of finding and executing an exception handler is sometimes called unwinding the exception. Exception handlers are kept on an internal stack as the function call...

Finding Uninitialized Memory Writes

Nothing is worse than a crash that occurs out of nowhere and that doesn't seem to match up with any known code path. If you're experiencing these symptoms, you're probably looking at an uninitialized write, which is also known as a wild write. The cause is a pointer variable that hasn't been initialized and happens to be pointing to valid memory. You'll mostly see this with stack-based pointers, in other words, local variables. Since the stack is being changed all the time as your program is...

Handling Toolbar Button Issues

After getting the Add-In wizard-generated code straightened out, probably the next problem you'll run into is getting custom toolbar bitmaps to show up correctly. It's not that hard it's just undocumented. It took me a while to get the magic incantations figured out, so hopefully this discussion will save you some time and frustration. For custom toolbar bit maps to be loaded, they have to be placed in a Win32 satellite DLL there's no way to use managed embedded bitmaps as toolbar bitmaps. When...

Common Debugging Question How can I break only when a specific thread calls a method

To set a per-thread breakpoint, you need a way to uniquely identify the thread. Fortunately for us, the Microsoft .NET Framework team was thinking ahead and provided the Name property on the System.Threading.Thread class to make identifying a thread a trivial task, so you can simply set a conditional expression breakpoint with something like ThreadlWantToStopOn Thread. CurrentThread. Name. Of course, this is assuming you always set the Name property inside your code whenever you start a thread....