Custom Startup Logic and Load Time Deadlocks

DllMain in native as well as mixed-code DLLs must be implemented with special care. If you do not follow certain rules, a deadlock can occur while a DLL is being loaded. DllMain is called either directly or indirectly via the DLL's PE entry point. Before calling the PE entry point of a DLL, the operating system's image loader acquires a special critical section with the famous name loader lock. This loader lock can affect the PE entry point itself as well as all functions that are directly or indirectly called by the PE entry point. In native DLL scenarios, these are typically_DllMainCRTStartup, DllMain, and all functions called by DllMain. For mixed-code

DLLs, this is also _CorDllMain. The functions _CorDllMain and _DllMainCRTStartup have been implemented with the loader lock in mind. For DllMain, it is your responsibility to avoid deadlocks. The Win32 API documentation does not define precisely what is allowed and what isn't. The following are two restrictions described in the official documentation:

• "Because DLL notifications are serialized, entry-point functions should not attempt to communicate with other threads or processes. Deadlocks may occur as a result."

• "The entry point function should perform only simple initialization or termination tasks. It must not call the LoadLibrary or LoadLibraryEx function (or a function that calls these functions), because this may create dependency loops in the DLL load order."

The documentation mentions only a few operations explicitly that can safely be called within your code as follows:

• "The entry-point function can call functions in Kernel32.dll that do not load other DLLs. For example, DllMain can create synchronization objects such as critical sections and mutexes, and use TLS."

If you implement a mixed-code DLL, the two restrictions quoted previously imply another very important restriction: It is illegal to execute managed code in DllMain or a function called directly or indirectly by DllMain. Executing managed code can easily load other assemblies, which would cause a call to LoadLibraryEx. Furthermore, managed execution can silently create dependencies to other threads. This can occur especially during garbage collection. These dependencies can cause deadlocks. In the C++ Managed Extensions language from Visual Studio .NET 2002 and 2003, this potential deadlock was known as the "mixed DLL loading problem." Visual Studio 2005 and .NET 2.0 provide much more help to detect, prevent, and solve this problem.

To understand how this additional support works, let's have a look at some code that demonstrates how you should not implement DllMain:

// compile with "cl /clr /LD lib4.cpp" #include <windows.h>

BOOL APIENTRY DllMain(HINSTANCE, DWORD, LPVOID) {

System::Console::WriteLine("DllMain called");

extern "C" __declspec(dllexport) void f() {

Since Lib4.cpp is compiled with /clr, DllMain would be compiled to managed code. The compiler is smart enough to give you a warning here:

warning C4747: Calling managed '[email protected]': Managed code may not be run under loader lock, including the DLL entrypoint and calls reached from the DLL entrypoint

For now, let's ignore this warning and create a client for this application: // Lib4Client.cpp

// compile with "CL /MD Lib4Client.cpp"

#pragma comment (lib, "Lib4.lib") extern "C" __declspec(dllimport) void f();

When Lib4Client.exe is started from the command line, Lib4.dll is loaded and initialized. During this initialization, _CorDllMain calls_DllMainCRTStartup. When

_DllMainCRTStartup tries to call DllMain, the CRT implementation detects that managed code would be executed next. To prevent the execution of managed code, the dialog box shown in Figure 12-2 pops up, and the application is terminated.

Figure 12-2. Don't call managed code in DllMain.

In contrast to the deadlock, the display of this error message is reproducible. As soon as managed code is about to be executed either directly or indirectly in DllMain, a dialog box like this one appears and the application terminates with error level 255. Should you execute managed code in your DLL initialization or uninitialization code, it is very likely that you will detect this early in the development phase due to this dialog box. Therefore, this dialog can be seen as a helpful tool to avoid the mixed DLL loading problem.

If you run Lib4Client.exe in a debugger like WinDBG or the Visual Studio .NET debugger, you will likely see different behavior. Unless you have changed the default configuration, the execution of the debugger will stop as if it has entered a breakpoint, and the output window will show you the following text:

<mda:msg xmlns:mda="http://schemas.microsoft.com/QR/2004/10/mda"> <!--

DLL 'C:\Data\Books\AppliedCPPCLI\sources\chapter9\lib5.dll' is attempting managed execution inside OS Loader lock. Do not attempt to run managed code inside a DllMain or image initialization function since doing so can cause the application to hang.

<mda:loaderLockMsg break="true"/> </mda:msg>

The breakpoint and the output are caused by a managed debugging assistant (MDA). MDAs are assertion-like constructs in the CLR and the base class library. There are various ways to turn MDAs on or off. Figure 12-3 shows how you can configure MDAs in Visual Studio 2005 via the menu item Debug > Exceptions.

Imageloader
Figure 12-3. Configuring the loader lock MDA in Visual Studio .NET

To provide further protection, the Visual C++ project wizards for DLLs generate the following code:

#ifdef _MANAGED

#pragma managed(push, off)

#endif

BOOL APIENTRY DllMain( HMODULE hModule,

DWORD ul_reason_for_call, LPVOID lpReserved

#ifdef _MANAGED #pragma managed(pop) #endif

When the file containing this code is compiled with /clr, the compilation model is temporarily switched to native compilation to compile DllMain. After that, the current compilation model is restored. Switching the compilation model inside a file via #pragma [un]managed is the source of some evil pitfalls (which will be discussed shortly). To reduce the problems, I prefer the following alternative:

#if _cplusplus_cli

#error DllMain must be compiled to native code #endif

BOOL APIENTRY DllMain( HMODULE hModule,

DWORD ul_reason_for_call, LPVOID lpReserved

The code you see here is a step in the right direction, but it is not the solution to all loader lock problems. You can still call a managed function from DllMain. In addition to that, there is another potential for executing managed code, which is discussed in the next section.

0 0

Post a comment