1. Computing
Delphi memory manager problems in dynamic libraries
Page 2: The solution, or how to prevent an "Invalid pointer operation" exception.
 Win prizes by sharing code!
Do you have some Delphi code you want to share? Are you interested in winning a prize for your work?
Delphi Programming Quickies Contest
 More of this Feature
• Page 1: The problem
 Join the Discussion
"Post your views, comments, questions and doubts to this article."
Discuss!
 Related Resources
• DLLs and Delphi (intro)
• DLL related articles

• Windows API and Delphi

Now that you know what is and why Delphi needs the ShareMem unit, in a DLL project, if your DLL exports any procedures or functions that pass strings as parameters or function results, we can continue our discussion on "Invalid pointer operation" exceptions...

What are heaps?
For those unfamiliar with heaps and how they are used in Delphi, a heap is a region of memory in which dynamically allocated memory is stored.
In most structured languages like C/C++, Delphi and even Microsoft's C#, a programmer may use two kinds of memory: static and dynamic. Basic data types, also called value types, are static, and their memory requirements are known and fixed at compile-time. Delphi's integers, enumerated types, records and static arrays are examples of statically allocated variables. In C, all data types are static, and the programmer must explicitly allocate dynamic memory through some form of allocation routine, like malloc().
Dynamic memory, on the other hand, may have its size readjusted at run-time. Examples of this are Delphi's long strings, class types and dynamic arrays. In Visual Basic, many data types are dynamically allocated, among them variants and dynamic arrays. As a rule of thumb, any data type whose size may be changed at run-time is dynamically allocated.
From a compiler's point of view, these two kinds of memory are very distinct, and "live" in completely separate sections of an application's memory: Global statically-allocated variables live in a global static data area, local statically-allocated variables live on the stack, and dynamic memory blocks live on a heap. This separation of memory objects is actually ingrained very deeply into the fabric of modern programming, extending deep into the operating system and down into the hardware itself. This is why many chips (such as the Intel x86 family) have support for explicit data and stack segments.
PChar over String?!
The last line in the auto-generated comment (Delphi ... New ... DLL) deserves attention: pass string information using PChar or ShortString parameters. It seems to suggest that using PChar's or shortstrings would "solve" the problem. This is very misleading, and can lull developers into a false sense of security. But consider:

In a DLL:

 function GetPChar: PChar;
 begin
   Result := StrAlloc(13);
   StrCopy(Result,'Hello World!');
 end;
        
In an EXE:

 var p: PChar;
 ...
 p := GetPChar;
 // do something...
        
 // DLL heap possibly corrupted;
 // "Invalid pointer operation" possibly thrown
 
 StrDispose(p); 

Again, the errors. There is the perception about PChars, that since "the Windows API does it this way, it must be right". But the windows API very seldom allocates PChars for passing to applications. It requires the caller to allocate the PChar and pass a parameter specifying its length, and the API then writes to this buffer. In fact, there is very little advantage to using PChars in Delphi, since the reference-counted string type is much safer and more efficient. Only very advanced users who have a clear idea of their reasons for doing so, should use them.

Passing objects doesn't help either:

In a DLL:

 function GetStringList: TStringList;
 begin
   Result := TStringList.Create;
   Result.Add('foo');
 end;
 
In an EXE:

 var obj: TStringList;
 ...
 obj := GetStringList;
 // do something
 
 // may corrupt DLL heap; may free > 1 blocks
 obj.Free; 

Depending on what the EXE did to the object, it may cause corruption to both heaps. Note that, as of Delphi 6, simply having a module free memory from another module's heap does not seem to actually corrupt the heap per se. The heap manager keeps the free memory blocks in a linked-list, and during deletion, attempts to merge two adjacent free blocks. The "Invalid pointer operation" exception only occurs when a module attempts to free the last allocated block of another module's free list, and, failing to recognize an invalid element, attempts to merge the free block with (what seems to be) a marker element (or might simply be garbage), which causes the error. Though the error only occurs in this instance, the implementation of the heap manager is not guaranteed to stay this way, and in future versions the heap corruption could occur at any point.

The above PChar example could be "fixed" this way:

In DLL:

 function GetPChar: PChar;
 begin
   Result := StrAlloc(13);
   StrCopy(Result, 'Hello World!');
 end;

 procedure FreePChar(p: PChar);
 begin
   StrDispose(p);
 end;

In EXE:

 var p: PChar;
 ...
 p := GetPChar;
 // do something
 
 // heap-friendly free
 FreePChar(p); 

There is no equivalent "fix" for the TStringList version, since strings created within the EXE's heap may be freed by the TStringList's destructor, causing corruption in the EXE's heap.

So what is a proper solution? There are several options:

  • Use Sharemem.pas/Borlndmm.dll, and proceed normally: This seems to be the simplest solution, and requires no special precautions. There is a tradeoff, however: all allocation calls are diverted into the Borlndmm.dll module, and may be 2-8 times slower than "normal" allocation. It also requires you to distribute Borlndmm.dll with your application.

  • Never pass dynamic data across modules: This is a difficult proposition, and requires a deep understanding of the Delphi type system. It may be especially difficult to enforce when multiple developers are involved.

  • Treat all DLLs as if they were written in other languages and use API-style management: this is somewhat reasonable, in that if a project is written in more than one language anyway, then simply not using any Delphi-specific constructs across DLLs, may not add any more complexity. A disadvantage is that many Delphi constructs will not be available (objects, strings, dynamic arrays, etc).

  • Use COM objects within DLLs: this an extreme solution, especially if COM will be used only to provide cross-DLL comunication. It also imposes COM performance penalties and significantly more effort.

There is a fifth option. The FastSharemem unit is an attempt an alternative solution. It is simple to use, no more complex than including a single unit, and incurs no performance penalty.

And that's all!

First page > The problem > Page 1, 2

©2014 About.com. All rights reserved.