Archive for Delphi

Inducing The Great Divide

One of the most important principles in building complex software systems, is detaching  the business logic from the screens that allow users to view and edit information.

You’ll notice I said “most important” and not “most popular.” Sad but true – all modern IDEs make it so easy for a programmer to simply whack together a bunch of user interface controls, that there are probably far more forms with embedded business logic than without.

The fact is that for simple systems, this can often be convenient and harmless. But for more complex ones, it nearly always spells disaster (or at least chronic pain). The gold standard for these kinds of systems is to have a set of business objects and a set of screens that merely displays and edits them.

Building a proper business object framework is a complex topic all of its own, but Delphi (and all the other modern languages) give us more than enough riches in the object model to implement what we need in this regard.

What environments like Delphi don’t do very well, is give developers the tools to build a form that can capably display and edit business objects without knowing too much about them. And very generic entry forms only work well for very generic data. Invariably, specialised data require specialised forms that give the user appropriate functionality and a sensible layout.

So you end up with lots and lots of code like this:

procedure TCustomerForm.LoadForm(Customer: TCustomer); 
begin 
  NameEdit.Text := Customer.Name; 
  EmailEdit.Text := Customer.Email; 
end; 

procedure TCustomerForm.PostForm(Customer: TCustomer); 
begin 
  Customer.Name := NameEdit.Text; 
  Customer.Email := EmailEdit.Text; 
end; 

And this kind of boiler-plate code is yet another reason while lots of programmers throw structure to the hounds and just integrate the business logic into the form.

Well, no need.

Delphi’s RTTI framework has always allowed one to access and manipulate all the published properties and fields on an object and I have been using a framework based on that for over ten years now. Simply publish all the properties you want to represent on the form and you can match object properties to screen controls by name and type.

Delphi 2010 has of course overhauled the RTTI framework completely and I quickly adapted my code to use that instead. Truth be told, the new RTTI is a little finicky and brittle, but it is also far richer in functionality and I found it well worth the effort to overhaul my classes.

What I am about to present here is not based in any way on the framework in our system, but illustrates very nicely how that works. Having said that, what I will show here is also simplified to the point that you will definitely want to extend it a little before you go gold with it.

Exploring The Cliff Faces

There is no real limit to the data types that you can have on your business object, but VCL controls almost all use simpler types to represent their values. So the first thing you’ll need is some sort of mapping of how you will want to represent information.

Here is a quick table of reasonable mappings:

Business Object Property Possbile VCL Controls
string, TObject TEdit, TComboBox, TLabel
Boolean TCheckBox
Integer TEdit, TUpDown, TTrackBar
Double TEdit
TDateTime TDateTimePicker
Enumerated type TComboBox, TListBox
Set type TCheckListBox

I’m sure you can think of a few more and probably disagree with me on a few, but you get to pick your own.

The plan is to have a completely autonomous business object and a very simple screen that can display and modify the business object without knowing much about it.

Here is the business object I am going to use:

TPerson = class
private
  FName: string;
  FAge: Integer;
  FOccupation: string;
public
  property Name: string read FName write FName;
  property Age: Integer read FAge write FAge;
  property Occupation: string read FOccupation write FOccupation;
end;

And my form is as simple as they get too:

image

The load button predictably loads the content of a TPerson instance into the form, the Save button does the reverse and Clear Form… well OK.

Here is the code for the Load and Save buttons:

procedure TCustomerForm.LoadButtonClick(Sender: TObject);
begin
  Binding.Load;
end;

procedure TCustomerForm.SaveButtonClick(Sender: TObject);
begin
  Binding.Save;
end;

That is about as simple as it gets, right? An obvious enhancement is to make the form validate the values before assigning them to the form, but let’s not get ahead of ourselves.

The Binding variable above is of the TObjectBinding class which links together any TWinControl and any object. Before I show that – here are a few things you probably know, but let’s cover them anyway:

  1. All controls placed on a form at design time get their own published fields. This lets the streaming system know what should go into the form file and what shouldn’t.
  2. Not all public properties on your business object necessarily need to be displayed on the form. And none of those properties that you want to display will be published unless you made them so.
  3. Properties have names. So do screen controls. They may even match, but lots of programmers prefer to have their controls named something like AgeEdit or EdtAge instead of just Age.
  4. Not all properties are writeable. In fact, you could (probably shouldn’t) even create properties that can be written to but not read.

Clearly then, we can find the relevant controls by using Delphi’s new RTTI framework to iterate the published fields that are controls. Finding the relevant business object fields can be as simple as grabbing all the public and published properties or it can involve checking attributes. Or only properties with an even-length name. My class has a method to pick out valid fields and another to pick out valid properties. Modify these to your heart’s content.

And in production code you may want to pick up some of the form’s properties as well. Works well when you need to override some of the basic functionality.

Building The Narrow Bridge

Here is the class declaration – I’ll discuss the methods as we go along:

TObjectBinding = class
private
  // Quick way to get from the object
  // field to its screen control.
  PropFieldMapping: TDictionary<TRttiProperty, TRttiField>;
  // Needed for RTTI.
  Context: TRttiContext;
  ControlType: TRttiType;
  ObjType: TRttiType;
  // The control (normally form) and
  // the object it represents.
  Control: TWinControl;
  Obj: TObject;
  // Finds the object properties that have corresponding
  // fields and stores them in the dictionary.
  procedure CreateMappings;
  function FindField(Prop: TRttiProperty; out Field: TRttiField): Boolean;
  function FieldClass(Field: TRttiField): TClass;
  // Modify these to change the rules about what
  // should be matched.
  function IsValidField(Field: TRttiField): Boolean;
  function IsValidProp(Prop: TRttiProperty): Boolean;
  // Modify these to change the mappings of property type
  // to VCL control class.
  procedure AssignField(Prop: TRttiProperty; Field: TRttiField);
  procedure AssignProp(Prop: TRttiProperty; Field: TRttiField);
  // Used from AssignField/AssignProp. Extend these to
  // support a wider range of properties.
  function GetPropText(Prop: TRttiProperty): string;
  procedure SetPropText(Prop: TRttiProperty; const Text: string);
public
  procedure Load;
  procedure Save;

  constructor Create(Control: TWinControl; Obj: TObject);
  destructor Destroy; override;
end;

Load and Save can easily be supplemented here with Validate, which will use attributes specified on the business object properties to decide whether the current screen values are valid. If not, we don’t call Save at all.

The constructor and destructor do what you expect. They create and destroy the objects held in those private fields. The constructor also calls CreateMappings, which is where a lot of the fun stuff happens.

constructor TObjectBinding.Create(Control: TWinControl; Obj: TObject);
begin
  inherited Create;

  Self.Control := Control;
  Self.Obj := Obj;
  Context := TRttiContext.Create;
  ControlType := Context.GetType(Control.ClassInfo);
  ObjType := Context.GetType(Obj.ClassInfo);
  PropFieldMapping := TDictionary<TRttiProperty, TRttiField>.Create;
  CreateMappings;
end;

destructor TObjectBinding.Destroy;
begin
  PropFieldMapping.Free;
  ObjType.Free;
  ControlType.Free;
  Context.Free;

  inherited;
end;

procedure TObjectBinding.CreateMappings;
var
  Props: TArray<TRttiProperty>;
  Prop: TRttiProperty;
  Field: TRttiField;
begin
  Props := ObjType.GetProperties;
  for Prop in Props do
    if IsValidProp(Prop) and FindField(Prop, Field) then
      PropFieldMapping.Add(Prop, Field);
end;

CreateMappings scans through all the properties on the business object and attempts to find fields on the form to match them to. If a match is found, they are stored in the PropFieldMapping dictionary for future reference.

Notice the IsValidProp function. That is where you should impose any additional rules about whether or not a property should be taken into account. The default implementation simply grabs all the public and published properties.

function TObjectBinding.IsValidProp(Prop: TRttiProperty): Boolean;
begin
  Result := Prop.Visibility >= mvPublic;
end;

Also, there is a FindField function. This little guy does a bit more than just grab all the published fields. Again, there is an IsValidField function for you to toy with. But it first attempts to find a control with a name that matches exactly with that of an approved property. If not, it also checks for a few common variants of the name (AgeEdit and EdtAge as mentioned before). This list will obviously need to grow if you support many more controls.

function TObjectBinding.FieldClass(Field: TRttiField): TClass;
begin
  Result := GetTypeData(Field.FieldType.Handle).ClassType;
end;

function TObjectBinding.IsValidField(Field: TRttiField): Boolean;
begin
  Result := (Field <> nil) and (Field.Visibility = mvPublished) and
    (Field.FieldType.TypeKind = tkClass) and
    (FieldClass(Field).InheritsFrom(TControl));
end;

function TObjectBinding.FindField(Prop: TRttiProperty; out Field: TRttiField): Boolean;
const
  Embelishments: array [0..5] of string =
    ('Edt', 'Edit', 'Combo', 'ComboBox', 'Lookup', 'Lkp');
var
  Emb: string;
begin
  Field := ControlType.GetField(Prop.Name);
  if IsValidField(Field) then
    Exit(True)
  else for Emb in Embelishments do
  begin
    Field := ControlType.GetField(Prop.Name + Emb);
    if not IsValidField(Field) then
      Field := ControlType.GetField(Emb + Prop.Name);
    if IsValidField(Field) then
      Exit(True);
  end;
  Result := False;
end;

Now that we have a dictionary of business object properties and the screen fields that we want to link them to, we can take a look at the very similar-looking Load and Save methods.

procedure TObjectBinding.Load;
var
  Prop: TRttiProperty;
begin
  for Prop in PropFieldMapping.Keys do
    AssignField(Prop, PropFieldMapping[Prop]);
end;

procedure TObjectBinding.Save;
var
  Prop: TRttiProperty;
begin
  for Prop in PropFieldMapping.Keys do
    AssignProp(Prop, PropFieldMapping[Prop]);
end;

AssignProp and AssignField are probably the two most difficult functions to write in this entire class. The problem is that Delphi’s TValue structure used in the new RTTI framework makes no attempt whatsoever to convert data. Instead, you need to do any conversions yourself and assign to the exact type you need. That isn’t too much of a problem, but it means lots of individually-crafted conversion blocks.

To keep my example manageable, I opted to convert everything to strings and back using GetPropText and SetPropText. And I only support string, Integer, Double and TDateTime. I didn’t use Double and TDateTime on my TPerson business object, but they do require a special trick that I wanted to show.

procedure TObjectBinding.AssignField(Prop: TRttiProperty; Field: TRttiField);
var
  NestedControl: TControl;
  PropText: string;
begin
  NestedControl := Field.GetValue(Control).AsObject as TControl;

  PropText := GetPropText(Prop);
  if NestedControl is TCustomEdit then
    TCustomEdit(NestedControl).Text := PropText
  else if NestedControl is TCustomComboBox then
    TComboBox(NestedControl).Text := PropText;
end;

procedure TObjectBinding.AssignProp(Prop: TRttiProperty; Field: TRttiField);
var
  NestedControl: TControl;
  FieldText: string;
begin
  NestedControl := Field.GetValue(Control).AsObject as TControl;

  if NestedControl is TCustomEdit then
    FieldText := TCustomEdit(NestedControl).Text
  else if NestedControl is TCustomComboBox then
    FieldText := TComboBox(NestedControl).Text
  else
    FieldText := '';

  SetPropText(Prop, FieldText);
end;

procedure TObjectBinding.SetPropText(Prop: TRttiProperty; const Text: string);
var
  V: TValue;
begin
  case Prop.PropertyType.TypeKind of
    tkInteger: V := StrToIntDef(Text, 0);
    tkFloat:
      if Prop.PropertyType.Handle = TypeInfo(TDateTime) then
        V := StrToDateDef(Text, 0)
      else
        V := StrToFloatDef(Text, 0);
    tkUString: V := Text;
    // And other types handled in similar ways
  else
    Exit; // Or some reasonable default action
  end;

  Prop.SetValue(Obj, V);
end;

function TObjectBinding.GetPropText(Prop: TRttiProperty): string;
var
  V: TValue;
begin
  V := Prop.GetValue(Obj);
  case Prop.PropertyType.TypeKind of
    tkInteger: Result := IntToStr(V.AsInteger);
    tkFloat:
      if Prop.PropertyType.Handle = TypeInfo(TDateTime) then
        Result := DateToStr(V.AsType<TDateTime>)
      else
        Result := FloatToStr(V.AsType<Double>);
    tkUString: Result := V.AsString;
    // And other types handled in similar ways
  else
    Result := ''; // Or some reasonable default
  end;
end;

Firstly, the AssignProp and AssignField functions each has a check for every supported VCL control base class. So to support TCheckBox or TCheckListBox this is where you would add the additional support code.

Then take a look at GetPropText and SetPropText. They mirror one another of course, so we can learn what we need by inspecting GetPropText.

First thing to notice is that string properties use the tkUString type and not tkString as in pre-Unicode versions of Delphi. Also, we can simply read TValue.AsInteger when reading an integer value.

The trouble arises when we look at Double and TDateTime. TDateTime is actually a type alias for Double and they both show up as tkFloat. This is why I compare the type info from my property with the type info for TDateTime. The alternative would be to check the type name, but that thought makes me queasy. The same kind of thing will be needed to tell Boolean from other enumerated types.

Also, both Double and TDateTime are incompatible with TValue.AsExtended, which is of course a floating point type of a different size. So taking care of all these type conversions become really cumbersome, but of course you only need to write the code once.

And that’s it

Hooking all of this together, you can now have business objects that are completely detached from their editing screens. You’ll find that the screens become really trivial to develop – most cases involve little more than slapping on the controls, naming them and calling Load and Save. And that last bit can be done is a base class.

You also get to take advantage of all the other advantages that come with a proper split. You will find it easier to modify your business rules, easily use the same business logic in other contexts (say a bulk import versus the capture screen) and also be able to validate your business rules using automated tests. Good idea!

I can now show you my super-complex test app in action. Note that the first screen shot shows the names of both edit boxes and the combo box. Despite the different naming conventions, they all link up automatically to the correct properties.

1. Starting up.

image

2. Click Load Form.

image

3. Mess around with the values and click Save to Object.

image

4. Click Clear Form.

image

5. And finally, load again.

image

Advertisements

Comments (22)

Getting Data From Anywhere, part 2

My last post underwhelmed everyone a little and I understand why. After all, using events or anonymous methods for this kind of thing is standard fare and it doesn’t even do the job that I set out to do in the first place.

It works, in that I am able to take data from any enumerator, but it is unintuitive to use and requires the use of boilerplate code.

Well, you’d be happy to know that I was also unhappy with these solutions (hence the previous post was called “Part 1”) and I desperately wanted to get it to work with RTTI instead. Because if I could just grab pointers to the enumerator’s MoveNext method and Current property you could completely obviate the need for code being passed from the caller.

Unfortunately, Delphi’s RTTI had a deal-breaking limitation, namely RTTI was only generated for published members. None of the GetEnumerator methods I could find was published and none of those enumerators had published MoveNext methods and Current properties.

Just as I was ready to resign myself to using the techniques described in part 1, Delphi 2010 became available. Hurray! I immediately set to work to use the new RTTI system to provide the alternative that I wanted and found it dead-easy.

So, without further delay, here is the new and improved way of getting data from anywhere.

Technique 3: Delphi 2010 RTTI

We needed to do two things, remember? First, find the public MoveNext method which takes no parameters and returns a Boolean. Second, find the public Current property which returns a type we can figure out how to use.

I’ve decided not to give an exhaustive description of how the new RTTI system works, because Robert Love has done such a fabulous job demystifying it in the last week or so. Go check out some of his posts to delve deeper into the inner workings.

Here is my complete method:

   1  class procedure TStringsFiller.Fill(Strings: TStrings;
   2    Enumerator: TObject);
   3  var
   4    Context: TRttiContext;
   5    EnumType: TRttiType;
   6    Current: TRttiProperty;
   7    MoveNext: TRttiMethod;
   8    Value: TValue;
   9  begin
  10    Context := TRttiContext.Create;
  11    try
  12      EnumType := Context.GetType(Enumerator.ClassType);
  13  
  14      // Find the Current property
  15      Current := EnumType.GetProperty('Current');
  16      if (Current = nil) or
  17        not (Current.PropertyType.TypeKind in
  18          [tkString, tkUString, tkClass]) then
  19        raise Exception.Create('Invalid Current property');
  20  
  21      // Find the MoveNext property
  22      MoveNext := EnumType.GetMethod('MoveNext');
  23      if (MoveNext = nil) or (Length(MoveNext.GetParameters) > 0) or
  24        (MoveNext.MethodKind <> mkFunction) or
  25        (MoveNext.ReturnType.Handle <> TypeInfo(Boolean)) then
  26        raise Exception.Create('Invalid MoveNext method');
  27  
  28      // while MoveNext do
  29      while MoveNext.Invoke(Enumerator, []).AsBoolean do
  30      begin
  31        // Value := Current
  32        Value := Current.GetValue(Enumerator);
  33        case Value.Kind of
  34          tkClass: Strings.Add(Value.AsObject.ToString);
  35          tkUString, tkString: Strings.Add(Value.AsString);
  36          tkClassRef: Strings.Add(Value.AsClass.ClassName);
  37          // Any other types you want to support go here
  38        end;
  39      end;
  40    finally
  41      Context.Free;
  42    end;
  43  end;

We have a fair bit more code than previously, so let’s do a quick walkthrough. Lines 10 and 12 are the standard statements you need to get your hands on the TRttiType object that contains all the good stuff.

Lines 15 through 19 gets the Current property and checks that it is of one of the types we’re going to use in this example. If not, we raise an exception. Here we could also add code to check that the property is not write-only and that it is not indexed.

Lines 22 through 26 gets the MoveNext method. It has to be a zero-parameter function that returns Boolean or we’re simply not interested.

There is an interesting scenario that I have not catered for here – what if the object has multiple overloaded MoveNext methods? My tests show that this code will then only pick up the one that was declared first.

To correctly handle that scenario, we really should call EnumType.GetMethods and iterate the returned array to find the method with the matching signature.

Finally, lines 29 through 39 use the returned MoveNext and Current to construct the loop we need to fill the list. Notice that we’re getting a TValue back from Current.GetValue(Enumerator) and that we need to check individually for all the types we know how to use – strings, objects and class references in this case.

Having much more code in the Fill method does have its payoff though: Calling the function is now far nicer and requires absolutely no boilerplate:

TStringsFiller.Fill(ListBox.Items, Beatles.GetEnumerator);
TStringsFiller.Fill(ListBox.Items, Memo.Lines.GetEnumerator);

Both calls in the above code looks the same, even though the first supplies an object list and the second supplies a TStrings.

And we could clean it up even more by using the RTTI to check for a public GetEnumerator method that returns a class with Current and MoveNext. That way, the call to GetEnumerator in the two lines above could go away as well.

When the Delphi 2010 beta bloggers first started showing off the enhanced RTTI functionality, responses seemed to be split between “Great!” and “But what is it good for?” This post shows one of the simpler scenarios where proper RTTI use can significantly simplify your code.

I have another post coming up that shows a different use of RTTI that can literally save you thousands of lines of code and many, many hours of hard labour.

As always, watch this space.

Update: The promised follow-up is now available. Unfortunately, deadlines, exams and two separate bouts of flu conspired to make it over a month late, but I think you’ll like it.

Leave a Comment

Getting Data From Anywhere, part 1

Delphi 2006 introduced enumerators – a way to iterate any kind of collection with the for-in loop. And the VCL is chock-full of enumerators: TStrings has one, so do TComponent, TWinControl, TList, TObjectList and the generic TList<T> and TObjectList<T>.

So I recently set about writing a grid control (the main reason for the long delay since my last post). It’s not the first time that I did this and my previous attempt is a nice-looking and quite capable DB-aware grid that replaces TDBGrid.

This time round, I thought it would be cool to take any old data source, not just TDataSource to feed my grid. I wanted to be able to take both TDataSource objects and also any object that will work with the for-in construct.

This has many complications but I thought I’d show three strategies that I came up with to iterate any enumerator and which one I think is better.

If you are not familiar with how enumerators work, here is a quick description from the official documentation:

“To use the forin loop construct on a class or interface, the class or interface must implement a prescribed collection pattern. A type that implements the collection pattern must have the following attributes:

  • The class or interface must contain a public instance method called GetEnumerator(). The GetEnumerator() method must return a class, interface, or record type.
  • The class, interface, or record returned by GetEnumerator() must contain a public instance method called MoveNext(). The MoveNext() method must return a Boolean.
  • The class, interface, or record returned by GetEnumerator() must contain a public instance, read-only property called Current. The type of the Current property must be the type contained in the collection.”

In my simple example, I am going to fill in a TStrings (a TListBox’s Items property to be exact) with text which I get from my enumerator. To simplify matters a little, I’m going to use Delphi’s new ToString property. And I am going to skimp a little on error checking.

The list that I’d like to iterate is this:

type
  TBeatle = class
  private
    FName: string;
  public
    function ToString: string; override;
    constructor Create(const Name: string);
  end;

  TBeatles = class(TObjectList<TBeatle>);

with this trivial implementation:

{ TBeatle }

constructor TBeatle.Create(const Name: string);
begin
  FName := Name;
end;

function TBeatle.ToString: string;
begin
  Result := FName;
end;

and the list is populated with the obvious choices:

Beatles := TBeatles.Create(True);

Beatles.Add(TBeatle.Create('John'));
Beatles.Add(TBeatle.Create('Paul'));
Beatles.Add(TBeatle.Create('George'));
Beatles.Add(TBeatle.Create('Ringo'));

Technique 1: Events

Let us first consider a Delphi 2006-compatible way to step through the list and add its content to a TStrings.

The first thing you’ll need is a pair of event types – one to read the Current property and one to advance the list with MoveNext:

TCallGetCurrentEvent = procedure (Enumerator: TObject; 
  var Value: string) of object;
TCallMoveNextEvent = procedure (Enumerator: TObject; 
  var Result: Boolean) of object;

Actually, both of these should probably be function of object, but I always end up with constructs where the compiler can’t quite determine whether I’m calling the function or taking its address.

Also, here is the declaration of my class method that populates a TStrings with data from an arbitrary enumerator. And yes, I am aware that TStringsFiller is a really, really stupid name.

TStringsFiller = class
public
  class procedure Fill(Strings: TStrings; 
    Enumerator: TObject;
    GetCurrent: TCallGetCurrentEvent;
    MoveNext: TCallMoveNextEvent);
end;

This method is implemented in exactly the way you’d expect:

class procedure TStringsFiller.Fill(Strings: TStrings;
  Enumerator: TObject;
  GetCurrent: TCallGetCurrentEvent;
  MoveNext: TCallMoveNextEvent);
var
  MoveResult: Boolean;
  Current: string;
begin
  MoveNext(Enumerator, MoveResult);
  while MoveResult do
  begin
    GetCurrent(Enumerator, Current);
    Strings.Add(Current);
    MoveNext(Enumerator, MoveResult);
  end;
end;

Usage is pretty simple too:

procedure TForm1.HandleGetCurrent(Enumerator: TObject; 
  var Value: string);
begin
  Value := (Enumerator as TList<TBeatle>.TEnumerator)
    .Current.ToString;
end;

procedure TForm1.HandleMoveNext(Enumerator: TObject; 
  var Result: Boolean);
begin
  Result := (Enumerator as TList<TBeatle>.TEnumerator)
    .MoveNext;
end;

procedure TForm1.OldStyleButtonClick(Sender: TObject);
begin
  TStringsFiller.Fill(ListBox.Items, Beatles.GetEnumerator,
    HandleGetCurrent, HandleMoveNext);
end;

Technique 2: Anonymous Methods

The technique above is quite simple and is very familiar to almost any Delphi programmer. It could be somewhat cumbersome to use though. Needing to add those event handlers and passing them through is the kind of boilerplate code that we all dislike.

So next up is a variation on the above technique, but one that requires Delphi 2009. Instead of the two event types, we have anonymous method types:

TCallGetCurrent = TFunc<TObject, string>;
TCallMoveNext = TFunc<TObject, Boolean>;

For some reason, I don’t get the same compiler confusion when using anonymous functions as I do when using events, so the declarations and the Fill method all become a little simpler:

class procedure TStringsFiller.Fill(Strings: TStrings;
  Enumerator: TObject;
  GetCurrent: TCallGetCurrent; MoveNext: TCallMoveNext);
begin
  while MoveNext(Enumerator) do
    Strings.Add(GetCurrent(Enumerator));
end;

Also, using it is – I think – a bit more intuitive and natural. I guess it is still a little boilerplate, but it just feels a tad less cumbersome to me.

TStringsFiller.Fill(ListBox.Items, Beatles.GetEnumerator,
  function (Enumerator: TObject): string
  begin
    Result := (Enumerator as TList<TBeatle>.TEnumerator)
      .Current.ToString;
  end,
  function (Enumerator: TObject): Boolean
  begin
    Result := (Enumerator as TList<TBeatle>.TEnumerator)
      .MoveNext;
  end);

I could really live with either of these two solutions, but what I wanted was a way to just assign any old list to my Source property and let the property setter code figure it out.

If you’re already guessing what solution I went for, good for you. If not, look out for part 2 in a few days’ time. And here’s a clue:

Perfect timing.

Comments (3)

Better than you’ve been told…

Strangely, the Delphi 2010 hype machine have slowed down somewhat as we got closer to release date. But I just had my first full day with the new IDE and it’s good.

The installer, I have to say, deflated me somewhat. It was slow, and it kept prompting me for the various bits that needed to be installed – DBPack, Boost and Rave. Unfortunate that they managed to regress in that regard, because the new black look of the installer looks really sharp. Quite unlike 2007/2009’s dove-gray slug.

But i wasn’t let down by the actual product.

All the much-hyped bits are there, and they all seem to work. As noted elsewhere, IDE Insight can do with some synonyms and other little tweaks. But it pops up in under a second and it found everything I went looking for. In time, I’m sure it will become even more handy.

I really wish I had some place to use the new touch features, but that is simply not what my day-job entails. I decided to give it a whack in any case. Literally under a minute’s worth of work to get it going.

I even tried designing a gesture of my own. Not only was it dead simple and intuitive, but the gesture editor even gave me let me know when my gesture was too close to another one. Real slick. Useless to my day-to-day work, but really slick.

What also pleased me are the little gems I found as I went along – Microsoft’s one stated aim with Windows 7 was to delight “early and often” and Delphi 2010 certainly emulated that second bit. Here are the little bits I stumbled upon that I didn’t see mentioned often or at all:

  1. Tab indents a block of code, Shift-Tab unindents. How obvious is that? I use lots of editors (Notepad++ and Visual Studio 2008 both very often) and it’s nice to see Delphi adopt something that’s become a convention.
  2. DBGrid is themed! Fine, they are a little late on this one but a simple recompile of your app can give it an instant UI update.
  3. Class constructors. I know Alan mentioned them after Delphi Live but I didn’t see any mention about them in the recent flurry of information. These are really cool, and I can think of a few places that will receive immediate attention.
  4. Various different views and new icons in Project Manager. Much more intuitive now, and they have even put in some effort to reduce that ridiculously long context menu. The one in the editor though, still sucks.
  5. Find in files now support file groups. So Delphi source folders go into one group, my project source go into another and all the Delphi demos and my own test/play projects go into another. Or something like that. This looks real handy.
  6. New icons for the IDE and applications. This sounds silly, but I presently have icons for Delphi 2006, 2007, 2009 and 2010 on my quick launch bar. Guess which is the only one to look distinctive. And I think the new icon is very cool.
  7. Windows 7 ITaskbarList3. You know, for all the cool jump lists, progress bars and thumbnail toolbars.

The real winner for me though, is the new RTTI library. I know all the rave reviews have been about attribute support – and I’m sure that rocks – but I was already blown away before I got to it. Just querying objects for their characteristics has now become so rich that it is bound to change the way many things are done.

I have a particular example in mind that I’ll show in a few days.

Comments (6)

Delphi’s broken bookmarks

Delphi 2009’s blue chip features were Unicode support, generics and anonymous methods. Most developers expected Unicode support to be the one to introduce the bulk of the migration headaches. But as with most Delphi versions, Delphi 2009 also saw a number of changes that didn’t quite make it into the marketing material.

Perhaps I’ll get into some of the beneficial ones in a later post – possibly just in time to follow it up with a Delphi 2010 version. But for now I’d like to look at one of the more perplexing ones.

Delphi’s TDataSet class has for the longest times supported bookmarks. If you still waste your time with TDBGrid, you’ll use these whenever you allow the user to multi-select.

Myself, I use these extensively with TVirtualStringTree – one free control worth a serious look if you don’t use it already – to display TDataSet records as rows in a grid.

So on Delphi 2007 and before, you used bookmarks like so:

Bookmark := CDS.GetBookmark;

and a little later:

CDS.GotoBookmark(Bookmark);

where CDS is a TDataSet – in my case usually a TClientDataSet. And Bookmark is a Pointer.

On Delphi 2009 this doesn’t quite work as expected. You see, in Delphi 2007 GetBookmark returned a Pointer and you would declare Bookmark as a Pointer.

In Delphi 2009, GetBookmark returns a TBookmark, which is an alias for TBytes. And TBytes is a dynamic array of Byte. If it hasn’t sunk in yet, read it again.

That’s right. In Delphi 2007, GetBookmark returned a regular, unmanaged pointer. In Delphi 2009, it returns a managed dynamic array pointer.

So what? Well, in the past you would get the bookmark and do with it as you please until you were done with it. And you could still do the same, providing that you declare my Bookmark variable from above as TBookmark.

If you don’t, the bookmark gets freed at the end of the method that you called GetBookmark from. Which makes it impossible to return to the bookmarked record from another record.

In case you don’t know Virtual Treeview, here’s the rundown: You add nodes and attach data to those nodes. Instead of storing strings, the tree would store only the data you give it. And the text displayed in the grid gets picked up in event handlers and not stored anywhere.

So if you simply used to store the pointer to the bookmark (like me) you’d suddenly keep getting annoying “Record not found” exceptions.

I can think of two possible solutions. The first is obvious – declare your bookmark as TBookmark. But that doesn’t work in my Virtual Treeview scenario and there are possibly many other people who stored bookmarks as untyped pointers in TList or something similar. In that case, you should simply cast when you do the assignment:

TBookmark(Bookmark) := CDS.GetBookmark;

or in Virtual Treeview:

Bookmark := nil;// Required so your assignment doesn't attempt to free a non-existent array
TBookmark(Bookmark) := CDS.GetBookmark;
Tree.AddChild(<strong>nil</strong>, Bookmark);

It’s a simple fix, but one that you will need to make at every place where you store a bookmark. Either that, or changing the bookmark declaration.

I can’t even begin to guess what possessed our friends at Embarcadero to do this. In fact, I don’t quite see what they intended to fix by making the change. If the idea was to replace the untyped pointer, then PByte – an unmanaged type – would be a far better alternative to TBytes.

But it’s an undocumented caveat. Or it was – until now.

Comments (8)

Hellishly Horrible Hack #1: Changing an object’s functionality without recompiling

Yup, you read that right. This one comes with a big disclaimer. It’s been tested and works but don’t read on if you are squeamish.

The set-up is like this:

We have an application that is deployed via XCopy and a second team in another city who is watching this with a hawk-eye. They have a stable system and don’t want changes that they are not absolutely certain they want. Lots to complain about in this kind of scenario, but that’s what we have to work with.

Now a requirement landed on my desk to change a piece of functionality without sending them a new version of the package (which is used everywhere) that it resides in. Fortunately, this unit (call it One) is only called from one other unit (call it Two).

Unit Two in turn is being used from the main executable – a very tiny little thing that basically loads all the runtime packages that do the actual work. So our hawkish friends are happy to take a change to the main executable, but not to this core package.

My first thought was to add the new functionality to a copy of One, which can be called directly from the executable and be folded back into the core package for the next major version of the application.

This seemed fine, until I realised that the object in unit Two is globally accessible and referenced from all over. Lots and lots of callers that cannot be recompiled. Yikes.

After the laughter gave way to despair, I finally decided that this calls for a hack. You know, the kind of thing you always sagely warn others against. After all, exactly how else do you change the functionality of Two without recompiling it?

Let me rephrase that. How do I call my new code in One, while leaving the internal state of Two identical to how it would have been if I’d used it to call One?

So what is an object anyway?

In Delphi, as you certainly know, object variables are really pointers to the actual objects, so MyList: TList creates a new pointer variable to which you can assign the memory address of a TList object. Some programmers prefer the word reference simply because Delphi handles the dereferencing of the pointer on your behalf, but a pointer is a pointer no matter what you call it.

So what does the object pointer point to exactly?

Well, it points to a record structure. Seriously. The first field of this record structure is a pointer to your virtual method table. Incidentally, if you declare a variable of a class reference type (TClass, TPersistClass…) that is also actually a pointer to the VMT. And if you call ClassType on any object variable, you also get a pointer to the VMT (typecast to a TClass – a pointer is a pointer, right?)

The second field in your record structure is the first field of your class. So, the hypothetical

TMyObject = class
private
  ID: Integer;
  Age: Integer;
  Income: Double;
end;

is identical to

TMyObject = ^TMyRec;
TMyRec = record
  VMTPtr: TClass;
  ID: Integer;
  Age: Integer;
  Income: Double;
end;

Before the flame war starts, let me point out that these two declarations are different in semantics, purpose and the warm fuzzy feelings they give to programmers, but they are identical in memory.

I can get more into this in a future post, but the thing to note here is that our object variable is in the end just another block of memory with a predictable layout. So if I needed to change the internal state of my object, I could do it without paying attention to scope identifiers, methods, implemented interfaces, or the weather.

So I could create a second class with my new functionality as long as the in-memory position of the fields I want to manipulate are the same.

TMyNewObject = class
private
  FID: Integer;
  FAge: Integer;
  FIncome: Double;
  FMonthlyWage: Boolean;
public
  procedure CalculateIncome(ID: Integer; IsMonthly: Integer);
end;

Notice that I have added a field to my new class as well as a method that can implement my new functionality. I’ve kept the other fields the same but could have changed them in any way that would leave the positions the same for those I use in TMyNewObject.

So now my executable can create an instance of TMyNewObject, call CalculateIncome and… then what? Well, remember the whole thing about the TMyObject instance being globally accessible? I need to find a way to assign the values stored in TMyNewObject back to TMyObject without being hindered by the private scope of those fields. But an object is just another block of memory, right?

To copy memory from the one to the other, I need three things:

  1. Two object instances, namely Obj: TMyObject and NewObj: TMyNewObject.
  2. The in-memory size of the data used by the TMyObject instance.
  3. A function to copy from NewObj^ to Obj^.

Requirement 1 is easily met, requirement 2 is met by calling Obj.InstanceSize and requirement 3 is met by using the following:

Move(Pointer(NewObj)^, Pointer(Obj)^, Obj.InstanceSize);

And that’s it! Well, sort of.

This will work just dandy as long as you use only simple types like Integer, Char, Double or Boolean. It does not, however work for managed types like strings or referenced types like objects. And it has a little type confusion issue as well.

String, interface and dynamic array types are managed types and if we just copy NewObj’s data over Obj’s data, we lose Obj’s pointer to this managed memory which means we leak memory. Also, and possibly worse, the memory manager has the impression that only one reference to your managed object exists, so it will free that object as soon as you free either Obj or NewObj. If you then free the other one, you’ll likely receive the dreaded “Invalid pointer operation” message.

A similar issue exists with any pointer or object types, because you lose the pointer to Obj’s data.

The solution here is to not copy the memory from NewObj to Obj, but rather to swap it. You know, like switching the values of two integer variables:

C := A;
A := B;
B := C;

We’ll put this in a reusable function. We’ll also let this new function automatically ensure that it only copies the memory the two objects have in common – so the lesser of Obj.InstanceSize and NewObj.InstanceSize.

The type confusion issue I mentioned relates to the VMT pointer. Remember I said that is actually used for the class type? So if you copy the entire memory block from NewObj to Obj, you are actually changing the runtime type. Certain constructs will still work as expected (confusingly, the is-operator is just fine) but others return type info from TMyNewObject (like ClassInfo or ClassName). Worse, your virtual methods could be all messed up.

So instead of casting like I did in the Move-call above, I’ll use this inline function to skip the VMT pointer:

function GetObjData(Obj: TObject): Pointer; inline;
begin
  Result := Pointer(Integer(Obj) + SizeOf(TClass));
end;

And then we get a perfectly generic function that doesn’t leak memory or cause pointer errors:

procedure SwitchObjects(Object1, Object2: TObject);
var
  Buffer: Pointer;
  BufSize: Integer;
  BuffOffset: Integer;
begin
  BuffOffset := SizeOf(TClass);
  BufSize := Min(Object1.InstanceSize, Object2.InstanceSize) - BuffOffset;
  GetMem(Buffer, BufSize);
  try
    Move(GetObjData(Object1)^, Buffer^, BufSize);
    Move(GetObjData(Object2)^, GetObjData(Object1)^, BufSize);
    Move(Buffer^, GetObjData(Object2)^, BufSize);
  finally
    FreeMem(Buffer);
  end;
end;

So for my nightmare scenario that I described at the start of this post, the solution turned out to be simple. Instead of directly calling the needed method on my global object, I passed it as parameter to a function that:

  1. Creates my new functionality object.
  2. Calls the new functions.
  3. Switches content with the global object.
  4. And returns.

This is about as clean as the hack can get, I’d say. Obviously, the new functionality will be folded back into the core package for the next major release of our software, but for now everyone is happy.

What truly ugly hacks have you had to apply?

Comments (3)

Translating flags in API calls, part 2

As promised, this is part 2 of my article on translating flags in API calls. If you haven’t read part 1 yet, please do so now.

Sometimes, you get API calls that make real hefty use of flags. One such example is DrawFrameControl, which has one flag to specify the type of control to draw as well as a second flag to specify the style of that control. The values are all declared as DFC_ or DFCS_ constants and they may be freely combined. Only, not all combinations work together and a lot of developers combine the flags using the plus-operator instead of or, which certainly doesn’t help.

Following on from the last post, we can create several overloads, each of which contain only the list of values relevant to the type of control:

function DrawFrameControl(DC: HDC; const Rect: TRect;
  uType: TDrawFrameControlType; uState: TDrawFrameControlCaptionStyles): BOOL; stdcall; overload; external user32 name 'DrawFrameControl';
function DrawFrameControl(DC: HDC; const Rect: TRect;
  uType: TDrawFrameControlType; uState: TDrawFrameControlMenuStyles): BOOL; stdcall; overload; external user32 name 'DrawFrameControl';
function DrawFrameControl(DC: HDC; const Rect: TRect; uType: TDrawFrameControlType; uState: TDrawFrameControlScrollStyles): BOOL; stdcall; overload; external user32 name 'DrawFrameControl';
function DrawFrameControl(DC: HDC; const Rect: TRect; uType: TDrawFrameControlType; uState: TDrawFrameControlButtonStyles): BOOL; stdcall; overload; external user32 name 'DrawFrameControl';

Once you then specify your control in uType, the values you pass to uStyle will select the correct overloaded version of this function.

Of course, if it was that simple I wouldn’t have done a separate post about this. In the case of this function (and several others), the Windows API conspires against us.

First, we have the shared states. All your control types can have the following states:

DFCS_INACTIVE = $100;
DFCS_PUSHED = $200;
DFCS_CHECKED = $400;
DFCS_TRANSPARENT = $800;
DFCS_HOT = $1000;
DFCS_ADJUSTRECT = $2000;
DFCS_FLAT = $4000;
DFCS_MONO = $8000;

There is no really sensible way of combining these with your other control-specific states, other than to add these to each of your types. So the State type for button controls will look like this:

TDfcButtonStyle = (DFCSBUTTONRADIOIMAGE, DFCSBUTTONRADIOMASK,
  DFCSBUTTONRADIO, DFCSBUTTON3STATE, DFCSBUTTONPUSH,
  DFCSINACTIVE = 8, DFCSPUSHED,
  DFCSCHECKED, DFCSTRANSPARENT, DFCSHOT,
  DFCSADJUSTRECT, DFCSFLAT, DFCSMONO,
  DFCSBUTTONRANGE = 31);
TDfcButtonStyles = set of TDfcButtonStyle;

And you’d have something similar for all four types (buttons, captions, menus/popup menus and scroll bars). Once again, notice that DFCSINACTIVE gets a value of 8 instead of $100, because we are interested in the bit position.

The issue then of course becomes on of naming the values. You see, if you have DFCSFLAT in both TDfcButtonStyle and TDfcCaptionStyle the compiler will see them both and complain about the identifier being re-declared.

My favourite solution to this, is using the {$SCOPEDENUMS} compiler directive. This basically makes the Delphi compiler treat enumerated types in a way that is very similar to that of C# – the actual enumerated values are only in scope when you specify the type. I don’t like needing to think up unique two-character prefixes for each type anyhow, but in this case I find scoped enumerations to be invaluable.

Not only is TDfcButtonStyle.DFCSHOT now a distinct value from TDfcCaptionStyle.DFCSHOT, but I can in fact clean the names up to look more like it was thought up with mortals in mind:

TDfcButtonStyle = (ButtonRadioImage, ButtonRadioMask,
  ButtonRadio, Button3State, ButtonPush,
  Inactive = 8, Pushed, Checked, Transparent, Hot,
  AdjustRect, Flat, Mono, Range = 31);
TDfcButtonStyles = set of TDfcButtonStyle;

No doubt this technique will have lots of traditionalists up in arms, but I think it looks much cleaner and more readable.

The second issue is a little worse and in a twist of irony, is even worse when scoped enumerations are used.

You’ll notice there is no DFCBUTTONCHECKED or ButtonChecked in the examples above. That’s because this Windows constant has a value of zero. And Delphi set types don’t handle zero at all. You could pass [] for that parameter, but then it is no longer obvious that you are trying to draw a checkbox. Also, the compiler will have a torrid time distinguishing between the DrawFrameControl overloads.

I found two ways of getting this to work, but they both have the word hack scribbled all over them.

Firstly, you could add ButtonCheck to the above enumeration at some unused position, say 6. This works and is transparent to the client code, but the success of the technique depends entirely on the API function you’re calling. If it has an undocumented feature, or it changes in the next Windows version or it keels over when an unknown flag is passed in, you’re toast. Experience shows it to be mostly reliable, but it is still not to be fully trusted.

The second solution – the one that doesn’t like scoped enumerations – is to declare the missing value as a typed constant:

DFCSBUTTONCHECK: TDrawFrameControlButtonStyles = [];

This is future proof but has another issue – inconsistent use. Compare the calls to draw a checked radio button and a checked check box:

DrawFrameControlTS(PaintBox.Canvas.Handle, PaintRect,
  DFCBUTTON, DFCSBUTTONCHECK + [DFCSCHECKED]);
DrawFrameControlTS(PaintBox.Canvas.Handle, PaintRect,
  DFCBUTTON, [DFCSBUTTONRADIO, DFCSCHECKED]);

And no, I don’t have a magic trick that will make this work better.

So the take-home for this two-part article is this:

It is possible to create type-safe imports for Windows API functions that are every bit as natural to use as native Delphi functions. Some functions make strange use of flags and these can either use some of the described techniques to work around the hardship, or could be imported using good ol’ Cardinal.

As always, the situation and your own better judgement should serve as a guide.

Comments (2)

Older Posts »