An Essential Principle

As a friend has written in a book I am proofreading: “…if you don’t know how to write good code in simple scenarios, you will certainly never learn to write good code in complex ones.”

Ponder that, don’t just rush on.

I know that many developers give greater attention to complex coding tasks than to the simple ones. But in writing our code, we condition our brains. We impress on our own mental processes a collection of habits which we follow without having to think much about them. Truly, that accounts for a lot of thoughtless coding.

Until we fully understand that the little things are at least important as the complex ones, we are unlikely to raise our level of praxis. And if we do not, then we are not practicing design, nor development, but merely writing code.

We should always strive to do the best work of which we are capable. And as we do that, we will elevate the quality of all our work.

Those pesky details…

The little things in code are easily overlooked, and can be the source of problems which are difficult to diagnose. Consider the conversion of a string to a number:

  d := StrToFloat(NumericString);

What could go wrong? Well, many things, actually. In writing this, you no doubt were thinking of the problem at hand, rather than the general class of such problems. But truly, that is a mistake. You would always ensure that the content of the argument NumericString is of the correct form to be converted. But others may not. So although you would expect to pass in a value like ‘2345.27’, someone else may pass in ‘2,345.27’, provoking an exception. One approach to guarding against that could be:

  try
     d := StrToFloat(NumericString);
  except
  end;

But that merely hides the exception. Would you not really prefer a better solution? The empty except clause is itself a code design error. There are many alternatives, among them:

var
  d: Double;
  code: Integer;
begin
  Val(NumericString, d, code);
  if code  0 then
    Exit;

Of course, this code ignores whatever is the purpose of the routine. However, it presents some advantages:

• Improper strings will not throw an exception.
• The value in code is zero if all is well, or is the index to the problem character in the string.

One thing to note is that Val() is a procedure which has been in Delphi forever. Now that we no longer get printed manuals, we really should spend some time in the libraries, or at least in the related help, to be fully aware of what we get from Delphi, without having to write our own code.

What other approaches could we consider in resolving this in a general way? We must first think of the possibilities, and of good design and coding practices. We could:

• Use a different conversion routine from the library, like TryStrToInt().
• Remove from the string all characters which are not in the set: [‘0’..’9′, ‘.’].
• Write a filter which handles the majority of reasonable possibilities.

Using TryStrToInt protects us from throwing an exception, and yields zero if the input was bad, but it’s not a real solution, in that it does nothing to help us understand the real problem.

The second and third possibilities are very similar, and worth considering:

function NumOnly(ANumStr: string): string;
var
  s: string;
  cnt, idx, n: Integer;
begin
  n := ANumStr.Trim.Length;
  SetLength(s, n);
  cnt := 0;
  for idx := 1 to n do
  begin
    if CharInSet(ANumStr[idx], ['0'..'9', '.']) then
    begin
      Inc(cnt);
      s[cnt] := ANumStr[idx];
    end;
  end;
  SetLength(s, cnt);
end;

This looks good, and we are then able to write something like:

  d := StrToInt(NumOnly(NumericString));

It will handle a wide range of numeric strings, and will return a good value. But it has weaknesses. What if the string looks like: ‘2235.30 – 27.57’? Well, it is not intended to be an expression evaluator, and it will then throw an exception, as it tries to convert ‘2235.3027.57’ to a number. So it is not a perfect solution, but it is a reasonable solution to a wide range of input.

In this kind of code, keep in mind that this should be in a library, so that it can serve your needs anywhere in the application. It should not appear inline or in a nested function, since that approach will lead inevitably to copy and paste.

In pursuing the solution for your own needs, you will have to consider many issues, but the point is that doing such analysis is the difference between designing and merely coding.

On Naming

We often hear that naming in software is difficult, and it certainly can be. But equally, it is an essential part of the job, so the proper attitude would be to spend some time and effort and improve our skills. Or, we could find another line of work. It’s just that important.

I have done extensive work in legacy projects for numerous employers. And by legacy, I do not mean that the product is no longer in development, but that it has been many years in development. Myriad challenges are altogether normal in old code, and naming is just one of them, but it is a serious one, as bad naming impedes comprehension.

It is common in legacy code to find very long routines, as well as badly named routines. The combination is particularly challenging for maintenance. Of course, it is difficult in such cases to improve the naming, since it is difficult to discern what the routine is doing. But you can approach it in a methodical manner.

First, extract nested routines. Good candidates will include lines of assignment statements. Or blocks which are saving to a database. Group these operations as cleanly as possible, and put them in nested routines. Practice good naming on those new routines. Keep at it, until you find no more obvious candidates for removal to small routines. You should find that the outer routine is becoming more comprehensible. You may also find that you can now improve on the names of those routines.

Next, look for other large routines in the same module which offer such possibilities. Repeat the process above.

Keep in mind that in this rework you must avoid altering the sequence of operations. If you can do that, the risk of introducing defects is small.

With luck, you may find that some of these nested routines are nearly identical to one another. Where that happens, consider creating a private class member routine which can replace the duplicates. That’s a significant improvement, and well worth the effort. Over time, you may also find that some of these private members match routines in other classes. At that point, you must consider whether to create a new module which consolidates those routines, and allows removing the duplicated code. Whether to use a class, or individual routines is a call you must make.

What of new code?

In new code work, we hope you are following good practices:

• Keep routines short and focused.
• Make your names document the functions.
• DRY: Don’t Repeat Yourself.

But for now, and the near future, give particular attention to naming. Don’t be the one who just says “Naming is hard!” Do better!

Database Design Tools

I do not do a lot of database design, but when I must, I want to use a tool which makes it easy and lets me produce a good diagram of tables and their relations. It would be very nice to have such a tool as a desktop app, but those I have seen fall short in a number of areas, or are best when linked to an already designed database.

I have used Volus Pencil, and though it is simple, it feels too much like using a graphics tool, and requires too much effort. Further, as it has not been updated in nearly a year, the author is apparently content with the tool in its current form.

QuickDBD

I only recently discovered QuickDBD, and I like very much what I see. I can create my tables by typing in the specification using a very terse syntax, and the diagram is created as I type. Since the specification is what I really want, and the diagram is chiefly for communicating what I have done, this is very attractive.

The online app is seeded with a schema:

The diagram it produces:

This is a small, but useful database, and the entry needed to produce a useful diagram is minimal. Once the diagram is created, you are also able to rearrange the positions of the tables to suit yourself.

Online documentation is provided, and includes sample schema examples. I’d like to have PDF documentation, but what is there seems to be concise and thorough. The examples presented appear sufficient to clarify the more complex issues in using the tool.

Import and export capabilities are present, albeit with support for a modest number of specific databases, including:
ANSI SQL (export only)
MySQL/MariaDB (import beta)
Oracle (12c+) (import beta)
PostgreSQL (export only)
SQL Server (import beta)

I have done only limited testing so far, but the exported SQL looks good, as do the RTF and PNG file output. Note that I used a screen capture tool for the images in this article, simply because I was more comfortable with that, and it simplified creating this article. That said, here is an exported PNG file:

The URL was inserted by the export process. The SVG was more troublesome:

My very old PaintShop Pro would not open it, but Affinity Designer did. Clearly, the PNG is closer to ready for prime time, though I have not made any effort to discover what may be going on with the SVG. Inkscape also opened the SVG, but with odd results. As there are aspects of the program which are labeled as beta, I am not overly concerned. I fully expect they will address the SVG issues, and the PNG will be perfectly adequate to my needs for now.

The ability to save a schema appears to be limited to the licensed Pro version, but you can export the diagram to PNG or SVG files. You are able also to export your specification to an RTF file or to a PDF document.

Right now, the tool is offered for free if you write a review or article , or tweet about it. That makes it pretty hard not to at least try it out!

About Attributes

From 2011 to late in 2013, I worked in Delphi XE, exclusively, and for the last 5 months, I have been working in Delphi 2007. Now, however, I am on the brink of leaping forward to Delphi XE5. Compared to D2007, The changes are very large, indeed:

• Unicode
• Generics
• Attributes
• Live Bindings

I have already been accustomed to the shift to Unicode, from working in XE. Similarly, I did make use of generics in XE, though not extensively. Attributes and Live Bindings, on the other hand, are uncharted territory.

I expect to dive first into attributes, and suspect they may help me to deal with some currently thorny problems. I say suspect, as I am still mainly scratching my head over how attributes can and should be used. Attributes can be applied to many types in Delphi, as seen in this article. But the mere fact that they can be applied to something does not mean that they should be. My slowly increasing understanding of them–some features simply elude my comprehension for a time, until I see a compelling example, then the light goes on–includes an appreciation of two important considerations:

1. Making use of attributes requires using RTTI, which brings with it significant performance penalties, so if performance is an issue, it may be best to avoid their use.
2. Using attributes provides late binding, which in turn means that the compiler cannot protect us from our own stupidity: if we misuse attributes, then we may run afoul of run-time errors.

One challenge is that in numerous examples, all too often the use of the attributes seems to be almost a bit of magic. Not being a fan of clever code, I hope to avoid using them in ways which make the code less comprehensible.

More later, as I find something worth demonstrating in code.

A small adventure in RTF

I have need of fixing the RTF comments contained in some 48,000 records in a database. These were imported from another database some years ago, and the original data is no longer available. The import process damaged at least some of them, generally by copying into a too small container, thus truncating the RTF script. On my form, I have placed a TMemo (memPlain) and a TRichEdit (memRich). I fill them with this code:

procedure TfrmMain.FillMemoes;
var
str: TMemoryStream;
s: string;
begin
str := TMemoryStream.Create;
try
s := dsLookup.DataSet.FieldByName( 'vComment' ).AsString;
Label2.Caption := s;
memPlain.Lines.Text := s;
str.Write( PChar( s )^, Length( s ) );
str.Position := 0;
memRich.Lines.LoadFromStream( str );
finally
str.Free;
end;
if not chkDisableRepairs.Checked then
MakeRepairs;
end;

The MakeRepairs procedure is simple:

function TfrmMain.MakeRepairs: Boolean;
begin
Result := False;
if CheckPlainAndFix then
Exit( True );
if CleanRTFAndFix then
Exit( True );
end;

A little function to make things a bit less cluttered:

function TfrmMain.HasRTFDelims(const text: string): Boolean;
begin
Result := ( PosEx( '{', text ) > 0 ) or
( PosEx( '}', text ) > 0 ) or
( PosEx( '\', text ) > 0 ) ;
end;

At this point, I must say that I had gotten ahead of myself. I wound up abandoning CheckPlainAndFix, and instead added some visual tools to let me learn about the actual problems in the data. What I learned changed my direction entirely.

There are 48,799 records in the database. Of these, it turned out that RTF was damaged in 59 of them. Annoying, but a much smaller incidence of problems than my client thought existed. My suggestion was that I could simply manually correct these few, and save the updates. It made little or no sense to code a solution, both because so few records were involved, and because among that small number, there were several different pathologies observed, at least one of which would have required a good deal of experimentation to resolve.

My client countered with the decision that we would make no repairs. These data are a few years old. It may be that the users will never open them, and as the reporting from the app is done for only the current year, they will have no impact there.

Lessons (re)learned:

1. Next time I am told that we “have a big problem” I shall go no further than to measure the actual magnitude of the problem.
2. Coding any sort of fix, however minor, for any sort of encoded stream prior to completing step 1 is just silly.

Those points are pretty fundamental, and I had certainly learned them years ago. But in this case, my client told mew a) that the damage was widespread and b) that he had tried to apply some repairs in SQL, but will little success. That said to me that he had done some analysis, and that it was a substantial problem in need of a clean solution. However, as I tripped over issues in debugging my code, I began moving toward trying to quantify the problem, and if necessary, to quantify the categories of pathologies, in terms of the difficulty it might involve to code a repair. With a total of only 59 damaged records, very little coding would have been justified.

In the final analysis, the only routines with value in my little project were:

1. The small routine which recognized RTF delimiters in the visible text of the TRichEdit.
2. The code I added to count the number of damaged records.
3. The visual items added which let me see the list of damaged records and click on each ID to reload the TRichEdit, allowing very rapid determination of pathologies.

Therefore, I have not developed any sort of RTF code repair tool. The need may surface someday, or I may decide to pursue it on my own, for the experience, but my client has no need of it.

On the other hand, one of the discoveries I did make was that a very small number of users had copied and pasted from Word to the app which originally managed the data. As might have been anticipated, Word exports in RTF (just as it does in HTML) a rather large number of elements which it would be nice to remove. Dozens of RGB color specifiers, for example. Now that may well be a project for me do undertake at some point. Even in the records affected in this way, the colors had not been used, so there is no reason whatever to retain them. It is just MS-bloat.

Shifting priorities

I’ve been quiet for months. Much of my work has been afflicted with the challenge of overcoming breaking changes in third-party products which are pervasive in my apps. Not much fun, and as the project is finally winding down, so is my stamina. I need a change.

I’ve just acquired a Raspberry Pi, on my way to getting an irrigation system installed at my home. I was originally looking at an AVR based controller, but now, from the same vendor, I have the option of an add-on card to the Raspbery Pi to fulfill the same requirements.

Honestly, I thought this over for weeks. First, we’ve had so much rain this year that it’s not urgent to get a system installed–but I know it will be needed, soon enough. But second,  I have been an AVR fan for years, and it was initially attractive to have a controller based on an AVR. However, the more I thought about it, the more I was drawn to the Raspberry Pi solution. 

Both products are open source, but the AVR version is using Arduino, which would not be my first choice. the Raspberry Pi version controller is written in Python, also not high on my list, but as RPi is a Linux machine, and many languages are available for it, I have the option of designing my own controller, in C, C++, or Pascal, as I decide.

Further, I have no desire to deal with TCP/IP on the AVR. It may actually be well shielded from the user by the Arduino approach, but at best, this is a niche solution. The RPi on Linux follows a more general path.

As I said above, I have received my RPi, but as I have not yet had time to do more than look at it, I have little to say so far. In my next articles, I expect to document the process of getting it operational, and getting a development environment in place. More to come….

Frameworks, everywhere… and no documentation

Even as many claim that Delphi is on the wane, we see more and more open source frameworks cropping up. Large or small, they all being something to the table, and some are very powerful. All have been created in response to a perceived need, necessity being the mother of invention.

Unfortunately, one characteristic most of them share is a lack of documentation. It is all well and good to believe that you write self-documenting code, but it is best to remember that you are, after all, immersed in the work you are doing, so of course, once you are there, it seems obvious.

To many of the rest of us, it is far less obvious. Write comments, at the very least. Better yet, write actual documentation. It is becoming a lost art. And if or when you do write documentation, don’t be a syntax fanatic in your code and dismissive of syntax in your writing. They are of equal importance.

You may write a framework that none of us should be without, but if we can’t quickly get a sense of why it exits and how to use it, many of us will simply turn away, as we don’t need yet another hobby, or worse, a science project.

Just another software blog…

So, I needed a place to store articles, and had not used WordPress before. Who knows where this may lead? Settle back, and enjoy the ride.

A COM Rant

COM. To know it is to love/hate it.

From COM we gain the ability to access tools in a language-agnostic way. In-process or out-of-process servers, servers running as services, servers which provide the interface to exotic hardware. In theory, it’s great. In practice, it can be very frustrating.

Vendors: Just because you publish a COM interface doesn’t mean you’re done. In fact, COM tends to be in greater need of documentation than many other technologies. Sample applications can help, but they also will be written in a programming language which may not be the one your customer prefers. We all know you won’t write the apps in multiple languages—even though this would be a terrific method of enhancing the documentation—and no sample app will ever obviate the need for a manual which presents the design philosophy. Even a single line of information about each method call is better than none.

Developers: Just because you have toughed your way through numerous adventures with ill-conceived and undocumented interfaces is no reason to put up with more of the same. Agitate for documentation, where none is given, or for better, where some exists.

An aside: Why is it that so few developers, despite living in a world where syntax is everything, and compilers utterly unforgiving, give so little attention to the quality of their prose? In documentation, as in code, syntax matters! Clarity is essential. A compiler may put up with spaghetti code, but most readers will soon tire of misspelled words, ambiguous and incorrect antecedents, disagreement of number, and confused tense.

It has been my experience that projects using COM are invariably plagued with confusion at the front end, and although they usually resolve well in the end, the cost in time, money, and frustration is larger than with more traditional solutions. As I am at the front end of such a project now, wrestling with wholly inadequate documentation, insufficient sample applications, and no documentation of design philosophy, I will have more to say. Stay tuned….