Chapters

  1. Introduction
  2. Structure
  3. Packaging
  4. Logging
  5. Configuration
  6. Debugging EXEs
  7. Handling errors
  8. Testing
  9. Documentation
  10. Make
  11. Providing help
  12. Scheduled Tasks
  13. Windows Services
  14. Windows Event Log
  15. Windows Registry
  16. Creating SetUp.exe
  17. Regular Expressions
  18. Acre
  19. GUI
  20. Git

Appendices

  1. Windows environment vars
  2. User commands
  3. aplcores & WS integrity
  4. Development environment
  5. Special characters

Misc

We want our logging and error handling to be configurable. In fact, we will soon have lots of state settings. Thinking more widely, an application’s configuration includes all kinds of state: e.g. folders for log files and crashes, a debug flag, a flag for switching off error trapping, an email address to report to – you name it.

Several mechanisms are available for storing configuration settings. Microsoft Windows has the Windows Registry. There are also cross-platform file formats to consider: XML, JSON – and good old INI files.

The Windows Registry is held in memory, so it is fast to read. It has been widely used to store configuration settings. Some would say, abused. However, for quite some time it was considered bad practice to have application-specific config files.

Everything was expected to go into the Windows Registry. The pendulum started to swing back the other way now for several years, and application-specific config files become ever more common. We follow a consensus opinion that it is best to minimise the use of the Registry.

Settings needed by Windows itself have to be stored in the Registry. For example, associating a file extension with your application, so that double-clicking on its icon launches your application.

The APLTree classes WinRegSimple and WinReg provide methods for handling the Windows Registry. We will discuss them in their own chapter.

MyApp doesn’t need the Windows Registry at this point. We’ll store its configurations in configuration files.

Information

The Windows Registry is still an excellent choice for saving user-specific stuff like preferences, themes, recent files etc. However, you have to make sure that your user has permission to write to the Windows Registry – that's by no means a certainty.

Three formats are popular for configuration files: INI, JSON and XML. INI is the oldest, simplest, and most crude. The other formats offer advantages: XML can represent nested data structures, and JSON can do so with less verbosity.

Both XML and JSON depend upon unforgiving syntax: a single typo in an XML document can render it impossible to parse.

We want configuration files to be suitable for humans to read and write, so you might consider the robustness of the INI format an advantage. Or a disadvantage: a badly-formed XML document is easy to detect, and a clear indication of an error.

Generally, we prefer simplicity and recommend the INI format where it will serve.

By using the APLTree class IniFiles we get as a bonus additional features:

We will discuss these features as we go along.

In the chapter on Logging, we considered the question of where to keep application logs. The answer depends in part on what kind of application you are writing. Will there be single or multiple instances?

For example, while a web browser might have several windows open simultaneously, it is nonetheless a single instance of the application. Its user wants to run just one version of it, and for it to remember her latest preferences and browsing history.

But a machine may have many users, and each user needs her own preferences and history remembered.

Our MyApp program might well form part of other software processes, perhaps running as a service. There might be multiple instances of MyApp running at any time, quite independently of each other, each with quite different configuration settings.

Where does that leave us? We want configuration settings:

As defaults for the application in the absence of any other configuration settings, for all users

These must be coded into the application (‘Convention over configuration’), so it will run in the absence of any configuration files.

But an administrator should be able to revise these settings for a site. So they should be saved somewhere for all users. This filepath is represented in Windows by the ALLUSERSPROFILE environment variable. So we might look there for a MyApp\MyApp.ini file.

For invocation when the application is launched
We could look in the command-line arguments for an INI.
As part of the user’s profile

The Windows environment variable APPDATA points to the individual user’s roaming profile, so we might look there for a MyApp\MyApp.ini file. Roaming means that no matter which computer a user logs on to in a Windows Domain [1], her personal settings, preferences, desktop etc. roam with her.

The Windows environment variable LOCALAPPDATA on the other hand defines a folder that is saved just locally. Typically APPDAATA points to something like C:\Users\{username}\AppData\Roaming and LOCALAPPDATA to C:\Users\{username}\AppData\Local.

Information

Note that when a user logs on to another computer all the files in APPDATA are synchronised first. Therefore it is not smart to save in APPDATA a logfile that will eventually grow large – put it into LOCALAPPDATA.

From the above we get a general pattern for configuration settings:

  1. Defaults in the program code
  2. Overwrite from ALLUSERSPROFILE if any
  3. Overwrite from USERPROFILE
  4. Overwrite from an INI specified on the command line
  5. Overwrite with the command line

However, for the Cookbook we keep things simple: we look for an INI file that is a sibling of the DYAPP or the EXE for now but will allow this to be overwritten via the command line with something like INI='C:\MyAppService\MyApp.ini.

We need this when we make MyApp a Windows Scheduled Task, or run it as a Windows Service.

Save a copy of Z:\code\v04 as Z:\code\v05 or copy v05 from the Cookbook website. We add one line to MyApp.dyapp:

...
Load ..\AplTree\FilesAndDirs
leanpub-insert-start
Load ..\AplTree\IniFiles
leanpub-insert-end
Load ..\AplTree\OS
...

and run the DYAPP to recreate the MyApp workspace.

You can read the IniFiles documentation in a browser with ]ADoc #.IniFiles.

This is the content of the newly introduced code\v05\MyApp.ini:

localhome = '%LOCALAPPDATA%\MyApp'

[Config]
Debug       = ¯1    ; 0=enfore error trapping; 1=prevent error trapping;
Trap        = 1     ; 0 disables any :Trap statements (local traps)

Accents     = ''
Accents     ,='ÁÂÃÀÄÅÇÐÈÊËÉÌÍÎÏÑÒÓÔÕÖØÙÚÛÜÝ'
Accents     ,='AAAAAACDEEEEIIIINOOOOOOUUUUY'

[Folders]
Logs        = '{localhome}\Log'
Errors      = '{localhome}\Errors'

If you have not copied v05 from the website make sure you create an INI file with this content as a sibling of the DYAPP.

Notes:

We create a new function CreateConfig for that:

∇ Config←CreateConfig dummy;myIni;iniFilename
⍝ Instantiate the INI file and copy values over to a namespace `Config`.
  Config←⎕NS''
  Config.⎕FX'r←∆List' 'r←{0∊⍴⍵:0 2⍴'''' ⋄ ⍵,[1.5]⍎¨⍵}'' ''~¨⍨↓⎕NL 2'
  Config.Debug←A.IsDevelopment
  Config.Trap←1
  Config.Accents←'ÁÂÃÀÄÅÇÐÈÊËÉÌÍÎÏÑÒÓÔÕÖØÙÚÛÜÝ' 'AAAAAACDEEEEIIIINOOOOOOUUUUY'
  Config.LogFolder←'./Logs'
  Config.DumpFolder←'./Errors'
  iniFilename←'expand'F.NormalizePath'MyApp.ini'
  :If F.Exists iniFilename
      myIni←⎕NEW ##.IniFiles(,⊂iniFilename)
      Config.Debug{¯1≡⍵:⍺ ⋄ ⍵}←myIni.Get'Config:debug'
      Config.Trap←⊃Config.Trap myIni.Get'Config:trap'
      Config.Accents←⊃Config.Accents myIni.Get'Config:Accents'
      Config.LogFolder←'expand'F.NormalizePath⊃Config.LogFolder myIni.Get'Folders:Logs'
      Config.DumpFolder←'expand'F.NormalizePath⊃Config.DumpFolder myIni.Get'Folders:Errors'
  :EndIf
  Config.LogFolder←'expand'F.NormalizePath Config.LogFolder
  Config.DumpFolder←'expand'F.NormalizePath Config.DumpFolder
∇

What the function does:

Notes:

The built-in function ∆List is handy for checking the contents of Config:

      Config.∆List
 Accents      ÁÂÃÀÄÅÇÐÈÊËÉÌÍÎÏÑÒÓÔÕÖØÙÚÛÜÝ  AAAAAACDEEEEIIIINOOOOOOUUUUY
 Debug                                                                  0
 DumpFolder                          C:\Users\kai\AppData\Local\MyApp\Log
 LogFolder                           C:\Users\kai\AppData\Local\MyApp\Log
 Trap                                                                   1

Now that we have moved Accents to the INI file we can lose these lines in the MyApp script:

⍝ === VARIABLES ===
    Accents←'ÁÂÃÀÄÅÇÐÈÊËÉÌÍÎÏÑÒÓÔÕÖØÙÚÛÜÝ' 'AAAAAACDEEEEIIIINOOOOOOUUUUY'
⍝ === End of variables definition ===

Where should we call CreateConfig from? Surely that has to be Initial:

leanpub-start-insert
∇ (Config MyLogger)←Initial dummy
⍝ Prepares the application.
  Config←CreateConfig ⍬
  MyLogger←OpenLogFile Config.LogFolder
  MyLogger.Log'Started MyApp in ',F.PWD
  MyLogger.Log #.GetCommandLine
  MyLogger.Log↓⎕FMT Config.∆List
∇

Note that we also changed what Initial returns: a vector of length two, the namespace Config but also an instance of the MyLogger class.

Initial was called within StartFromCmdLine, and we are not going to change this but we must change the call as such because now it returns something useful:

leanpub-start-insert
∇ {r}←StartFromCmdLine arg;MyLogger;Config
⍝ Needs command line parameters, runs the application.
  r←⍬
  (Config MyLogger)←Initial ⍬
  r←TxtToCsv arg~''''
∇

Although both MyLogger and Config are global and not passed as arguments, it’s good practice to assign them this way rather than bury their creation somewhere down the stack. This way it’s easy to see where they are set.

Specifying an INI file on the command line

We could pass the command line parameters as arguments to Initial and investigate whether it carries any INI= statement. If so the INI file specified this way should take precedence over any other INI file. However, we keep it simple here.

We now need to think about how to access Config from within TxtToCsv.

The configuration parameters, including Accents, are now collected in the namespace Config. That namespace is not passed explicitly to TxtToCsv but is needed by CountLetters which is called by TxtToCsv.

We have two options here: we can pass a reference to Config to TxtToCsv, for example as left argument, and TxtToCsv in turn can pass it to CountLetters. The other option is that CountLetters just assumes the Config is around and has a variable Accents in it:

CountLetters←{
    {⍺(≢⍵)}⌸⎕A{⍵⌿⍨⍵∊⍺}Config.Accents U.map A.Uppercase ⍵
}

Yes, that’s it. Bit of a compromise here. Let’s pause to look at some other ways to write this.

Passing everything through function arguments does not come with a performance penalty. The interpreter doesn’t make ‘deep copies’ of the arguments unless and until they are modified in the called function (which we hardly ever do) – instead the interpreter just passes around references to the original variables.

So we could pass G as a left argument of TxtToCsv, which then simply gets passed to CountLetters.

No performance penalty for this, as just explained, but now we’ve loaded the syntax of TxtToCsv with a namespace it makes no direct use of, an unnecessary complication of the writing. And we’ve set a left argument we (mostly) don't want to specify when working in session mode.

The matter of encapsulating state – which functions have access to state information, and how it is shared between them – is very important. Poor choices lead to tangled and obscure code.

From time to time you will be offered (not by us) rules that attempt to make the choices simple. For example: never communicate through global or semi-global variables. [2].

There is some wisdom in these rules, but they masquerade as satisfactory substitutes for thought, which they are not.

Just as in a natural language, any rule about writing style meets occasions when it can and should be broken.

Following style ‘rules’ without considering the alternatives will from time to time have horrible results, such as functions that accept complex arguments only to pass them on unexamined to other functions.

Think about the value of style ‘rules’ and learn when to follow them.

One of the main reasons why globals should be used with great care is that they can easily be confused with local variables with similar or – worse – the same name.

If you need to have global variables then we suggest encapsulating them in a dedicated namespace Globals. With a proper search tool like Fire [3] it is easy to get a report on all lines referring to anything in Globals.

Sometimes it’s only after writing many lines of code that it becomes apparent that a different choice would have been better.

And sometimes it becomes apparent that the other choice would be so much better than it’s worth unwinding and rewriting a good deal of what you’ve done. (Then rejoice that you’re writing in a terse language.)

We share these musings here so you can see what we think about when we think about encapsulating state; and also that there is often no clear right answer.

Think hard, make your best choices, and be ready to unwind and remake them later if necessary.

We have used the most important features of the IniFiles class, but it has more to offer. We just want to mention some major topics here.

We need to change the Version function:

∇ r←Version
   ⍝ * 1.2.0:
   ⍝   * The application now honours INI files.
   ⍝ * 1.1.0:
   ⍝   * Can now deal with non-existent files.
   ⍝   * Logging implemented.
   ⍝ * 1.0.0
   ⍝   * Runs as a stand-alone EXE and takes parameters from the command line.
      r←(⍕⎕THIS)'1.2.0' '2017-02-26'
∇

And finally we create a new standalone EXE as before and run it to make sure that everything keeps working. (Yes, we need test cases)


Footnotes

  1. https://en.wikipedia.org/wiki/Windows_domain

  2. So-called semi-globals are variables to be read or set by functions to which they are not localised. They are semi-globals, rather than globals, because they are local to either a function or a namespace. From the point of view of the functions that do read or set them, they are indistinguishable from globals – they are just mysteriously ‘around’.

  3. Fire stands for Find and Replace. It is a powerful tool for both search and replace operations in the workspace. For details see https://github.com/aplteam.Fire. Fire is discussed in the chapter Useful user commands.