pymsi - a msi alternative written in python

Introduction

msi is the established EPICS tool to do macro substitutions on the build host.

Macros and the files where they should be substituted are defined in substitution files. When the msi tool is applied to such an substitution file, it substitutes all macros in the mentioned files, the template files, and writes the result to the console or a new file. This substitution process is used to generate EPICS database files that are then loaded by the IOC.

Although msi is well tested and widely used there are some problems, especially with the question in what area of the substitution file a macro is defined after the definition statement. This discussion on how msi variable scoping works and how to extend msi in order to be able to define global variables brought about the idea if it was possible to do this much easier in python.

The basic idea is that a substitution file is no longer a special file format but simply valid python code. This has the following advantages:

However, the new format for substitution files should not be too different from the old one. So we want to keep the "file" and "pattern" keywords as well as the simple way to specify macros without the need to enclose the macro names in quotes. With pymsi this is now possible, here is an example of the new file format:

do(
    file("adimovhgbl.template",
         subst(
           DESCVAR="adimovhgbl_ins.mac",
           GBASE="U3IV:",
           DRV="V",
           AdiMopVer="9",
           TRIG1="U3IV:AdiVGblPvr.PROC",
           TRIG2="U3IV:AdiVGblMvr.PROC",
         ),
         subst(
           DESCVAR="adimovhgbl_ins.mac",
           GBASE="U3IV:",
           DRV="H",
           AdiMopVer="9",
           TRIG1="U3IV:AdiHGblPvr.PROC",
           TRIG2="U3IV:AdiHGblMvr.PROC",
         ),
         pattern(
           ("GBASE","DRV","AdiMopVer","TRIG1","TRIG2"),
           ("U3IV","C","9","U3IV:AdiCGblPvr.PROC", "U3IV:AdiCGblMvr.PROC"),
           ("U3IV","P","9","U3IV:AdiPGblPvr.PROC", "U3IV:AdiPGblMvr.PROC"),
         ),
    ),
    file("adimogbl.template",
         subst(
           DESCVAR="adimogbl_ins.mac",
           GBASE="U3IV:",
           TRIG1="U3IV:AdiMoVGblTrg.PROC",
           TRIG2="U3IV:AdiMoHGblTrg.PROC",
           TRIG3="",
           TRIG4="",
         ),
    ),

)

The whole structure is a call to the function "do" with a list of parameters, which are calls to other functions. All functions except "do" return functions or lists of functions. Only "do" is different, it returns nothing but executes all the functions it has been given. As you see, the indentation is arbitrary here, since python has no strict rules for the indentation of function arguments.

We call the new format of the substitution file the "pymsi language". It actually is just a set of functions defined in the "msilib" python module. The substitution file is valid python code, adding the "from msilib import *" statement at the top of the file would make it valid and executable python code.

For convenience we have a small script, "msi.py", supplied with the package. It imports the msilib module and executes the substitution file with the python interpreter.

Format of the template file

In this chapter we describe the basics of msi macros and commands in template files. Note that the format of template commands is different from the original format in msi. However, pymsi can be switched to a msi compatible mode where it can parse the original msi template commands.

Macros in the template file

pymsi uses the maclib, a library that is part of the EPICS libCom library. By this the macro substitution mechanism and the format of macros in the template file is fully compatible with msi. Macros in the template file have the following form:

$(macroname)
$(macroname=default-value)
${macroname}
${macroname=default-value}

The default value is taken if the macro is not defined at the time of the expansion.

If you want to have a string like '$(A)' as a literal in your template file you have to precede the dollar sign with a backslash like this '\$(A)'. Then a macro will not be substituted. Pymsi will remove each backslash that is immediately followed by a dollar sign, so '\$(A)' becomes '$(A)'. This is different from msi which would leave the backslashes in place.

If you use pymsi in the compatible template format, backslashes are not removed and template files are interpreted exactly the same way msi does.

Commands in the template file

Two commands can be used in template files, "include" and "substitute". pymsi supports these template commands in two different formats, the new pymsi format and the old backwards compatible msi format. In the original msi definition the commands are just bare words followed by a whitespace and a quoted string that have to be placed in a single line. If such a line would appear in the template file for some other reason msi could not be stopped interpreting this as a command.

In the new pymsi format, commands always start with a dollar ("$") sign and their arguments are enclosed in brackets like a function call. By this they are easily recognizable in the file. Since we use a dollar sign at the start, we can use the same escape mechanism as it is used with macros. If you do not want to have a string like '$include("..")' interpreted as a command you have to precede the "$" with a "\". Backslashes immediately followed by a dollar sign are removed in the output. So '\$include("..")' becomes '$include("..")'.

With pymsi, template commands do not have to be in lines of their own, they can appear anywhere in the text. This is different from msi.

The include command

The include command works much like an #include statement in C. It continues the macro substitution in a new file and, when it reaches the end of that file, continues at the place it left off.

Here is an example of the include command:

$include("myfile.db")

Note that '\$include("myfile.db")' would not be interpreted but just printed as '$include("myfile.db")'.

The substitute command

The substitute command is used to define macros within the template file. Definitions here have the form "name=value", several definitions can be given in a comma separated list. Here is an example:

$substitute("var1=value1,var2=value2")

Note that '\$substitute("var=value")' would not be interpreted but printed as '$substitute("var=value"). Different from the pymsi language, values do not have to be enclosed in quotes here but the quotes have to enclose the whole definition string.

Basics of the pymsi language

This section describes the basic parts of the pymsi language that implement the same functionality as the old substitution files.

The pymsi language is just a set of functions defined in the "msilib" module. All statements that are valid in python can be used here. Since all functions of "msilib" are imported in the global namespace they can be used without a preceding module name.

All functions listed except "do" return functions that do the actual work. This concept is important to remember. Only the returned function actually use the PyMacLib module and define macros or substitute macros in files.

Another important concept to mention here is scope, this means the area in the program code where a macro definition is valid. You can imagine a scope like a '{..}' block in C. Scopes can be nested and macros defined in a scope become undefined once the scope is left as in C at the place where matching '}' is. Scope are implemented by calls to the pushScope and popScope method of the PyMacLib.Macros object.

Here is a list of the basic functions of the pymsi language:

do

Every pymsi substitution file has to contain at least one call to the "do" function. "do" simply calls all it's arguments without any parameters. It makes no assumption on what these functions do, but usually the arguments are functions returned by calls to "file" functions. "do" executes everything in a single scope.

file

The "file" function expects a filename followed by functions and/or lists of functions. It returns a function that calls each of the given functions with that filename as the first and only parameter. The returned function makes the calls to the functions in a new scope.

subst

The "subst" function expects a number of named parameters. Each pair of parameter name and parameter value defines a new macro. It returns a function that expects a filename as parameter, defines all the macros in a new scope and expands the file for that filename. The expanded file is printed to the console unless the global function "output_to" has been used to specify a file where the output go.

pattern

The "pattern" function expects 2 or more unnamed arguments, every argument must be a tuple or a list of strings. The first argument defines macro names, all following arguments define macro values. It returns a list of functions that expect a filename a parameter, define their macros in a new scope and expand the file for that filename. The expanded file is printed to the console unless the global function "output_to" has been used to specify a file where the output go.

New features

Here we list functions and variables that implement features that are not known from the original substitution files and msi. Global macro definitions and proper scoping.

var

The "var" function is similar to the "subst" function. It expects a number of named parameters. Each pair of parameter name and parameter value defines a new macro. It returns a function that defines all the macros but NOT in a new scope. This function can be used at any place within a call to "file" or "do" or "scope" to define new macros. These macros are then defined and valid within the enclosing scope. Here is an example:

do(
    file("test.template",
         var(
           DESCVAR="adimovhgbl_ins.mac",
           GBASE="U3IV:",
           AdiMopVer="9"
         ),
         subst(
           DRV="V",
           TRIG1="U3IV:AdiVGblPvr.PROC",
           TRIG2="U3IV:AdiVGblMvr.PROC",
         ),
         subst(
           DRV="H",
           TRIG1="U3IV:AdiHGblPvr.PROC",
           TRIG2="U3IV:AdiHGblMvr.PROC",
         ),
        ),
  )

This defines "DESCVAR", "GBASE" and "AdiMopVer" for the file "test.template" globally. Note that, due to the scoping rules, these macros are not defined outside the call to "file" with filename "test.template".

macros

This is the name of the global msilib variable that holds all macro definitions. You usually do not access this variable directly but there may be cases when you want to read it. With respect to reading a value, the "macros" variable can be used like a python dictionary. So you can retrieve a macro value like this:

macros["macro-name"]

See also the chapter The Power of the Python for an example how to use this.

scope

The "scope" function expects a number of functions and/or lists of functions as parameters. It returns a function that calls each of the given functions with all parameters itself has been called. When the returned function is executed, it makes the calls in a new scope. This function is used for proper scoping, meaning that macros defined in the argument list of "scope" are not defined outside of "scope". Here is an example:

do(
    file("test.template",
         scope(
               var(
                 DESCVAR="adimovhgbl_ins.mac",
                 GBASE="U3IV:",
                 AdiMopVer="9"
               ),
               subst(
                 DRV="V",
                 TRIG1="U3IV:AdiVGblPvr.PROC",
                 TRIG2="U3IV:AdiVGblMvr.PROC",
               ),
               subst(
                 DRV="H",
                 TRIG1="U3IV:AdiHGblPvr.PROC",
                 TRIG2="U3IV:AdiHGblMvr.PROC",
               ),
          ),
          subst(
            DRV="H",
            TRIG1="U3IV:AdiHGblPvr.PROC",
            TRIG2="U3IV:AdiHGblMvr.PROC",
          ),
        ),
  )

In this example, "DESCVAR", "GBASE" and "AdiMopVer" are defined for the first two "subst" functions, but not for the third "subst" function, since the call to "var" is enclosed in a call to "scope". In the third "subst" call, "DESCVAR", "GBASE" and "AdiMopVer" are undefined.

Miscellaneous

enable_errors

This function is used to change the handling of undefined macros. It gets a single boolean parameter that, if True, causes pymsi to raise an exception when there are undefined macros in template files. Otherwise undefined macros are simply not expanded (note that the newest version maclib adds ",undefined" at the end of the macro name). The default is enable_errors(False).

output_to

This function is used to define the name of the file where the output of pymsi should be stored. It gets the filename as a single parameter which should be a string or the special python value None. If None is given all output is printed to the console, which is the default. If a file was given, the first following call to "do" deletes this file and writes it's output to that file. All following calls to "do" append to this file.

add_path

This function expects one or more strings or lists of strings as parameter. Each string may be a single path or a colon separated list of paths. Each of these paths is added to the internal list of search paths. When the program tries to find a template file specified in the call to a "file" function, it searches these paths if the file is not found in the current directory.

searchpaths

This is the global list variable that holds all file search paths. You usually do not access this variable directly but use add_path to append new file search paths to the list.

set_debug

This function is used to set the debug mode on or off. It gets a single boolean parameter. If that parameter is True, additional information is stored internally when macros are defined. Note that this makes new macro definitions slower. When set_debug(True) is called before the first macro definition, the method "macros" of the "macro" object can be used to retrieve the currently defined macros as a python dictionary at any place in the program. "set_debug(True)" should also be called if set_dry_run is to be used later. As a default, the debug mode is switched off.

set_dry_run

This function is used to set the dry run mode on or off. It gets a single boolean parameter. If that parameter is True, the macro substitution mechanism operates in dry run mode. This means that a list of all defined macros is printed together with the name of the template file that should be expanded. Note that the template files are not opened, so template file commands have no effect in this case. "set_debug(True)" must have been called before dry run mode is enabled. As a default, dry run mode is switched off.

set_old_template_format

This function is used to switch the old template file format on or off. It gets a single boolean parameter. If that parameter is True, the old fully msi compatible format for template files is used. If the parameter is False, the new template file format is used. See also Commands in the template file. The default mode is off, meaning that the new format will be used if set_old_template_format is never called.

The Power of the Python

See also http://xkcd.com/353/ ... :)

Here are some examples on how python can be used to make your substitution files more smart.

loops

Since substitution files are just nested calls to functions, we cannot place a statement like "for x in ..." in the argument list. We can, however, use the power of the "map" function:

do(
    file("test.template",
         var(
           DESCVAR="adimovhgbl_ins.mac",
           GBASE="U3IV:",
           AdiMopVer="9"
         ),
         map( lambda d:
                subst(
                  DRV=d,
                  TRIG1="U3IV:Adi%sGblPvr.PROC" % d,
                  TRIG2="U3IV:Adi%sGblMvr.PROC" % d,
                ), ["V","H"]
            )
        ),
  )

In this example, we take a "subst" call and embed it in an anonymous function using the "lambda" statement. This function is applied to the list ["V","H"]. The result is that the file test.template it expanded two times with slightly different sets of macros. This method can reduce redundancy in substitution files.

Here is the same using the python list comprehension:

do(
    file("test.db",
         var(
           DESCVAR="adimovhgbl_ins.mac",
           GBASE="U3IV:",
           AdiMopVer="9"
         ),
         [ subst(
                  DRV=d,
                  TRIG1="U3IV:Adi%sGblPvr.PROC" % d,
                  TRIG2="U3IV:Adi%sGblMvr.PROC" % d,
                ) for d in ["V","H"]
         ],
        ),
  )

Outside the call to "do" we are free to use any python constructs, so here is another implementation of a loop:

lst= []
for d in ["V","H"]:
    lst.append( subst(
                DRV=d,
                TRIG1="U3IV:Adi%sGblPvr.PROC" % d,
                TRIG2="U3IV:Adi%sGblMvr.PROC" % d,
                )
              )
do(
    file("test.db",
         var(
           DESCVAR="adimovhgbl_ins.mac",
           GBASE="U3IV:",
           AdiMopVer="9"
         ),
         lst
        ),
  )

calculation of macro values

We have to distinguish between macro values that are calculated before the execution of the function that "do" returns and macros that are calculated in the function "do" returns. The first variant must not access the "macros" variable since before the execution of the function returned by "do" it is empty.

However, macro values are allowed to be callable python objects. These are called in the function "do" returns, so they can access the global "macros" variable directly. Here is an example of that:

do(
    file("test.db",
         var(
           DESCVAR="adimovhgbl_ins.mac",
           GBASE="U3IV:",
           driveno="9"
         ),
         subst(
                AXNO= lambda: int(macros["driveno"])*2,
                DRV="V",
                TRIG1="U3IV:AdiVGblPvr.PROC",
              ),
        ),
  )

The macro "AXNO" has now twice the value of the macro "driveno".

If the macro value to be calculated does not access the global "macros" variable, it may be calculated before the execution of the function "do" returns. In this case we do not have to use "lambda" and everything looks a bit more straight forward:

drives= ["V","H"]
def letter(idx):
    return(chr(ord("A")+idx))

do(
    file("test.db",
         var(
           DESCVAR="adimovhgbl_ins.mac",
           GBASE="U3IV:",
           AdiMopVer="9"
         ),
         [ subst(
                  DRV=drives[i],
                  TRIG1="U3IV:Adi%sGblPvr.PROC" % drives[i],
                  TRIG2="U3IV:Adi%sGblMvr.PROC" % drives[i],
                  INP1 ="U3IV:BasePmGap.%s" % letter(i),
                ) for i in range(0,len(drives))
         ],
        ),
  )

We first define a function "letter" that converts a number to a character, it returns "A" for 1, "B" for 2 and so on. Below in the substitution part we use this function to calculate the value of the macro "INP1". This could be used for example, it we want to interface fields of a subroutine record, where we have fields "A","B","C" and so on.

extending the pymsi language

We can even extend the pymsi language. In the following example we add a "files" statement. It does almost the same as "file" but substitutes the macros in a list of files:

def files(filelist, *funcs):
    return [ file(f, *funcs) for f in filelist ]

do(
    files(("test.db","test2.db"),
         var(
           DESCVAR="adimovhgbl_ins.mac",
           GBASE="U3IV:",
           AdiMopVer="9"
         ),
         [ subst(
                  DRV=d,
                  TRIG1="U3IV:Adi%sGblPvr.PROC" % d,
                  TRIG2="U3IV:Adi%sGblMvr.PROC" % d,
                ) for d in ["V","H"]
         ],
        ),
  )

In this example we add an "incr" function that increments a macro value (assumed that it can be interpreted as an integer). Since we access the "macros" object, we have to use a function as value. "**" unpacks the dictionary to named arguments which the "var" function expects:

def incr(macname):
    return var(** {macname: lambda: int(macros[macname])+1})

do(
  file("t-macros.db",
       var(NO=0),
       subst(
             NAME="U3IV:Pos1",
             DTYP="lowcal",
             OUT="@f C 2 3 c7 87 12 10 1f4 0",
            ),
       incr("NO"),
       subst(
             NAME="U3IV:Pos1",
             DTYP="lowcal",
             OUT="@f C 2 3 c7 87 12 10 1f4 0",
            ),
       incr("NO"),
       subst(
             NAME="U3IV:Pos1",
             DTYP="lowcal",
             OUT="@f C 2 3 c7 87 12 10 1f4 0",
            )
      )
  )

Each 'incr("NO")' statement increments the variable "NO" before the next expansion of the template file, so "NO" runs from "0" to "2" in this example.

Building and testing

Before you install pymsi on your system in the appropriate directories, you may want to test it. This section describes how.

First you have to specify the environment variable "EPICS_BASE" that describes where your local EPICS base installation can be found. Secondly EPICS_HOST_ARCH should be set, that is needed in order to find the EPICS libraries.

You can pass these variables to setup.py on the command line.

For testing, we install everything in a directory named "TEST". If you want to install this module properly on your system, you should use a different directory, the next section contains a description how to do this.

For now, we create "TEST" and install pymsi there:

export PYMSITESTDIR=TEST
mkdir $PYMSITESTDIR
EPICS_BASE=[your EPICS base path] EPICS_HOST_ARCH=[your EPICS host arch] \
python setup.py build_ext install --home=$PYMSITESTDIR

Now you have to adapt your PYTHONPATH and PATH variable in order to find the pymsi libraries and scripts:

export PYTHONPATH=`pwd`/$PYMSITESTDIR/lib/python:$PYTHONPATH
export PATH=`pwd`/$PYMSITESTDIR/bin:$PATH

Now you can test msi.py with your epics application. Change to the directory with your application. The application should already be built. Now look for a substitution file:

find . -name '*.substitutions'

msi.py substitution files have a different format. The are in fact python program code, so you have the full power of the python scripting language in the substitution file. Substitution files in the old format can be converted with the msi-convert.py program. It's output should in most cases be suitable for msi.py. However there may be cases where some editing of the generated file is needed. Please contact me if this happens, I will then adapt msi-convert.py to cover these cases, too.

For now we want to see what msi-convert.py generates from our substitution file:

msi-convert.py -S [substitution-file] | less

The generated output is in fact a call to a function "do". This function simply executes all it's arguments, which must be functions. All other functions here, like "file" or subst" return functions or lists of functions. So without the enclosing "do" nothing would happen. You should, however, recognize your substitution file since msi-convert.py does not change the formatting.

Now we can use the generated output to feed it to msi.py, don't be confused by the error message that follows now, I will explain it in the next paragraph:

msi-convert.py -S [substitution-file] | msi.py -S -

You probably see a kind of stack trace (actually the output of a python exception) with "AssertionError: file "xxxx.template" not found" at the end. You have to provide msi.py, like the original msi, with paths where to find the template files. Enter this command:

find . -name "[template-filename-from-the-error-message]"

Select the correct path where to find the template file and add it to the call to msi.py:

msi-convert.py -S [substitution-file] | msi.py -I [path where to find templates] -S -

If there is still an error message, you have to add a second or third path, with a new "-I" option. Your call now could look like this:

msi-convert.py -S [substitution-file] | msi.py -I [path1] -I [path2] -S -

After all needed paths are given on the command line, msi.py creates a database file, just like the original msi does.

This is just a proof that msi.py could be used with your project. In order to use it in the intended way, you would have to convert all your substitution files with msi-convert.py only once, then change your build rules to call msi.py instead of msi.

Building and installing

pymsi uses the distutils package of python. Simply call setup.py with the appropriate parameters in order to install the package. The generic documentation how to do this is here:

http://docs.python.org/install/

Before you install the package I recommend you that have a short look at this site.

If you simply want to place the scripts in one directory and everything else into another, this is the way how to do it:

We assume that the scrips should be installed in [scriptdir] and the python libraries should be installed in [libdir], then the call to setup.py should be like this:

python setup.py build_ext install --install-scripts [scriptdir] \
       --install-purelib [libdir] --install-platlib [libdir]

Scripts

Substitution files could be, in principle, valid python scripts. All the functionality is there but the user has to program python. But the specification for example of file search paths by function calls in the substitution file is a bit clumsy and inflexible.

In order to make pymsi easy to use, msi.py and msi-convert.py are part of the package. The options of msi.py are mostly compatible with msi and msi-convert.py can be used to convert old-style substitution files to the new format.

Here is a short description of the two scripts:

msi.py

This script can replace msi. It performs macro substitutions as specified in the substitution file and writes the result to the console or a new file.

Here is a link to the command line options of msi.py.

msi-convert.py

This script converts substitution files from the old format to the new one.

Here is a link to the command line options of msi-convert.py.

Internals

This section describes how pymsi works.

pymsi components

pymsi consists of these parts:

PyMacLib
A python interface to the EPICS maclib.
msilib
A python module that implements all the functions needed to interpret substitution files.
msi.py
A python script that implements command line options for search paths and file names which calls "execfile" on the substitution file. The interpretation of the substitution file is done by the python interpreter.
msi-convert.py
This is a helper script than converts old-style substitution files to the new form.

msi.py

Substitution files in pymsi are valid python expressions. If you would add "import msilib" and "add_path([searchpaths]) at the top of these files you could directly execute them in python.

msi.py is just a utility that does this for you. It imports msilib and defines searchpaths according to the given command line parameters. Then the given substitution file is executed in python by a call to the python "execfile" function.

It also has support for parsing standard-in and for writing the output to a file instead of the console, but all of this could also be performed by adding some more statements to the substitution file and interpreting it directly with python.

Notable is the "--dry-run" option of msi.py. It just prints to the console the set of macros for each template file that would be used to expand it.

Here is a link to the command line options of msi.py.

msi-convert.py

This is a tool to convert substitution files from the old msi format to the new pymsi format. It is not a complete substitution parser but uses regular expression matching to find and replace some parts in the substitution file. This has the advantage that the formatting of the file, whitespaces and newlines, are mostly left intact. The disadvantage is that the generated output may in some cases not be valid for pymsi where it has to be edited by hand.

Here is a link to the command line options of msi-convert.py.

msilib.py

This is the library that defines all the functions that you use in your substitution files. Functions like "do", "file" or "subst" are defined here. If you have a look at the source of this file you will see that most functions are very short. This is due to the fact that the actual macro substitution is done by the PyMacLib python module.

The concept here is that all functions that define the pymsi language return, when executed, just functions or lists of functions. This makes the implementation very easy and, as a side effect, indentation in the substitution file doesn't matter. Python has no rules on how to indent parameters of a function. At the end, of course, somehow the generated functions have to be called anyway, this is done by the "do" function. It calls all functions and functions in lists of functions it has been given.

Most generated functions call the pushScope() and popScope() functions of PyMacLib. This means that macro definitions within the scope have no effect outside the scope. These are the scoped functions:

  • do
  • scope
  • file
  • subst

"pattern" returns a list of functions that are generated by "subst", but it is not directly scoped. "var" is not scoped at all.

"var" and "subst" use named python parameters in order to define macros. The advantage of this is that the macro names do not have to be enclosed in quotes, the disadvantage is that the macro names have to be valid python identifiers.

For the macro definitions, msilib defines a single object of the class PyMacLib.Macros, named "macros". You could, in principle, call member functions of this object directly in your substitution file, but this is not recommended. However, if you want to extend the pymsi language, you will probably have to deal with the "macros" object.

Here is a link to the embedded documentation of msilib.

PyMacLib

This python module is an interface to the EPICS maclib, which is part of libCom. This EPICS library provides functions to define macros, scopes and to substitute macros in strings. PyMacLib uses SWIG to interface this C-Library. It also defines a class "Macros" that encloses all of the functionality of maclib. See also PyMacLib.i, at the end of the file is the definition of the class "Macros" and some embedded documentation.

Since maclib only supports strings as values of macros, the values of macro definitions are always converted to strings using the python function "str".

When maclib is initialized with macCreateHandle, it returns a handle to the new created macro structure. In PyMacLib, this handle is stored in the Macros object. All member functions of the Macros object use that handle.

Since the class "Macros" implements the __setitem__ and __getitem__ methods, the object can be used similar to a dictionary:

m= PyMacLib.Macros()
m[macname]= "value"
if m[macname] == "value":
  print "OK"

Here is a link to the embedded documentation of PyMacLib.

Speed

msi.py is slower than the original msi tool. This has to be the case since most of the functionality is implemented in a scripting language. Here is the result of a measurent of msi.py with a real EPICS application.

The source substitution file has 3964 lines. The "file" statement is listed 61 times, 49 different template files are used. According to the "cProfile" python profiler 18784 lines are processed from all the template files. On a Intel(R) Core(TM)2 Duo CPU, E8200 with 2.66GHz and on a local filesystem the program used about 0.25 CPU seconds. This is the command line that was used to do this measurement:

python -m cProfile ~/net/project/pymsi/TEST/bin/msi.py \
       -I ./idcpApp/collector/O.Common \
       -I ./idcpApp/protocols/scripts/O.vxWorks-ppc603 -S T.subst -o DELME