|Powered by QM on a Rpi server|
KnowledgeBase 00010: Object Code Management
This article describes the principles of the mechanisms used to manage object code within QM.
QM uses a virtual machine architecture in which the source program is translated to an instruction set that is then executed by the Run Machine, part of the main QM executable program. This technique enables programs to be moved between different processor types without the need to recompile. It also produces very small programs by comparison with systems in which the program is compiled to the native instruction set of the machine.
The instruction set and the architectural details are not published. There is no decompiler for QM object code and the terms of the software licence preclude development of such a tool by users.
The RUN Command
The simplest way to run a program is by use of the RUN command. The object code is loaded into memory and execution commences. Internally, the RUN command always resolves the name of the program into an operating system pathname.
QM supports three modes of cataloguing; local, private and global.
Local cataloguing is performed using the LOCAL keyword to the CATALOGUE command. A VOC entry is added to point to the compiled program in the compiler output file. If the program is recompiled, there is no need to repeat the CATALOGUE command as the pointer is already in place. If, however, the program is recompiled and has errors, live users of the application would follow the pointer to the now faulty or, perhaps, non-existent object code.
Private cataloguing is performed by using the CATALOGUE command with no mode qualifier. The program is copied to the account's private catalogue directory (cat). There is no VOC pointer from a privately catalogued program. If the program is recompiled, the CATALOGUE command must be repeated to copy the new version to the private catalogue directory. A failed compilation will not affect live users who will continue to use the previous version.
Global cataloguing makes a subroutine available to all accounts and is achieved by using the GLOBAL keyword to the CATALOGUE command or by adding on of the reserved prefix characters listed below to the program's catalogue name. The compiled program is copied to the global catalogue directory (gcat under the QMSYS account). There is no VOC pointer for a globally catalogued item. As with private cataloguing, it is necessary to repeat the CATALOGUE command whenever the program is rebuilt.
For compatibility with other database systems, QM supports a set of reserved prefix characters for a subroutine name that imply that it is in the global catalogue. These are:
Programs with the same name can exist in all three catalogue areas. The search process checks for a program in the order local, private, global unless the name being located commences with one of the reserved global catalogue prefix characters.
Catalogued programs may be executed from the command prompt by typing their catalogue name.
Programs may be catalogued automatically after compilation by including the $CATALOGUE compiler directive in the program or by use of the CATALOGUE element of the $BASIC.OPTIONS record.
The Object Cache
To optimise performance, QM caches object code. When a program or subroutine exits, the object code is not discarded. Instead it is retained in memory cache on the basis that it is likely to be called again fairly soon.
When a request is made to load a program, the object code manager searches the cache to see if the desired program is already in memory. If so, it is moved to the head of the cache. If not, the program is located via the catalogue system or, for a run file, from its operating system file, loaded into memory and added to the head of the cache.
By using the cache, QM can minimise the number of times that programs must be loaded into memory. Under normal operating conditions, all inactive programs are removed from the cache when the process returns to the command prompt. Configuration options described later allow earlier removal of cached programs to reduce memory requirements at the expense of a possible performance degradation caused by the need to reload the program. Programs to be removed are taken from the end of the cache list and are hence the least recently used items.
In a development environment it is important that if one user recompiles a program, another user ceases to use a cached copy of the previous version of the program. To achieve this, the BASIC and CATALOGUE commands both cause all processes to flush inactive programs from their object cache.
How CALL works
The previous sections have used the phrase "inactive programs" several times. So, what is an inactive program? Perhaps it is easier to describe an active program.
The QMBasic language supports two styles of CALL, direct and indirect. An indirect call references a variable that contains the name of the program or subroutine to be called. When the CALL is first executed, the cache and catalogue search mechanisms described above are used to locate the program, loading it into memory if it is not already there. The character string variable holding the subroutine name is then transformed into a subroutine reference variable which contains a pointer to the subroutine in memory. This variable can still be referenced as a string but any operation that changes it will convert it back to a string or other data type, effectively removing the reference to the subroutine. It can therefore be seen that effective use of an indirect calling mechanism should only overwrite the indirection variable when the name of the referenced subroutine is to be changed. Placing the indirection variable in named common may give best performance. The DELETE.COMMON command can be used to remove the variable if necessary.
A direct call is actually compiled as an indirect call using a local hidden indirection variable.
A program is "active" so long as there is at least one subroutine reference variable pointing to its memory image. As soon as the final one is destroyed, the program becomes inactive and is a candidate for removal from memory.
The object code management system is affected by two configuration parameters. These take their initial value from the configuration file but can be changed within a QM session using the CONFIG command.
The OBJECTS configuration parameter restricts the number of inactive programs that will be retained in cached memory. The OBJMEM configuration parameter restricts the total size of inactive programs that will be retained in cached memory, specified in multiples of 1kb. For example, a value of 1024 would sets a 1Mb limit. For both parameters, a value of zero implies no limit.
When a program that is not already in memory is required to be loaded, the system checks whether the total number of loaded programs exceeds the value of the OBJECTS parameter. It also checks whether the total size of the loaded programs exceeds the value of the OBJMEM parameter. If either condition is met, the inactive programs that have not been referenced for longest are discarded to bring the cache below the limit set by these parameters.
Flushing the Object Cache
For most systems, the caching mechanism described above is adequate. An inactive program will be flushed from the cache if it is recompiled or recatalogued.
In some situations, particularly in development environments, it may be desirable to force reload of all programs, including active ones, when the program is recompiled or recatalogued. This can be achieved by setting the FORCE.RELOAD mode of the OPTION command in the process that is to have object code loaded in this way (which is probably not the process in which the compilation or catalogue operation was performed).
The effect of setting this option is that when a process is notified that a program has been recompiled or recatalogued, all subroutine links within that process are discarded, forcing reload of the object code on the next call to the subroutine. Note that this reloads all subroutines, not just the one that has been updated. Use of this option carries the risk that a simple loop that calls a subroutine may call a different version of the routine on successive iterations of the loop.