|Powered by QM on a Rpi server|
KnowledgeBase 00107: Virtual File System Changes at Release 3.2-0
QM release 3.2-0 includes a change to the Virtual File System (VFS) that is incompatible with previous releases and will require VFS users to make a small change to their VFS handler programs. This article sets out the reasons for this change and the modifications that are required to VFS handler programs.
Why the Change?
The VFS allows users to access files in external file systems using the same QMBasic file handling operations as for QM files. In typical usage this may require the VFS handler program to supply authentication details when connecting to the external file system. In past releases, the username and password were hard-coded into the VFS handler or derived programmatically within the handler.
There has been concern raised that this is not an ideal situation. Firstly, the authentication details may be visible to users who should not have access to them, weakening system security. Secondly, without additional coding, all users of the local QM system will use the same authentication details for the remote system whereas it would be better if the remote authentication details were dependent on the identity of the QM user on the local system.
Both of these issues were present in early versions of the QMNet subsystem. QM release 2.10-5 (September 2010) enhanced the user authentication mechanism for QMNet but did not address the similar issues in the VFS.
In addition, the previous implementation of the VFS led to a situation in which changes to the connection details for a remote server could require many VOC entries to be updated.
Defining VFS Servers
From release 3.2-0, VFS server connection data is stored in the same database as QMNet connection data. In addition to the server address, username and password used for QMNet style definitions, a server name is linked to a specific VFS handler program. When using a external VFS handler, the name of the handler program should be entered with a prefix of EXT and a space. As in previous releases, the actual program name of an external VFS handler in the bin subdirectory of the QMSYS account has a prefix of vfs_ added internally by QM when starting the program.
VFS server definitions are created or modified using the SET.VFS.SERVER command. Because this command does not allow null entries for the network address and username, a dummy value such as "none" must be entered when defining a VFS handler that does not require this data.
The LIST.SERVERS command shows VFS server definitions in a separate section after QMNet servers.
VFS VOC Entries
The format of an F-type VOC entry that references a VFS file has been changed such that the handler name and network address are now replaced by the server name used in the SET.VFS.SERVER command. For example, the published VFS handler for accessing U2 files via the InterCall API might result in the VOC entry below.
1: F 2: VFS:EXT U2:DEMO:ORDERSThis references a VFS server definition named U2 and passes the remainder of the definition into the VFS handler as the qualifying data string. In this case, they are the account name and filename.
The External VFS Handler
The v_open() function in an external VFS handler changes to be declared as
static int v_open(int file_id, /* Unique id for this file */ char * server, /* Server network name or IP address */ char * username, /* Server username... */ char * password, /* ...and password */ int port, /* Server port number */ char * qualifier) /* Qualifying data from VOC entry */Note that there are also changes in the standard parts of the VFS handler to support this revised interface. It is recommended that users reintegrate their own coding with the template handler.
The Internal VFS Handler Class Module
The V$OPEN function is revised to be declared as
public function v$open(server, username, password, port, qualifier)There are no changes required elsewhere in this VFS handler.
Reducing File Alias Errors
In past releases, the internal QM file table that tracks all open files used the pathname string in the VOC as the filename, for example,
VFS:EXT U2:172.16.0.53:DEMO:ORDERSThis led to a potential problem if another VOC entry referenced the same file as, for example,
VFS:EXT U2:DEV:DEMO:ORDERSwhere DEV was the network name that resolved to 172.16.0.53. Although both VOC items reference the same file, QM would have no knowledge of this which could lead to locking issues.
Because the file table entry now references the server name instead of the handler name and network address, the risk of aliased names is reduced. It is, however, still important that any one handler/connection combination is only ever referenced as a single VFS server name.
Extended Filename Syntax
The changes that support the revised VFS definition described above also allow the introduction of a new extended filename syntax,
VFS:server:qualifierwhich, in the context of the earlier example, would allow use of
VFS:U2:DEMO:ORDERSBecause some VFS handlers may need to be able to process filenames that contain spaces (e.g. DICT name), any tilde characters (~) in the qualifier will be replaced by spaces.
In a similar manner to other extended filename syntaxes, the availability of this feature is controlled by additive value 8 in the FILERULE configuration parameter.
The VFS Cache
To improve performance of programs that repeatedly open and close the same files (as may happen with use of TRANS() or T-conversions), there is now an optional VFS file cache. If the cache size is set to a non-zero value using the VFSCACHE private configuration parameter, closing a VFS file does not actually cause it to be closed on the remote server, instead moving the file variable into the cache, possibly displacing some other file if the cache size limit has been reached.
When opening a VFS file, the cache is searched and, if the file is found, the file variable is restored rather than opening a new file.
The cache is automatically flushed on return to the command prompt or by use of the FLUSH.DH.CACHE QMBasic statement (so called because originally it only affected the dynamic hashed file cache).