KnowledgeBase 00106: Handling Binary Data in QM

This article was originally published as a Tip of the Week.

Introduction

Historically, multivalue environments have often required programmers to use special encodings when storing binary data such as transforming the data to a stream of hexadecimal character values. This is because, for example,

• Some systems follow the C programming convention and use the ASCII null character (character 0) as a string terminator internally.
• Some systems cannot store data containing the item mark (also known as the record mark or segment mark).
• The media format used by the Pick T.DUMP and ACCOUNT.SAVE tools reserves sequences such as an item mark followed by a text mark for internal use.

Binary Data in the QM File System

QM does not reserve any special character sequences within the file system. It is therefore possible to save binary data such as scanned images without any encoding. When using directory files, it is important that the file is used with mark mapping disabled so that the default action of treating field marks in the data as newlines in the file is suppressed.

QM itself makes use of the ability to store binary data. Compiled QMBasic programs are normally stored in directory files. C/I-type dictionary items and A/S-type items containing correlatives contain embedded object code.

QM provides a limited emulation of the Pick data save tools. It is important to remember that these may not be able to save all data constructs found in QM because they must adhere to the Pick media format definition.

Use of Null Terminators

Although the QM file system is "binary clean", the underlying operating system requires use of null terminated strings in some places. For example, a filename cannot include the ASCII null character.

QMClient

The QMClient API for Visual Basic is binary clean in that it handles strings using the BSTR data type which has a length associated with the actual string data. Unfortunatly, Microsoft have modified Visual Basic to use null terminators internally so the API is effectively no longer binary clean.

The C version of the QMClient API, also used by several wrappers for other languages, has never been binary clean as it was designed to work with null terminated C strings.

Related Articles

None.