The System Editor contains a menu setting called Minimum Delimiter Value. The possible settings are shown below:
In the OpenInsight implementation, we have the extremely critical criteria of the support of the Pick style system delimiters (@fm, @vm, @svm, @tm, etc.) in a 100% backwards-compatible manner. This was considered such an important criteria that if it could not be met, then there was no point in even attempting Unicode support. After investigating all the different ways system delimiters are used throughout by the OpenInsight system code and by end developers, there was no getting around the fact that system delimiters are the specific, single byte values Dick Pick declared them to be. There are just too many places where these system delimiter values are hard coded as literals and then used against data in many different ways. Another criteria was not to create an implementation that was specific to Unicode as implemented by Windows NT. The 16-bit wide Unicode character format has already been superceded by a 32-bit wide character format in the latest Unicode specifications.
Our UTF8 implementation has met both these criteria. Both the system code and your code can continue to operate on all of your data in the same exact manner it has always done, as strings of single-byte values with the Pick system delimiters continuing to provide the organization of the data. In addition, we are in position to support the Unicode implementations of other operating systems.
Originally, the conversion routines assumed that the high 16 ANSI characters were all "system delimiters". This means that char(240) through char(255) were maintained as that single byte sequence. It also meant that when they were displayed in a Unicode enabled windows, that this range (0xF0-0xFF) was converted to Unicode 0XFFF0 through 0xFFFF. We then realized that we needed to add a feature to the conversion routines such that they can specify what the lowest system delimiter is that should be recognized.
Therefore, the System Editor now features a Minimum Delimiter Value submenu under the View menu, where the minimum value to be used as the System Delimiter (from amongst the last 16 values listed above) can be selected, or no System Delimiter can used at all. The default System Delimiter is "@STM" (char(250)). Once a Minimum System Delimiter has been selected, OpenInsight checks if any ANSI value being used is a System Delimiter, and if it is, it stays that value. If it is not a System Delimiter, it takes the 8-bit ANSI value and converts it to a 16-bit Unicode value by using the current code page in effect on that machine. It then converts that 16-bit Unicode value to the appropriate UTF8 byte sequence. The UTF8 character sequence will be between 1 and 3 bytes long.
