GT.M V5.3-003 Release Notes

Release Notes: GT.M 5.3-003

Legal Notice

GT.M™ is a trademark of Fidelity National Information Services, Inc.

Unicode™ - "Unicode is a trademark of Unicode, Inc."

Unicode® - "Unicode is a registered trademark of Unicode, Inc."

GT.M and its documentation are provided pursuant to license agreements containing restrictions on their use. They are the copyrighted intellectual property of Fidelity National Information Services, Inc. and Sanchez Computer Associates, LLC (collectively "Fidelity") and are protected by U.S. copyright law. They may not be copied or distributed in any form or medium, disclosed to third parties, or used in any manner not authorized in said license agreement except with prior written authorization from Fidelity.

This document contains a description of Fidelity products and the operating instructions pertaining to the various functions that comprise the system. It should not be construed as a commitment of Fidelity. Fidelity believes the information in this publication is accurate as of its publication date; such information is subject to change without notice. Fidelity is not responsible for any inadvertent errors.

October 15, 2010

Revision History
Revision 1.1October 15, 2010
In C9I09-003044, changed LUS to MLG and fixed an incorrect LOCK example.
Revision 1.0December 15, 2008
First published version.

Contact Information

GT.M Group
Fidelity National Information Services, Inc.
2 West Liberty Boulevard, Suite 300
Malvern, PA 19355
United States of America

GT.M Support: +1 (610) 578-4226
Switchboard: +1 (610) 296-8877
Fax: +1 (484) 595-5101
Website: http://fis-gtm.com
Email: gtmsupport@fnis.com


Table of Contents

Conventions Used in This Document
Bulletin Overview
Platforms
Migrating to 64-bit platforms
Recompile
Rebuild Shared Libraries or Images
Additional Installation Instructions
Database Compatibility for GT.M V5.3-003
Managing M mode and UTF-8 mode
Setting the environment variable TERM
Installing Compression Libraries
Change History
V5.3-003
V5.3-000
M-Database Access
M-Other Than Database Access
Utilities- MUPIP
Utilities-Other Than MUPIP
Error Messages
BACKUPKILLIP
DBCOMMITCLNUP
KILLABANDONED
LOGTOOLONG
MMNOBFORRPL
REPLTRANS2BIG

Conventions Used in This Document

Command Syntax: UNIX syntax (that is, lowercase text and "-" for flags/qualifiers) is used throughout this document. OpenVMS accepts both lowercase and uppercase text; flags/qualifiers on OpenVMS should be preceded with "/".

Reference Number: The reference numbers used to track software enhancements and customer support requests appear in parentheses ().

Platform Identifier: If a new feature or software enhancement does not apply to all platforms, the relevant platform or platforms appear in brackets [ ].

Bulletin Overview

GT.M V5.3-003 provides support for a new PIPE device on the UNIX/Linux platforms as a mnemonic space device. An OPEN of the device starts a sub-process. Data written to the device by the M program is available to the process on its STDIN.  The M program can read the STDOUT and STDERR of the sub-process. This facilitates output only applications, such as printing directly from a GT.M program to an lp command; input only applications, such as reading the output of a command such as ps; and co-processing applications, such as using iconv to convert data from one encoding to another.  Refer to the PIPE IO Technical Bulletin for details. [UNIX] (C9H05-002859)

The release more comprehensively addresses the UNIX/Linux platforms security issues previously identified and patched in the Security Bulletin date 23 October 2008. For details, see S9I09-002700.

The release provides an interface for UNIX/Linux platforms so you may provide a compression library for the replication facility, which can reduce bandwidth requirements and improve performance. For details, see C9I08-003020.

The release implements $VIEW("GVSTATS" <region>,) to report process specific operational data. For details, see C9I09-003044.

For a comprehensive list of the fixes and enhancements in this release, see section Change History.

Platforms

As of the publication date, FIS supports this release on the following hardware and operating system versions. Contact FIS for a current list of supported platforms.

Platform

Supported Versions

Notes

Hewlett-Packard Integrity IA64 HP-UX

11V3(11.31)

-

IA64 GNU/Linux - Red Hat Enterprise Linux

Red Hat Enterprise Linux 5.2

GT.M should also run on recent releases of other major Linux distributions with a contemporary 2.6 Linux kernel, glibc (version 2.5-24 or later) and ncurses (version 5.5-24 or later). We have verified that GT.M passes comprehensive testing on RHEL 5.x on machines that have single cells (no more than 8 CPUs). Multi-cell machines are not considered viable until they are certified.

Hewlett-Packard HP-PA HP-UX

11.11

GT.M supports UTF-8 mode and M mode on this platform subject to the following:

[Tip]

While GT.M should support both UTF-8 mode and M mode on this platform, there are problems with the version of HP-UX that we currently use for supporting HP-UX that prevent FIS from testing 4 byte UTF-8 characters in UTF-8 mode on the HP PA-RISC HP-UX platform. FIS understands that the issue is resolved in HP-UX 11.31. At this time, HP-UX 11.31 is not formally supported (as it is untested); however, there is no identified reason that would prevent GT.M V5.2-000 (and above) from working correctly on that newer version.

Running GT.M on HP-UX 11i requires that patch PHKL_28475 be applied to the system. This patch fixes a problem with the lseek64() C library call that GT.M uses. A system without this patch gives fairly consistent database errors of varying types, structural damage, and in general does not work correctly for any but the most simplistic usage. The "swlist -p" command (as root) can be used to determine if this patch has been applied. Note that recent "BATCH" and "GOLDEN" patches may contain this patch therefore your system may already have this patch applied but may not list it separately. Contact an HP service representative for more information.

Hewlett-Packard Alpha/AXP Tru64 UNIX

5.1B

GT.M supports M mode but not UTF-8 mode on this platform.

Hewlett-Packard Alpha/AXP OpenVMS

7.3-1/7.3-2/8.2

GT.M supports M mode but not UTF-8 mode on this platform.

Although OpenVMS 8.3 remains an officially unsupported release until GT.M is tested on it, we are not aware of any reason that GT.M V5.3-003 would not run on it.

If you need to work with external calls written in C with Version 6.x of the Compaq C compiler on Alpha OpenVMS, then you must carefully review all the provided kits for that product and apply them appropriately.

IBM eServer pSeries AIX

5.2/5.3

Since GT.M processes are 64-bit, FIS expects 64-bit AIX configuration to be preferable.

Although AIX 5.2 and 5.3 are the officially supported releases, we are not aware of any reason why GT.M V5.3-003 will not run on AIX 6.x.

[Tip]

While GT.M supports both UTF-8 mode and M mode on this platform, there are problems with the AIX ICU utilities that prevent FIS from testing 4-byte UTF-8 characters as comprehensively on this platform as we do on others.

Sun SPARC Solaris

9 (Update 3 and above) and 10

GT.M supports the deprecated DAL calls in M mode but not in UTF-8 mode. Please refer to the Integrating External Routines chapter in the Programmer’s Guide for appropriate alternative solutions.

x86_64 GNU/Linux

Red Hat Enterprise Linux 5.2; Ubuntu 7.10 (Gutsy Gibbon) and 8.04 (Hardy Heron)

To run 64-bit GT.M processes requires both a 64-bit kernel as well as 64-bit hardware.

GT.M should also run on recent releases of other major Linux distributions with a contemporary 2.6 Linux kernel, glibc (version 2.5-24 or later) and ncurses (version 5.5 or later).

x86 GNU/Linux

Red Hat Enterprise Linux 4

This 32-bit version of GT.M runs on either 32- or 64-bit x86 platforms; we expect the X86_64 GNU/Linux version of GT.M to be preferable on 64-bit hardware.

GT.M should also run on recent releases of other major Linux distributions with a contemporary 2.6 Linux kernel, glibc (version 2.3.4-2 or later) and ncurses (version 5.4-1 or later). The minimum CPU must have the instruction set of a 686 (Pentium Pro) or equivalent. Contact FIS for support of older CPUs.

Migrating to 64-bit platforms

The same application code runs on both 32-bit and 64-bit platforms. Please note that:

  • You must compile the application code separately for each platform. That is, although the M source code is exactly the same, the generated object modules are different.

  • Parameter-types that interface GT.M with non-M code using C calling conventions must match the data-types on their target platforms. Mostly, these parameters are for call-ins, external calls, internationalization (collation), and environment translation and are listed in the tables below. Note that most addresses on 64-bit platforms are 8 bytes long and require 8 byte alignment in structures whereas all addresses on 32-bit platforms are 4 bytes long and require 4-byte alignment in structures.

Call-ins and External Calls

Parameter type

32-Bit

64-bit

Remarks

gtm_long_t

4-byte (32-bit)

8-byte (64-bit)

gtm_long_t is much the same as the C language long type, except on Tru64 UNIX, where GT.M remains a 32-bit application.

gtm_ulong_t

4-byte

8-byte

gtm_ulong_t is much the same as the C language unsigned long type.

gtm_int_t

4-byte

4-byte

gtm_int_t has 32-bit length on all platforms.

gtm_uint_t

4-byte

4-byte

gtm_uint_t has 32-bit length on all platforms

[Caution]

If your interface uses gtm_long_t or gtm_ulong_t types but your interface code uses int or signed int types, failure to revise the types so they match on a 64-bit platform will cause the code to fail in unpleasant, potentially dangerous and hard to diagnose ways.

Internationalization (Collation)

Parameter type

32-Bit

64-bit

Remarks

gtm_descriptor in gtm_descript.h

4-byte

8-byte

Although it is only the address within these types that changes, the structures may grow by up to 8 bytes as a result of compiler padding to meet platform alignment requirements.

[Important]

Any collation routines require only a recompile, assuming other aspects of their code are 64-bit capable.

Environment Translation

Parameter type

32-Bit

64-bit

Remarks

gtm_string_t type in gtmxc_types.h

4-byte

8-byte

Although it is only the address within these types that changes, the structures may grow by up to 8 bytes as a result of compiler padding to meet platform alignment requirements.

[Important]

Any environment translation routines require only a recompile, assuming other aspects of their code are 64-bit capable.

[Note]

Although any existing M code must be compiled for the 64-bit environment, the source is unchanged from 32-bit environments.

Recompile

  • Recompile all M and C source files.

Rebuild Shared Libraries or Images

  • Rebuild all Shared Libraries (UNIX) or Shareable Executable Images (OpenVMS) after recompiling all M and C source files.

Additional Installation Instructions

For installing GT.M, see the "Installing GT.M" section in the GT.M Administration and Operations Guide.

UNIX

  1. Fidelity strongly recommends installing each version of GT.M in a separate (new) directory, rather than overwriting a previously installed version. If you must overwrite an existing GT.M installation with a new version you must first shut down all processes using the old version. Since the the Linux Assigned Names and Numbers Authority (http://lanana.org) has assigned GT.M the package name lsb-gtm, FIS recommends installing GT.M V5.3-003 in /opt/lsb-gtm/V5.3-003 on Linux systems.

  2. Use the MUPIP RUNDOWN command of the old GT.M version to ensure all database files are cleanly closed.

  3. In UNIX editions, make sure gtmsecshr is not running. If gtmsecshr is running, first stop all GT.M processes including the DSE, LKE and MUPIP utilities and then perform kill <pid of gtmsecshr >.

    [Caution]

    Never replace the binary image on disk of any executable file while it is in use by an active process. It may lead to unpredictable results depending on the operating system. These results include but are not limited to the denial of service (that is, system lockup) and damage to files that these processes open (that is, database structural damage).

Additional Information for AIX

GT.M for IBM pSeries AIX requires the Asynchronous IO facility to be available and configured. Before installing GT.M on IBM pSeries AIX, run the following command to check the filesets of this facility:

  • lslpp -l bos.rte.aio

If there are no filesets, then install them from AIX installation media.

Then, use SMIT to configure the Asynchronous IO facility. Use SMIT in one of the following modes:

  • smit aio (for gui mode) or

  • smitty aio (for text mode)

For system with a "posixaio" option instead of "aio", use SMIT in one of the following modes:

  • smit posixaio (for gui mode) or

  • smitty posixaio (for text mode)

  • Select "Configure AIO now" which gives a message such as "aio0 has been created". This means that there is no further need of setup at this time.

In addition to configuring the aio0 device, select "Change/Show characteristics of Asynchronous I/O" change the value of "State to be configured at system restart" from "defined" to "available". This ensures that the aio0 device is available when you next reboot the system.

If you expect a database file to exceed 2GB, then you MUST configure its file system to permit files larger than 2GB. Alternatively, should you choose to place journal files on file systems with a 2GB limit, since GT.M journal files can grow to a maximum size of 4GB, you MUST then set the journal auto switch limit to less than 2 GB.

OpenVMS

To upgrade from a GT.M version prior to V4.3-001, you must update any customized copy of GTM$DEFAULTS to include a definition for GTM$ZDATE_FORM.

You can ignore the following section if you choose the standard GT.M configuration or answer yes to the following question:

Do you want to define GT.M commands to the system

If you define GT.M commands locally with SET COMMAND GTM$DIST:GTMCOMMANDS.CLD in GTMLOGIN.COM or other command file for each process which uses GT.M, you must execute the same command after installing the new version of GT.M before using it. If you define the GT.M commands to the system other than during the installation of GT.M, you must update the system DCLTABLES with the new GTMCOMMANDS.CLD provided with this version of GT.M. See the OpenVMS "Command Definition, Librarian, and Message Utilities Manual" section on "Adding a system command." In both cases, it is important for each process to match the proper GTMCOMMANDS.CLD with the version of GT.M it runs.

Database Compatibility for GT.M V5.3-003

The GT.M database consists of four types of components:database files, journal files, global directories and replication instance files. Whenever you upgrade to a new GT.M release, you may need to execute certain upgrade procedures on one or all four components to make your database compatible with your new GT.M release. For example, although the database and journal file formats remain unchanged when you upgrade an instance from 32-bit GT.M V5.3-003 x86 Linux to 64-bit GT.M V5.3-003 x86_64 Linux, you need to:

  • Upgrade global directory files.

  • For secondary instances, create new instance files and restart the receiver server with the UPDATERESYNC qualifier.

The following subsections describe the upgrade procedures for each of these 4 components. Whenever you upgrade to a new GT.M release, you should always refer to these subsections to check how your GT.M upgrade impacts each database component and then execute the relevant procedures to make your database compatible with your new GT.M release.

General Principles

  • A change in major release identifier (for example V4 to V5) signifies a change in database block format.  Although FIS tries to minimize the effort involved in upgrading, changes in block format may require special procedures and planning.

  • Changes to the database file header may occur in any release.  GT.M automatically upgrades database file headers as needed.  Any changes to database file headers are upward and downward compatible within a major database release number, that is, although processes from only one GT.M release can access a database file at any given time, processes running different GT.M releases with the same major release number can access a database file.

  • Journal format changes always occur with a major releases numbers, but may occur in any release. Simply create a new backup with fresh journal files. Recovery by one GT.M release, using journal files created by another GT.M release is not supported and is not necessary if you follow recommended procedures.

  • Global directory and instance file changes occur as needed. GDE typically upgrades global directories whenever it opens (and rewrites) an older version.  FIS recommends that you retain readable scripts to recreate your global directories.

  • Instance file formats are recreated with MUPIP when documented in the release notes.  For secondary instances, a new instance file requires the receiver server to be shut down cleanly with the prior GT.M version and restarted with the UPDATERESYNC qualifier the first time using the new GT.M version.

Database Files

From prior to V5.0-000 to V5.3*

To upgrade from a GT.M version prior to V5.0-000, including field test versions prior to V5.0-000, you need to perform a database file upgrade. See Database Migration Technical Bulletin for more information.

[Note]

The global directory in use at the time of database upgrade MUST have a mapping of globals to database files that includes ALL globals that are actually resident in those database files. If you use a global directory that does not map all the global variables in a database file, the database upgrade procedure fails because database certification does not correctly process all the database file. If this happens, a subsequent MUPIP REORG -UPGRADE or a GT.M process can fail with the DYNUPGRDFAIL message after the V5 upgrade.

After you complete the database upgrade procedure, execute the MUPIP REORG -UPGRADE command. This command upgrades all blocks to V5 format and the file header to the current V5.* format.

From V5.0*/V5.1*/V5.2* to V5.3*

No database file upgrade procedure is necessary if you upgrade from GT.M V5.3-000 or later.

Global Directory

Global directory formats differ for 32-bit and 64-bit GT.M platforms. This means that:

  • The global directory format differs between GT.M on x86 GNU/Linux and GT.M on x86_64 GNU/Linux.

  • On AIX, the global directory format differs between GT.M on V5.3-001 and prior versions.

  • On Solaris, the global directory format differs between V5.3-002 and prior versions.

Furthermore, on Itanium platforms, there is a difference in global directory formats between V5.3-000 and V5.3-001.

Except for the above cases, you do not require a global directory upgrade when moving up from GT.M V5.0-000 or later.

Moving up from a GT.M version prior to V5.0-000, from a 32-bit global directory to a 64-bit global directory, or on Itanium platforms migrating from V5.3-000 to V5.3-001 requires a global directory upgrade. Opening a global directory with the GDE utility program from the latest GT.M version (or the 64-bit x86_64 format for that platform), followed by an EXIT command automatically, even with no other intervening commands, upgrades the global directory.

[Note]

FIS strongly recommends you make a copy of any global directory before upgrading it. There is no way to downgrade a global directory to an earlier format.

If you inadvertently open a global directory in an earlier format, with no intention of upgrading it, execute the QUIT command rather than the EXIT command.

If you inadvertently upgrade a global directory, perform the following steps:

  1. Open the global directory with GDE from the current version.

  2. Execute the SHOW ALL command.

  3. Create a command file, or manually enter the commands corresponding to the output, into GDE from the prior GT.M version.

Changes to Journal File Format

The journal file format is changing in GT.M V5.3-003. This means journal files created by GT.M V5.3-003 will not be readable by older versions and vice-versa. Switch journal files right away using MUPIP SET -JOURNAL on all databases immediately after installing this version.

[Important]

Always generate new journal files every time you upgrade to a new GT.M release. Your new version cannot share the journal files of the previous version. Since MUPIP cannot recover database files using journal files from a prior GT.M release, FIS recommends backing up database files when upgrading from one GT.M release to another.

Replication Instance File

If you are running a database replication configuration, then you need to recreate the replication instance file using the MUPIP REPLICATE -INSTANCE_CREATE command whenever your upgrade changes GT.M from a 32-bit implementation to a 64-bit implementation (or potentially vice versa on the x86 platform). If your upgrade activity does not change between a 32- and 64-bit implementation then you do not need to recreate the replication instance file. You have to recreate the replication instance file only for the following upgrade scenarios:

[Important]

Users upgrading from a 32-bit GT.M version to a 64-bit GT.M version need to recreate the replication instance files. This includes upgrades from V5.3-000 or prior versions to GT.M V5.3-001 or later on AIX or 64-bit Linux and upgrades from V5.3-001 or prior versions to GT.M V5.3-002 or later on Solaris. GT.M version upgrades on 32-bit Linux do not need to recreate instance files. After recreating replication instance files for a replication secondary (or tertiary in a multi-site replication environment) always start it with the -UPDATERESYNC qualifier.

Using pre-existing instance files (as opposed to creating new instance files) could cause any process that reads the instance file (which includes the source server, receiver server, update process and GT.M processes on primary) to abnormally terminate with errors ranging from REPLINSTSECMTCH to a SIG-11 (which would create a corefile).

  1. On AIX systems, if you upgrade from 32-bit pre-V5.3-001 to 64-bit V5.3-001/V5.3-001A/V5.3-002/V5.3-003

  2. On Linux systems, if you upgrade from a 32-bit pre-V5.3-001 to 64-bit V5.3-001/V5.3-001A/V5.3-002/V5.3-003 or from a 64-bit V5.3-001/V5.3-001A/V5.3-002 to a 32-bit V5.3-003

  3. On Sun SPARC Solaris, if you upgrade from 32-bit pre-V5.3-003 to 64-bit V5.3-003.

In these three scenarios, your source server process terminates abnormally if you do not recreate the replication instance file. Shut down all receiver servers on other instances looking for updates from this instance, shut down this instance, recreate the instance file and then restart the receiver server on this instance with the -UPDATERESYNC qualifier.

[Note]

The UPDATERESYNC qualifier unconditionally synchronizes this secondary instance with the primary.

Unless you are changing from 32-bit to 64-bit or vice versa, you do not need to recreate the replication instance file. For example, on Linux systems, you do not have to recreate the replication instance file if you upgrade from 32-bit pre V5.3-001 to 32-bit V5.3-001/V5.3-001A/V5.3-002/V5.3-003.

Managing M mode and UTF-8 mode

GT.M requires International Components for Unicode (www.icu-project.org) version 3.6 in order to enable Unicode support. On a system that does not have ICU 3.6 installed, the GT.M installation only provides support for GT.M M mode.

On a system that has ICU installed, GT.M installs support for both M mode and UTF-8 mode, including a utf8 subdirectory of the directory where GT.M is installed. The following paragraph describes the technique that GT.M uses to separate M mode and UTF-8 mode object files.

From the same source file, depending upon the value of the environment variable gtm_chset, the GT.M compiler generates an object file either for M mode or UTF-8 mode. GT.M generates a new object file when an object file is older than the source file and was generated with the same setting of gtm_chset/$ZCHset. GT.M processes trigger an error if they encounter an object file generated with a different setting of gtm_chset/$ZCHset than the current run-time value.

Always generate an M object module with a value of gtm_chset that matches the value that a process executing that module will have. As the GT.M product contains utility programs written in M, their object files must also conform to this rule. In order to use utility programs in both M mode and UTF-8 mode, the GT.M installation ensures that both M and UTF-8 versions of object modules exist, the latter in the utf8 subdirectory. This technique of segregating the object modules by their compilation mode prevents both frequent recompiles and errors in installations where both modes are in use. If your installation uses both modes, consider a similar pattern for structuring application object code repositories.

GT.M is installed in a parent directory and a utf8 subdirectory as follows:

  • GT.M executable programs (mumps, mupip, dse, lke, and so on) are in the parent directory, that is, the location specified for installation.

  • Object files for programs written in M (GDE, utilities) have two versions-one compiled with support for Unicode™ in the utf8 subdirectory, and one compiled without support for Unicode™ in the parent directory. Installing GT.M generates both the versions of object files. Note that GT.M generates the object files for utf8 subdirectory if and only if you have a ICU 3.6 installation on your system. You may have multiple versions of ICU installations on your system but GT.M looks only for ICU 3.6 to generate the object files for utf8 subdirectory even if ICU 3.6 is not the current version.

  • The utf8 subdirectory has files called mumps, mupip, dse, and so on, which are relative symbolic links to the executables in the parent directory (for example, mumps is the symbolic link ../mumps).

  • When a shell process sources the shell scripts gtmprofile or gtmcshrc, the behavior is as follows:

    • If gtm_chset is "m", "M" or undefined, there is no change from the previous GT.M versions to the value of the environment variable gtmroutines.

    • If gtm_chset is "UTF-8",

      • $gtm_dist is set to the utf8 subdirectory (that is, if GT.M is installed in /opt/lsb-gtm/gtm_V5.3-001, then $gtm_dist is set to /opt/lsb-gtm/gtm_V5.3-001/utf8).

      • The last element of $gtmroutines is $gtm_dist($gtm_dist/..) so that the source files in the parent directory for utility programs are matched with object files in the utf8 subdirectory.

Use the following instructions to compile ICU on HP PA-RISC HP-UX:

Version: 11.31 (11iv3)

Compiler: cc HP C/aC++ B3910B A.06.12, aCC HP C/aC++ B3910B A.06.15, GNU Make 3.81

  1. Ensure that system environment variable PATH includes the location of all the compilers mentioned above.

  2. Download the source code of ICU version 3.6 for C from http://icu.sourceforge.net/download/3.6.html#ICU4C

  3. At the shell prompt, execute the following commands:

    gunzip -d < icu4c-3_6-src.tgz | tar -xf - 
    cd icu/source/
    chmod +x runConfigureICU configure install-sh
    runConfigureICU --enable-debug HP-UX/ACC --enable-64bit-libs  --enable-rpath -disable-threads
    gmake
    gmake check
    gmake install
  4. Set the environment variable LD_LIBRARY_PATH to point to the location of ICU. HP-UX uses the environment variable LD_LIBRARY_PATH to search for dynamically linked libraries to be loaded.

ICU is now installed in the /usr/local directory.

[Note]

By default, ICU is installed in /usr/local. If you install ICU in a different directory, type:

runConfigureICU HP-UX/ACC --prefix=<install_path> --enable-64bit-libs  --enable-rpath -disable-threads 

Then execute the gmake commands, and set the environment variable LD_LIBRARY_PATH to point to the appropriate location.

Setting the environment variable TERM

The environment variable TERM must specify a terminfo entry that accurately matches the terminal (or terminal emulator) settings. Refer to the terminfo man pages for more information on the terminal settings of the platform where GT.M needs to run.

[Important]

Some terminfo entries may seem to work properly but fail to recognize function key sequences or position the cursor properly in response to escape sequences from GT.M. GT.M itself does not have any knowledge of specific terminal control characteristics. Therefore, it is important to specify the right terminfo entry to let GT.M communicate correctly with the terminal. You may need to add new terminfo entries depending on their specific platform and implementation. The terminal (emulator) vendor may also be able to help.

GT.M uses the following terminfo capabilities. The full variable name is followed by the capname in parenthesis.

auto_right_margin(am), clr_eos(ed), clr_eol(el), columns(cols), cursor_address(cup), cursor_down(cud1), cursor_left(cub1), cursor_right(cuf1), cursor_up(cuu1), eat_newline_glitch(xenl), key_backspace(kbs), key_dc(kdch1), key_down(kcud1), key_left(kcub1), key_right(kcuf1), key_up(kcuu1), key_insert(kich1), keypad_local(rmkx), keypad_xmit(smkx), lines(lines).

GT.M sends keypad_xmit before terminal reads for direct mode and READs (other than READ *) if EDITING is enabled. GT.M sends keypad_local after these terminal reads.

Installing Compression Libraries

If you plan to use the option compression facility for replication, you must provide the compression library. The GT.M interface for compression libraries accepts the zlib compression libraries without any need for adaptation. These libraries are included in many UNIX distributions and are downloadable from http://www.zlib.net. If you prefer to use other compression libraries, you need to configure or adapt them to provide the same interfaces used by zlib. Simple instructions for compiling zlib on a number of platforms follow.

Solaris/cc compiler from Sun Studio:

./configure --shared 
make CFLAGS="-KPIC -m64"

HP-UX(IA64)/HP C compiler:

./configure --shared 
make CFLAGS="+DD64" 

AIX/XL compiler:

./configure --shared 
Add -q64 to the LDFLAGS line of the Makefile 
make CFLAGS="-q64"

Linux/GCC:

./configure --shared 
make CFLAGS="-m64"

Then use make install or copy to place the zlib library as described by the following paragraph:

By default, GT.M searches for the libz.so (libz.sl in HPUX HPPA) shared library in the standard system library directories (for example, /usr/lib, /usr/local/lib, /usr/local/lib64). If the shared library is installed in a non-standard location, before starting replication, the operator must ensure that the environment variable LIBPATH (in AIX) and LD_LIBRARY_PATH (in other UNIX platforms) includes the directory holding the library. The source and receiver server link in the shared library at runtime. If this fails for any reason (such as file not found, or insufficient authorization), the replication logic logs a DLLNOOPEN error and continues with no compression.

Change History

V5.3-000

An update to the V5.3-000 Release Notes:

M-Database Access

  • GT.M allows TP transactions to be nested i.e. TSTART can be specified within a transaction to start a sub-transaction. The nested TP transaction can be rolled back or committed using TROLLBACK or TCOMMIT. Nested TP transactions now guard against a possible rare spurious conflict in internal representation. In previous versions of GT.M, in very rare cases nested TP transactions could fail with a TPFAIL or GTMASSERT error and also cause database damage. (S9I07-002693)

    GDS Block Certification is a debugging feature that can currently be turned on by a VIEW "GDSCERT":1 command. This causes GT.M to verify the integrity of the block after every update. This feature now works correctly for TP transactions that do both SETs and KILLs of globals inside the same transaction. In previous versions of GT.M, in rare cases this could incorrectly fail with a GTMASSERT error.

  • If a runtime error occurs in the midst of committing a transaction, GT.M prints the error code in the DBCOMMITCLNUP message sent to the syslog (operator log in VMS). Previous versions of GT.M sent a DBCOMMITCLNUP message which did not contain the error code. (S9I09-002701)

  • The update process issues "Number of transactions from Primary which did NOT update JNLSEQNO on Secondary is xxx" message whenever it applies updates from the primary to non-replicated databases on the secondary. Previously such updates caused a "Secondary ahead of Primary by xxx" message in the update process log. Both new and old versions provide one such message for every such unreplicated transaction. (S9I11-002712)

  • The MM database access method is again an option on UNIX editions of GT.M. This access method bypasses the BG buffer pool and relies entirely on the operating/file system to manage traffic between memory and disk. Because with MM, GT.M has no control over the timing of disk updates, before-image journaling is not an option with MM; attempts to use these two facilities together produce an error. MM has been a continuing option on VMS, where it can provide improved performance for smaller regions with lower volatility (such as code tables). However on UNIX systems, the benefits proved so marginal that it was not used, Fidelity ceased testing it and it was not properly maintained. As UNIX file systems improved and GT.M replication has become more common, we found real benefit in reviving support for the MM option as it can provide a modest performance improvement under a range of circumstances. [UNIX] (C9904-001024)

  • The logical names GTM_TPRESTART_LOG_FIRST and GTM_TPRESTART_LOG_DELTA now reliably control the frequency of logging of TPRESTART messages. GTM_TPRESTART_LOG_FIRST indicates the number of TP restarts to log after GT.M invocation. Once GT.M has logged that number of TP restarts, it logs one such message, for every GTM_TPRESTART_LOG_DELTA TP restarts. Previous versions of GT.M occasionally used different values for the two parameters than those the user had specified in the logical name, causing unintended logging behavior. [VMS] (C9B11-001824)

  • The GT.CM GNP server now guards against external signals (including MUPIP STOP) interrupting it while it is in the middle of running down the database on behalf of a client process by deferring the signal handling until the database rundown is complete. Previously, the server attempted to handle the signal right away and would try to rundown the database already in the process of being rundown resulting in SIG-11 (UNIX) or ACCVIO (VMS) errors. (C9E02-002509)

  • GT.M manages shared resources through the use of shared memory. As processors, caches, and multi-processor interconnects have evolved, computer architectures have become more prone to memory update patterns that appear to a reading process in a different order than they do in the code of the updating process (commonly called out-of-order stores). GT.M uses platform specific primitives (commonly called memory barriers) to prevent out-of-order store issues where they pose problems. GT.M adopted memory barriers some time ago in selected places within the database and journal logic. We recently found that the replication code, which is subject to less intense inter-process contention, needs some as well. This release adds a few memory barriers to the receiver server and update process logic to prevent rare out-of-order store related problems including a very rare potential for the primary and secondary databases to become out of sync (although no actual customer or in-house environment has experienced or reported any such issue). The likelihood of encountering this issue is a function of both the particular hardware (different models of the same CPU line may have different characteristics) and the level of contending activity. [AIX, HP-UX HPPA, HP-UX Itanium, Tru64, OpenVMS] (C9E04-002588)

  • GT.M now handles MUPIP STOPs appropriately even if it is cleaning up after a TP transaction. In previous versions, MUPIP STOP of a GT.M process that is cleaning up after a TP transaction could cause it to abnormally terminate with SIG-11 (ACCVIO in VMS) errors. (C9E11-002660)

  • Whenever GT.M does a cache recovery (one way of triggering this is by DSE CACHE -RECOVER), it now waits appropriately for all concurrent writer processes to complete before proceeding with manipulating the shared memory structures. Previous versions had some timing issues in the wait logic which in very rare situations could result in GTMASSERT errors. (C9I03-002973)

  • Databases created by any GT.M V5 version no longer require MUPIP REORG -UPGRADE when upgrading to GT.M V5.3-003. Previously, upgrading from a pre-V5.3-000 GT.M version to V5.3-000 or higher version would cause periodic DBVERPERFWARN2 messages to show up in the syslog (operator log in VMS) until a MUPIP REORG -UPGRADE was run on that database. (C9I05-002987)

  • On UNIX systems, the replication source server can now compress the journal records before sending them across in the replication stream. The receiver process detects whether the incoming records are compressed, and if so, decompresses these journal records before placing them in the receive pool. [UNIX] (C9I08-003020)

    • GT.M dynamically links the "zlib" compression library (a widely used freely available compression library at http://www.zlib.net) and uses it to perform compression and decompression. By default, GT.M searches for the libz.so (libz.sl in HPUX HPPA) shared library in the standard system library directories (for example, /usr/lib, /usr/local/lib, /usr/local/lib64). If the shared library is installed in a non-standard location, before starting replication, the operator must ensure that the environment variable LIBPATH (in AIX) and LD_LIBRARY_PATH (in other UNIX platforms) includes the directory holding the library. The source and receiver server link in the shared library at runtime. If this fails for any reason (such as file not found, or insufficient authorization), the replication logic logs a DLLNOOPEN error and continues with no compression.

    • MUPIP REPLIC -SOURCE -START (the source server startup command) and MUPIP REPLIC -RECEIV -START (the receiver server startup command) support the -CMPLVL=N qualifier, where N is a positive integer value indicating the level of compression desired. Level 0 offers no compression. Level 1 offers the least compression while Level 9 (as of version 1.2.3.3 of the zlib library) offers the most compression (at the cost of the most CPU usage). Specifying the CMPLVL qualifier without an accompanying START qualifier triggers an error. In the case of the source server, if N specifies any value outside of the range accepted by the zlib library or if CMPLVL is not specified, the compression level defaults to zero (0). In the case of the receiver server, as long as N is non-zero the decompression feature is enabled; since the source server setting determines the actual level, any legal non-zero value enables compressed operation at the receiver.

    • Alternatively, an environment variable "gtm_zlib_cmp_level" can specify the desired compression level (in the same value range as N above) and the source server can then be started without the -CMPLVL qualifier. This has the same effect as starting it with -CMPLVL specified. An explicitly specified value on the command line overrides any value specified by the environment variable.

    • Whenever the source and receiver server connect with each other, if the source server was started with a valid non-zero compression level, they first determine whether the receiver server is running a version of GT.M which handles compressed records and has been started with a non-zero compression level. Only if this is true, do they agree to use compressed journal records. They also verify with a test message that compression/decompression works correctly before sending any compressed journal data across. They automatically fall back to uncompressed mode of transmission if this test fails or if, at any point, either side detects that compression or decompression has failed. That is, any runtime error in the compression/decompression logic results in uncompressed replication (thereby reducing replication throughput) but never jeopardizes the functional health of replication.

    • The source and receiver servers log all compression related events and/or messages in their respective logs. The source server also logs the length of the compressed data (in addition to the uncompressed data length) in its logfile.

    • Existing features of replication like Cross endian replication, Internal filters and External filters work independent of whether or not the replication stream uses compression.

    • Replication external filters now work on the secondary when receiving from an opposite endian primary. In previous versions of GT.M, such usage caused replication to hang with the receiver server indefinitely logging "REPL ERROR - Journal records did not endian convert properly" errors and the update process indefinitely logging "Bad trans" messages.

  • GT.M journaling now recovers more gracefully from additional runtime errors. If the control structures in the journal buffer (in database shared memory) get set to invalid values (such as an unanticipated out-of-design condition), GT.M journal buffer flushing logic now detects this situation and ensures that journaling gets turned OFF by one and only one process. In previous versions of GT.M, more than one process could attempt to turn journaling off at the same time. While in UNIX this was not an issue, in VMS this would cause every process except for the first one to terminate abnormally with a GTMASSERT error. [VMS](C9I08-003022)

  • GT.M now correctly handles situations in MM access mode when extending database files. In V53002 it was possible for a file extension in MM access mode to result in GTMASSERT failures. [UNIX](C9I08-003025)

  • In VMS, GT.M journaling logic now waits appropriately (a maximum of 2 minutes) for GT.M and the file system to flush the journal buffers in database shared memory to the journal file on disk even in the case where there is no other process doing the journal flush. Prior versions of GT.M, in these cases, could occasionally terminate prematurely with a GTMASSERT error without giving the writes adequate time to happen. [VMS] (C9I10-003045)

  • GT.M now correctly handles VIEW "NOISOLATION" commands even in the case the global names specified involve different regions (in one or more global directories) that map to the same database file. In rare situations, GT.M V5.3-001A and V5.3-002 did not handle this properly and could terminate with SIG-11 (ACCVIO in VMS) errors. (C9I11-003055)

  • GT.M now does a timely flush (one second unless a longer delay is specified) of the journal file to disk when used with the MM access method. Previous versions could wait more than 4 minutes after starting a timer before flushing the file. [UNIX] (C9I11-003056)

  • Database file extension for MM database files requires standalone access with MUPIP on the HP-UX PA-RISC platform, whereas on other UNIX/Linux platforms where GT.M now supports MM allow for automatic extension during use, as they do for database using the BG access method. On the HP-UX PA-RISC platform, GT.M now returns a GBLOFLOW error rather than extending an MM database during normal use. It previously returned the generic, and less meaningful, DBFILERR message. [HP-UX PA-RISC] (C9I12-003058)

M-Other Than Database Access

  • $VIEW("GVSTATS", <region>) now records SET operations even if they are inside TP transactions. For details on all the statistics recorded, see C9I09-003044. (S9709-000529)

  • A GT.M process on OpenVMS will occasionally terminate on receipt of a MUPIP INTRPT.  The probability of process termination appears to depend on how likely the process is to be processing an error or a transaction restart. As far as we can determine, the failure appears to be limited to the process receiving the interrupt without database damage or other issues.  Given the impact of the issue, and the effort to fix it (which would involve the use of different operating system facilities that are believed to be more robust than the facilities used by GT.M), FIS does not intend to address this in the foreseeable future. [OpenVMS] (S9E08-002477)

  • GT.M, DSE, MUPIP, LKE properly handle external UNIX signals while writing to any device or file. In previous versions of GT.M, in rare cases, using a MUPIP STOP or Ctrl-C (for example) to interrupt a DSE DUMP BLOCK or MUPIP REORG command while output was being written to the terminal would cause an indefinitely repeating sequence of the same messages. Although this had been observed only during terminal writes, this could also have occurred during writes to a file. [UNIX] (S9H05-002657)(S9I10-002706)

  • GT.M now uses a “wrapper” to guard gtmsecshr. The wrapper starts gtmsecshr in a clean environment and allows gtmsecshr to have permissions such that a non-root user can’t access it except though the mechanism used by the wrapper. This is an enhanced version of the mechanism described in the security bulletin of 23 October 2008. The subsections below describe other changes (applied to UNIX unless otherwise specified) that accompany this change. In prior versions, gtmsecshr resided in the main distribution directory and other GT.M components started gtmsecshr directly. [UNIX] (S9I10-002703)

    • Both the installation and the open source build (make for Linux and Tru64) now deal with the gtmsecshr and wrapper.

    • GT.M now correctly handles overly long environment variables by issuing a LOGTOOLONG error, which indicates the maximum allowed length. Previously it could overwrite memory and cause unpredictable results including abnormal process termination due to SIG-11 errors (ACCVIO in VMS), process hangs and creation of core dumps. This was particularly dangerous with the gtmsecshr executable, which runs as root, where it resulted in a security vulnerability that could lead to execution of exploit code.

    • GT.M now correctly handles any errors encountered at process startup. Previously such errors could cause abnormal process termination with SIG-11 error and core dumps.

    • GT.M now restricts the maximum message length limit for the DISTPATHMAX error to 1024 bytes across all platforms. Previously it allowed a much higher limit on Linux. [Linux]

    • The GT.CM GNP server now properly processes its command line. Previous versions did not null-terminate the end of the command line which could cause them to add more unintended bytes to the end of the command line. For example if the command line ended with a -trace usage, the server startup could fail indicating -trace<??> was not specified, where <??> could be any arbitrary sequence of unspecified bytes.

    • DBCERTIFY now works correctly on Itanium platforms. In previous versions, it had an incorrect view of the V4 database file header format, which could cause it to terminate abnormally if used to migrate a V4 format database to the V5 format. Note that GT.M support for these platforms started in a V5 version so one would normally not have a V4 version in use on these platforms. Therefore, this error could only occur if you copied a V4 database from another platform. [Linux Itanium, HPUX Itanium]

    • GT.CM servers no longer recognize the environment variable (logical in VMS) GTCM_GDSCERT. This used to control GDS block certification for the GT.CM GNP server. They continue to recognize the environment variable gtm_gdscert (logical GTM_GDSCERT in VMS) to control the same function. Previous versions unnecessarily allowed both approaches.

  • Invoking the $ZJOBEXAM() function at process startup no longer causes GT.M to abnormally terminate with a "Segmentation fault" or "SIG-11" (ACCVIO in VMS) error. (S9I11-002713)

  • GT.M now restricts the maximum message length limit for the DISTPATHMAX error to 1024 bytes across all platforms. Previously it allowed a much higher limit on Linux. [Linux] (S9I10-002703)(C9B11-001783)

  • [V5.3-000]The update process now writes out dirty buffers when it is idle (when the backlog on the receiver side is zero). (C9G01-002769)

  • GT.M V5.3-003 provides support for a new PIPE device on the UNIX/Linux platforms as a mnemonic space device. An OPEN of the device starts a sub-process. Data written to the device by the M program is available to the process on its STDIN.  The M program can read the STDOUT and STDERR of the sub-process. This facilitates output only applications, such as printing directly from a GT.M program to an lp command; input only applications, such as reading the output of a command such as ps; and co-processing applications, such as using iconv to convert data from one encoding to another.  Refer to the PIPE IO Technical Bulletin for details. [UNIX] (C9H05-002859)

  • GT.M now again reports error context correctly on x86 and x86_64 Linux. A change made in V5.2-001 could cause misreporting for certain limited kinds of address/location context. As the misreporting was typically confined to fatal errors, the potential impact would have been minimal. [Linux x86 and x864_64] (C9I08-003016)

  • GT.M now detects end-of-line when the source line contains a string literal missing its closing quote. In addition, this change moves the LITNONGRAPH warning to the conventional place after the source line display in stead of before. The changes for S9I07-002688 in V5.3-002 caused GT.M to miss the end-of-line in this case and produce many inappropriate errors and warnings. (C9I09-003032)

  • GT.M now properly handles the negation of an undefined variable. Starting with version V5.3-001, negation of an undefined variable caused a SEG-V (SIG-11 on UNIX, and an ACCVIO on VMS)(C9I09-003041)

  • GT.M now provides per-process statistics for database and LOCK-related operations. Previously similar statistics accumulated since file creation were only available using $VIEW("GVSTATS",<region>) or DSE on a database region. The file statistics have been adjusted to better align with the process statistics. DSE now has a facility to reset the file statistics. (C9I09-003044)

    The ZSHOW command supports a new information code "G" that displays statistics for each type of global variable access done by the current process for each database file that it has open. The statistics reflect the total activity since the process startup. Each database file's statistics are prefixed by the full pathname of the global directory and region to which they correspond. In addition, a line containing aggregated statistics (across all database files) displays with a prefix of * for the global directory and region name.

    The counts are displayed in the following order in a comma-separated list where each statistic has its mnemonic followed by a colon and the count. The characters used to arrive at each mnemonic are highlighted in upper-case in the corresponding description. The usage TP applies those database operations that are surrounded by a TSTART/TCOMMIT. The usage non-TP applies other operations, also called "mini-transactions."

    • SET : # of SET   operations (TP and non-TP) 
    • KIL : # of KILl  operations (kill as well as zwithdraw, TP and non-TP) 
    • GET : # of GET   operations (TP and non-TP) 
    • DTA : # of DaTA  operations (TP and non-TP) 
    • ORD : # of ORDer operations (TP and non-TP) 
    • ZPR : # of ZPRevious (reverse order) operations (TP and non-TP) 
    • QRY : # of QueRY operations (TP and non-TP) 
    • LKS : # of LocK calls (mapped to this db) that Succeeded 
    • LKF : # of LocK calls (mapped to this db) that Failed 
    • CTN : Current Transaction Number of the database for the last committed read-write transaction (TP and non-TP) 
    • DRD : # of Disk ReaDs from the database file by this process (TP and non-TP, committed and rolled-back) 
    • DWT : # of Disk WriTes to the database file by this process (TP and non-TP, committed and rolled-back) 
    • NTW : # of Non-tp committed Transactions that were read-Write on this database 
    • NTR : # of Non-tp committed Transactions that were Read-only on this database 
    • NBW : # of Non-tp committed transaction induced Block Writes on this database 
    • NBR : # of Non-tp committed transaction induced Block Reads on this database 
    • NR0 : # of Non-tp transaction Restarts at try 0 
    • NR1 : # of Non-tp transaction Restarts at try 1 
    • NR2 : # of Non-tp transaction Restarts at try 2 
    • NR3 : # of Non-tp transaction Restarts at try 3 
    • TTW : # of Tp committed Transactions that were read-Write on this database 
    • TTR : # of Tp committed Transactions that were Read-only on this database 
    • TRB : # of Tp read-only or read-write transactions that got Rolled Back (incremental rollbacks are not counted) 
    • TBW : # of Tp transaction induced Block Writes on this database 
    • TBR : # of Tp transaction induced Block Reads on this database 
    • TR0 : # of Tp transaction Restarts at try 0 (counted for all regions participating in restarting tp transaction) 
    • TR1 : # of Tp transaction Restarts at try 1 (counted for all regions participating in restarting tp transaction) 
    • TR2 : # of Tp transaction Restarts at try 2 (counted for all regions participating in restarting tp transaction) 
    • TR3 : # of Tp transaction Restarts at try 3 (counted for all regions participating in restarting tp transaction) 
    • TR4 : # of Tp transaction Restarts at try 4 and above (restart counted for all regions participating in restarting tp transaction) 
    • TC0 : # of Tp transaction Conflicts at try 0 (counted only for that region which caused the tp transaction restart) 
    • TC1 : # of Tp transaction Conflicts at try 1 (counted only for that region which caused the tp transaction restart) 
    • TC2 : # of Tp transaction Conflicts at try 2 (counted only for that region which caused the tp transaction restart) 
    • TC3 : # of Tp transaction Conflicts at try 3 (counted only for that region which caused the tp transaction restart) 
    • TC4 : # of Tp transaction Conflicts at try 4 and above (counted only for that region which caused the tp transaction restart)

    The use of comma-separated pieces allows for future releases of GT.M to provide additional data while facilitating upward compatibility of application code. Since FIS reserves the right to change the order in which statistics are reported in future versions of GT.M, application programs should use the names (mnemonics) when picking pieces from the string instead of relying on field position or ordering.

    $Order(xxx,1) operations get recorded under the ORD category while $Order(xxx,-1) operations get reported under ZPR.

    The DWT statistic in the ZSHOW "G" output always reports 0 for databases that use the MM (memory-mapped) access method as this has no real meaning in that mode.

    KILL/GET/DATA/QUERY/ORDER/ZPREVIOUS operations on globals that never existed are not counted (nor were they previously) while the same operations on globals that once existed but have since been killed are counted (as they were previously).

    Name-level ORDER/ZPREVIOUS operations (for example, $ORDER(^a) with no subscripts) increment the count for each transition into a region as they process the global directory map up to the point they find a global with data (as they were previously).

    Data for M locks refers to M locks mapped to the shared memory of a database file. If "local" M locks are mapped to a file, the data for M locks includes these local locks.

    These statistics report on global variable operations performed by a process. If an operation is performed inside a TP transaction, and not committed as a consequence of a rollback, or an explicit or implicit restart, GT.M still counts it.

    If two or more regions (in the same or different global directories) map to the same physical database file, the ZSHOW "G" reports identical statistics for those two regions, but counts them only once towards the aggregated statistics across all database files in the line beginning with GLD:*,REG:*.

    The aggregated statistics line in the ZSHOW "G" output (GLD:*,REG:* line) always reports the value for the CTN category as 0 because this statistic makes sense only for individual database files.

    ZSHOW "G" reports process private global access statistics only for regions whose corresponding segments have an access method of BG or MM in the global directory. For regions with other access methods, for example GT.CM GNP, which maps a region/segment to a remote database file, ZSHOW "G" does not report any process private statistics even though aggregated statistics (across all processes) will continue to be gathered in the remote database file header.

    Processes with read-only access to the database have their global access statistics counted in the database file header as long as the last process to shut down the database has read-write access.

    The counters are 8-byte quantities (can get as high as 2**64). If these counters exceed 18 decimal digits (somewhere between 2**59 and 2**60), which is the current GT.M numeric representation precision threshold, their use in arithmetic expressions in GT.M results in loss of precision.

    As an example of use for this enhancement: during a benchmark, a process can make periodic commands to ZSHOW "G" and store the returned strings in a local variable - a fast storage mechanism in GT.M - for subsequent analysis.

    Alternatively, since the $ZJOBEXAM() function by default does a ZSHOW "*" which in turn automatically includes the "G" information code, invoking MUPIP INTRPT commands periodically on a particular process will now cause it to additionally record all global access statistics in the $ZJOBEXAM dump file.

    The following examples use 0 values for all counters.

    Assuming that a GT.M process uses the global directory "/tmp/x1.gld" and opens two regions REG1 and REG2 corresponding to two database files, the output of a ZSHOW "G" command performed before any database operation appears as follows.

    GLD:*,REG:*,SET:0,KIL:0,GET:0,DTA:0,ORD:0,ZPR:0,QRY:0,LKS:0,LKF:0,CTN:0,DRD:0,DWT:0,NTW:0,NTR:0,NBW:0,NBR:0,NR0:0,
    NR1:0,NR2:0,NR3:0,TTW:0,TTR:0,TRB:0,TBW:0,TBR:0,TR0:0,TR1:0,TR2:0,TR3:0,TR4:0,TC0:0,TC1:0,TC2:0,TC3:0,TC4:0
    GLD:/tmp/x1.gld,REG:REG1,SET:0,KIL:0,GET:0,DTA:0,ORD:0,ZPR:0,QRY:0,LKS:0,LKF:0,CTN:0,DRD:0,DWT:0,NTW:0,
    NTR:0,NBW:0,NBR:0,NR0:0,NR1:0,NR2:0,NR3:0,TTW:0,TTR:0,TRB:0,TBW:0,TBR:0,TR0:0,TR1:0,TR2:0,TR3:0,TR4:0,
    TC0:0,TC1:0,TC2:0,TC3:0,TC4:0
    GLD:/tmp/x1.gld,REG:REG2,SET:0,KIL:0,GET:0,DTA:0,ORD:0,ZPR:0,QRY:0,LKS:0,LKF:0,CTN:0,DRD:0,DWT:0,NTW:0,
    NTR:0,NBW:0,NBR:0,NR0:0,NR1:0,NR2:0,NR3:0,TTW:0,TTR:0,TRB:0,TBW:0,TBR:0,TR0:0,TR1:0,TR2:0,TR3:0,TR4:0,TC0:0,TC1:0,
    TC2:0,TC3:0,TC4:0

    The following is the result of a ZSHOW "G":zgbl done by the same process under the same circumstances, redirecting the output of ZSHOW into a local or global variable.

    zgbl("G",0)="GLD:*,REG:*,SET:0,KIL:0,GET:0,DTA:0,ORD:0,ZPR:0,QRY:0,LKS:0,LKF:0,CTN:0,DRD:0,DWT:0,NTW:0,NTR:0,
    NBW:0,NBR:0,NR0:0,NR1:0,NR2:0,NR3:0,TTW:0,TTR:0,TRB:0,TBW:0,TBR:0,TR0:0,TR1:0,TR2:0,TR3:0,TR4:0,TC0:0,TC1:0,
    TC2:0,TC3:0,TC4:0"
    zgbl("G",1)="GLD:/tmp/x1.gld,REG:REG1,SET:0,KIL:0,GET:0,DTA:0,ORD:0,ZPR:0,QRY:0,LKS:0,LKF:0,CTN:0,DRD:0,DWT:0,
    NTW:0,NTR:0,NBW:0,NBR:0,NR0:0,NR1:0,NR2:0,NR3:0,TTW:0,TTR:0,TRB:0,TBW:0,TBR:0,TR0:0,TR1:0,TR2:0,TR3:0,TR4:0,
    TC0:0,TC1:0,TC2:0,TC3:0,TC4:0"
    zgbl("G",2)="GLD:/tmp/x1.gld,REG:REG2,SET:0,KIL:0,GET:0,DTA:0,ORD:0,ZPR:0,QRY:0,LKS:0,LKF:0,CTN:0,DRD:0,DWT:0,
    NTW:0,NTR:0,NBW:0,NBR:0,NR0:0,NR1:0,NR2:0,NR3:0,TTW:0,TTR:0,TRB:0,TBW:0,TBR:0,TR0:0,TR1:0,TR2:0,TR3:0,TR4:0,TC0:0,
    TC1:0,TC2:0,TC3:0,TC4:0"

    The $VIEW("GVSTATS",<region-name>) intrinsic function continues to return the global access statistics across all processes, but with a change in its output format. It displays statistics in the same format as the ZSHOW "G" command as illustrated below.

    New output:

    SET:0,KIL:0,GET:0,DTA:0,ORD:0,ZPR:0,QRY:0,LKS:0,LKF:0,CTN:0,DRD:0,DWT:0,NTW:0,NTR:0,NBW:0,NBR:0,NR0:0,NR1:0,NR2:0,
    NR3:0,TTW:0,TTR:0,TRB:0,TBW:0,TBR:0,TR0:0,TR1:0,TR2:0,TR3:0,TR4:0,TC0:0,TC1:0,TC2:0,TC3:0,TC4:0

    Old output:

    SET:0,KIL:0,GET:0,ORD:0,QRY:0,ZPR:0,DTA:0,RT0:0,RT1:0,RT2:0,TP0:0,TP1:0,TP2:0,TP3:0,TP4:0,TP5:0,TP6:0,TP7:0,
    TP8:0,TP9:0,TP::0,TP;:0,TC0:0,TC1:0,TC2:0,TC3:0,TC4:0,TC5:0,TC6:0,TC7:0,TC8:0,TC9:0,TC::0,TC;:0

    The DSE DUMP -FILE -ALL command prints the global access statistics across all processes for the same categories as those reported for the process private global access statistics. It now lists it counters in the same order as $VIEW("GVSTATS",<region>), except for the CTN category, which appears as part of the basic DSE DUMP -FILE output.

    All ZSHOW "G" and $VIEW("GVSTATS",<region>) statistics display in DECIMAL while DSE DUMP -FILE -ALL statistics display in HEXADECIMAL (as they did previously).

    ZSHOW "*" is now equivalent to ZSHOW "IVBDLGSC" so ZSHOW "G" output appears after ZSHOW "L" output and before ZSHOW "S" output.

    The VIEW "RESETGVSTATS" command resets all the process-private global access statistics to 0. This is particularly useful for long running processes which would periodically like to restart the counting without requiring a shut down and restart.

    The DSE CHANGE -FILE -GVSTATSRESET command resets all the database file header global access statistics to 0. Note that this erases all statistics previously accumulated in the database file header.

    ZSHOW "L" now displays the following additional M-lock statistics in one line just before its current output.

    • MLG : # of M Lock commands Granted 
    • MLT : # of M Lock commands Timed-out

    Suppose a process runs LOCK (^SUCCESS1,^SUCCESS2) which succeeds and a LOCK +^FAIL:1 which times out due to another process holding that lock. A ZSHOW "L" at this point displays the following output.

    MLG:1,MLT:1
    LOCK ^SUCCESS1 LEVEL=1
    LOCK ^SUCCESS2 LEVEL=1
    [Note]

    Note that even though two lock resources ^SUCCESS1 and ^SUCCESS2 were specified in the LOCK command that succeeded, GT.M increments the MLG counter by only 1 because they are part of the same LOCK command. A ZSHOW "L":var by the same process (redirecting the output of ZSHOW into a local or global variable) would result in <var> holding the following contents.

    var("L",0)="MLG:1,MLT:1"
    var("L",1)="LOCK ^SUCCESS1 LEVEL=1"
    var("L",2)="LOCK ^SUCCESS2 LEVEL=1"

    For every user level lock request, GT.M increments the MLG or MLT statistic for the ZSHOW "L" output. Every user level lock request in turn translates to one or more calls to the database lock code (depending on the timeout and the number of lock names specified in the same lock command) which increments the LKS and/or LKF statistics of the ZSHOW "G" output appropriately.

    MUPIP ENDIANCVT now preserves the global statistics as part of endian converting the database file whereas previously it used to reset them to 0.

Utilities- MUPIP

  • MUPIP LOAD FORMAT=ZWR now handles decimal number values correctly. In prior versions it reported a format error for numerics containing a decimal and skipped records with such values. The workaround was to use %GI. (S9I07-002691)

    %GO now includes a UTF-8 stamp in the label of its output. In prior versions it omitted this indicator. The workaround was to add the indicator by editing the extract file.

  • MUPIP online BACKUP now releases locks on all the previous regions, before waiting for KILL(s) to complete. MUPIP BACKUP on multiple regions used to hold locks on all the previous regions, in the region list, while waiting for ongoing KILL(s) command(s) to complete on a region. (S9H12-002672)

  • GT.M properly handles MUPIP STOPs while opening a journaled database. In previous versions of GT.M, in rare cases, this could cause the process to get deadlocked and the resolution was to kill -9 this process.[UNIX] (C9903-000942)

  • A MUPIP STOP issued to a MUPIP JOURNAL process should now reliably trigger a graceful termination. In prior versions, it  could occasionally cause the process, to terminate abnormally an ACCVIO error. [VMS] (C9F08-002752)

  • MUPIP BACKUP and MUPIP INTEG REGION both interact with MUPIP REOROG and with the KILL command somewhat differently. Initiation of an online BACKUP or MUPIP INTEG on one or more REGIONs defers KILLs or REORG actions that free entire blocks until it manipulates the database state to record its start. BACKUP waits for any block-freeing KILLs or in progress REORG actions to complete. MUPIP BACKUP now waits for a maximum of one minute across all the regions. MUPIP BACKUP no longer errors out with BACKUP2MANYKILL error; if it waits more than one minute for a clean KILL state, it prints a BACKUPKILLIP warning message and proceeds. INTEG on REGION(S) now waits for any block-freeing KILLs or REORG actions currently in progress to complete; if it waits more than one minute for a clean block state, it gives a warning message and proceeds. If BACKUP or INTEG encounter any ABANDONED_KILLS, they give a warning and proceed. (C9G04-002783)(C9I12-003061)

    The startup logic of MUPIP BACKUP or INTEG REGION may defer block-freeing KILLs or REORG actions for a maximum of two (2) minutes after which the KILL logic clears the hold-off flag and proceeds normally. Normally MUPIP BACKUP and INTEG start nearly instantly and clear the hold-off flag immediately.

    In addition, work on this issue also changed the mechanism to differentiate between KILLs or REORG actions in progress (doing active cleanup) and abandoned KILLs or REORG actions where the process terminated without completing its cleanup (leaving incorrectly marked busy blocks). When a process that is having KILLs or REORG actions in progress is MUPIP STOP'ed at a stage where it may leave the database blocks in inconsistent state, the process cleanup logic converts the KILLs in progress flag to the abandoned KILLS flag.

    Work in this area removed a very slight chance that GT.M could cause OpenVMS to crash (BUGCHECK) if something corrupted memory in certain ways.

    DSE CHANGE FILEHEADER can now manipulate flags for ABANDONED_KILLS (in addition to KILL_IN_PROG). DSE DUMP FILEHEADER shows one sum for the combination of KILLs in progress and abandoned KILLs, but DUMP FILEHEADER ALL shows them separately. DSE can also examine and modify the INHIBIT_KILLS flag.

    Previously neither MUPIP REORG nor KILL would hold off to let BACKUP and FREEZE actions start, and GT.M did not differentiate between KILL in progress and abandoned KILL, where both include the “virtual” KILLs performed by MUPIP REORG in moving data to more optimal places.

  • Backward journal recovery (MUPIP JOURNAL RECOVER or ROLLBACK -BACKWARD) after an abnormal termination of the database now runs faster, especially for large databases. MUPIP achieves this by avoiding bit map reads. In previous versions of GT.M, recovery used to read ALL the local bitmaps of the database multiple times in order to fix certain fields in the file header. (C9H09-002906)

  • MUPIP ENDIANCVT no longer issues "wc_blocked is set - rundown needed" and "the database is frozen" messages. Instead it resets the corresponding fields in the file header and proceeds with the conversion. Additionally, it supports a new -OVERRIDE qualifier which if specified will enable it to proceed with the conversion is long as it had only issued one or more of these errors: [UNIX] (C9I06-002996)

    1. "minor database format is not the current version"

    2. "kills in progress"

    3. "a GT.CM server is accessing the database"

    The OVERRIDE qualifier has no effect for database fields whose state indicates serious issues that endanger the success of the conversion process (for example fields that indicate the database has integrity errors etc.).

  • MUPIP FREEZE defers KILLs that free entire blocks until the database state reaches a consistent state suitable for a FREEZE. FREEZE waits for any block-freeing KILLs currently in progress to complete; if it waits more than one (1) minute for a clean KILL state, it gives an error and terminates marking unfrozen any of the regions that were it had previously marked frozen. (C9I06-002997)

  • The replication instance file format changed in GT.M V5.3-003. Therefore, you must recreate all replication instance files when installing this version of GT.M. If GT.M encounters an old instance file, it reports a REPLINSTFMT error. Additionally, GT.M reports a REPLINSTFMT error whenever it encounters: a) an instance file created on a different endian system or b) an instance file created by a 32-bit (or 64-bit) version of GT.M differing from the current 64-bit (or 32-bit) version of GT.M. [UNIX] (C9I08-003021)

  • MUPIP CREATE now creates appropriately sized database files. In previous versions of GT.M, it could occasionally try to write out much more data than a local bitmap block can hold causing it to abnormally terminate with SIG-11 and/or IO related errors. [UNIX] (C9I11-003053)

  • GT.M now processes forward recovery while using the MM access method properly. In rare situations, previous versions could incorrectly return a DBTNTOOLRG error erroneously. [UNIX] (C9I11-003054)

    GT.M now gives valid results for the SHOW and VERIFY options of the DSE CACHE command when using the MM access method. Previous versions did not use the MM specific internal structures for the cache and could return incorrect information.

    GT.M now properly initializes the default value to NOBEFORE image journaling when running with the MM access method, and journaling characteristics have not been previously specified. Previous versions could have a default value of BEFORE image journaling with MM, leading to potential conflicts with other journaling attributes and resulting in an MMBEFOREJNL error.

  • GT.M now correctly handles MUPIP REORG for the MM access method. In previous versions the MUPIP process would occasionally terminate with a SIGMAPERR. [UNIX] (C9I12-003059)

  • The replication source server, receiver server and MUPIP JOURNAL -ROLLBACK -FETCHRESYNC now correctly handle two kinds of errors at connection establishment time. (C9I12-003060)

    1. if the operation to determine the TCP socket buffer size fails, they issue a ERR_REPLCOMM error, and

    2. if the remote instance resets the connection, they report a GETSOCKNAMERR error. Previous versions trying to display these relatively uncommon errors abnormally terminated with SIG-11 error on UNIX (ACCVIO in OpenVMS).

    The MUPIP REPLIC /RECEIV /START command now issues a proper error message if it is unable to start update helper processes. Previous versions could abnormally terminate with an ACCVIO error. [OpenVMS]

    When LKE encounters an error while opening the file specified in the -OUTPUT qualifier of a command (say because of permission issues), it now reports the cause of the error. Previous versions printed only a "Cannot create <filename>" message without accompanying detail.

Utilities-Other Than MUPIP

  • V5CBSU (a utility program that is used during a V4 to V5 GT.M database migration) now operates as described below. (S9I09-002700)

    1. It renames the input scan file (created by DBCERTIFY SCAN) with a _orig suffix (for example mumps.dbcertscan to mumps.dbcertscan_orig) as part of its processing. If a file with the _orig suffix already exists, that is deleted before renaming the input scan file. Previous versions used to overwrite the input scan file with new contents.

    2. It writes bypassed records to the output scan file. This way DBCERTIFY CERTIFY later upgrades any of those blocks requiring adjustment to the new requirements. Previously DBCERTIFY CERTIFY missed those blocks, which could later cause a DYNUPGRDFAIL error during V5 database operation (the only workaround then was to downgrade back to V4 and re-upgrade).

    3. Invoking the entryref DUMP^V5CBSU (instead of the usual ^V5CBSU) invokes a diagnostic mode of operation where V5CBSU only dumps the contents of the scan file. In this diagnostic mode it does not modify either the database or the scan file.

  • The GT.M installation now offers to give ownership of the distribution to a particular group, and, if this is option chosen, restricts access to the distribution to the designated group. Previously you could put such a restriction in place after the installation, but this makes a security related option more convenient. [UNIX] (C9I10-003048)

Error Messages

BACKUPKILLIP

BACKUPKILLIP: Kill in progress indicator is set for file ffff, backup database could have incorrectly marked busy integrity errors

Information Message: This indicates that one or more active process are performing KILL cleanup in database file ffff. Generally, BACKUP can wait for this to finish in order to get a consistent copy of the database. However, this indicates it waited several minute and is proceeding The resulting backup almost surely contains blocks incorrectly marked busy.

Action: Wait and perform the BACKUP when there are no large KILL operations triggering extensive cleanup. If this is not desirable, fix the errors in the backup copy (reported by an INTEG NOMAP) with DSE MAP FREE. If there are many such blocks you can edit the INTEG output to create a script to drive the DSE operations. Alternatively, if you can get standalone access, to the database you may use DSE MAP RESTORE. Do not use MAP RESTORE on an active database.

DBCOMMITCLNUP

DBCOMMITCLNUP: Pid dddd [hhhh] handled error (code = eeee) during commit of xxxx transaction in database file yyyy

Information Message: This message is output to the operator log and indicates that there was an error in the midst of committing a xxxx (TP or non-TP) transaction that involved the database file yyyy, but the process (pid = dddd in decimal and hhhh in hexadecimal) handled the error and completed the commit. If non-zero, the error code eeee is what triggered the error in the first place. If zero, accompanying syslog messages will contain information on the cause.

Action: In most cases the commit will be successfully completed. But in very rare cases, there might be errors that prevent the transaction from being successfully completed. To determine if there was any error, examine the following operator log messages. If there are more then 3 (three) DBCLNUPINFO messages for the same database file from the same process-id, then that particular database is suspect and an integrity check of that database needs to be done at the earliest. In addition, contact GT.M Support with the operator log messages.

KILLABANDONED

KILLABANDONED: Abandoned kills counter is greater than zero for file ffff, tttt

Information Message: This indicates a process terminated during KILL cleanup in database file ffff; tttt is text waring of the implications. Generally, this leaves a database with block incorrectly marked busy errors. Such errors are benign in that they only cause blocks to be inappropriately unavailable. Nonetheless they should be addressed promptly to avoid operators becoming desensitized to errors in INTEGs.

Action: Use DSE MAP to carefully FREE individual incorrectly marked busy block. If there are many blocks, you can edit the output of the integ (run with NOMAP) to create a script for driving repeated DSE MAP FREE. Alternatively, if you can get standalone access, to the database you may use DSE MAP RESTORE - never use MAP RESTORE on an active database.

LOGTOOLONG

LOGTOOLONG: Environment variable eeee is too long. Maximum length allowed is llll bytes.

Information Message: This error is triggered whenever the length of an environment variable that GT.M cares about exceeds the maximum allowed limit.

Action: The maximum allowed limit is indicated in the message. Specify a value for the environment variable within this length.

MMNOBFORRPL

MMNOBFORRPL: Replication cannot be used in database file ffff which uses MM access method and NOBEFORE image journaling

Information Message: You can't turn on replcation for MM access method database file ffff.

Action: Forgo replication for the file or change the access method to BG.

REPLTRANS2BIG

REPLTRANS2BIG: Transaction xxxx of size yyyy (pre-filter size ffff) too large to be accommodated in the zzzz pool

Information Message: This indicates that the size of the incoming transaction is larger than what the receive pool can accommodate. If a replication external filter is in use on the secondary side, the actual received data size is recorded in the pre-filter size ffff and the data size after the filter transaction is recorded in the size yyyy.

Action: The receiver server would have shut down because of this error. Shut down the update process and any helper process on the secondary using MUPIP REPLIC -RECEIV -SHUT -TIME=0 followed by a MUPIP RUNDOWN -REG "*". Restart the secondary with a bigger receive pool size enough to accommodate the size yyyy.