Journal Extract Formats

Journal Extract Formats
Prev	Chapter 6. GT.M Journaling	Next

Journal EXTRACT files always start with a label. For the current release of GT.M, the label is GDSJEX06 for a simple journal extract file. This label is necessary to identify the format of the file.

If the environment variable gtm_chset is set of UTF-8, then file format label is followed by another label called "UTF-8" to indicate UTF-8 mode.

After this label, the journal record extracts follow. These journal record extracts include fields or pieces delimited by a back slash (\).

The first piece of an -EXTRACT output record contains a two-digit decimal transaction record type (for example, 01 for a process initialization record). The second piece contains the full date and time of the operation, represented in the $HOROLOG format. The third piece contains a GT.M assigned number (database transaction number) which uniquely identifies the transaction within the time covered by the journal file. The fourth piece contains the process ID (PID) of the process that performed the operation, represented as a decimal number. The remainder of the record depends on the record type.

Records of type SET, KILL, ZKILL, TSTART, and TCOMMIT include the token_seq as part of the output. It is the sixth field in the output of the journal record extract. When replication is in use, token_seq is a journal sequence number (jsnum) that uniquely identifies each transaction(for more information on journal sequence number refer to Chapter 7: “Database Replication”). When replication is not in use and the transaction is a TP or ZTP transaction, token_seq is an 8-byte token that uniquely identifies the entire TP or ZTP transaction. For non-replicated, non-TP, and non-ZTP journal records, token_seq has a zero (0) value.

The format of the plain journal extract is as follows:

NULL    00\time\tnum\pid\clntpid\jsnum\strm_num\strm_seq
PINI(U) 01\time\tnum\pid\nnam\unam\term\clntpid\clntnnam\clntunam\clntterm
PINI(V) 01\time\tnum\pid\nnam\unam\term\mode\logintime\image_count\pname\clntpid\clntnnam\clntunam\clntterm\clntmode\clntlogintime\clntimage_count\clntpname
PFIN    02\time\tnum\pid\clntpid
EOF     03\time\tnum\pid\clntpid\jsnum
KILL    04\time\tnum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node
SET     05\time\tnum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node=sarg
ZTSTART 06\time\tnum\pid\clntpid\token
ZTCOM   07\time\tnum\pid\clntpid\token\partners
TSTART  08\time\tnum\pid\clntpid\token_seq\strm_num\strm_seq
TCOM    09\time\tnum\pid\clntpid\token_seq\strm_num\strm_seq\partners\tid
ZKILL   10\time\tnum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node
ZTWORM  11\time\tnum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\ztwormhole
ZTRIG   12\time\tnum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node

where:

01 record indicates a process/image-initiated update (PINI) into the current journal file for the first time.

02 record indicates a process/image dropped interest (PFIN) in the current journal file.

03 record indicates all GT.M images dropped interest in this journal file and the journal file was closed normally.

04 record indicates a database update caused by a KILL command.

05 record indicates a database update caused by a SET command.

06 record indicates a ZTSTART command.

07 record indicates a ZTCOMMIT command.

08 record indicates a TSTART command.

09 record indicates a TCOMMIT command.

10 record indicates a database update caused by a ZKILL command.

11 records indicates a value for/from $ZTWORMHOLE (when replication is turned on).

12 record indicates a ZTRIGGER command.

Journal extracts contain NULL records only in a multisite replication configuration where triggers or external M-filters are active. Here are two examples when NULL records are sent to the journal files:

An external filter on an instance transforms a SET record to a NULL record that has a different schema.
If the source side has triggers enabled and its receiver side either runs a pre-trigger version of GT.M or runs on a platform where triggers are not supported (for example, AXP VMS or HPUX HPPA), trigger definition journal records from the source side are transformed to NULL records on the receiver side.

	Important
	A NULL record does not have global information. Therefore, it resides in the alphabetically last replicated region of the global directory.

The format of the detail journal extract is as follows:

PINI(U) time\tnum\chksum\pid\nnam\unam\term\clntpid\clntnnam\clntunam\clntterm
PINI(V) time\tnum\chksum\pid\nnam\unam\term\mode\logintime\image_count\pname\clntpid\clntnnam\clntunam\clntterm\clntmode\clntlogintime\clntimage_count\clntpname
PFIN    time\tnum\chksum\pid\clntpid
EOF     time\tnum\chksum\pid\clntpid\jsnum
SET     time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node=sarg
KILL    time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node
ZKILL   time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node
ZTWORM  time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\ztwormhole
ZTRIG   time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node
TSTART  time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq
TSET    time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node=sarg
TKILL   time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node
TZKILL  time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node
TZTWORM time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\ztwormhole
TZTRIG  time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node
USET    time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node=sarg
UKILL   time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node
UZKILL  time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node
UZTWORM time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\ztwormhole
UZTRIG  time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node
TCOM    time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\partners\tid
INCTN   time\tnum\chksum\pid\clntpid\opcode\incdetail
EPOCH   time\tnum\chksum\pid\clntpid\jsnum\blks_to_upgrd\free_blocks\total_blks\fully_upgraded[\strm_num\strm_seq]...
PBLK    time\tnum\chksum\pid\clntpid\blknum\bsiz\blkhdrtn\ondskbver
AIMG    time\tnum\chksum\pid\clntpid\blknum\bsiz\blkhdrtn\ondskbver
NULL    time\tnum\pid\clntpid\jsnum\strm_num\strm_seq
ZTSTART time\tnum\chksum\pid\clntpid\token
FSET    time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node=sarg
FKILL   time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node
FZKILL  time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node
GSET    time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node=sarg
GKILL   time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node
GZKILL  time\tnum\chksum\pid\clntpid\token_seq\strm_num\strm_seq\updnum\nodeflags\node
ZTCOM   time\tnum\chksum\pid\clntpid\token\partners
ALIGN   time\tnum\chksum\pid\clntpid

where:

AIMG records are unique to DSE action and exist because those actions do not have a "logical" representation.

EPOCH records are status records that record information related to check pointing of the journal.

NCTN records are the transaction numbers of the sequence of critical sections in which the process and marked the database blocks of the globals as previously used but no longer in use in the bit maps.

PBLK records are the before image records of the bit maps.

ALIGN records pad journal records so every alignsize boundary (set with MUPIP SET -JOURNAL and is visible in DSE DUMP -FILEHEADER output) in the journal file starts with a fresh journal record. The sole purpose of these records is to help speed up journal recovery.

Legend (All hexadecimal fields have a 0x prefix. All numeric fields otherwise are decimal):

`tnum`	`Transaction number`
`chksum`	Checksum for the record.
`fully_upgraded`	1 if the db was fully upgraded (indicated by a dse dump -file -all) at the time of writing the EPOCH
`pid`	Process id that wrote the jnl record.
`clntpid`	If non-zero, clntpid is the process id of the GT.CM client that initiated this update on the server side.
`jsnum`	Journal sequence number.
`token`	Unique 8-byte token.
`strm_num`	If replication is true and this update originated in a non-supplementary instance but was replicated to and updated a supplementary instance, this number is a non-zero value anywhere from 1 to 15 (both inclusive) indicating the non-supplementary stream number. In all other cases, this stream # value is 0. In case of an EPOCH record, anywhere from 0 to 16 such "strm_num" numbers might be displayed depending on how many sources of supplementary instance replication have replicated to the instance in its lifetime.
`strm_seq`	If replication is true and this update originated in a non-supplementary instance but was replicated to and updated a supplementary instance, this is the journal sequence number of the update on the originating non-supplementary instance. If replication is true and this update originated in a supplementary instance, this is the journal sequence number of the update on the originating supplementary instance. In all other cases, this stream sequence number is 0. Note that the journal seqno is actually 1 more than the most recent update originating on that stream number. In case of an EPOCH record, anywhere from 0 to 16 such "strm_seq" numbers might be displayed depending on how many sources of supplementary instance replication have replicated to the instance in its lifetime.
`tid`	TRANSACTIONID string (BATCH or any string of descriptive text chosen by the application) specified as an argument of the corresponding TSTART command. If TRANSACTIONID is not specified with TSTART, GT.M sets `tid` to null. TRANSACTIONID can specify any value for `tid` but affects GT.M behavior only when TRANSACTIONID specifies BATCH or BA.
`token_seq`	If replication is turned on, it is the journal sequence number. If not, it is a unique 8-byte token.
`updnum`	=n where this is the nth update in the TP or ZTP transaction. n=1 for the 1st update etc. 0 for non-TP.
`nodeflags`	Decimal number interpreted as a binary mask.. Currently only 5 bits are used. 00001 (1) => update journaled but NOT replicated (For example, update inside a trigger) 00010 (2) => update to a global that had at least one trigger defined, even if no trigger matched this update 00100 (4) => $ZTWORMHOLE holds the empty string ("") at the time of this update or was not referenced during this update 01000 (8) => update did not invoke any triggers even if they existed (For example, MUPIP LOAD) 10000 (16) => whether the update (set or kill) is a duplicate. In case of a KILL, it is a kill of some non-existing node aka duplicate kill. Note that the dupkill occurs only in case of the Update Process. In case of GT.M, the KILL is entirely skipped. In both cases (duplicate set or kill), only a jnl record is written, the db is untouched. Combinations of the above bits would mean each of the individual bit characteristics. For example, 00011 => update within a trigger context, and to a global with at least one trigger defined. For example, 00011 => update inside a trigger and to a global with at least one trigger defined. Certain bit combinations are impossible. For example, 01001 since GT.M replicates any update that does not invoke triggers.
`node`	Key that is being updated in a SET or KILL.
`sarg`	Right-hand side argument to the SET (that is, the value that the key is being SET to).
`partners`	Number of journaled regions participating in this TP or ZTP transaction (TCOM/ZTCOM record written in this TP or ZTP) .
`opcode`	Inctn opcode. See gdsfhead.h inctn_opcode_t for all possible values.
`blknum`	Block number corresponding to a PBLK or AIMG or INCTN record.
`bsiz`	Block size from the header field of a PBLK or AIMG record.
`blkhdrtn`	Transaction number from the block header of a PBLK or AIMG record.
`ondskbver`	On disk block version of this block at the time of writing the PBLK or AIMG record. 0 => V4, 1 => V5.
`incdetail`	0 if opcode=1,2,3; blks2upgrd if opcode=4,5,6; blknum if opcode=7,8,9,10,11,12,13
`ztwormhole`	string corresponding to $ZTWORMHOLE
`blks2upgrd`	# of new V4 format bitmap blocks created if opcode=4,5; csd->blks_to_upgrd if opcode=6
`uname`	Name of the user that wrote this PINI record.
`clntunam`	If non-empty, clntunam is the name of the GT.CM client that initiated this update on the server side.

Prev	Up	Next
JOURNAL	Home	Chapter 7. Database Replication