Integration Toolkit Return Messaging

Introduction

Return messages traffic (messages that come FROM the vehicles) must somehow be received by the customer machine, processed, and handled appropriately. The Portal Integration Toolkit offers a return message integration mechanism that aims to accomplish these tasks.

Return message traffic includes:

Text messages
Macro messages
Position reports
Forward message dispositions
Entity definition synchronization notifications
Various events and notification

These messages are all conveyed via ESS Transactions. The entire return traffic stream, containing all message types listed above and more, flows through the Event Subscription Service system.

ESS Dequeue

ESS transactions enqueued by the Services Portal are available for retrieval by the customer via the ESS High Volume Dequeue API web service.

You may have noticed that the transaction formats returned by ESS are XML based, making them somewhat complex to parse via traditional iSeries technologies.

Also note, that a call to the ESS Dequeue web service must be issued repeatedly, in order to retrieve all the continually flowing transactions that ESS might have enqueued for the customer.

Solution

The Portal Integration Toolkit provides a return transaction mechanism to dequeue the customer-desired, published ESS transactions.

The mechanism is quite simple; it employs a batch job (labeled "Return Message Processor" above) which runs forever. The batch job does the following:

(1) Repeatedly calls the ESS Dequeue Web Service Wrapper.
(2) For each transaction within each transaction block fetched, it writes one or more records to a set of database files. The format of the data within these files is slightly simpler to digest than the original ESS transaction formats.
(3) A second batch job (labeled "Return Callback Stub" above), customized by integration engineers, empties the database files, and sends the data back to the customer's dispatch system as necessary.

Operating Parameters

Much of the behavior of the return message processor is governed by defaults contained within the Portal Toolkit Configuration file. See Integration Toolkit Configuration for more information. Pay special attention to:

DFMTDLY - The number of seconds to wait after being told by ESS that the transaction queue is empty before attempting another dequeue.
DFSUBID - Default ESS Subscriber ID to use to dequeue ESS transactions.
DFBLOCCNT - Default maximum number of ESS transactions to receive per dequeue.

Program Names

The "return message procesor" is named IIRTNPROC.
The "return callback stub" is named IIRTNCUST. It has both Omnitracs-authored logic (which should be kept) as well as places where customer-written logic should be inserted

Controlling the Return Message Processes

During the installation process, a job description (in the specified site library) and an autostart job entry (for the specified subsystem) for both IIRTNPROC and IIRTNCUST may have been optionally created by the engineer installing the Integration Toolkit. When starting the subsystem, the IIRTNPROC will be started as well as IIRTNCUST (if the stub program was created in the site library)

If the need should arise for a manual start of the return message process(es), the SBMRTNPROC command can be issued.

To end both the return message process(es) gracefully, issue the IIENDRPROC command.

Inside the Return Transaction Tables

It should be noted that customers may wish to make use of the return message processor to fetch and store the ESS transactions, but may then wish to use an entirely different mechanism to retrieve the transactions out of the return transaction tables.

Whether that is the customer's desire or not, it is helpful to understand the contents and structure of these return transaction tables.

IITRAN - Transaction Table

Quite simply, this table contains one record per transaction fetched. The record structure is also quite simple:

Field	DataType	Length	Precision	Description
Transaction ID	Decimal	21	0	Unique transaction ID. The file is keyed by this field.
Transaction Type	Character	20	n/a	Look at the page describing all the ESS Transactions. This field will be the value contained under the "Transaction Type" column on that page. This field tells you what kind of transaction you have.
Processed?	Character	1	n/a	Has this transaction been seen and its data been given to the dispatch system? A simple Y/N flag tells you whether you've already handled this transaction.
Macro Direction	Character	1	n/a	If this transaction is a macro, this is the direction ("F"orward or "R"eturn). Otherwise blank.
Macro Number	Character	3	n/a	If this transaction is a macro, this is the macro number. Otherwise blank.
Macro Version	Character	3	n/a	If this transaction is a macro, this is the macro version. Otherwise blank.
Macro Name	Character	50	n/a	If this transaction is a macro, this is the macro name. Otherwise blank.
Transaction Body	Character	32000	n/a	This is a generic character field containing either a fixed-format version of the data parsed from the return XML-based ESS transaction OR the raw, unparsed XML version of the ESS transaction. Transaction types having external data structures defined in the source file called TSTRANDS will be written to the transaction body field in fixed format form. The appropriate transaction type-specific fixed-format structure definition may then be mapped over these fixed format transactions to provide field-level access to their content. Transaction types such as DriverWorkflow and others which don't have associated structure definitions in the TSTRANDS file will be written to the transaction body field in their unparsed XML forms (eg. <T.6.04.0>...</T.6.04.0>. These XML transactions will need to be parsed via custom written logic in order to extract their contents.

IIMACDAT - Macro Data Table

In the event the return transaction type corresponds to a return macro transaction (T.2.RM01.0 through T.2.RM63.0), a record will be written to the IIMACDAT table for each macro field in the given transaction. Each of the macro field records will reference the transaction ID of their parent transaction record in the IITRAN table. Each macro field record will also contain the macro field's data and the customer-assigned dictionary tag ID of the macro field. Integrators may use the dictionary tag ID values of each macro field to identify and process the data from specific macro fields. The macro field dictionary tag IDs are maintained and viewable by the customer from the Services Portal macro template editor web pages.

The IIMACDAT table contains one record for each macro field associated with a given macro transaction record in the IITRAN table. The record structure is shown below:

Field	DataType	Length	Precision	Description
Transaction ID	Decimal	21	0	Transaction ID reference to a record in the IITRAN table.
Sequence Number	Decimal	3	0	Sequence number indicating the relative order of the macro fields (records) associated with the given transaction record in the IITRAN table.
Field Dictionary Tag	Character	50	n/a	Customer defined/assigned dictionary tag name associated with the macro field. This value was specified at the time the corresponding macro template definition was created or modified. The dictionary tag value may be used during integration to identify specific macro fields in a given return macro transaction.
Group Dictionary Tag	Character	50	n/a	Customer defined/assigned dictionary tag name associated with one or more macro fields (i.e. a macro field grouping). Macro field groups are optional and may be specified at the time the macro template definition is created or modified. The dictionary tag value may be used during integration to identify a group of related macro fields in a given return macro transaction. A blank value indicates the macro field is not associated with a macro field group.
Field Data	Character	100	n/a	A generic character field used to contain the actual macro field data. The field may contain all blanks, indicating no macro field data value was specified.

Inside the Return Callback Stub

The delivered stub program (IIRTNCUST) is intended to be modified for integrators to suit the customer's needs. However, despite needing on-site custom integration, it also contains quite a bit of delivered code. This code will be helpful in processing the return data stream. Logic includes:

Mechanism to obey an external "shut down" request
Mechanism to read from IITRAN
When finding IITRAN empty, a mechanism to wait a configurable amount of time before trying again
Mechanism to delete a transaction from IITRAN once it has been processed (IIDLTTRAN)

Modifying the Return Callback Stub

TSTRANDS is a delivered copy member source file containing transaction type structure definitions which may mapped directly over the transaction body field of each record in the IITRAN table. The structures in the copy member each correspond to one of the ESS Transaction Types published by the Services Portal. The IIRTNCUST stub makes use of this copy member to map the data contained in the transaction body field of IITRAN table records. NOTE: Any ESS transaction type that does not correspond to one of the data structures in TSTRANDS will be written in raw XML form to TRBODY in the IITRAN table.
The IIRTNCUST stub may be modified to detect/map/handle specific transaction types requiring custom integration logic. Within IIRTNCUST, you can see examples of how the program tests a transaction as it is pulled from the IITRAN data file, and then puts the transaction into one of the formats defined inside IITRANDS, if an appropriate transaction type structure is provided. In certain cases, such as DriverWorkflow transaction types, the transaction body field in the IITRAN table will contain the unparsed XML version of the transaction and no equivalent fixed format structure is available. In these cases, the XML must be parsed via custom logic which is not provided as part of the integration toolkit.
Inside IIRTNCUST, the IIDLTTRAN API is called to delete the transaction from IITRAN once it has been processed. If this does not take place, the same transaction will be read from IITRAN over and over.
Once the manual modifications to IIRTNCUST have been completed, the IIRTNCUST module and program must be created in the customer's site-specific library. It can then be submitted as a batch or auto-start process, where it will wake, process any existing IITRAN transaction data, and then sleep for a short period before repeating the cycle.
The following is an example of the CRTPGM command needed:
CRTPGM PGM(CustomerSiteSpecificLibrary/IIRTNCUST) ACTGRP(IIACTGRP)