ESS High-Volume Dequeue

Last updated
Save as PDF

High Volume Dequeue (HVDQ)

Given a subscriber identifier, retrieves a block of transactions from the ESS queue for that subscriber.

This webservice differs from the original Dequeue in that it can potentially return a much larger block of transactions. Original Dequeue returned up to 64k worth of transactions. High-Volume Dequeue (HVDQ) can return many times that number. The maximum transaction block size is determined on a per-company basis, typically around the 1mb range.

This method is used by subscribers to retrieve ESS transactions. The actual number of transactions dequeued and returned in the transaction block is entirely determined by how many can fit into the return transaction block. Since ESS transaction type formats vary widely, a precise number of transactions that can be returned in one call cannot be determined.

ESS supports acknowledged dequeuing, which requires the subscriber to provide positive acknowledgement for a set of successfully received transactions dequeued in a prior call. The acknowledgement is provided by the subscriber in the form of the transactionIdOut input parameter indicating the highest value transaction ID successfully processed in the prior call. Upon receipt of a non-zero acknowledged transaction ID, ESS will permanently delete all of the subscriber's transactions whose transaction IDs are less than or equal to the acknowledged transaction ID. A zero-value acknowledged transaction ID indicates no prior transactions are being acknowledged.

Implementation Recommendations

Dequeue Calling Interval - Optimal client-side and server-side efficiency and performance can be acheived by ensuring each dequeue web service call returns the maximum number of allowable transactions possible (i.e. a full return transaction block). This can be accomplished by optimizing the time interval between successive dequeue web service calls. The optimal time interval will differ for each customer and is directly related to the volume of customer transactions. Generally:
- If calls result in a transaction block containing only a handful of transactions, the interval between calls needs to be increased.
- If calls result in a completely full transaction block, the interval between calls can be decreased.
- In order to achieve the highest overall throughput, the interval between calls may be reduced to zero so long as full blocks are being returned. However, as soon as less than full blocks are being returned, then the interval between calls should be increased.

Multi-Threaded Clients - Particular care should be taken when implementing multi-threaded clients capable of issuing concurrent calls to the dequeue web service for the same subscriber. If the threads do not synchronize their acknowledgement of the transactions dequeued from their shared subscriber, duplicate transaction data may be subsequently dequeued. While this will not result in any data loss or corruption on the server-side, it does complicate the client-side implementation and negatively impacts the efficiency of the overall process.
In-Line Processing - In order to maintain the highest throughput levels, it is necessary for clients to be written such that they avoid in-line processing of the transactions between each dequeue call. Upon receipt of each transaction block from dequeue, the client should persist the transactions locally, and then either sleep, or call again. Secondary processes should be written to perform client-side processing. Otherwise, the client's ability to keep up during periods of high activity (e.g. backlog draw downs) will be adversely impacted.

Calling Interface

TransactionBlock2 dequeue2 (int subscriberId, long transactionIdIn) throws WSException

subscriberId: The identifier of the subscriber from whose queue transactions will be retrieved. The valid range for this field is 1-999999999. ESS subscriber IDs are unique, read-only identifiers (integers) that are automatically assigned by the SP when you create a new subscriber via the user interface. These are referred to as the "reference" in the UI. Each subscriber you create is given its own transaction queue on the system. The system enqueues transactions to each subscriber's queue based upon the set of transaction type subscriptions held by the subscriber at that time. Customers dequeue transactions from a specific subscriber's queue by specifying its subscriber ID value when calling this web service. Typically, customers will create a single subscriber and a corresponding set of transaction type subscriptions. Although, based upon your integration needs, you may opt to create multiple subscribers, each having its own set of subscriptions. Each subscriber may have identical or different subscriptions if desired.
transactionIdIn: A long integer indicating the identifier of the last transaction successfully dequeued and processed by the subscriber. ESS will permanently delete all transactions for this subscriber whose identifiers are less than or equal to this value. ESS will return transactions that follow this transaction identifier in the queue. A zero-value acknowledged transaction ID indicates no prior transactions are being acknowledged. The valid range for this field is 0-99999999999999999 (17 digits).
returns TransactionBlock2

Authorization Requirements

The credentials supplied with this method call (as specified in the accompanying WS-Security header) must be authorized to the following application(s) and their associated role-based permissions.

Required Application Licenses
- Remote Integration

Required User Permissions
- Can dequeue transactions

Customer integrators are authorized to their own company's data only.

Sample XML Request

Only the SOAP body is shown. To see a sample SOAP Envelope and Header, click here.

<soap:Body>
   <dequeue2>
      <subscriberId>54</subscriberId>
      <transactionId>87235</transactionId>
   </dequeue2>
</soap:Body>

Sample XML Response

<soapenv:Body>
   <dequeue2Response>
      <dequeue2Return>
         <count>6</count>
         <transactionIdOut>58744654</transactionIdOut>
         <transactions>PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz48dHJjaz4=</transactions>
      </dequeue2Return>
   </dequeue2Response>
</soapenv:Body>

Exceptions

001 Unknown Error.
002 Required field not specified.
003 Company not authorized for integration.
004 User not authorized.
005 Subscriber ID not valid.
006 Company not found
007 Acknowledged transaction ID is invalid