Exactly-Once Delivery

Exactly-once delivery sounds like the perfect promise: every business event happens once, never skipped, never doubled. IBM MQ is excellent at not losing persistent messages and at redelivering when consumers fail—but that strength naturally produces at-least-once behavior unless you design above the queue manager. Vendors and slide decks sometimes say MQ delivers exactly-once; architects know the honest story is exactly-once processing is an application and data problem built on reliable messaging. This page defines the semantics, shows failure timelines where duplicates appear, explains idempotency keys and MsgId deduplication, contrasts with Kafka and database transactions, and gives MQ patterns beginners can implement in a lab without pretending the queue manager alone deduplicates your payroll.

Three Delivery Semantics Compared

Delivery guarantees in distributed systems
SemanticMeaningTypical MQ role
At-most-onceMessage may be lost; usually not redelivered
At-least-onceNo loss under rules; duplicates possible
Exactly-onceBusiness effect once despite duplicates

Why MQ Defaults Toward At-Least-Once

Consider a consumer that MQGETs a persistent message under syncpoint, writes to a database, then crashes before MQCMIT. The message returns to the queue (or remains available for redelivery depending on get options and backout count). A second instance gets the same message again. MQ did its job—no silent loss. Your order service might ship two parcels unless charge logic checks whether OrderId 8842 was already fulfilled. Exactly-once processing means the second delivery does not double-charge or double-ship even though MQ delivered twice.

Failure Timeline: Duplicate Delivery

  1. Producer MQPUT persistent message; commit succeeds. Message on ORDERS.IN.
  2. Consumer MQGET under syncpoint; message invisible to others.
  3. Consumer updates DB: status=SHIPPED for order 8842.
  4. Process crashes before MQCMIT.
  5. Syncpoint backs out the get; message available again.
  6. Second consumer gets same message—at-least-once transport.

Exactly-once processing at step 6 detects order 8842 already shipped (natural key) or recognizes MsgId already in PROCESSED_MSG table, and commits MQ without repeating side effects.

Building Blocks for Exactly-Once Processing

  • Idempotent operations: setting balance=100 twice has the same effect as once if you use absolute state rather than increment-by-10 twice.
  • Deduplication table: insert MsgId or CorrelId with unique constraint; duplicate insert fails safely.
  • Transactional outbox: DB row and outbound message in one transaction; separate relay publishes to MQ once.
  • Natural business keys: OrderId, payment reference, or idempotency token from the client.

Message Descriptor Fields You Will Use

MsgId uniquely identifies a message instance on the queue manager (applications can set it for request-reply matching). CorrelId links replies to requests. For deduplication, many teams store MsgId plus PutDate or a hash of the payload. Be careful resetting MsgId on republish—that creates a new identity and bypasses dedup. Expiry and report options do not replace idempotent design.

Syncpoint Without Illusions

MQCMIT under global transaction coordinates MQ with Db2, Oracle, or other XA resources so either all commit or all roll back. That is atomicity, not exactly-once across time. After successful commit, a later duplicate delivery is still possible if a downstream system republishes or if a duplicate message was put at the producer. Teach your team: syncpoint prevents torn updates inside one unit of work; idempotency handles redelivery across units of work.

When Teams Claim Exactly-Once

Single-threaded consumer, no syncpoint, auto-commit get—might process once in the happy path but loses messages on failure (drifting toward at-most-once). Kafka exactly-once streams use broker transactions and consumer offsets in specialized setups; MQ integrators achieve similar goals with different primitives. Document your estate honestly in architecture decision records: MQ provides durable at-least-once; services implement exactly-once business semantics.

Tutorial: Dedup Table Pattern (Conceptual SQL)

sql
1
2
3
4
5
-- After MQGET under syncpoint, before side effects: -- INSERT INTO processed_messages (msg_id, processed_at) -- VALUES (:MsgIdFromMQMD, CURRENT_TIMESTAMP); -- If unique violation -> duplicate delivery -> skip business logic, MQCMIT anyway -- If insert ok -> run business logic -> MQCMIT with DB + MQ in same UoW when using XA

Explain Like I'm Five: Exactly-Once

The teacher might read your homework request twice because the first time the classroom door slammed (crash). Exactly-once means you only write one answer in your notebook even if you hear the question twice—you check if you already did homework for that worksheet number before starting again.

Practice Exercises

Exercise 1: Classify the Flow

Non-persistent metrics, no syncpoint, auto-commit get. Which semantics apply for transport and for business processing?

Exercise 2: Design Dedup

Payment messages carry client idempotency key PAY-abc-123. Sketch tables and commit order with MQGET syncpoint.

Exercise 3: Audit Slide

Rewrite the marketing claim MQ guarantees exactly-once into accurate architecture language for compliance reviewers.

Frequently Asked Questions

Frequently Asked Questions

Test Your Knowledge

Test Your Knowledge

1. IBM MQ under failure typically gives consumers:

  • Exactly-once without application code
  • At-least-once unless you add idempotency
  • At-most-once always
  • No redelivery ever

2. A consumer crashes after updating the database but before MQCMIT. On restart:

  • The message is gone forever
  • The message may be delivered again
  • The queue manager deletes the queue
  • MsgId always changes

3. Storing processed MsgIds in a table is an example of:

  • Channel tuning
  • Application-level deduplication
  • DEFPSIST
  • Listener configuration

4. Exactly-once processing usually requires:

  • Only non-persistent messages
  • Idempotent handlers or dedup
  • Disabling syncpoint
  • MAXDEPTH(1)
Published
Read time15 min
AuthorMainframeMaster
Verified: IBM MQ 9.3 documentation