Aggregator

The aggregator is the patient clerk at the counter who waits for three signed forms before stamping the application approved. In IBM MQ integration, workers on credit, inventory, and tax queues each return a reply message; the aggregator service reads those messages, matches them using CorrelId or custom properties, and only when the expected number of distinct replies has arrived does it build the composite response and put to ORDER.RESPONSE. Without a disciplined aggregator, replies pile up unread or the client receives incomplete data. Scatter-gather tutorials describe firing parallel work; this page focuses on gather logic: state machines, timeouts, merging payloads, duplicate detection, and operational metrics. Beginners implement gather as a single loop that blocks forever on the third missing message—production needs timers and poison-message policies. Mainframe and distributed workers can all publish to GATHER.IN as long as correlation rules are identical.

Aggregator State Machine

Merge Strategies

Concatenate JSON objects into one document with legs as keys credit, stock, tax. XML wrapper with three child elements. Fixed binary record if all legs share copybook layout. Choose schema version in merged output so clients parse predictably. Transformation may run inside aggregator after merge if response format differs from leg formats.

1
2
3
4
5
6
7
8
9
10
11
12
State key = CorrelId R1
expectedLegs = {CREDIT, STOCK, TAX}
received = {}
 
On GET GATHER.IN (CorrelId=R1, property leg=CREDIT):
  received[CREDIT] = body
  if size(received) == 3:
    merged = mergeJson(received)
    PUT ORDER.RESPONSE CorrelId=R1 body=merged
    delete state R1
  else:
    save state R1 with TTL 30s

Explainer: Waiting for All Signatures

The aggregator is the teacher collecting permission slips from three parents. The field trip bus leaves only when every slip is in the folder with the same student name on it—or when the bell rings and the trip is cancelled.

Correlation and Selectors

MQI MQGET with MQMO_MATCH_CORREL_ID retrieves only messages for one request. Aggregator loop may use multiple concurrent correlation keys in state map. JMS consumers use message selector on JMSCorrelationID. Ensure scatter stamped the same bytes the aggregator filters—endian and padding matter in binary CorrelId.

Timeout Handling

• Scheduler scans state store for expired OPEN or PARTIAL entries.
• PUT timeout response to client with list of missing legs.
• Optionally publish compensating message to CANCEL queues.
• Alert operations if one leg consistently times out—upstream SLA issue.

Persistence of Gather State

In-memory HashMap loses state on pod kill. Redis with TTL keyed by CorrelId survives restarts. Database row per active gather suits audit. TTL must exceed worst-case worker latency but not grow unbounded—garbage collect completed and timed-out keys.

Step-by-Step: Build a Minimal Aggregator

1. Create GATHER.IN and RESPONSE queues.
2. POST three test messages manually with same CorrelId and leg property.
3. Run aggregator that merges on third GET.
4. Verify RESPONSE body structure.
5. Test duplicate leg message ignored.
6. Test timeout with only two messages sent.

Transactions and Failure

If merge PUT fails after GET, message is lost unless GET was under syncpoint and backs out—reply redelivers. If GET commits before merge succeeds, duplicate merge risk rises. Prefer UOW: GET reply, update state, PUT response, commit once when complete. Partial GET batching complicates—start simple.

Explain Like I'm Five: Aggregator

The aggregator is the friend who collects everyone's answers to the scavenger hunt and only rings the doorbell when the whole team has every item on the list.

Practice Exercises

Exercise 1

Design JSON merge schema for three leg replies.

Exercise 2

Choose Redis versus database for state; justify one paragraph.

Exercise 3

Write timeout user story for missing stock leg.

Frequently Asked Questions

Test Your Knowledge