Message Transformation

Message transformation answers a problem every integrator meets: the warehouse system expects fixed-length COBOL copybook bytes while the mobile app publishes JSON, and both teams refuse to change before next year's budget. IBM MQ faithfully delivers whatever bytes the producer put; transformation is the middleware step that converts those bytes into something the consumer understands, usually in a program or integration flow between two queues. Transformation pairs naturally with message routers—route first by content, then map per destination—and with canonical models that reduce N-squared mappings between N systems. Beginners embed fifty lines of string manipulation in each consumer; when the partner adds a field, five consumers break. This tutorial covers inbound versus outbound maps, MQMD Format and encoding fields, tools from App Connect mapping to XSLT and custom Java, transactional boundaries, large message handling, versioning schemas, testing golden messages, performance of XML parsing, and operational monitoring when transform error queues grow.

Inbound, Outbound, and Canonical Hub

Transformation placement
StyleFlowBenefit
Inbound adapterPartner → canonical → MQ hubIsolate partner quirks
Outbound adapterCanonical → partner formatOne hub, many tails
Point-to-pointFormat A → Format B directSimple two-party
EnrichAdd data from databaseLookup codes to descriptions

MQMD and Format Fields

After transformation, set MQMD.Format to match payload—MQSTR for string JSON, MQHRF2 with appropriate rules for RFH2, or leave binary for copybook. CodedCharSetId and Encoding must reflect UTF-8 versus EBCDIC consumers. Wrong descriptor fields cause consumers to mis-parse even when bytes are correct. JMS clients map properties automatically in some paths; MQI programs set fields explicitly.

text
1
2
3
4
5
6
7
GET TRANSFORM.IN (syncpoint) Read partner XML Map to canonical JSON SET MQMD.Format = MQSTR SET MQMD.CodedCharSetId = UTF-8 PUT CANONICAL.ORDERS.IN (syncpoint) COMMIT

Mapping Technologies

  • Graphical maps in IBM App Connect / ACE Mapping node.
  • XSLT for XML-to-XML in integration engines.
  • Java or Python scripts for JSON and CSV.
  • COBOL or PL/I on z/OS for mainframe-side transform before PUT.

Explainer: Translator Between Languages

Transformation is a translator standing between two people who speak different languages, reading the letter in French and writing the reply in English while the postal service (MQ) only carries envelopes.

Enrichment and Decomposition

Enrichment adds customer name from Db2 after reading accountId in payload. Decomposition splits one inbound order message into line-item messages on separate queues for parallel fulfillment—often combined with router. Each output PUT may share one syncpoint or use separate transactions depending on partial failure tolerance.

Large and Binary Messages

Transforming multi-megabyte payloads in memory risks OOM. Consider message segmentation, reference messages pointing to shared storage, or transform only headers while passing bulk via file transfer. MQ MaxMsgLength caps single message size—plan splits if maps cannot shrink content.

Versioning and Compatibility

schemaVersion field in canonical JSON lets maps branch logic. Deprecate v1 after consumers upgrade. Store sample messages per version in Git for regression tests in CI. Document breaking changes in interface catalog.

Error Handling

  1. Validation failure → DLQ with ReasonCode property set by app.
  2. Retryable partner timeout → backout and rely on MQ redelivery with limit.
  3. Poison message → move to BACKOUT.DLQ after threshold; alert.
  4. Never commit partial multi-put without business approval.

Performance and Scale

Pool transform workers as competing consumers. CPU-bound XML parsing may need more instances than IO-bound MQGET. Cache reference data locally with TTL. Avoid logging full payload at INFO level in production.

Explain Like I'm Five: Message Transformation

Message transformation is rewriting a note from picture drawings into words so the next friend can read it, before putting it in their mailbox.

Practice Exercises

Exercise 1

Sketch canonical JSON for one order; list two partner formats mapping to it.

Exercise 2

Which MQMD fields change when converting EBCDIC copybook to UTF-8 JSON?

Exercise 3

Define DLQ handling for validation failure versus system failure.

Frequently Asked Questions

Frequently Asked Questions

Test Your Knowledge

Test Your Knowledge

1. Transformation changes:

  • Message format/structure
  • Queue manager name
  • Disk RPM
  • JES class

2. Canonical model is:

  • Internal standard format
  • Random bytes
  • Empty queue
  • Channel type

3. Failed map with syncpoint:

  • Message backs out
  • Auto-commit
  • Delete QM
  • Skip DLQ

4. Transform service sits:

  • Between queues
  • Inside DNS
  • Only mainframe
  • In firewall only
Published
Read time21 min
AuthorMainframeMaster
Verified: Enterprise Integration Patterns