Teams modernizing integration architecture often ask whether to standardize on IBM MQ or Apache Kafka—or whether both belong in the estate. The answer is usually contextual: they solve overlapping but not identical problems. IBM MQ excels at reliable, transactional messaging between operational applications, especially where z/OS, CICS, and strict delivery governance matter. Kafka excels at high-throughput event streaming, log retention, and fan-out to many analytical consumers. This page compares models, persistence, ordering, operations, mainframe fit, and practical hybrid patterns so beginners can choose or combine technologies with clear eyes.
IBM MQ is message-oriented middleware built around queue managers, queues, and channels. Producers put messages; consumers get them. Messages leave the queue when consumed (unless browsing or specific configurations). Pub/sub uses topics and subscriptions with the queue manager routing copies to subscriber queues. The unit of work can participate in syncpoint with Db2 and other resources.
Kafka is a distributed commit log organized into topics partitioned across brokers. Producers append records; consumers track offsets per partition. Messages remain in the log according to retention policy (time or size), and many consumer groups can read the same data independently. Throughput and horizontal scale across partitions are first-class design goals.
| Topic | IBM MQ | Apache Kafka |
|---|---|---|
| Primary pattern | Queues, point-to-point and pub/sub | Partitioned append-only logs |
| Consumption | Get removes message (typical) | Consumer offset; message retained in log |
| Transactional integration | Syncpoint, XA, mature mainframe | Transactions exist; different operational profile |
| Replay | Not default; requires design or mirroring | Native via offset reset |
| Retention | Until consumed + persistent storage | Policy-based log retention for all consumers |
| z/OS core banking | Decades of production use | Often off-platform analytics path |
Persistent IBM MQ messages are written to queue manager logs and queue storage before the put completes (subject to syncpoint). Administrators reason about depth, backout counts, and dead-letter queues. Delivery is commonly at-least-once; exactly-once business effects require idempotent consumers.
Kafka replicates partitions across brokers for durability. Producers acknowledge based on acks settings; consumers commit offsets after processing. Duplicates can occur on failure unless applications use idempotency or transactional APIs carefully. Beginners should not equate "durable log" with "exactly-once business processing" without reading both products' guarantees for their version and configuration.
MQ ordering is typically preserved within a single queue for a single consumer sequence. Multiple consumers on one queue share work but may process out of order relative to each other unless the application enforces sequencing keys. Channels and clusters scale connectivity; queue depth and CPU on the queue manager remain capacity focal points.
Kafka ordering is guaranteed within a partition, not across partitions. Scale-out adds partitions; keys route related events to the same partition. Very high event rates for telemetry, clickstreams, or market data feeds often land on Kafka; mission-critical low-latency payment instructions often stay on MQ in regulated environments.
MQ operations teams use runmqsc, MQ Explorer, channel monitors, and z/OS SMF data. Skills include channel troubleshooting, certificate management, and queue sharing groups. Kafka operations focus on broker health, partition leadership, consumer lag, and ZooKeeper or KRaft quorum depending on version. Running both means two operational models unless a managed cloud service absorbs part of the burden.
IBM MQ is commercial software with enterprise support agreements and mainframe entitlements that reflect decades of regulated production use. Apache Kafka is open source at the core; organizations often pay for Confluent or other vendors for support, schema registry, and tooling. Total cost is not license alone: MQ teams on z/OS are expensive to hire but already exist in banks; Kafka teams for stream processing are expensive in cloud-native shops. A standards document should list which organization owns each product and which architecture review board approves new topics versus new queues.
Kafka ecosystems frequently adopt Schema Registry with Avro or JSON schemas and compatibility rules (backward, forward). IBM MQ applications traditionally use COBOL copybooks, XSD, JSON contracts in repositories, or enterprise service bus canonical models. Neither product removes the need for versioned message contracts. Hybrid bridges must translate schema versions at the boundary and handle unknown fields safely. Beginners should not assume Kafka schema enforcement fixes MQ-side producers that still send legacy fixed-length records.
MQ operations monitor queue depth, channel status, oldest message age, and AMQERR. Kafka operations monitor consumer lag, partition leadership, under-replicated partitions, and broker disk. Alerting standards differ: lag one million on a Kafka partition is normal in some analytics pipelines; depth one million on a payment MQ queue is a Sev1. When both products exist, dashboards should label which technology owns which SLO so on-call does not apply Kafka runbooks to MQ incidents.
| Metric | IBM MQ focus | Kafka focus |
|---|---|---|
| Backlog signal | CURDEPTH, XMITQ depth | Consumer lag per partition |
| Connection path | Channel CHSTATUS | Consumer group coordinator |
| Typical Sev1 | Payment queue full | Broker loss with min ISR |
| Config store | MQSC, object defs | Topic config, ACLs |
CICS, IMS, and batch on z/OS use MQI against queue managers on the LPAR with syncpoint coordination to Db2. Kafka clients on z/OS exist but are not the default integration style for core ledger programs written in COBOL. Standards for mainframe-led enterprises usually state: operational transactions on MQ; optional Kafka for downstream analytics fed by replication. Moving CICS PUT to a remote Kafka producer without redesigning two-phase commit semantics is a multi-year program, not a weekend migration.
A common pattern publishes operational truth on MQ, then a bridge process copies selected messages to Kafka topics for analytics. The operational path does not depend on Kafka availability; if the tap fails, core processing continues. Schema registry and compatibility rules apply on the Kafka side; MQ side keeps copybook or JSON contracts. IBM and partner products offer Kafka connectors; custom bridges are also widespread.
123456[CICS Payment] --put--> [MQ PAYMENTS.IN] --get--> [Ledger Service] | +--> [Bridge] --produce--> [Kafka payments.events] | +--> [Fraud analytics] +--> [Data lake]
IBM MQ is the armored truck between buildings. Kafka is the security camera recording everyone who entered the plaza all day so analysts can replay foot traffic. You would not replace the truck with cameras only; you might install cameras because the trucks already run.
IBM MQ is like the school office inbox: each letter is taken out when someone handles it, and important letters are filed in a safe until delivered. Kafka is like a long scroll on the wall that everyone can read: new lines are added at the bottom, and many kids can read the same scroll at different speeds, even reread old lines if the teacher leaves the scroll up long enough. Schools need both kinds of systems for different jobs.
For each scenario pick MQ, Kafka, or both: (a) authorize card payment from CICS, (b) train a fraud model on six months of transactions, (c) notify one shipping service to pack an order. Explain ordering and replay needs.
Sketch a bridge from MQ queue ORDERS.PLACED to Kafka topic orders.placed. What happens if Kafka is down for two hours? What happens if the bridge duplicates a message?
1. Kafka topics are best thought of as:
2. A typical reason to keep IBM MQ for core banking is:
3. Message consumption in classic IBM MQ point-to-point is:
4. High-volume analytics over historical events often favors: