Event streaming has become shorthand for platforms that append events to durable logs, retain them for days or years, and let many consumer groups read at independent offsets with replay. IBM MQ predates this vocabulary but underpins event-driven integration in banks and insurers worldwide. Beginners hear "streaming" and wonder if MQ is obsolete—it is not, but the concepts differ. This page teaches streaming vocabulary (events, logs, partitions, offsets, replay), contrasts them with MQ queues and topics, introduces event sourcing and CQRS at a high level, and shows how hybrid architectures combine MQ operational messaging with streaming analytics without conflating delivery guarantees.
An event is a record that something happened: AccountOpened, PaymentSettled, SensorThresholdExceeded. In MQ, events are messages on queues or publications on topics. In Kafka, events are records in a topic partition with key, value, timestamp, and headers. The payload format (JSON, Avro, COBOL copybook) is a contract separate from the transport. Streaming discussions emphasize immutable append: new facts arrive as new records rather than overwriting old rows in the bus itself.
| Aspect | Log-based stream | IBM MQ queue |
|---|---|---|
| Retention | Time/size policy keeps history | Until consumed (typical get) |
| Multiple readers | Consumer groups with offsets | Competing consumers share one queue |
| Replay | Reset offset to reprocess | Not default; mirror or redesign |
| Primary enterprise fit | Analytics, stream processing | Transactional operations, z/OS |
Streaming platforms partition topics for scale; ordering holds within a partition via message key. MQ ordering holds on a single queue for a single consumer sequence. Both require design when parallel consumers process related events—partition by account ID, or use multiple queues keyed by hash. Mainframe MQ hubs may not need partition counts in the thousands; cloud telemetry might. Match technology to volume and operational skills.
Event sourcing persists business changes as events. The authoritative history is the stream of events; projections build read models (balances, dashboards). Command Query Responsibility Segregation (CQRS) separates write models from read models. IBM MQ can carry commands to write-side services and publish resulting events to topics consumed by read-side projectors. The event store might be Db2 tables, Kafka, or both; MQ is the reliable courier between bounded contexts in many legacy-modernization programs.
Replay reprocesses historical events—useful after fixing a bug in a projector or rebuilding a data mart. Streaming offsets make replay explicit. MQ-centric systems replay by re-reading from a database outbox, resending from archive files, or mirroring into a stream first. Replayed events duplicate side effects unless handlers are idempotent (natural keys, upserts, dedup tables). Regulators may require audit trails; streaming retention policies must align with data retention law—not infinite logs by default.
MQ topics fan out current events to subscribers. Retained publications give late joiners the latest value, not full history. For near-real-time operational reactions—fraud hold, inventory decrement—MQ pub/sub remains appropriate. For data science over six months of clicks, add a streaming tier. IBM offers integration paths; architecture reviews should document which events are mirrored and whether mirroring is at-least-once.
Stream processors (Kafka Streams, Flink, Spark Streaming) window, aggregate, and join event streams in flight. MQ applications traditionally process one message at a time in COBOL or Java consumers. Complex event processing can still run on messages from MQ queues if latency requirements allow. Choosing stream processing implies staffing and platform cost—justify with analytics or real-time aggregation requirements MQ alone does not meet.
Imagine a long comic strip on the wall where every new box is something that happened in the school day. You can start reading from today's box, or go back to earlier boxes to remember what happened. IBM MQ is more like personal inboxes: when you take a letter out, it is usually gone from the slot. Streaming keeps the comic on the wall for a while; MQ delivers letters to the right people quickly and reliably.
A bug corrupted weekly revenue reports. Would you replay from MQ queues, a Kafka topic, or Db2 event tables? Explain.
Draw MQ for payment authorization and a stream for fraud analytics from the same business event.
1. In a Kafka-style log, messages are typically:
2. IBM MQ queue consumption usually:
3. Event sourcing means:
4. A hybrid MQ + streaming design often: