GitOps

GitOps applies the discipline developers already use for application code—pull requests, reviews, tags, rollback—to IBM MQ infrastructure and configuration. Instead of an operator running runmqsc on a bastion without audit trail, QueueManager custom resources, Helm values, ConfigMaps with MQSC, and Kustomize overlays live in a repository Argo CD or Flux watches. When main branch updates, the tool applies manifests to the staging cluster; promotion merges to prod-branch triggers production sync. Beginners confuse GitOps with simply storing YAML in Git without automation; true GitOps closes the loop with reconciliation, diff views, and optional self-heal when someone ALTERs a channel attribute by hand at 2 a.m. This tutorial covers repository layout for multi-environment MQ, Argo CD Application patterns, Flux HelmRelease and Kustomization, combining GitOps with MQ Operator and Helm, managing MQSC and REST-defined objects, secrets handling, drift and self-heal policies, promotion workflows DEV to PROD, coordination with Terraform that creates clusters, and incident response when Git and production must temporarily diverge.

Repository Layout

A common monorepo structure: clusters/prod/mq/ contains kustomization.yaml referencing Helm chart version pin; environments/dev/values-dev.yaml overrides storage size; config/mqsc/queues.mqsc holds DEFINE QLOCAL statements applied by init job or config operator. Separate repos for platform (cluster) versus applications (queues) reduce blast radius. Tag releases correlating Git tag with deployed MQ fix pack for audits.

Argo CD Application Example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: mq-finance-prod
  namespace: argocd
spec:
  project: mq
  source:
    repoURL: https://git.example.com/mq-gitops.git
    targetRevision: prod-2026.05.17
    path: clusters/prod/finance-qm
    helm:
      valueFiles:
        - values-prod.yaml
  destination:
    server: https://kubernetes.default.svc
    namespace: mq-prod
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

automated sync applies Git within minutes; selfHeal reverts manual kubectl edits that differ from Git—dangerous for emergency tuning unless scoped. prune removes resources removed from Git—verify you do not delete PVCs unintentionally with finalizers. Use ApplicationSet for many queue managers from generator matrix.

Flux Pattern

Flux v2 uses GitRepository source plus HelmRelease or Kustomization pointing at paths. Reconciliation interval polls Git; helm-controller upgrades releases when chart version in Git changes. Notification controllers alert Slack on sync failure. Flux fits teams already standardized on CNCF GitOps Toolkit without Argo UI.

Explainer: Git as Remote Control

GitOps is a remote control whose buttons are merge commits. Press merge (approve PR) and the TV (cluster) switches channel to the show listed in the guide (manifest). If someone changes the channel with the TV buttons (kubectl), the remote can change it back when self-heal is on.

MQSC and Object Definitions in Git

Store MQSC in Git with idempotent patterns: define queues if missing using scripts that tolerate existing objects, or use REST admin Terraform provider resources. Non-idempotent CREATE fails on second run—use ALTER or check dspmqobj in wrapper scripts. Review MQSC in PRs like application code; link to change tickets. Export brownfield with dumpmqcfg, sanitize secrets, commit baseline, then Git becomes truth.

Secrets and Compliance

• Sealed Secrets encrypted to cluster controller only.
• SOPS with PGP or cloud KMS for values files.
• External Secrets pull from Vault at sync time.
• Audit log: Git commit SHA on every sync event.

Promotion DEV to PROD

1. Merge feature branch to develop; Argo syncs dev cluster.
2. Run integration tests against dev QM.
3. Tag or promote commit to prod branch with same manifest path pattern.
4. Prod Application requires manual sync or approval gate in Argo.
5. Post-sync smoke test channels and depths.

Drift, Break-Glass, and Terraform Boundaries

Terraform provisions VPC, AKS, or OpenShift cluster; GitOps deploys MQ inside. Avoid both Terraform and Argo managing the same Secret without coordination. Break-glass: disable selfHeal temporarily, manual fix incident, backport fix to Git within SLA. Drift reports from Argo UI become toil if teams never fix root cause—culture matters as much as tooling.

Troubleshooting

Sync failed — invalid YAML, CRD not installed, or RBAC denied Argo SA. OutOfSync loop — fields ignored by apply annotation; add ignoreDifferences for status fields. MQSC job failed — syntax error; check job logs. Wrong cluster — destination server misconfigured in Application.

Explain Like I'm Five: GitOps

GitOps is keeping a shared instruction book in the library. When the book updates, a robot reads it and arranges the MQ toys exactly as the book says, and tells grown-ups if someone moved a toy without updating the book.

Practice Exercises

Exercise 1

Deploy Argo CD app for lab MQ; change Git values; watch sync.

Exercise 2

Manually ALTER a queue; observe drift in UI; decide heal policy.

Exercise 3

Document promotion path from dev branch to prod tag.

Frequently Asked Questions

Test Your Knowledge