Integrating traditional JCL with modern DevOps practices for enhanced agility, reliability, and efficiency
The mainframe world of JCL and the modern DevOps landscape may seem worlds apart, but organizations are increasingly finding ways to bring these technologies together. DevOps principles like automation, continuous integration, and version control can be applied to JCL to create more agile, reliable, and efficient mainframe operations.
DevOps for JCL doesn't mean abandoning mainframe strengths like reliability and security. Instead, it means enhancing those strengths with modern practices for increased agility and efficiency.
Continuous Integration (CI) involves automatically building and testing code changes to detect issues early. While traditionally focused on distributed systems, CI can be extended to include JCL.
Pipeline Stage | Mainframe JCL Activities | Common Tools |
---|---|---|
Source Control | Store JCL in version control system | Git for z/OS, Endevor, ChangeMan ZMF |
Build | Generate environment-specific JCL, validate syntax | JCLCheck, JCLPLUS, Zowe CLI |
Test | Run jobs in test environment, verify results | Zowe API, JUnit, custom test harnesses |
Analyze | Check for coding standards, security issues | SonarQube with mainframe plugins, custom analyzers |
Deliver | Prepare JCL for deployment to higher environments | UrbanCode Deploy, Jenkins with z/OS plugin |
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152# Jenkins pipeline example for JCL pipeline { agent any stages { stage('Checkout') { steps { // Get JCL from source control checkout scm } } stage('Syntax Check') { steps { // Run syntax validation using Zowe CLI sh 'zowe jobs submit lf "./jcl/syntax-check.jcl" --wait' } } stage('Generate Test JCL') { steps { // Apply test environment parameters sh 'node ./scripts/apply-test-params.js' } } stage('Execute Test Jobs') { steps { // Submit test job and capture output sh 'zowe jobs submit lf "./jcl/test-job.jcl" --wait --output-file ./results.txt' } } stage('Analyze Results') { steps { // Check job results sh 'node ./scripts/validate-job-results.js' } } stage('Deploy to QA') { when { branch 'main' } steps { // Prepare JCL for QA sh 'node ./scripts/prepare-qa-deployment.js' } } } post { always { // Store results archiveArtifacts artifacts: 'results.txt', fingerprint: true } } }
This example shows a Jenkins pipeline that checks out JCL from source control, validates syntax, generates test-specific JCL, executes the job, analyzes results, and prepares for deployment to QA if all tests pass.
Important: Even with automation, ensure that changes to critical production JCL undergo appropriate review and validation. Automation enhances but doesn't replace sound governance processes.
Automated deployment is a cornerstone of DevOps that brings consistency, reliability, and efficiency to the process of moving JCL from development to production.
Use base JCL templates with environment-specific parameters injected at deployment time.
1234567891011//REPORTJB JOB (ACCT),'DAILY REPORT',CLASS=A, // MSGCLASS=X,NOTIFY=&SYSUID //* // SET ENV=%%ENVIRONMENT%% // SET DATE=%%CURRENT_DATE%% //* //STEP01 EXEC PGM=REPORTER //INPUT DD DSN=&ENV..DATA.MASTER,DISP=SHR //OUTPUT DD DSN=&ENV..REPORT.&DATE, // DISP=(NEW,CATLG,DELETE), // SPACE=(CYL,(10,5),RLSE)
Group related JCL changes into deployment packages with dependencies and rollback instructions.
1234567891011121314151617181920{ "packageId": "PAYROLL-2023-05", "description": "May 2023 Payroll System Updates", "components": [ { "name": "PAY001", "type": "JCL", "path": "/jcl/payroll/PAY001.jcl", "dependencies": ["PAYLIB"] }, { "name": "PAYRPT", "type": "JCL", "path": "/jcl/reports/PAYRPT.jcl", "dependencies": [] } ], "deployOrder": ["PAY001", "PAYRPT"], "rollbackInstructions": "..." }
A typical JCL deployment pipeline includes these key stages:
Stage | Purpose | Automation Activities |
---|---|---|
Preparation | Ready JCL for target environment | Parameter substitution, dependency resolution |
Validation | Ensure JCL is valid before deployment | Syntax checking, dependency validation |
Backup | Create safety net for rollback | Copy existing JCL, capture current state |
Deployment | Move JCL to target environment | Copy to libraries, update procedures |
Verification | Confirm successful deployment | Validation tests, smoke tests |
Notification | Inform stakeholders | Email, Slack, status updates |
When automating JCL deployment, implement progressive deployment strategies. Start with lower environments, then move to production with appropriate approval gates. Always include rollback capabilities for when issues occur.
Testing JCL changes is essential to prevent issues in production. DevOps practices introduce more rigorous, automated testing approaches.
A key DevOps principle is automating tests to ensure consistency and enable continuous integration.
1234567891011121314151617181920212223242526272829303132333435363738394041424344// Example Node.js script for automated JCL testing using Zowe CLI const { execSync } = require('child_process'); const fs = require('fs'); // 1. Submit JCL for testing console.log('Submitting test job...'); const jobOutput = execSync('zowe jobs submit lf "./test-jobs/PAYROLL.jcl" --wait').toString(); // 2. Extract job ID const jobIdMatch = jobOutput.match(/job id is (\w+)/i); const jobId = jobIdMatch ? jobIdMatch[1] : null; if (!jobId) { console.error('Failed to submit job'); process.exit(1); } // 3. Get job output console.log('Retrieving job output...'); execSync('zowe jobs view spool-file-by-id ' + jobId + ' --spisploc *'); // 4. Check return code const jobDetailsRaw = execSync('zowe jobs view job-status-by-jobid ' + jobId).toString(); const jobDetails = JSON.parse(jobDetailsRaw); // 5. Validate results if (jobDetails.retcode.endsWith('CC 0000')) { console.log('Job completed successfully'); // 6. Validate output datasets console.log('Validating output datasets...'); const outputCheck = execSync('zowe files list ds "PAYROLL.TEST.OUTPUT*"').toString(); if (outputCheck.includes('PAYROLL.TEST.OUTPUT.REPORT')) { console.log('Output dataset created successfully'); // 7. Further validation of content could be done here } else { console.error('Expected output dataset not found'); process.exit(1); } } else { console.error('Job failed with return code ' + jobDetails.retcode); process.exit(1); }
This example demonstrates using Zowe CLI in a Node.js script to automate JCL testing: submitting a job, retrieving output, checking return codes, and validating expected outcomes.
Effective JCL testing requires appropriate test data that:
Important: Never use actual production data containing sensitive information for testing. Implement data masking or synthetic data generation to maintain security and compliance.
For automated testing, clearly define success criteria:
Version control is fundamental to DevOps, providing history tracking, collaboration capabilities, and the foundation for continuous integration.
Traditional JCL environments often rely on library management systems, but modern approaches leverage Git-based workflows:
Organizing JCL in Git repositories requires thoughtful structure:
1234567891011121314151617181920212223242526mainframe-jcl/ ├── README.md # Documentation ├── .gitattributes # Git settings for proper line endings ├── applications/ # Application-specific JCL │ ├── payroll/ │ │ ├── daily/ # Daily job streams │ │ ├── monthly/ # Monthly processing │ │ └── quarterly/ # Quarterly processing │ └── accounting/ │ ├── ... ├── utilities/ # Shared utility JCL │ ├── backups/ │ ├── maintenance/ │ └── reports/ ├── procedures/ # Cataloged procedures │ ├── standard/ │ └── specialized/ ├── templates/ # JCL templates with parameters │ ├── batch-jobs/ │ ├── reports/ │ └── data-processing/ └── config/ # Environment configuration ├── dev/ ├── test/ ├── qa/ └── prod/
1234567891011121314# Treat JCL files as text with specific line ending handling *.jcl text eol=lf *.JCL text eol=lf *.cntl text eol=lf *.CNTL text eol=lf # Treat REXX files as text *.rexx text eol=lf *.REXX text eol=lf # Binary files that should not be modified *.load binary *.LOAD binary *.objlib binary
When transitioning to Git for JCL, focus first on establishing proper file conversion and synchronization processes before implementing complex workflows. This ensures the integrity of your JCL throughout the transition.
Effective change management processes ensure that JCL modifications are properly controlled, documented, and implemented with minimal risk.
Modern JCL change management balances governance with automation:
Traditional Approach | DevOps Approach | Benefits |
---|---|---|
Manual change request forms | Ticket system integrated with version control | Automated tracking, linking changes to requirements |
Change approval board meetings | Pull request reviews with automated checks | Faster approvals, consistent review criteria |
Manual impact analysis | Automated dependency and impact scanning | More thorough analysis, fewer missed impacts |
Weekend deployment windows | Smaller, more frequent deployments | Reduced risk, faster time to market |
Manual rollback procedures | Automated rollback capabilities | Faster recovery from issues |
A pull request (PR) workflow provides structured reviews and approvals:
1234567891011121314151617181920212223242526272829303132333435363738394041424344# JCL Change Request ## Description [Describe the purpose and scope of this JCL change] ## Related Ticket [Link to the ticket/issue in your tracking system] ## Type of Change - [ ] Bug fix - [ ] New feature - [ ] Performance improvement - [ ] Maintenance update ## JCL Files Modified - [List modified JCL files] ## Datasets/Resources Affected - [List datasets, databases, or other resources affected] ## Testing Performed - [ ] Syntax validation - [ ] Dev environment testing - [ ] Test environment validation - [ ] Performance impact assessment ## Test Results [Summarize test results, include job logs if relevant] ## Risk Assessment - Impact Level: [Low/Medium/High] - Rollback Plan: [Describe how to rollback if needed] ## Approvals Required - [ ] Technical Lead - [ ] Operations - [ ] Security (if applicable) - [ ] Business Owner (if applicable) ## Deployment Window [Specify when this should be deployed] ## Additional Notes [Any other relevant information]
Important: While automating change management, ensure compliance with regulatory requirements. Document approval workflows and maintain audit trails of all changes to satisfy auditors.
Beyond just tools and processes, modern JCL development adopts contemporary programming practices and principles.
JCL is a form of Infrastructure as Code, defining how batch workloads operate. Modern approaches:
Using templates with parameters improves consistency and reduces errors:
1234567891011121314//{{JOB_NAME}} JOB ({{ACCOUNT}}),'{{DESCRIPTION}}', // CLASS={{JOB_CLASS}}, // MSGCLASS={{MSG_CLASS}}, // NOTIFY=&SYSUID //* // SET ENV={{ENVIRONMENT}} //* //STEP01 EXEC PGM={{PROGRAM_NAME}} //INPUT DD DSN={{ENV}}.{{INPUT_DATASET}},DISP=SHR //OUTPUT DD DSN={{ENV}}.{{OUTPUT_DATASET}}, // DISP=(NEW,CATLG,DELETE), // SPACE=({{SPACE_TYPE}},({{PRIMARY}},{{SECONDARY}})), // DCB=(RECFM={{RECFM}},LRECL={{LRECL}}, // BLKSIZE={{BLKSIZE}})
123456789101112131415161718192021222324252627282930313233343536{ "dev": { "JOB_NAME": "DEVJOB", "ACCOUNT": "DEV#ACCT", "DESCRIPTION": "DEVELOPMENT JOB", "JOB_CLASS": "A", "MSG_CLASS": "X", "ENVIRONMENT": "DEV", "PROGRAM_NAME": "TESTPGM", "INPUT_DATASET": "TEST.INPUT", "OUTPUT_DATASET": "TEST.OUTPUT", "SPACE_TYPE": "CYL", "PRIMARY": "10", "SECONDARY": "5", "RECFM": "FB", "LRECL": "80", "BLKSIZE": "27920" }, "prod": { "JOB_NAME": "PRODJOB", "ACCOUNT": "PROD#ACCT", "DESCRIPTION": "PRODUCTION JOB", "JOB_CLASS": "P", "MSG_CLASS": "A", "ENVIRONMENT": "PROD", "PROGRAM_NAME": "PRODPGM", "INPUT_DATASET": "LIVE.INPUT", "OUTPUT_DATASET": "LIVE.OUTPUT", "SPACE_TYPE": "CYL", "PRIMARY": "50", "SECONDARY": "20", "RECFM": "FB", "LRECL": "80", "BLKSIZE": "27920" } }
Breaking JCL into reusable components improves maintainability:
Several tools help integrate JCL with modern development pipelines:
Modern JCL development bridges traditional mainframe strengths with contemporary DevOps practices. This hybrid approach allows organizations to maintain the reliability of mainframe systems while gaining the agility of modern development methodologies.
Apply your understanding of JCL and DevOps with these practical exercises:
Convert the following JCL into a template with parameterized values:
12345678910111213141516//PAYROLL JOB (ACCT#),'PAYROLL PROCESS',CLASS=A, // MSGCLASS=X,NOTIFY=USER01 //* //STEP01 EXEC PGM=PAYRUN //SYSPRINT DD SYSOUT=* //INPUT DD DSN=PROD.PAYROLL.INPUT,DISP=SHR //OUTPUT DD DSN=PROD.PAYROLL.OUTPUT, // DISP=(NEW,CATLG,DELETE), // SPACE=(CYL,(50,10)), // DCB=(RECFM=FB,LRECL=200,BLKSIZE=0) //CONFIG DD DSN=PROD.PAYROLL.CONFIG,DISP=SHR //* //STEP02 EXEC PGM=PAYRPT //SYSPRINT DD SYSOUT=* //INPUT DD DSN=PROD.PAYROLL.OUTPUT,DISP=SHR //REPORT DD SYSOUT=A
Task: Create:
Design a continuous integration pipeline for JCL changes:
Task: Create a detailed pipeline design that includes:
Create a flowchart or diagram to represent your pipeline visually.
Develop a simple framework for automated JCL testing:
Task: Design and document:
Focus on practical implementation that could work in a real mainframe environment.
Integrating JCL with DevOps practices bridges traditional mainframe strengths with modern development approaches:
While implementing DevOps for JCL presents challenges, the benefits are substantial: faster delivery, improved quality, better collaboration, and enhanced visibility. Organizations that successfully integrate JCL with DevOps gain competitive advantages through more responsive, reliable mainframe operations.
The journey to JCL DevOps is incremental. Start with version control, then add automated testing, followed by deployment automation. Each step provides benefits, and the cumulative effect transforms mainframe operations while preserving the stability that makes mainframes essential to enterprise computing.
1. Which DevOps principle is most directly related to JCL version control?
2. What is a key challenge when implementing continuous integration for JCL?
3. Which tool can be used to convert JCL to a more DevOps-friendly format?
4. What is the recommended approach for testing JCL changes in a DevOps pipeline?
5. Which statement about modern JCL development is FALSE?