COBOL Parallel Execution

Parallel execution in COBOL allows multiple instances of a program or multiple independent jobs to run simultaneously, significantly reducing processing time for large workloads. While COBOL is traditionally a sequential language, mainframe systems provide several mechanisms to achieve parallel processing, including QUICKSTART, multi-job parallelism, Parallel Sysplex, and BatchPipes. Understanding these techniques is essential for optimizing performance in mainframe environments.

What is Parallel Execution in COBOL?

Parallel execution refers to running multiple COBOL program instances or jobs concurrently to process data faster. Unlike sequential execution where one task completes before the next begins, parallel execution divides work across multiple processors or systems, allowing simultaneous processing. This is particularly valuable for batch processing large datasets, where dividing the work can dramatically reduce elapsed time.

Key benefits of parallel execution:

• Reduced Elapsed Time: Multiple tasks run simultaneously, completing faster than sequential execution
• Better Resource Utilization: Leverages multiple CPUs and system resources effectively
• Scalability: Can scale processing by adding more parallel instances
• Fault Tolerance: If one instance fails, others can continue processing
• Throughput Improvement: Processes more data in the same time period

Parallel Processing with QUICKSTART

QUICKSTART enables multiple instances of a COBOL application to run simultaneously within a single job step. This approach divides the workload into smaller segments, leveraging multi-processor CPUs by operating in a multiple Task Control Block (TCB) structure.

How QUICKSTART Works

QUICKSTART allows a single COBOL program to spawn multiple parallel instances, each processing a portion of the data:

• Multiple TCBs: Each instance runs in its own Task Control Block
• Workload Division: Data is divided among parallel instances
• Independent Execution: Each instance processes its portion independently
• Resource Sharing: Instances can share DB2 plans or use different ones
• Error Isolation: If one instance abends, others continue

QUICKSTART Configuration

QUICKSTART is typically configured through JCL parameters and program design:

1
2
3
4
5
6
7
8
9
10
//STEP1    EXEC PGM=QUICKSTRT
//SYSPRINT DD  SYSOUT=*
//SYSIN    DD  *
  PARM='PARALLEL=4'
/*
//INPUT    DD  DSN=INPUT.DATA,DISP=SHR
//OUTPUT1  DD  DSN=OUTPUT1.DATA,DISP=(NEW,CATLG)
//OUTPUT2  DD  DSN=OUTPUT2.DATA,DISP=(NEW,CATLG)
//OUTPUT3  DD  DSN=OUTPUT3.DATA,DISP=(NEW,CATLG)
//OUTPUT4  DD  DSN=OUTPUT4.DATA,DISP=(NEW,CATLG)

Key Configuration Elements:

• PARALLEL=n: Specifies the number of parallel instances to create
• Input Division: Input data must be divided or each instance processes different records
• Output Separation: Each instance typically writes to separate output datasets
• DB2 Considerations: Each instance can use the same or different DB2 plan names

QUICKSTART Program Design

Programs designed for QUICKSTART parallel processing must handle data division and instance coordination:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
IDENTIFICATION DIVISION.
PROGRAM-ID. PARALLEL01.
AUTHOR. Mainframe Master.
DATE-WRITTEN. 2024.
 
ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
    SELECT INPUT-FILE ASSIGN TO INPUTDD
        ORGANIZATION IS SEQUENTIAL
        ACCESS MODE IS SEQUENTIAL
        FILE STATUS IS WS-INPUT-STATUS.
    SELECT OUTPUT-FILE ASSIGN TO OUTPUTDD
        ORGANIZATION IS SEQUENTIAL
        ACCESS MODE IS SEQUENTIAL
        FILE STATUS IS WS-OUTPUT-STATUS.
 
DATA DIVISION.
FILE SECTION.
FD  INPUT-FILE
    RECORDING MODE IS F
    RECORD CONTAINS 80 CHARACTERS.
01  INPUT-RECORD          PIC X(80).
 
FD  OUTPUT-FILE
    RECORDING MODE IS F
    RECORD CONTAINS 80 CHARACTERS.
01  OUTPUT-RECORD         PIC X(80).
 
WORKING-STORAGE SECTION.
01  WS-INPUT-STATUS       PIC XX.
01  WS-OUTPUT-STATUS      PIC XX.
01  WS-RECORD-COUNT       PIC 9(8) VALUE ZERO.
01  WS-INSTANCE-ID         PIC X(8).
01  WS-END-OF-FILE        PIC X(1) VALUE 'N'.
 
PROCEDURE DIVISION.
MAIN-PROCESSING.
    * Get instance identifier (if available)
    ACCEPT WS-INSTANCE-ID FROM ENVIRONMENT 'INSTANCE_ID'
    
    * Open files
    OPEN INPUT INPUT-FILE
    OPEN OUTPUT OUTPUT-FILE
    
    * Process records
    PERFORM UNTIL WS-END-OF-FILE = 'Y'
        READ INPUT-FILE
            AT END
                MOVE 'Y' TO WS-END-OF-FILE
            NOT AT END
                * Process the record
                PERFORM PROCESS-RECORD
                ADD 1 TO WS-RECORD-COUNT
        END-READ
    END-PERFORM
    
    * Close files
    CLOSE INPUT-FILE
    CLOSE OUTPUT-FILE
    
    DISPLAY 'Instance ' WS-INSTANCE-ID 
            ' processed ' WS-RECORD-COUNT ' records'
    
    STOP RUN.
 
PROCESS-RECORD.
    * Business logic for processing each record
    MOVE INPUT-RECORD TO OUTPUT-RECORD
    * Add processing logic here
    WRITE OUTPUT-RECORD.

Multi-Job Parallelism with JCL

Multi-job parallelism involves breaking a large sequential process into multiple independent jobs that run concurrently. This approach uses JCL to coordinate parallel execution of separate COBOL programs.

JCL Parallel Job Design

Design JCL to run multiple jobs simultaneously, each processing a portion of the data:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
//PARALLEL JOB (ACCT),'PARALLEL PROCESSING',CLASS=A
//*
//* Job 1: Process records 1-1000000
//JOB1     EXEC PGM=COBPROG1
//STEPLIB  DD  DSN=PROD.LOADLIB,DISP=SHR
//INPUT    DD  DSN=INPUT.DATA(1:1000000),DISP=SHR
//OUTPUT   DD  DSN=OUTPUT1.DATA,DISP=(NEW,CATLG)
//SYSOUT   DD  SYSOUT=*
//*
//* Job 2: Process records 1000001-2000000
//JOB2     EXEC PGM=COBPROG1
//STEPLIB  DD  DSN=PROD.LOADLIB,DISP=SHR
//INPUT    DD  DSN=INPUT.DATA(1000001:2000000),DISP=SHR
//OUTPUT   DD  DSN=OUTPUT2.DATA,DISP=(NEW,CATLG)
//SYSOUT   DD  SYSOUT=*
//*
//* Job 3: Process records 2000001-3000000
//JOB3     EXEC PGM=COBPROG1
//STEPLIB  DD  DSN=PROD.LOADLIB,DISP=SHR
//INPUT    DD  DSN=INPUT.DATA(2000001:3000000),DISP=SHR
//OUTPUT   DD  DSN=OUTPUT3.DATA,DISP=(NEW,CATLG)
//SYSOUT   DD  SYSOUT=*
//*
//* Merge step: Combine all outputs
//MERGE    EXEC PGM=SORT
//SORTIN   DD  DSN=OUTPUT1.DATA,DISP=SHR
//         DD  DSN=OUTPUT2.DATA,DISP=SHR
//         DD  DSN=OUTPUT3.DATA,DISP=SHR
//SORTOUT  DD  DSN=FINAL.OUTPUT,DISP=(NEW,CATLG)
//SYSIN    DD  *
  SORT FIELDS=(1,10,CH,A)
/*

Data Division Strategies

Effective data division is critical for multi-job parallelism:

• Range-Based Division: Divide data by record ranges (e.g., records 1-1000, 1001-2000)
• Key-Based Division: Divide by key ranges (e.g., customer IDs A-M, N-Z)
• Date-Based Division: Divide by date ranges (e.g., by month or quarter)
• Hash-Based Division: Use hash functions to distribute records evenly
• File-Based Division: Process different input files in parallel

Coordinating Parallel Jobs

Use JCL job dependencies and condition codes to coordinate parallel execution:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
//PARALLEL JOB (ACCT),'COORDINATED PARALLEL',CLASS=A
//*
//* Step 1: Prepare data division
//PREPARE  EXEC PGM=DATASTAG
//*
//* Step 2-4: Run parallel processing (all can run simultaneously)
//JOB1     EXEC PGM=COBPROG1,COND=(0,NE,PREPARE)
//INPUT    DD  DSN=INPUT.DATA(PART1),DISP=SHR
//OUTPUT   DD  DSN=OUTPUT1.DATA,DISP=(NEW,CATLG)
//*
//JOB2     EXEC PGM=COBPROG1,COND=(0,NE,PREPARE)
//INPUT    DD  DSN=INPUT.DATA(PART2),DISP=SHR
//OUTPUT   DD  DSN=OUTPUT2.DATA,DISP=(NEW,CATLG)
//*
//JOB3     EXEC PGM=COBPROG1,COND=(0,NE,PREPARE)
//INPUT    DD  DSN=INPUT.DATA(PART3),DISP=SHR
//OUTPUT   DD  DSN=OUTPUT3.DATA,DISP=(NEW,CATLG)
//*
//* Step 5: Merge results (waits for all parallel jobs)
//MERGE    EXEC PGM=SORT,COND=((0,EQ,JOB1),(0,EQ,JOB2),(0,EQ,JOB3))
//SORTIN   DD  DSN=OUTPUT1.DATA,DISP=SHR
//         DD  DSN=OUTPUT2.DATA,DISP=SHR
//         DD  DSN=OUTPUT3.DATA,DISP=SHR
//SORTOUT  DD  DSN=FINAL.OUTPUT,DISP=(NEW,CATLG)
//SYSIN    DD  *
  SORT FIELDS=(1,10,CH,A)
/*

IBM Parallel Sysplex

IBM Parallel Sysplex allows multiple mainframe systems to function as a single system image, enabling parallel processing across systems. This provides both performance benefits and high availability.

Parallel Sysplex Architecture

Parallel Sysplex consists of multiple z/OS systems working together:

• Multiple Systems: Two or more z/OS systems in a sysplex
• Shared DASD: Common storage accessible by all systems
• Coupling Facility: High-speed shared memory for coordination
• Workload Distribution: Automatic workload balancing across systems
• Data Sharing: Multiple systems can access the same data

Benefits of Parallel Sysplex

• Horizontal Scaling: Add systems to increase capacity
• High Availability: If one system fails, others continue
• Workload Balancing: Automatic distribution of work
• Continuous Operations: Maintenance without downtime
• Performance: Aggregate processing power of all systems

BatchPipes for Concurrent Processing

BatchPipes is a utility that enables concurrent processing by allowing data to be "piped" between jobs. Traditionally, if data is written to a sequential dataset, it cannot be read concurrently by another job. BatchPipes overcomes this limitation.

How BatchPipes Works

BatchPipes creates a virtual pipeline between jobs:

• Producer Job: Writes data to the pipe
• Consumer Job: Reads data from the pipe simultaneously
• Buffering: Data is buffered in memory for efficient transfer
• No Intermediate Storage: Data flows directly between jobs
• Reduced Elapsed Time: Jobs run concurrently instead of sequentially

BatchPipes JCL Example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
//PIPELINE JOB (ACCT),'BATCHPIPES EXAMPLE',CLASS=A
//*
//* Producer job: Generate and write data
//PRODUCER EXEC PGM=COBPROG1
//OUTPUT   DD  DSN=&&PIPE1,DISP=(,PASS),
//             DCB=(RECFM=FB,LRECL=80,BLKSIZE=8000),
//             UNIT=SYSDA,SPACE=(CYL,(10,10))
//SYSOUT   DD  SYSOUT=*
//*
//* Consumer job: Process data as it's produced
//CONSUMER EXEC PGM=COBPROG2,COND=(0,NE,PRODUCER)
//INPUT    DD  DSN=&&PIPE1,DISP=(OLD,DELETE)
//OUTPUT   DD  DSN=FINAL.OUTPUT,DISP=(NEW,CATLG)
//SYSOUT   DD  SYSOUT=*

Designing Programs for Parallel Execution

Programs must be designed with parallelism in mind to work effectively:

1. Stateless Design

• Avoid maintaining state between records
• Each record should be processable independently
• Minimize dependencies on processing order
• Use WORKING-STORAGE for temporary data only

2. Data Independence

• Ensure records can be processed in any order
• Avoid dependencies on previous records
• Design for record-level processing
• Handle data division cleanly

3. Resource Management

• Manage file access carefully (read-only vs write)
• Coordinate database access if using DB2
• Avoid contention on shared resources
• Use appropriate locking mechanisms

4. Error Handling

• Isolate errors to individual instances
• Log errors with instance identification
• Continue processing even if one instance fails
• Provide recovery mechanisms

Performance Considerations

Effective parallel execution requires careful performance planning:

Optimal Parallelism Level

• Too Few Instances: Underutilizes system resources
• Too Many Instances: Creates overhead and contention
• Optimal: Balance based on CPU count, I/O capacity, and workload
• Rule of Thumb: Start with number of CPUs, adjust based on performance

I/O Considerations

• Ensure sufficient I/O bandwidth for parallel access
• Distribute I/O across multiple volumes when possible
• Consider I/O contention when determining parallelism
• Monitor I/O wait times

Memory Requirements

• Each parallel instance requires memory
• Ensure sufficient storage for all instances
• Monitor memory usage and paging
• Adjust parallelism if memory constrained

Best Practices for Parallel Execution

1. Start Simple

• Begin with a small number of parallel instances
• Measure performance improvements
• Gradually increase parallelism
• Stop when benefits diminish

2. Monitor Performance

• Track elapsed time vs CPU time
• Monitor resource utilization
• Identify bottlenecks
• Adjust based on metrics

3. Test Thoroughly

• Test with production-like data volumes
• Verify correctness of parallel results
• Test error scenarios
• Validate data integrity

4. Document Design

• Document parallelism strategy
• Explain data division approach
• Document coordination mechanisms
• Maintain runbooks

Explain It Like I'm 5 Years Old

Exercises

Quiz