Easytrieve Batch Processing Pattern

Canonical JOB Shell

1
2
3
4
5
6
7
8
9
10
11
JOB
 
PERFORM INIT-JOB
 
JOB INPUT DRIVER-FILE
  ADD 1 TO READ-COUNT
  PERFORM PROCESS-RECORD
END-JOB
 
PERFORM TERM-JOB
STOP

INIT-JOB opens auxiliary files, zeros counters, reads control parameters. PROCESS-RECORD holds business logic—validate, transform, WRITE output, optional PERFORM UPDATE. JOB INPUT implicitly reads DRIVER-FILE each iteration until EOF. TERM-JOB DISPLAYs or writes READ-COUNT, OK-COUNT, ERR-COUNT. STOP ends the program. END-JOB is alternative terminator depending on release and shop style; be consistent within your library.

JOB Activity Phases

JOB INPUT Versus GET

JOB INPUT

Best when exactly one input record per loop iteration drives processing. Easytrieve reads automatically at each JOB pass. After JOB INPUT block body executes, the product fetches the next record. Test IF EOF DRIVER-FILE inside the block before using buffer fields when the last iteration might process empty buffer on some designs—know your release EOF semantics on the final cycle.

GET

Use when only some iterations read—skip records based on prior logic—or when reading multiple files in one iteration. Manual GET inside PROCESS-RECORD with explicit IF EOF after each GET prevents reading when not needed. High-volume jobs sometimes GET from a presorted work file produced by an earlier SORT activity in the same program.

PROCESS-RECORD PROC

1
2
3
4
5
6
7
8
9
10
PROCESS-RECORD. PROC
  PERFORM VALIDATE-RECORD
  IF VAL-STATUS NE ZERO
     PERFORM WRITE-ERROR
     EXIT
  END-IF
  PERFORM TRANSFORM-FIELDS
  WRITE OUT-FILE
  ADD 1 TO WRITE-COUNT
END-PROC

Keep PROCESS-RECORD at one level of abstraction—PERFORM subtasks, do not inline fifty lines. EXIT on validation failure avoids WRITE. TRANSFORM-FIELDS holds MOVE, COMPUTE, and MASK preparation. Separate PROCs make unit testing with DISPLAY checkpoints easier during development.

Embedding SORT Activity

Batch pattern often runs SORT before the read loop when CONTROL reports or merge logic require ordered input. SORT activity belongs outside JOB INPUT loop—sort once, then JOB INPUT the sorted work file. Anti-pattern: SORT inside each iteration rebuilds entire file millions of times. Correct sequence: PERFORM INIT-JOB, SORT activity with USING and GIVING work files, JOB INPUT sorted file, PERFORM TERM-JOB.

1
2
3
4
5
6
7
SORT USING UNSORTED-FILE GIVING SORTED-WORK
  KEY DEPT ASCENDING EMP-ID ASCENDING
END-SORT
 
JOB INPUT SORTED-WORK
  ...
END-JOB

Embedding REPORT Activity

REPORT can run after processing completes—exception listing from ERROR-FILE via JOB INPUT on errors—or detail can print inside JOB INPUT using LINE in a REPORT tied to the same input. Pattern for exception-only report: first JOB pass writes ERROR-FILE, second JOB INPUT ERROR-FILE drives REPORT EXCEPTIONS with no UPDATE logic. Single-pass pattern: REPORT subactivity with JOB INPUT same file prints detail as records are read—efficient when input is presorted for CONTROL.

Multi-File Batch Designs

Counters and Control Totals

READ-COUNT, WRITE-COUNT, ERR-COUNT, SKIP-COUNT belong in working storage with RESET or explicit zero in INIT-JOB. TERM-JOB compares READ-COUNT to input trailer record when file carries batch header/trailer pattern. Mismatch triggers error-handling pattern hard stop. DISPLAY counts for operations; optionally WRITE statistics record to audit file keyed by run date and job name.

Restart and Idempotency

Batch jobs rerun after abends. Design PROCESS-RECORD idempotent when possible—UPDATE with same values twice should not double-apply financial effect. Use transaction keys and status fields on output records. Simulated checkpoint: every 10000 records WRITE restart key to CKPT-FILE; INIT-JOB reads CKPT-FILE and skips records until key matches—requires sorted input and documented rerun JCL. Many shops prefer simple full rerun from backup when idempotency is hard.

Performance Habits

• Minimize GET inside tight loops on large reference files—use table load in INIT-JOB or indexed READ keyed by driver.
• SORT once; never per record.
• Avoid DISPLAY every record in production—guard with DEBUG-FLAG.
• Match buffer sizes and BLKSIZE in JCL to file sizes—see performance tutorial section.
• Split enormous jobs into extract-transform-load phases when memory limits bite.

Batch Versus Online

Batch pattern uses JOB, JOB INPUT, file WRITE, REPORT, STOP. Online uses SCREEN, BEFORE-SCREEN, AFTER-SCREEN, terminal GOTO. Shared Library PROCs for validation and date formatting cross both worlds when called from appropriate activity. Do not PERFORM screen-only PROCs from batch JOB—activity scope rules forbid crossing boundaries. Duplicate thin wrappers if needed.

Testing Batch Jobs

1. Empty file: READ-COUNT zero, clean TERM, no abend.
2. Single good record: WRITE-COUNT one, output matches expected.
3. Single bad record: ERR-COUNT one, no UPDATE side effect.
4. Trailer mismatch: hard stop when configured.
5. Large file sample: runtime within SLA; counters match reference job.

Common Mistakes

• No INIT-JOB—counters carry garbage on rerun in same region.
• Processing records after EOF without guard.
• SORT inside JOB INPUT loop.
• STOP in middle of JOB without closing files—use TERM-JOB first.
• Mixing driver files without documenting which JOB INPUT is primary.

Explain It Like I'm Five

Batch processing is a factory belt. Boxes (records) arrive one at a time. A worker checks each box, fixes what is inside, puts it on the outbound belt, and counts boxes. When no boxes are left, the worker writes the totals on a clipboard and goes home. The pattern is the factory layout—where the counter sits, where checks happen—so every new product uses the same belt instead of inventing a new factory each time.

Exercises

1. Write INIT-JOB and TERM-JOB with four counters documented.
2. Sketch JOB with SORT then JOB INPUT sorted work file.
3. When would you replace JOB INPUT with manual GET? Write two scenarios.
4. Design two-phase batch: phase one writes work file, phase two reports exceptions.
5. List batch phases and map one statement type to each phase.