Easytrieve Memory Optimization

REGION and Address Space Size

JCL REGION controls how much virtual storage your job step can consume below the 2 GB bar on z/OS. Easytrieve modules themselves grew over releases; Broadcom 11.6 release notes recommend REGION=0M so the job is not artificially capped while VFM and report processing expand. A fixed REGION that worked in 2005 may abend today when the same logic processes ten times the rows. Work with operations: 0M does not mean infinite memory—it means use available partition resources subject to site limits and MEMLIMIT policies.

1
2
3
4
//EZTPAY   EXEC PGM=EZTPA00,REGION=0M
//SYSPRINT DD SYSOUT=*
//SYSIN    DD DSN=PAYROLL.SOURCE(EZTPAY01),DISP=SHR
//EZTVFM   DD UNIT=VIO,SPACE=(CYL,(50,20))

Virtual File Manager Buffer Pool

VFM holds temporary file images while Easytrieve processes JOB input, SORT keys, and REPORT lines. The VFMSPAC site option and related PARM parameters define how much core storage seeds the buffer pool. Environmental options documentation suggests sizing VFMSPAC to a fraction of the partition—for example roughly 25 to 40 percent on larger regions. Too small a pool forces early overflow to EZTVFM, increasing I/O. Too large a pool can starve other work in the same address space. Tune with a representative job before changing production JCL globally.

EZTVFM Overflow Behavior

When VFM cannot hold all intermediate data, Easytrieve allocates space through EZTVFM. Performance articles note that multiple large reports sharing one overflow pack can contend for space and degrade elapsed time. Mitigations include WORKFILE on individual REPORT statements so each report uses its own sequential work dataset, spreading EZTVFM across faster DASD or VIO, and reducing input volume with SELECT before expensive activities. Treat EZTVFM contention as a symptom: the root cause is usually too much data kept in memory-oriented paths at once.

WORKFILE Versus VFM for Reports

Report processing builds intermediate line images and control-break accumulators. Keeping everything in VFM is convenient for small extracts but expensive for million-line reports. WORKFILE tells Easytrieve to use temporary sequential disk files for report work data. WORKFSPA (site option or PARM) sets cylinder allocation for dynamically allocated work files. On non-CICS batch, coding FILE on the REPORT statement can also dedicate a work file instead of sharing VFM. Compare elapsed time and EXCP counts in test with both approaches.

1
2
3
4
5
6
7
8
9
PARM WORKFILE
FILE MASTER FB(200 20000)
%MASTER
JOB INPUT MASTER
  PRINT BIGRPT
*
REPORT BIGRPT WORKFILE
  LINE MASTER-KEY MASTER-AMT
  TOTAL MASTER-AMT

Tables Versus VSAM KSDS

In-memory TABLE files load lookup data into the address space. Small reference tables are fast; very large tables increase search time and REGION requirements. Broadcom best practices recommend VSAM KSDS when estimated entries exceed 32767, when record sizes vary, or when segmented areas are needed—capabilities tables handle poorly. SEARCH against an indexed table file with ARG equal to the key can compile to a keyed READ, which is both faster and lighter on memory than scanning a flat table. Migration from TABLE to KSDS is a common memory optimization project on aging payroll and insurance codes.

TABLE max-entries reserves slots at initiation whether or not the external reference file is full. Ten tables each reserving eight thousand rows with forty-byte DESC fields can consume megabytes before the first input record arrives. INSTREAM TABLE embeds rows at compile time—fine for fifty status codes, wrong for a vendor file that grows weekly. When row counts approach the classic indexed limit near 32767, initiation may fail or performance degrades unpredictably depending on release and FLDMAX interactions. VSAM KSDS moves bytes to DASD and pays per-lookup I/O—a fair trade when TABLE cannot load at all. Db2 SQL FILE is the third path when reference data is shared across COBOL and Easytrieve or changes without recompile windows.

Table SEARCH Cost and Memory Together

Memory optimization and SEARCH CPU optimization overlap but are not identical. A TABLE that fits comfortably in REGION may still dominate CPU when JOB logic calls SEARCH eight times per input row on two million records—that is sixteen million binary searches per run. Shrinking DESC from eighty to thirty bytes saves initiation memory but does not reduce call count. Consolidate decode tables where business rules allow, cache hot keys in W fields when consecutive rows share department codes, and move mega-lookups to KSDS GET when I/O per row is acceptable. DISPLAY inside lookup PROC on every row adds SYSPRINT I/O that feels like a memory problem in elapsed time graphs but is actually diagnostic overhead—remove DEBUG DISPLAY before production promotion.

RETAIN, VIRTUAL MEMORY, and Forgotten Spill

FILE SORTWRK VIRTUAL MEMORY RETAIN keeps sort output in VFM across activities until CLOSE or job end. Correct when JOB and REPORT both consume the same ordered extract; expensive when you RETAIN a million-row file that downstream logic never reads again. Each RETAIN chain holds VFMSPAC pool bytes and may extend EZTVFM until STOP. FILE VIRTUAL DISK accepts earlier spill when you know MEMORY will exceed REGION—sometimes stabilizing a step that pages when MEMORY overcommits on a busy LPAR. Neither MEMORY nor DISK removes the EZTVFM DD requirement in typical 11.6 JCL; they change spill timing, not the need for overflow space on peak month-end volumes.

1
2
3
4
5
6
FILE SORTWRK FB(180 18000) VIRTUAL MEMORY
SORT TRANSIN TO SORTWRK USING (DEPT, ACCT) SIZE 1500000
JOB INPUT SORTWRK
  PRINT SUMMARY
CLOSE SORTWRK
* CLOSE releases VFM when RETAIN not coded — verify CLOSE rules

FILE Buffers and Working Storage at Initiation

Each FILE declaration allocates buffer space proportional to record length and block factor. DEFINE OCCURS multiplies element storage by occurrence count in W or FILE layouts. Installation FLDMAX caps any single field allocation—oversized OCCURS fails at compile. Sum FILE buffers, TABLE reservations, large W arrays, and estimated VFM peak when writing capacity documents. Sequential scans on multi-million-row inputs reuse one record buffer rather than loading the entire file into REGION; beginners sometimes confuse row count with address-space footprint. Multiple OPEN files each hold buffers until CLOSE. Retire unused FILE definitions from the Library section—initiation may still build structures for declared files even when JOB never reads them.

Scenario: Month-End Memory Pressure

A daily job succeeds with REGION=8M; month-end abends during TABLE initiation before SORT starts. Input quadrupled; six TABLE files still use max-entries 8000; two vendor files now exceed that count. SORT uses VIRTUAL MEMORY RETAIN; three REPORT blocks copy wide W amount fields into report work storage. Corrective path: raise max-entries only where justified, migrate oversized tables to KSDS, switch production from compile-and-go to link-edited load module, set REGION=0M per operations, increase EZTVFM CYL allocation, CLOSE RETAIN chains promptly, enable WORKFILE on the largest REPORT, and strip DEBUG STATE from PARM. Re-test on archived peak file before calendar close. Failures often combine TABLE, VFM, and REGION rather than a single mis-set knob.

Memory Anti-Patterns

Compile-and-Go Versus Link-Edit

Compile-and-go compiles and executes in one step—ideal for developers, not for nightly production throughput. Each run repeats compiler work and may allocate differently than a stable load module. Link-edit once, store the load module in a production library, and execute the module repeatedly. This does not shrink VFM requirements for huge reports, but it removes compile overhead and aligns with change-control expectations for memory-sensitive schedules.

Compiler symbol tables and parse trees can consume multiple megabytes on large source programs with hundreds of FILE and DEFINE statements. That footprint disappears on execute-only steps where PGM=yourmodule loads just runtime support from STEPLIB. When triaging S822 or storage ABEND, compare the same input on compile-and-go versus load module execute before tuning VFMSPAC—sometimes the fix is promotion, not options table change. Emergency compile-and-go on truncated input remains valid; schedule full-volume retest after link-edit promotion.

Coordinating With Operations

Application teams own accurate record counts, TABLE max-entries justification, and RETAIN chain documentation. Operations owns REGION job-class limits, EZTVFM volume selection, VFMSPAC and WORKFSPA in the options table, and paging policy on the LPAR. Effective tickets include job name, program module, peak input row count, record length, VIRTUAL file names with RETAIN, TABLE names with max-entries versus actual rows, SYSPRINT excerpts, and SMF step statistics. Avoid requesting global VFMSPAC increases without evidence from overflow timing and EZTVFM high-water marks on representative peak runs.

Diagnostic Overhead and Production PARM

DEBUG options such as STATE, FLOW, and FLDCHK consume additional storage and CPU during test. Broadcom advises removing verbose DEBUG and ABEXIT SNAP from production jobs after diagnosis. A program with ten thousand executable lines and STATE enabled carries tens of kilobytes of trace storage alone—small compared with a bad TABLE design but still worth removing when you no longer need abend correlation.

Memory Tuning Workflow

1. Baseline job with REGION=0M and current JCL on representative data.
2. Review SYSPRINT and SMF for VFM or EZTVFM stress indicators.
3. Enable WORKFILE for the largest REPORT; remeasure.
4. Replace oversized TABLE lookups with VSAM KSDS where appropriate.
5. Adjust VFMSPAC or EZTVFM only with operations approval.
6. Strip development DEBUG from the production PARM profile.

Explain It Like I'm Five

Your Easytrieve job carries work in a backpack (VFM). When the backpack is full, extra papers go into a locker (EZTVFM). If you bring too many papers, you walk slower because you keep running to the locker. WORKFILE is like putting each big project in its own box on a shelf instead of stuffing everything in the backpack. REGION=0M is asking for a bigger backpack allowance so you are not forced to stop early.

TABLE is a cheat sheet glued inside the cover—fine for fifty codes, but if you try to memorize every phone number in the city the cover rips. Put the big phone book in the library (VSAM) and look up only the numbers you need. Compile-and-go is doing homework and field trip in one morning; link-edit is finishing homework at home and bringing only the finished project on trip day.

Exercises

1. Identify which activities in a sample program create VFM intermediates (JOB, SORT, REPORT).
2. Estimate TABLE entry count and decide TABLE versus KSDS using the 32767 guideline.
3. Draft JCL with REGION=0M and EZTVFM on VIO for a test LPAR.
4. Add WORKFILE to one REPORT and list metrics you would compare before and after.
5. List DEBUG PARM options you would remove before production promotion.