DFSORT Parallel Processing

What Is Parallel Processing in DFSORT?

In a single-threaded sort, one logical stream of work runs: input is read, records are sorted (in memory and/or sortwork), and output is written. In parallel processing, DFSORT can use multiple threads or engines so that several streams of work run concurrently. For example, different portions of the sort or merge can be handled by different threads, each potentially using a different CPU. The goal is to shorten elapsed time by using more of the system’s processing capacity. The exact implementation (how work is split, how many threads, and how they coordinate) depends on your DFSORT or z/OS Beyond Sort version and installation.

When Does Parallelism Help?

Parallel processing tends to help when:

• The sort or merge is large enough that the work can be meaningfully split across threads. Very small sorts may not benefit because the overhead of coordinating threads can outweigh the gain.
• The system has multiple CPUs or logical processors available so that threads can actually run concurrently.
• There is enough memory for the product to use multiple threads without excessive paging or contention.
• The job is not already limited by something else (e.g. I/O). If the bottleneck is disk, adding more CPU threads may not reduce elapsed time.

So parallelism is one lever among many: you still need sufficient memory (SIZE/MOSIZE), sortwork (DYNALLOC or SORTWK), and a realistic FILSZ. Enabling THREADS (or the equivalent) can then reduce elapsed time when the workload and system are suitable.

Factors That Affect Benefit

How Parallelism Is Controlled

When your DFSORT or z/OS Beyond Sort installation supports parallel processing, it is typically controlled by an option in the OPTION statement or by a PARM (or DFSPARM) value on the EXEC. A common option name is THREADS; the syntax might be THREADS=n to specify the number of threads, or a keyword like THREADS=YES. The exact option name, allowed values, and default behavior are product- and installation-dependent. For details on the THREADS option and syntax, see the THREADS and parallelism tutorial. Here we focus on the concepts: what parallelism is and when to consider it.

Example: OPTION THREADS (Illustrative)

The following is illustrative; your product may use a different keyword or syntax.

1
2
  OPTION THREADS=4
  SORT FIELDS=(1,20,CH,A)

This might request that DFSORT use up to four threads for sort work. Whether four is optimal depends on your system (number of CPUs, memory, and the size of the sort). Some installations leave the default (e.g. OPTIMAL or a product-chosen value) and only override when tuning for a specific workload.

Interaction with Memory and Sortwork

Parallel processing does not replace the need for adequate memory and sortwork. The product still needs enough central storage (SIZE/MOSIZE) and enough sortwork (DYNALLOC or SORTWK) to complete the sort. Parallelism spreads the work across threads; each thread may use memory and may generate I/O to sortwork. So tuning typically involves setting FILSZ, SIZE, and sortwork appropriately, and then enabling or tuning THREADS (or equivalent) if supported. If the job fails with capacity or resource errors, address memory and sortwork first; parallelism is an optimization on top of that.

I/O-Bound vs CPU-Bound

If your sort is I/O-bound—meaning the CPUs are often idle waiting for data from disk—adding more threads may not reduce elapsed time much, because the bottleneck is the disk, not the CPU. In that case, improving I/O (e.g. more or faster sortwork volumes, better BLKSIZE, or more memory to reduce sortwork I/O) may help more than increasing THREADS. If the job is CPU-bound and has enough memory and sortwork, then parallelism can help. Understanding your job’s profile (e.g. from SMF or performance tools) helps decide where to tune.

Product and Version Differences

Not all DFSORT environments support parallel processing. Support and option names (e.g. THREADS, PARALLEL) vary by product (e.g. DFSORT vs z/OS Beyond Sort) and release. Some installations may have the feature disabled or may use different PARM conventions. Always check your DFSORT Application Programming Guide or installation documentation to see whether parallelism is available and how to enable or tune it.

Explain It Like I'm Five

Imagine sorting a huge pile of cards. If one person does it, it takes a long time. If four friends help and each sorts a part of the pile, then you combine the parts, the job can finish sooner—but only if you have four friends (CPUs) and enough table space (memory) for everyone to work. If the slow part is running to get more cards (I/O), then having more friends doesn’t help until the running is faster. Parallel processing is like having more friends help with the sort; it works when the job is big enough and when you have enough helpers and space.

Exercises

1. Your large sort is I/O-bound. Would increasing THREADS alone be likely to help? What might you tune instead?
2. Why might a very small sort not benefit from parallel processing?
3. Where would you look to find the exact option name and syntax for parallel processing in your DFSORT installation?