Range Filtering with INCLUDE and OMIT

Inclusive Range: GE and LE

An inclusive range means the boundary values are included. For example, "between 100 and 500 inclusive" includes 100 and 500. In COND= you express that as:

• Field >= low — use GE (greater than or equal) with the low value.
• Field <= high — use LE (less than or equal) with the high value.
• Connect the two conditions with AND so both must be true.

Example: keep records where a 4-byte packed-decimal field at position 20 is between 100 and 500 inclusive:

1
INCLUDE COND=(20,4,PD,GE,100,AND,20,4,PD,LE,500)

The first condition (20,4,PD,GE,100) keeps records with value >= 100. The second (20,4,PD,LE,500) keeps records with value <= 500. AND means both must hold, so the result is 100 <= value <= 500. The same (start,length,format) is repeated because you are testing the same field twice. For a character field—for example a 4-byte code at position 1—you would use CH and character constants:

1
INCLUDE COND=(1,4,CH,GE,C'A100',AND,1,4,CH,LE,C'A999')

This keeps records where the character field collates between A100 and A999 (inclusive). Character order follows the EBCDIC collating sequence (or AC for ASCII if specified).

Exclusive Range: GT and LT

An exclusive range means the boundary values are excluded. "Strictly between 0 and 100" would include 1–99 but not 0 or 100. Use GT (greater than) for the low bound and LT (less than) for the high bound:

1
INCLUDE COND=(20,4,PD,GT,0,AND,20,4,PD,LT,100)

So only values 1 through 99 are kept. This is useful when 0 or 100 have special meaning (e.g. missing or invalid) and you want to exclude them. GT and LT never include the comparison value; GE and LE do.

Half-Open Ranges

Sometimes you want one boundary included and the other excluded. For example, "from 100 up to but not including 500" is a half-open interval [100, 500). Use GE for the low and LT for the high:

1
INCLUDE COND=(20,4,PD,GE,100,AND,20,4,PD,LT,500)

That keeps 100, 101, … 499 but not 500. Conversely, (20,4,PD,GT,100,AND,20,4,PD,LE,500) keeps 101–500 but not 100. Choosing GE/LE vs GT/LT lets you control exactly which endpoints are in the range.

Range Type Summary

Omitting Records in a Range

To omit records that fall inside a range (and keep everything else), use OMIT with the same range condition:

1
OMIT COND=(20,4,PD,GE,100,AND,20,4,PD,LE,500)

This drops every record where the field is between 100 and 500 inclusive. The output contains only records with value < 100 or value > 500. So OMIT with a range is the complement of INCLUDE with that range.

Keeping Records Outside a Range

If you want to keep only records that are outside a range (below the low or above the high), you have two options. Option 1: use OMIT for the range (as above); the remaining records are outside the range. Option 2: use INCLUDE with OR: (field < low OR field > high). In COND= that is:

1
INCLUDE COND=(20,4,PD,LT,100,OR,20,4,PD,GT,500)

So you keep records where the value is less than 100 or greater than 500. Both approaches yield the same set; choose the one that reads more clearly for your case (OMIT range vs INCLUDE outside range).

Numeric vs Character Ranges

For numeric fields (PD, ZD, BI, etc.), the comparison is numeric: 99 is less than 100, and 501 is greater than 500. For character (CH) fields, the comparison is byte-by-byte collating order. So the "range" is in character order, not necessarily numeric. For example, with CH, "9" can collate higher than "10" depending on encoding, so numeric-looking character fields should often be converted to a numeric format (or normalized in INREC) before range filtering if you want numeric semantics. For truly numeric data stored as PD or ZD, use PD or ZD in COND= so the range is numeric.

Combining a Range with Other Conditions

You can combine a range with additional conditions using AND or OR. For example, keep records where amount is between 100 and 500 and region is "NY":

1
INCLUDE COND=(20,4,PD,GE,100,AND,20,4,PD,LE,500,AND,5,2,CH,EQ,C'NY')

The range is one logical condition (GE low AND LE high); the third condition (region = NY) is ANDed with it. For more complex logic (e.g. (range AND code=A) OR (range AND code=B)), use parentheses as allowed by your DFSORT product; see the AND/OR and complex conditional expressions pages.

Explain It Like I'm Five

Imagine a number line. You want to keep only the numbers between 100 and 500. You put a "fence" at 100 and one at 500. "Between" can mean two things: (1) including the fences—so 100 and 500 are allowed (inclusive). (2) not including the fences—so only 101, 102, … 499 (exclusive). In DFSORT we don't have one word for "between"; we say "at least 100" (GE 100) and "at most 500" (LE 500), and we connect them with AND. If we want to leave out the fences, we say "more than 100" (GT 100) and "less than 500" (LT 500). Same idea for letters or codes: we say "from A100 up to A999" by using GE and LE (or GT and LT) on that field.

Exercises

1. Write INCLUDE COND= to keep records where a packed-decimal field at 30, length 4, is between 1 and 9999 inclusive.
2. Write OMIT COND= to drop records where a 2-byte character field at 10 is between C'01' and C'99' (inclusive).
3. Write INCLUDE COND= to keep records where the same PD field at 20, length 4, is strictly between 0 and 100 (exclude 0 and 100).
4. Keep records where amount (20,4,PD) is in [100, 500] and status (8,1,CH) is 'A'. Write the full COND=.