ICETOOL JCL example using ALLDUP

ICETOOL JCL example using ALLDUP

---
Posted anonymously from tutorial: /tutorials/dfsort/dedup-separate-files

To extract all duplicates you could use the following ICETOOL:

SELECT FROM(IN) TO(DUPES) ON(1,10,CH) ALLDUPS DISCARD(UNIQUE)

IN = Input File
Dupes = First output file contain duplicate records.
Unique = A second output file containing the rest of the non duplicate records.

On(1,10,CH) = the field or full record to match on for duplicates.

If you want to match on multiple fields you can do something like ON(1,10,CH) ON(15,5CH) and have multiple ON statements.