What is data qualification in COBOL?

The process of validating, verifying, and ensuring data meets specific criteria, standards, and business rules before processing or storage.

What are the types of data qualification?

Format validation, range checking, business rule validation, referential integrity checks, and data completeness verification for comprehensive data quality.

How do I implement data qualification?

Use validation procedures, qualification rules, error handling, and quality metrics to ensure data meets requirements before processing or storage.

MainframeMaster

COBOL Data Qualification

Data qualification validates, verifies, and ensures data meets specific criteria, standards, and business rules before processing or storage. Learn comprehensive validation techniques and quality assurance procedures.

Data Format Qualification

cobol

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
       WORKING-STORAGE SECTION.
       01 FORMAT-QUALIFICATION.
           05 INPUT-DATA.
               10 CUSTOMER-ID PIC X(10).
               10 PHONE-NUMBER PIC X(15).
               10 EMAIL-ADDRESS PIC X(40).
               10 DATE-FIELD PIC X(10).
           05 QUALIFICATION-RESULTS.
               10 ID-FORMAT-VALID PIC X.
               10 PHONE-FORMAT-VALID PIC X.
               10 EMAIL-FORMAT-VALID PIC X.
               10 DATE-FORMAT-VALID PIC X.
           05 QUALIFICATION-FLAGS.
               88 FORMAT-VALID VALUE 'Y'.
               88 FORMAT-INVALID VALUE 'N'.
       
       PROCEDURE DIVISION.
       PERFORM FORMAT-QUALIFICATION-CHECK
       
       FORMAT-QUALIFICATION-CHECK.
           MOVE '123456' TO CUSTOMER-ID
           MOVE '(123) 456-7890' TO PHONE-NUMBER
           MOVE 'john@email.com' TO EMAIL-ADDRESS
           MOVE '12/25/2023' TO DATE-FIELD
           
           *> Validate customer ID format (numeric, 6 digits)
           IF CUSTOMER-ID IS NUMERIC AND FUNCTION LENGTH(CUSTOMER-ID) = 6
               MOVE 'Y' TO ID-FORMAT-VALID
           ELSE
               MOVE 'N' TO ID-FORMAT-VALID
           END-IF.
           
           *> Validate phone number format
           PERFORM VALIDATE-PHONE-FORMAT
           
           *> Validate email format
           PERFORM VALIDATE-EMAIL-FORMAT
           
           *> Validate date format
           PERFORM VALIDATE-DATE-FORMAT
           
           DISPLAY 'Format qualification results:'
           DISPLAY 'ID Format Valid: ' ID-FORMAT-VALID
           DISPLAY 'Phone Format Valid: ' PHONE-FORMAT-VALID
           DISPLAY 'Email Format Valid: ' EMAIL-FORMAT-VALID
           DISPLAY 'Date Format Valid: ' DATE-FORMAT-VALID.
       
       VALIDATE-PHONE-FORMAT.
           *> Check for valid phone format (XXX) XXX-XXXX
           IF PHONE-NUMBER(1:1) = '(' AND PHONE-NUMBER(5:1) = ')' 
               AND PHONE-NUMBER(6:1) = ' ' AND PHONE-NUMBER(10:1) = '-'
               MOVE 'Y' TO PHONE-FORMAT-VALID
           ELSE
               MOVE 'N' TO PHONE-FORMAT-VALID
           END-IF.

Implement format qualification to validate data structure, patterns, and formats. Check numeric formats, phone number patterns, email formats, and date formats for data consistency.

Range and Value Qualification

cobol

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
       WORKING-STORAGE SECTION.
       01 RANGE-VALUE-QUALIFICATION.
           05 INPUT-VALUES.
               10 AGE PIC 9(3).
               10 SALARY PIC 9(8)V99.
               10 GRADE PIC X.
               10 SCORE PIC 9(3).
           05 QUALIFICATION-RULES.
               10 MIN-AGE PIC 9(3) VALUE 18.
               10 MAX-AGE PIC 9(3) VALUE 65.
               10 MIN-SALARY PIC 9(8)V99 VALUE 30000.00.
               10 MAX-SALARY PIC 9(8)V99 VALUE 200000.00.
               10 VALID-GRADES PIC X(10) VALUE 'ABCDEF'.
               10 MIN-SCORE PIC 9(3) VALUE 0.
               10 MAX-SCORE PIC 9(3) VALUE 100.
           05 QUALIFICATION-RESULTS.
               10 AGE-VALID PIC X.
               10 SALARY-VALID PIC X.
               10 GRADE-VALID PIC X.
               10 SCORE-VALID PIC X.
       
       PROCEDURE DIVISION.
       PERFORM RANGE-VALUE-QUALIFICATION-CHECK
       
       RANGE-VALUE-QUALIFICATION-CHECK.
           MOVE 25 TO AGE
           MOVE 75000.00 TO SALARY
           MOVE 'B' TO GRADE
           MOVE 85 TO SCORE
           
           *> Validate age range
           IF AGE >= MIN-AGE AND AGE <= MAX-AGE
               MOVE 'Y' TO AGE-VALID
           ELSE
               MOVE 'N' TO AGE-VALID
           END-IF.
           
           *> Validate salary range
           IF SALARY >= MIN-SALARY AND SALARY <= MAX-SALARY
               MOVE 'Y' TO SALARY-VALID
           ELSE
               MOVE 'N' TO SALARY-VALID
           END-IF.
           
           *> Validate grade value
           IF GRADE = 'A' OR GRADE = 'B' OR GRADE = 'C' 
               OR GRADE = 'D' OR GRADE = 'F'
               MOVE 'Y' TO GRADE-VALID
           ELSE
               MOVE 'N' TO GRADE-VALID
           END-IF.
           
           *> Validate score range
           IF SCORE >= MIN-SCORE AND SCORE <= MAX-SCORE
               MOVE 'Y' TO SCORE-VALID
           ELSE
               MOVE 'N' TO SCORE-VALID
           END-IF.
           
           DISPLAY 'Range and value qualification results:'
           DISPLAY 'Age Valid: ' AGE-VALID
           DISPLAY 'Salary Valid: ' SALARY-VALID
           DISPLAY 'Grade Valid: ' GRADE-VALID
           DISPLAY 'Score Valid: ' SCORE-VALID.

Implement range and value qualification to ensure data falls within acceptable ranges and contains valid values. Check numeric ranges, valid value sets, and business rule constraints.

Business Rule Qualification

cobol

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
       WORKING-STORAGE SECTION.
       01 BUSINESS-RULE-QUALIFICATION.
           05 BUSINESS-DATA.
               10 ACCOUNT-BALANCE PIC 9(9)V99.
               10 CREDIT-LIMIT PIC 9(9)V99.
               10 TRANSACTION-AMOUNT PIC 9(9)V99.
               10 ACCOUNT-STATUS PIC X.
               10 CUSTOMER-TYPE PIC X.
           05 BUSINESS-RULES.
               10 MIN-BALANCE PIC 9(9)V99 VALUE 100.00.
               10 MAX-TRANSACTION PIC 9(9)V99 VALUE 10000.00.
               10 OVERDRAFT-LIMIT PIC 9(9)V99 VALUE 500.00.
           05 QUALIFICATION-RESULTS.
               10 BALANCE-RULE-VALID PIC X.
               10 TRANSACTION-RULE-VALID PIC X.
               10 OVERDRAFT-RULE-VALID PIC X.
               10 STATUS-RULE-VALID PIC X.
       
       PROCEDURE DIVISION.
       PERFORM BUSINESS-RULE-QUALIFICATION-CHECK
       
       BUSINESS-RULE-QUALIFICATION-CHECK.
           MOVE 500.00 TO ACCOUNT-BALANCE
           MOVE 5000.00 TO CREDIT-LIMIT
           MOVE 2000.00 TO TRANSACTION-AMOUNT
           MOVE 'A' TO ACCOUNT-STATUS
           MOVE 'P' TO CUSTOMER-TYPE
           
           *> Validate minimum balance rule
           IF ACCOUNT-BALANCE >= MIN-BALANCE
               MOVE 'Y' TO BALANCE-RULE-VALID
           ELSE
               MOVE 'N' TO BALANCE-RULE-VALID
           END-IF.
           
           *> Validate transaction amount rule
           IF TRANSACTION-AMOUNT <= MAX-TRANSACTION
               MOVE 'Y' TO TRANSACTION-RULE-VALID
           ELSE
               MOVE 'N' TO TRANSACTION-RULE-VALID
           END-IF.
           
           *> Validate overdraft rule
           IF ACCOUNT-BALANCE >= (0 - OVERDRAFT-LIMIT)
               MOVE 'Y' TO OVERDRAFT-RULE-VALID
           ELSE
               MOVE 'N' TO OVERDRAFT-RULE-VALID
           END-IF.
           
           *> Validate status rule
           IF ACCOUNT-STATUS = 'A' OR ACCOUNT-STATUS = 'I' OR ACCOUNT-STATUS = 'S'
               MOVE 'Y' TO STATUS-RULE-VALID
           ELSE
               MOVE 'N' TO STATUS-RULE-VALID
           END-IF.
           
           DISPLAY 'Business rule qualification results:'
           DISPLAY 'Balance Rule Valid: ' BALANCE-RULE-VALID
           DISPLAY 'Transaction Rule Valid: ' TRANSACTION-RULE-VALID
           DISPLAY 'Overdraft Rule Valid: ' OVERDRAFT-RULE-VALID
           DISPLAY 'Status Rule Valid: ' STATUS-RULE-VALID.

Implement business rule qualification to validate data against specific business logic and constraints. Check account balances, transaction limits, overdraft rules, and status requirements.

Referential Integrity Qualification

cobol

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
       WORKING-STORAGE SECTION.
       01 REFERENTIAL-INTEGRITY-QUALIFICATION.
           05 REFERENCE-DATA.
               10 CUSTOMER-ID PIC 9(6).
               10 ACCOUNT-NUMBER PIC 9(8).
               10 PRODUCT-CODE PIC X(10).
               10 DEPARTMENT-ID PIC X(3).
           05 QUALIFICATION-RESULTS.
               10 CUSTOMER-EXISTS PIC X.
               10 ACCOUNT-EXISTS PIC X.
               10 PRODUCT-EXISTS PIC X.
               10 DEPARTMENT-EXISTS PIC X.
           05 INTEGRITY-STATUS.
               10 INTEGRITY-VALID PIC X.
                   88 INTEGRITY-CHECK-PASSED VALUE 'Y'.
                   88 INTEGRITY-CHECK-FAILED VALUE 'N'.
       
       PROCEDURE DIVISION.
       PERFORM REFERENTIAL-INTEGRITY-QUALIFICATION-CHECK
       
       REFERENTIAL-INTEGRITY-QUALIFICATION-CHECK.
           MOVE 123456 TO CUSTOMER-ID
           MOVE 87654321 TO ACCOUNT-NUMBER
           MOVE 'PROD001' TO PRODUCT-CODE
           MOVE 'IT' TO DEPARTMENT-ID
           
           *> Check customer exists in master file
           PERFORM CHECK-CUSTOMER-EXISTS
           
           *> Check account exists
           PERFORM CHECK-ACCOUNT-EXISTS
           
           *> Check product exists
           PERFORM CHECK-PRODUCT-EXISTS
           
           *> Check department exists
           PERFORM CHECK-DEPARTMENT-EXISTS
           
           *> Determine overall integrity status
           IF CUSTOMER-EXISTS = 'Y' AND ACCOUNT-EXISTS = 'Y' 
               AND PRODUCT-EXISTS = 'Y' AND DEPARTMENT-EXISTS = 'Y'
               MOVE 'Y' TO INTEGRITY-VALID
           ELSE
               MOVE 'N' TO INTEGRITY-VALID
           END-IF.
           
           DISPLAY 'Referential integrity qualification results:'
           DISPLAY 'Customer Exists: ' CUSTOMER-EXISTS
           DISPLAY 'Account Exists: ' ACCOUNT-EXISTS
           DISPLAY 'Product Exists: ' PRODUCT-EXISTS
           DISPLAY 'Department Exists: ' DEPARTMENT-EXISTS
           DISPLAY 'Overall Integrity: ' INTEGRITY-VALID.
       
       CHECK-CUSTOMER-EXISTS.
           MOVE 'N' TO CUSTOMER-EXISTS
           OPEN INPUT CUSTOMER-MASTER-FILE
           PERFORM UNTIL EOF OR CUSTOMER-EXISTS = 'Y'
               READ CUSTOMER-MASTER-FILE AT END SET EOF TO TRUE
               NOT AT END
                   IF CUSTOMER-MASTER-ID = CUSTOMER-ID
                       MOVE 'Y' TO CUSTOMER-EXISTS
                   END-IF
               END-READ
           END-PERFORM
           CLOSE CUSTOMER-MASTER-FILE.

Implement referential integrity qualification to ensure that referenced data exists in master files. Check customer existence, account validity, product codes, and department references.

Data Completeness Qualification

cobol

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
       WORKING-STORAGE SECTION.
       01 DATA-COMPLETENESS-QUALIFICATION.
           05 REQUIRED-FIELDS.
               10 CUSTOMER-ID PIC 9(6).
               10 CUSTOMER-NAME PIC X(30).
               10 CUSTOMER-ADDRESS PIC X(50).
               10 CUSTOMER-PHONE PIC X(12).
               10 CUSTOMER-EMAIL PIC X(40).
           05 COMPLETENESS-CHECKS.
               10 ID-PRESENT PIC X.
               10 NAME-PRESENT PIC X.
               10 ADDRESS-PRESENT PIC X.
               10 PHONE-PRESENT PIC X.
               10 EMAIL-PRESENT PIC X.
           05 COMPLETENESS-SCORE PIC 9(3)V99.
           05 COMPLETENESS-STATUS PIC X.
               88 DATA-COMPLETE VALUE 'Y'.
               88 DATA-INCOMPLETE VALUE 'N'.
       
       PROCEDURE DIVISION.
       PERFORM DATA-COMPLETENESS-QUALIFICATION-CHECK
       
       DATA-COMPLETENESS-QUALIFICATION-CHECK.
           MOVE 123456 TO CUSTOMER-ID
           MOVE 'JOHN SMITH' TO CUSTOMER-NAME
           MOVE '123 MAIN ST' TO CUSTOMER-ADDRESS
           MOVE '555-123-4567' TO CUSTOMER-PHONE
           MOVE SPACES TO CUSTOMER-EMAIL
           
           *> Check required field presence
           IF CUSTOMER-ID > 0
               MOVE 'Y' TO ID-PRESENT
           ELSE
               MOVE 'N' TO ID-PRESENT
           END-IF.
           
           IF CUSTOMER-NAME NOT = SPACES
               MOVE 'Y' TO NAME-PRESENT
           ELSE
               MOVE 'N' TO NAME-PRESENT
           END-IF.
           
           IF CUSTOMER-ADDRESS NOT = SPACES
               MOVE 'Y' TO ADDRESS-PRESENT
           ELSE
               MOVE 'N' TO ADDRESS-PRESENT
           END-IF.
           
           IF CUSTOMER-PHONE NOT = SPACES
               MOVE 'Y' TO PHONE-PRESENT
           ELSE
               MOVE 'N' TO PHONE-PRESENT
           END-IF.
           
           IF CUSTOMER-EMAIL NOT = SPACES
               MOVE 'Y' TO EMAIL-PRESENT
           ELSE
               MOVE 'N' TO EMAIL-PRESENT
           END-IF.
           
           *> Calculate completeness score
           MOVE 0 TO COMPLETENESS-SCORE
           IF ID-PRESENT = 'Y' ADD 20 TO COMPLETENESS-SCORE END-IF
           IF NAME-PRESENT = 'Y' ADD 20 TO COMPLETENESS-SCORE END-IF
           IF ADDRESS-PRESENT = 'Y' ADD 20 TO COMPLETENESS-SCORE END-IF
           IF PHONE-PRESENT = 'Y' ADD 20 TO COMPLETENESS-SCORE END-IF
           IF EMAIL-PRESENT = 'Y' ADD 20 TO COMPLETENESS-SCORE END-IF
           
           *> Determine completeness status
           IF COMPLETENESS-SCORE >= 80
               MOVE 'Y' TO COMPLETENESS-STATUS
           ELSE
               MOVE 'N' TO COMPLETENESS-STATUS
           END-IF.
           
           DISPLAY 'Data completeness qualification results:'
           DISPLAY 'ID Present: ' ID-PRESENT
           DISPLAY 'Name Present: ' NAME-PRESENT
           DISPLAY 'Address Present: ' ADDRESS-PRESENT
           DISPLAY 'Phone Present: ' PHONE-PRESENT
           DISPLAY 'Email Present: ' EMAIL-PRESENT
           DISPLAY 'Completeness Score: ' COMPLETENESS-SCORE '%'
           DISPLAY 'Data Complete: ' COMPLETENESS-STATUS.

Implement data completeness qualification to ensure all required fields are present and populated. Calculate completeness scores and determine if data meets minimum completeness requirements.

Data Quality Metrics

cobol

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
       WORKING-STORAGE SECTION.
       01 DATA-QUALITY-METRICS.
           05 QUALITY-METRICS.
               10 TOTAL-RECORDS PIC 9(8) VALUE 0.
               10 VALID-RECORDS PIC 9(8) VALUE 0.
               10 INVALID-RECORDS PIC 9(8) VALUE 0.
               10 QUALITY-SCORE PIC 9(3)V99.
           05 QUALITY-BREAKDOWN.
               10 FORMAT-ERRORS PIC 9(6) VALUE 0.
               10 RANGE-ERRORS PIC 9(6) VALUE 0.
               10 BUSINESS-RULE-ERRORS PIC 9(6) VALUE 0.
               10 INTEGRITY-ERRORS PIC 9(6) VALUE 0.
               10 COMPLETENESS-ERRORS PIC 9(6) VALUE 0.
       
       PROCEDURE DIVISION.
       PERFORM CALCULATE-DATA-QUALITY-METRICS
       
       CALCULATE-DATA-QUALITY-METRICS.
           *> Process records and calculate quality metrics
           PERFORM UNTIL EOF
               READ INPUT-FILE AT END SET EOF TO TRUE
               NOT AT END
                   ADD 1 TO TOTAL-RECORDS
                   PERFORM QUALIFY-SINGLE-RECORD
                   IF RECORD-QUALIFIED
                       ADD 1 TO VALID-RECORDS
                   ELSE
                       ADD 1 TO INVALID-RECORDS
                       PERFORM CATEGORIZE-QUALIFICATION-ERRORS
                   END-IF
               END-READ
           END-PERFORM
           
           *> Calculate overall quality score
           IF TOTAL-RECORDS > 0
               COMPUTE QUALITY-SCORE = (VALID-RECORDS / TOTAL-RECORDS) * 100
           ELSE
               MOVE 0 TO QUALITY-SCORE
           END-IF.
           
           DISPLAY 'Data quality metrics:'
           DISPLAY 'Total Records: ' TOTAL-RECORDS
           DISPLAY 'Valid Records: ' VALID-RECORDS
           DISPLAY 'Invalid Records: ' INVALID-RECORDS
           DISPLAY 'Quality Score: ' QUALITY-SCORE '%'
           DISPLAY 'Format Errors: ' FORMAT-ERRORS
           DISPLAY 'Range Errors: ' RANGE-ERRORS
           DISPLAY 'Business Rule Errors: ' BUSINESS-RULE-ERRORS
           DISPLAY 'Integrity Errors: ' INTEGRITY-ERRORS
           DISPLAY 'Completeness Errors: ' COMPLETENESS-ERRORS.

Calculate comprehensive data quality metrics including total records, valid/invalid counts, quality scores, and error categorization. Monitor data quality trends and identify improvement areas.

Qualification Error Handling

cobol

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
       WORKING-STORAGE SECTION.
       01 QUALIFICATION-ERROR-HANDLING.
           05 ERROR-CONTROL.
               10 ERROR-COUNT PIC 9(6) VALUE 0.
               10 MAX-ERRORS PIC 9(6) VALUE 1000.
               10 ERROR-THRESHOLD PIC 9(3)V99 VALUE 95.00.
           05 ERROR-RESPONSES.
               10 CONTINUE-PROCESSING PIC X VALUE 'Y'.
               10 STOP-PROCESSING PIC X VALUE 'N'.
               10 ERROR-ACTION PIC X.
                   88 CORRECT-ERROR VALUE 'C'.
                   88 REJECT-RECORD VALUE 'R'.
                   88 STOP-PROCESS VALUE 'S'.
       
       PROCEDURE DIVISION.
       PERFORM QUALIFICATION-ERROR-HANDLING-OPERATION
       
       QUALIFICATION-ERROR-HANDLING-OPERATION.
           PERFORM UNTIL EOF OR STOP-PROCESSING = 'Y'
               READ INPUT-FILE AT END SET EOF TO TRUE
               NOT AT END
                   PERFORM QUALIFY-RECORD
                   IF QUALIFICATION-FAILED
                       ADD 1 TO ERROR-COUNT
                       PERFORM HANDLE-QUALIFICATION-ERROR
                       
                       *> Check error threshold
                       IF ERROR-COUNT > MAX-ERRORS
                           MOVE 'Y' TO STOP-PROCESSING
                           DISPLAY 'Maximum error count exceeded - stopping processing'
                       END-IF
                   END-IF
               END-READ
           END-PERFORM
           
           DISPLAY 'Qualification error handling completed:'
           DISPLAY 'Total errors: ' ERROR-COUNT
           DISPLAY 'Processing stopped: ' STOP-PROCESSING.
       
       HANDLE-QUALIFICATION-ERROR.
           EVALUATE ERROR-SEVERITY
               WHEN 'LOW'
                   MOVE 'C' TO ERROR-ACTION
                   PERFORM CORRECT-ERROR-AUTOMATICALLY
               WHEN 'MEDIUM'
                   MOVE 'R' TO ERROR-ACTION
                   PERFORM REJECT-RECORD
               WHEN 'HIGH'
                   MOVE 'S' TO ERROR-ACTION
                   PERFORM STOP-PROCESSING
           END-EVALUATE.

Implement comprehensive error handling for qualification failures including error counting, threshold monitoring, and appropriate error responses. Handle different error severities with appropriate actions.

Implement comprehensive format and range qualification checks.
Validate data against business rules and constraints.
Ensure referential integrity and data completeness.
Calculate data quality metrics and monitor trends.
Handle qualification errors with appropriate responses.

Data Processing

Data manipulation and operations

Data Integrity

Data validation and consistency