DB2 optimizer cost model and query rewrite

Every SQL statement you BIND or PREPARE is handed to the DB2 optimizer. The optimizer does not execute your English intent. It builds an internal query tree, may rewrite that tree into a cheaper equivalent, estimates how many rows survive each predicate, enumerates join orders, and picks the access path with the lowest estimated cost. This page is the map of that process for Db2 for z/OS.

Optimizer
Progress0 of 0 lessons

Cost-based optimization, not folklore rules

A rule-based optimizer would say things like “an index is always better than a tablespace scan.” That fails when 99% of rows match STATUS = 'A' and the index would bounce randomly through almost every data page. Db2 for z/OS has used cost-based optimization (CBO) since the first release: enumerate strategies, estimate cost, choose the minimum.

Cost is a relative number used to rank plans. It is not a guaranteed elapsed time. Buffer hits, concurrent workload, and storage caching are mostly unknown at bind time. If plan A costs 500 and plan B costs 50,000, A is the winner; you should not read 500 as “500 milliseconds.”

Ingredients of the cost model
ComponentWhat the optimizer approximates
I/ORandom vs sequential page reads; prefetch; index and data pages
CPUPredicate evaluation, row copying, join logic, functions
Sort / work fileORDER BY, GROUP BY, DISTINCT, merge join, RID lists, materialization
ParallelismDegree vs sequential; more tasks can lower elapsed estimate but raise CPU
CommunicationDistributed (DDF) queries: shipping result rows

Sequential I/O is modeled as cheaper than random I/O, which is why clustering ratio and prefetch matter. CPU cost includes walking index keys and applying stage 1 versus stage 2 predicates. Sorts and materializations add work-file cost. Parallelism can improve the elapsed-time side of the estimate while increasing CPU.

Optimization phases

High-level optimizer phases
PhaseWhat happens
Parse and checkSyntax, authorization, object resolution, data types
Query rewritePredicate generation/simplification, subquery/join transforms, view merge, MQT match
Cardinality estimationFilter factors × table/index stats → estimated rows per step
Join enumerationCandidate join orders and methods (nested loop, merge, hybrid)
Access path selectionCost each candidate (I/O, CPU, sort, parallelism); pick the lowest
Plan generationInternal structures for runtime (package section or dynamic cache)

Beginners often jump straight to “which index?” Access path selection is the last costing step. If rewrite already turned an EXISTS subquery into a join, or merged a view so a local predicate became matching, the index question is asked about a different statement than the one you typed.

sql
1
2
3
4
5
6
7
EXPLAIN PLAN SET QUERYNO = 1001 FOR SELECT E.EMPNO, D.DEPTNAME FROM HR.EMPLOYEE E INNER JOIN HR.DEPARTMENT D ON E.WORKDEPT = D.DEPTNO WHERE D.DEPTNO = 'A00' AND E.SALARY > 50000;

After EXPLAIN, read PLAN_TABLE for methods and indexes, DSN_PREDICAT_TABLE for how predicates were used, and DSN_DETCOST_TABLE when you need mini-plan cost comparisons.

Query rewrite

IBM documents that Db2 sometimes modifies the internal form of SQL to improve access path efficiency. You should still write simple SQL; do not complicate a statement hoping to “trigger” a transform. Transformations are not guaranteed for any one statement.

Typical rewrite families (details on the predicate-and-query-transformations page):

  • Removal of unneeded or pre-evaluated predicates (literals like 1 = 1)
  • Generated predicates, including transitive closure
  • Join simplification / join elimination when a table is redundant
  • Subquery correlation, de-correlation, or conversion to a join
  • UNION ALL subselect simplification
  • View / nested table expression merge versus materialization
  • Automatic query rewrite onto MQTs for eligible dynamic read-only query blocks when the MQT covers the needed rows and columns and estimated cost is better

MQT rewrite is cost-based after the substitution is considered: Db2 keeps the rewritten query only if its estimated cost beats the original. Stale MQT data is your responsibility; the optimizer assumes the MQT matches the base tables for matching purposes.

Filter factors, selectivity, and cardinality estimation

A filter factor (FF) is IBM’s 0–1 estimate of the fraction of rows for which a predicate is true. Selectivity is the same idea in DBA English. Cardinality is the estimated row count after applying predicates: roughly CARDF × FF1 × FF2 × … when Db2 treats predicates as independent.

For a simple equality COL = constant with uniform distribution:

sql
1
2
3
-- Conceptual formula (uniform distribution, no frequency stats) -- FF(COL = value) ≈ 1 / COLCARDF -- Example: COLCARDF = 10 → FF ≈ 0.10 → 10% of rows

Filter factor also depends on the operator (equal, range, LIKE, NE) and on the literal when frequency or histogram statistics exist. A popular status code with FREQUENCYF = 0.80 should not be costed as 1/COLCARDF if COLCARDF is 5.

Combined predicates are the danger zone. CITY = 'CHICAGO' AND STATE = 'IL' are correlated. Multiplying two independent filter factors undercounts rows and can make a nested loop look falsely cheap. COLGROUP statistics exist to fix that. Missing stats (COLCARDF = -1) force default filter factors—historically 1/25 for equality—which is a guess, not your data.

Range predicates use HIGH2KEY and LOW2KEY (second-highest and second-lowest values) to interpolate where a literal sits in the domain. Histograms improve ranges when data is clustered in bands rather than uniform.

Join enumeration

For n tables, the number of join orders explodes. The optimizer searches a space of join sequences and, for each adjacent pair, considers nested loop, merge scan, and hybrid join when eligible. It uses join predicates to avoid Cartesian products until no join predicate remains.

Estimated cardinality of each intermediate composite drives later choices: a tiny outer favors nested loop with a matching inner index; two large ordered inputs favor merge scan; hybrid join sits between them with RID lists and list prefetch. Bad early cardinality (wrong FF) poisons the whole tree—this is why “the join method flipped after RUNSTATS” is normal, not mysterious.

Composite tables may be materialized between steps. That cost is part of the model, not an accident.

Access path selection

For each table in a given join position, candidates include:

  • Matching index scan (ACCESSTYPE I, MATCHCOLS > 0)
  • Nonmatching index scan / index screening
  • Index-only access when INDEXONLY = Y
  • Multiple index access (M / MX / MI / MU) with RID lists
  • Tablespace scan (R) with sequential or other prefetch
  • List, sequential, or dynamic prefetch
  • Parallel variants when DEGREE allows

The winner is the cheapest estimated combination, not the “most indexes.” A tablespace scan with sequential prefetch can beat an unclustered matching index that returns 40% of the table.

Optimizer statistics feedback

During access path selection, Db2 notices missing or conflicting statistics (for example COLCARDF that cannot reconcile with SYSCOLDIST CARDF). It externalizes recommendations:

  • SYSIBM.SYSSTATFEEDBACK — catalog table, populated on a statistics interval from BIND, REBIND, and PREPARE
  • DSN_STAT_FEEDBACK — EXPLAIN table, written when EXPLAIN runs through access path selection (not STMTCACHE/PACKAGE EXPLAIN shortcuts)

RUNSTATS can clear SYSSTATFEEDBACK rows once the requested stats are collected. DSN_STAT_FEEDBACK is not cleaned that way. If STATFDBK_PROFILE is YES, Db2 can maintain statistics profiles in SYSTABLES_PROFILES from that feedback so USE PROFILE jobs collect what the optimizer asked for. Feedback is not a substitute for a regular RUNSTATS schedule; histogram stats on ever-increasing keys go stale fast.

What you can influence (without fighting the model)

  • Statistics — RUNSTATS with the right COLGROUP / FREQVAL / HISTOGRAM
  • SQL shape — stage 1 predicates, avoid wrapping columns in functions
  • Indexes and clustering — give the enumerator a cheap matching path
  • REBIND after stats or version change for static SQL
  • Hints and profiles — last resort when the model is wrong and you cannot fix stats or SQL

Do not update catalog statistics by hand unless you understand interpolation formulas. IBM warns that invented HIGH2KEY/COLCARDF combinations produce worse plans than defaults.

Explain It Like I'm Five

You ask a grown-up to fetch snacks. A rule-based grown-up always uses the ladder because “ladders are best.” A cost-based grown-up looks at how many snacks, which shelf, and how heavy the ladder is. Query rewrite is the grown-up saying “you really meant the cookie jar next to the fridge” and then costing that shorter walk. Filter factor is guessing how many cookies match “chocolate.” If the guess is wrong, they bring a dumpster instead of a plate. Statistics are counting the cookies so the guess is not a wild one.

Exercises

  1. EXPLAIN a two-table join before and after RUNSTATS TABLESPACE with INDEX ALL. Note METHOD, MATCHCOLS, and DSN_STATEMNT_TABLE cost if available.
  2. Compute 1/COLCARDF for a status column and compare it to an actual COUNT(*) / CARDF for the most common status. Explain the gap.
  3. Find a view that is merged versus one that is materialized (TABLE_TYPE Q vs W) and relate that to rewrite versus later costing.
  4. Query SYSIBM.SYSSTATFEEDBACK for a busy table and map TYPE values to RUNSTATS keywords you would add.
  5. Change a stage 2 predicate (YEAR(HIREDATE) = 2020) to a range and EXPLAIN again. Which phase of the model changed—rewrite, FF, or join enumeration?

Quiz

Test Your Knowledge

1. Db2 for z/OS has used which kind of optimizer since the first release?

  • Rule-based only (always prefer any index over a scan)
  • Cost-based optimization using catalog statistics
  • Random plan selection
  • Only hints; no costing

2. What is a filter factor?

  • A JCL COND code
  • A number from 0 to 1 estimating the fraction of rows that satisfy a predicate
  • The number of buffer pools
  • Always exactly 0.5

3. When does query rewrite happen relative to access path costing?

  • Only after the query finishes executing
  • Db2 may transform the statement first, then cost candidate access paths on the rewritten form
  • Never; SQL text is executed literally token by token
  • Only during REORG

4. Default filter factor for COL = literal when COLCARDF is known and distribution is treated as uniform is:

  • 1 / COLCARDF
  • Always 1/25
  • Always 0
  • NPAGES / NLEAF

5. Optimizer statistics feedback is written to:

  • Only SYSUTIL
  • SYSIBM.SYSSTATFEEDBACK (BIND/REBIND/PREPARE) and DSN_STAT_FEEDBACK (EXPLAIN)
  • Only the BSDS
  • Only SDSNLOAD