v0.9.2¶
The screening IV on WOE bins leaves class-pure bins out of the IV sum by default. This fixes the known issue of 0.9.1. The binning, the WOE and the scores do not change. The upgrade guide lists every setting that reproduces earlier numbers.
Behavior-changing fixes (exception to RELEASING.md)
A patch release normally changes no numbers on healthy inputs. 0.9.2 changes the screening IV of features with a
class-pure bin, and therefore which features the IV threshold, the correlation stage and the truncation keep. It
ships as a patch release because the 0.9.1 IV was inflated by a number that eps=1e-6 alone sets; this is a
documented exception to the release policy, the same way as 0.9.1.
| Change | What changes | Reproduce 0.9.1 with |
|---|---|---|
screen_iv_convention="guarded" (new, the default): the screening IV on WOE bins leaves class-pure bins out of the sum (section 1) |
The IV of every feature with a bin that holds no bad or no good, in feature_screen with iv_use_woe_bins (and every weighted run with an engine), in the selection stage, ivks_summary and the group and per-target evidence of FeatureValidationPipeline, in the screening of CreditModelPipeline on WOE bins, and in VarExtractionInsights / CorrelationFilter with a WOE engine; hence the features that iv_threshold, the correlation stage and max_selected_features keep |
screen_iv_convention="raw" in FeatureScreenConfig, FeatureValidationPipelineConfig, CreditModelPipelineConfig, VarExtractionInsights or CorrelationFilter. Checked on 31 screening and pipeline configurations: the IV tables, selections, summaries and correlation decisions are identical to 0.9.1 |
The ranking of the gates (max_selected_features, the VIF tie-break) uses that IV also when iv_upper_threshold is set (section 1) |
Which features the truncation keeps, on WOE bins, when the upper limit is armed | screen_iv_convention="raw" (the zero-cell-floored ranking of 0.9.1) |
Further changes add information without changing a number: the columns iv, iv_raw, iv_sv_pure and n_sv_pure
and the list of class-pure special-value bins (section 2), the drop reason "iv_above_upper_sv_pure" of the leakage
gate (section 3; the gate drops the same features as before), the odps import error (section 5).
1. The Screening IV Leaves Class-Pure Bins Out of the Sum¶
The screening on WOE bins scores each bin with woe = ln((bad share + 1e-6) / (good share + 1e-6)) and sums
(bad share - good share) * woe. A class-pure bin (only goods or only bads), usually a special-value (SV) bin, gets a WOE
that the 1e-6 alone sets, and up to 0.9.1 it inflated the IV:
- 5,000 rows, a special value
-999on 150 rows that are all good (the reproduction of this release): the SV bin has a WOE of-10.50and an IV contribution of0.38; the feature's IV was1.74. - The test frame of the 0.9.1 known issue (a sentinel on 10% of the rows that holds only bads): IV
4.02, of which3.58came from that bin. The 0.9.1 notes rounded it to 4.03; the value is 4.0247.
screen_iv_convention="guarded" applies the iv_guard convention that MonotoneWOEBinner's by-group IV already uses: a
bin with zero bads or zero goods is left out of the IV sum. It stays in the denominators: its rows remain in the total
bad and good counts (the sv_total_basis="all" basis), so the WOE and IV of every other bin are exactly those of 0.9.1.
The rule covers special-value bins and ordinary bins that min_n_bins kept unmerged (a near-perfect separator). As in
the binner's convention, a special-value bin that the binner smoothed (sv_woe_smoothing="laplace" with
sv_smoothing_alpha > 0) is not left out: its WOE is finite and not set by eps, so it counts with that WOE,
(bad share - good share) * woe. Smoothing is not used as the fix: alpha acts as a pseudo-count on the whole bin, so
its effect fades as the bin grows (an all-bad SV bin of 200 rows only goes from a WOE of 9.9 to 8.1, one of 2,000 rows
stays at 10.4).
| Frame | iv (guarded, the default) |
iv_raw (0.9.1) |
iv_sv_pure |
n_sv_pure |
|---|---|---|---|---|
-999 example (150 rows, all good) |
1.3610 | 1.7407 | 0.3797 | 150 |
0.9.1 test frame (f_special, selection on INS) |
0.4437 | 4.0247 | 3.5809 | 197 |
Both screening paths use one shared implementation (Modeling_Tool/_utils/screen_iv.py). Class purity is judged on the
quantities that the IV uses: the weighted bad and good sums on a weighted run, the counts on an unweighted one. With unit
weights both paths flag the same bins and give the same IV (tested on both frames). A bin whose bads all carry weight 0
is class-pure on the weighted path only.
The guarded IV is what the screening decides on:
iv_threshold(the IV stage) andiv_cutof the reports;- the IV that decides between two correlated features (
CorrelationFilter, the weighted correlation stage); - the ranking of
max_selected_featuresand of the VIF tie-break. Up to 0.9.1 the ranking substituted the zero-cell-floored IV wheniv_upper_thresholdwas set; under"guarded"it ranks on the guarded IV, under"raw"as in 0.9.1; - the IV of the group-stability and per-target gates of
FeatureValidationPipeline(monthly_iv_min,monthly_iv_cv_max,per_target_iv_range), per group on the group's own counts.
Where the setting lives:
| Object | Setting |
|---|---|
FeatureScreenConfig (feature_screen, screen_config_from_mapping) |
screen_iv_convention |
FeatureValidationPipelineConfig |
screen_iv_convention (selection_params and corr_params may repeat it; a different value there raises ValueError) |
CreditModelPipelineConfig |
screen_iv_convention (feature_selection may repeat it; a different value there raises ValueError) |
VarExtractionInsights, CorrelationFilter |
screen_iv_convention (constructor) |
The paths without WOE bins (the decision-tree IV of VarExtractionInsights and the weighted equal-frequency IV) already
have their own class-pure handling (the decision-tree IV zeroes such bins, the weighted IV leaves them out through
iv_guard) and are not affected. CreditModelPipeline screens on WOE bins only when feature_selection sets
iv_use_woe_bins (its default woe_engine is "equal_freq"); FeatureValidationPipeline uses WOE bins by default.
2. New Columns and the List of Class-Pure Special-Value Bins¶
Every screening table that shows the IV gains three columns next to iv:
| Column | Meaning |
|---|---|
iv |
The IV under screen_iv_convention (guarded by default) |
iv_raw |
The IV with every bin counted, the 0.9.1 value |
iv_sv_pure |
The IV contribution of the class-pure bins (special-value or ordinary) |
n_sv_pure |
The rows in those bins |
They appear in FeatureScreenResult.iv_table (which keeps iv_weighted, equal to iv), the iv table of the
selection summaries (feature_selection_summary["iv"], selection_summary["iv"], iv_report.csv), ivks_summary, and
the report of VarExtractionInsights.get_var_analysis_report with a WOE engine. The paths without WOE bins report
iv_raw = iv and NaN for iv_sv_pure and n_sv_pure.
A separate table lists every class-pure special-value bin (a bin that the engine flags as special, or whose rows are
all missing): var, special_value, bin_label, n (rows), bad, good (weighted sums on a weighted run),
bad_rate, woe (the WOE that the binning gives the bin, the value apply_woe writes), iv (its contribution to
iv_raw) and low_count (n < 30). It is meant to be handed to the strategy team as candidate rules; nothing in the
package acts on it.
| Where | Name |
|---|---|
feature_screen |
result.stage_tables["iv_sv_pure_bins"] |
VarExtractionInsights |
insights.iv_sv_pure_bins after get_var_analysis_report |
FeatureValidationPipeline |
result.ivks_sv_pure_bins (global rows of ivks_summary, with a target column; table ivks_sv_pure_bins) and selection_summary["stage_tables"]["iv_sv_pure_bins"] (table selection_iv_sv_pure_bins) |
CreditModelPipeline |
feature_selection_summary["iv_sv_pure_bins"] and iv_sv_pure_bins.csv |
Each table is present only when there is at least one such bin.
3. The Leakage Gate Keeps Its Floored IV and Labels Drops Caused by a Pure SV Bin¶
Changed in 0.9.3
The limit was still off by default in 0.9.2, so a leak with an ordinary-looking guarded IV passed silently. Since
0.9.3 iv_upper_threshold defaults to 2.0, tests iv_raw as well on WOE bins under the guarded
convention (where the floored IV cannot be computed, 0.9.2 fell back to the guarded iv), and a warning flags every
feature whose iv_raw is far above iv.
A special value that holds only bads can be a label leak, so the gate (iv_upper_threshold) does not switch to the
guarded IV: it keeps the zero-cell-floored IV on equal-frequency bins of 0.9.1 and drops the same features under both
conventions. When a feature exceeds the limit but would not exceed it without its class-pure special-value bins (the
floored IV recomputed on the other rows), dropped_detail records the reason "iv_above_upper_sv_pure" instead of
"iv_above_upper", so that a human can review the bin. The feature is dropped all the same. A genuine leak, with or
without such a bin, keeps the reason "iv_above_upper" (tested on both paths: the guarded IV of the leak is 1.8, far
below the limit of 2.0, while its floored IV is 21).
4. Scoring Is Unchanged; Resolve Class-Pure Bins Before Training¶
screen_iv_convention changes the screening only. The WOE stored in the binning tables and used by apply_woe and
transform does not change: the -999 bin of the example keeps its WOE of -10.50, and apply_woe gives the 0.9.1
values (tested against values recorded with 0.9.1). Existing models score identically.
The WOE of a class-pure bin has no statistical meaning
The WOE of a bin without bads or without goods is set by eps=1e-6: it is about ln(1e-6 / share), so it grows
with the size of the bin, not with its risk. When you build a new model, resolve every bin flagged sv_class_pure
(the 0.9.1 warning, the sv_class_pure column of the SV table, or the lists of section 2) before training: give the
special value a separate rule, or set sv_small_policy="neutral" (WOE 0) or "merge_missing" (the WOE of the
[Missing] bin). A WOE cap is out of scope for this release.
5. Small Fixes¶
Modeling_Tool/Core/kDataFrame.py: the regular expression"\w+"ofconvert_strlist_to_listis a raw string (r"\w+"). Python 3.12 warns about the invalid escape (SyntaxWarning) and a future version will reject it. No behavior change.- pyodps: the
odpsextra (pip install 'supermodelingfactory[odps]', declared inpyproject.toml) installspyodps. ImportingModeling_Tool.Core.ODPS_Tool(ODPSRunner,ParallelODPSManager) without it now raises anImportErrorthat names the extra instead ofNo module named 'odps'. See Installation.
Tests¶
test_screen_iv_convention.py (new): both reproductions, weighted against unweighted with unit weights, the weighted
purity rule, "raw" against numbers recorded with 0.9.1, the threshold, correlation and truncation on the guarded IV,
the leakage gate, apply_woe against 0.9.1, both pipelines, and the two small fixes. Existing tests that assert 0.9.1
screening numbers pass screen_iv_convention="raw"; their assertions are unchanged.