v0.9.3¶

0.9.2 made the screening IV on WOE bins leave class-pure bins out of the sum (screen_iv_convention="guarded"). A near-perfect separator then reports an ordinary-looking iv (13.8 drops to 1.8 on the leak frame of the tests), and the only remaining guard, iv_upper_threshold, was off by default, so such a leak passed the screening without any sign. 0.9.3 closes that gap with a leak warning and a default upper limit, makes sure that no empty-stage fallback brings a dropped leak back, and warns about every leak the limit drops. Scoring does not change. The upgrade guide explains how to keep the selections of 0.9.2.

Behavior changes (exception to RELEASING.md)

A patch release normally changes no numbers on healthy inputs. 0.9.3 changes which features the screening keeps, because 0.9.2 let leaks through silently; this is a documented exception to the release policy, as in 0.9.1 and 0.9.2.

Change	What changes	Keep the 0.9.2 selections with
(a) `iv_upper_threshold` defaults to 2.0 (was `None`) in `FeatureScreenConfig`, `screen_config_from_mapping`, `weighted_feature_screen`, the selection of `FeatureValidationPipeline` and the `feature_selection` of `CreditModelPipeline`; on WOE bins under the guarded convention the limit gates on `iv_raw` (the larger of `iv_raw` and the zero-cell-floored IV), never on the guarded `iv` (section 2)	Features whose floored IV or `iv_raw` is above 2.0 are dropped (`dropped_detail` reason `iv_above_upper`, or `iv_above_upper_sv_pure` when the excess comes from class-pure special-value bins). This also applies to the default screening of `CreditModelPipeline` without WOE bins	`iv_upper_threshold=None`, or your previous explicit value (see the migration note)
(b) New leak warning and `iv_gap_flag` column (section 1)	A `UserWarning` per feature whose `iv_raw` is far above `iv`, and the boolean column `iv_gap_flag` in `iv_table` and the `iv` tables of the selection summaries. It never drops a feature	Nothing to restore (no number changes); filter the warning or set `iv_gap_ratio_warn` / `iv_gap_abs_warn` to `inf`
(c) The empty-stage fallback no longer revives leak-gate drops (section 3)	Under the default `on_empty_stage="keep_all_warn"`, a stage that would drop every feature never brings back a feature that the upper limit dropped; from the IV stage on it keeps only the `fallback_top_k` (5) other features with the highest screening IV (marked `fallback_kept` in `dropped_detail`, with a warning), and it raises `EmptyStageError` when only leak-gate drops are left. Before, it kept every feature, so with the limit on the selection could hold MORE features than without it	`on_empty_stage="keep_all"` (the old fallback, leak drops included; for comparisons with old results only)
(d) A warning for every leak-gate drop (section 4)	A `UserWarning` per feature that `iv_upper_threshold` drops ("possible target leakage; check the field's time point (PIT) and source before training"), on every screening path, next to the gap warning of (b) when both apply	Nothing to restore (no number changes); filter the warning or switch the limit off

1. Leak Warning and `iv_gap_flag`¶

Every screening path that reports both iv and iv_raw (feature_screen, feature_screen_from_dataframe, weighted_feature_screen, the selection of FeatureValidationPipeline and the screening of CreditModelPipeline on WOE bins) now checks each feature:

(iv > 0 and iv_raw / iv >= iv_gap_ratio_warn)  or  (iv_raw - iv >= iv_gap_abs_warn)

with iv_gap_ratio_warn=2.0 and iv_gap_abs_warn=0.5 by default (new fields of FeatureScreenConfig, also keys of the pipelines' selection_params / feature_selection and arguments of weighted_feature_screen). An iv at or below 1e-12 counts as zero, and then only the difference rule applies. A matching feature gets iv_gap_flag=True in iv_table and a UserWarning that names it:

f_leak: iv=1.8116, iv_raw=13.7705, iv_sv_pure=11.9589, n_sv_pure=1500 (iv_raw / iv >= 2 or iv_raw - iv >= 0.5):
possible leakage or unresolved class-pure bins; review before training

The rule only warns and flags; it never drops a feature. The flag is set from the rule alone, also for a feature that the upper limit drops, so the screening report shows both the drop reason (dropped_detail) and the flag (iv_table). CreditModelPipeline now also puts dropped_detail into feature_selection_summary (and feature_selection_dropped.csv). Under screen_iv_convention="raw", and on the paths without WOE bins, iv equals iv_raw, so nothing is flagged.

2. The Upper Limit Is On by Default and Gates on `iv_raw`¶

iv_upper_threshold defaults to 2.0; iv_upper_threshold=None switches it off (a dict key set to None does too; a missing key takes the default). The value the limit tests:

Path	Tested value
WOE bins, `screen_iv_convention="guarded"` (the default)	The larger of `iv_raw` (every WOE bin counted) and the zero-cell-floored IV on equal-frequency bins; `iv_raw` alone where the floored IV cannot be computed (non-numeric features). Never the guarded `iv`
WOE bins, `screen_iv_convention="raw"`	As in 0.9.2: the floored IV on equal-frequency bins, `iv_raw` where it cannot be computed
No WOE bins	As in 0.9.2: the floored IV, the reported IV where it cannot be computed

The floored IV keeps catching a near-perfect separator that the binner's merge folded into one ordinary bin (0.9.1); iv_raw catches a class-pure WOE bin that equal-frequency bins mix with other values. dropped_detail records which one decided (metric iv_floored or iv_raw) and keeps the 0.9.2 reasons: iv_above_upper, or iv_above_upper_sv_pure when the feature would not exceed the limit without its class-pure special-value bins.

Before and after¶

Frame	0.9.2 defaults	0.9.3 defaults
Leak frame, `f_leak` (`y * 3 + noise`)	`iv` 1.8116, `iv_raw` 13.7705: kept, no warning	Same IVs; `iv_gap_flag` True, warning; dropped (`iv_above_upper`, floored IV 21.30)
Leak frame, `f_sv` (12% of rows on an all-bad special value)	`iv` 0.4617, `iv_raw` 6.1464: kept	Flagged, warning; dropped (`iv_above_upper_sv_pure`)
Special value hidden in an equal-frequency bin (`x_sv`, 6% all-bad)	`iv` 0.10, `iv_raw` 2.83, floored IV below 2.0: kept, also with an explicit `iv_upper_threshold=2.0`	Flagged, warning; dropped (`metric=iv_raw`, `iv_above_upper_sv_pure`)
`-999` example (150 all-good rows on `-999`)	`iv` 1.3609, `iv_raw` 1.7407, `iv_sv_pure` 0.3797, `n_sv_pure` 150: kept	Identical numbers; tested value 1.7407 < 2.0: kept; ratio 1.28 and difference 0.38: not flagged, no gap warning and no leak warning
`x` (leak, `iv` = `iv_raw` far above 2.0) + `z` (noise, `iv` 0.0020 below 0.02), `selected_features`	`['x']` (no limit)	First 0.9.3 commit (7e601de): `['x', 'z']` (both dropped, the fallback revived both). Now: `['z']`; `x` stays dropped (`iv_above_upper`, leak warning), `z` is `fallback_kept` with the warning `IV stage empty; fallback kept z (iv=0.0020); ...`. `on_empty_stage="keep_all"`: `['x', 'z']`; `iv_upper_threshold=None`: `['x']`
`x` alone, `selected_features`	`['x']` (no limit)	7e601de: `['x']` (revived). Now: `EmptyStageError` naming `x (iv_above_upper)`, after the leak warning. `on_empty_stage="keep_all"`: `['x']`

The binning is unchanged: the -999 bin keeps its WOE of -10.4963 (IV 0.3797, ordinary bins 1.3609), and apply_woe gives it -10.4963.

3. The Empty-Stage Fallback Never Revives a Leak¶

A stage that would drop every feature used to keep all of them (on_empty_stage="keep_all_warn"). With the upper limit on, a leak x dropped by the limit and a noise feature z dropped by the lower limit emptied the IV stage, and the fallback brought both back: the selection was ['x', 'z'] with the limit and ['x'] without it. Since this release:

Hard drops are never revived. A feature that the leak gate dropped (dropped_detail reason iv_above_upper or iv_above_upper_sv_pure, the set HARD_DROP_REASONS of Modeling_Tool.Feature.Weighted_Screen) is a hard drop. No fallback brings it back: not the IV stage, not the VIF, group-stability or multi-target gate, not the CSV batch mode of FeatureValidationPipeline (its first pass keeps every candidate of an empty stage, but the leak drops among them never reach the global pass), and not the backward elimination of CreditModelPipeline, which falls back to the screened features only. When the screening of CreditModelPipeline fails for another reason after the IV stage (for example a G03 setting without evidence), every feature is kept except the leak drops found before the failure, listed in feature_selection_summary["hard_dropped"]. The correlation stage never empties (it keeps one feature of each correlated group).
The fallback keeps only the best soft drops. From the IV stage on, an empty stage keeps the fallback_top_k (new field of FeatureScreenConfig, default 5; also a key of feature_selection / selection_params and an argument of weighted_feature_screen) features with the highest screening IV, the iv column that the threshold tested (guarded by default), in that order. It never ranks by iv_raw, which would favour features with large class-pure special-value bins. It warns, for example IV stage empty; fallback kept z (iv=0.0008); review the data before training, and marks the kept features in dropped_detail with the reason fallback_kept: on their own row of the stage (the reason is replaced; the metric, value and threshold stay and show the cause), or on a new row with metric="iv", their IV and the stage threshold. The <stage>_fallback row of summary gives the number kept in n_out. The missing-rate and PSI stages run before any IV exists and before the leak gate: their fallback keeps every feature, as before.
Only hard drops left: the screening stops. When every feature that an empty stage would drop is a hard drop, it raises EmptyStageError (the existing ValueError subclass) with the feature names and reasons, for example ... every one of them was dropped by the leak gate, which no fallback may revive: x (iv_above_upper). The leak warning of each feature (section 4) is issued before the raise; it is not part of the message. CreditModelPipeline does not catch the error, and a CSV batch run whose candidates are all leak drops raises it as well.
The old fallback stays available. on_empty_stage="keep_all" (new value) keeps every feature, hard drops included, exactly as before this release. It can let a leak back in: use it only to compare with old results. on_empty_stage="raise" raises as before, and "keep_all_warn" keeps its name for compatibility.

4. A Warning for Every Leak-Gate Drop¶

A plain leak whose guarded iv and iv_raw are both far above 2.0 was dropped (iv_above_upper) without any warning: only the gap of section 1 warned, and it needs class-pure bins. Every feature that iv_upper_threshold drops now gets one UserWarning, on every screening path (unweighted and weighted, with and without WOE bins, and in both pipelines):

x: dropped by the IV upper limit (iv_above_upper): iv_floored=9.2127 > iv_upper_threshold=2 (iv=2.5592,
iv_sv_pure=nan, n_sv_pure=nan): possible target leakage; check the field's time point (PIT) and source before training

It names the value the limit tested (iv_floored or iv_raw, as metric in dropped_detail), the threshold, iv, iv_sv_pure and n_sv_pure (nan on the paths without WOE bins). A feature that also trips the gap rule gets both warnings, because the two point to different checks: the leak warning says the IV is above the upper limit (check the field's time point and source), the gap warning says the guarded and the raw IV are far apart (check for unresolved class-pure bins).

5. `weighted_feature_screen` Exposes the Convention and the Leak Settings¶

weighted_feature_screen gains screen_iv_convention (default "guarded"), iv_upper_threshold (default 2.0, not exposed before), iv_gap_ratio_warn and iv_gap_abs_warn, appended after the existing arguments and passed on to FeatureScreenConfig exactly as feature_screen uses them, and fallback_top_k (default 5) with the new on_empty_stage="keep_all" (section 3). With screen_iv_convention="raw" its IV numbers are those of 0.9.1 (tested on the -999 frame: 1.79 unweighted, 1.768988838079573 weighted).

6. Documentation Build¶

requirements-docs.txt of the documentation repository pins mkdocs<2 (MkDocs 2.0 removes the plugin and theme systems the site uses) and keeps pymdown-extensions<12.2 until mkdocstrings supports 12.2.

7. Code Cleanup¶

No behavior change: the unused imports weighted_rate (Feature/Weighted_Screen.py) and var_corr_filter (Feature/WOE_Engine_Feature_Patch.py) are removed, and the Iterable annotation of feature_validation.py is imported from collections.abc.

Out of Scope¶

Unchanged on purpose: the WOE of class-pure bins and the scoring (no WOE cap, no automatic "neutral"), the default sv_woe_smoothing="none", the IV thresholds and the gap-warning thresholds, the guarded and raw conventions, the default engine of CreditModelPipeline (equal_freq, which screens without WOE bins), and the first two known issues of 0.9.0. The third one (a stricter threshold can return a looser selection) is narrowed by section 3: from the IV stage on the fallback keeps at most fallback_top_k features and never a leak drop; the missing-rate and PSI stages still keep every feature.

Tests¶

test_leak_gate_defaults.py (new, 39 tests): the defaults, the gap rule, the leak frame (flag, warning, drop, both paths), the limit on iv_raw (the hidden special value), the -999 frame (exact numbers, no drop, no warning), iv_upper_threshold=None against numbers recorded with 0.9.2, weighted_feature_screen with "raw" against numbers recorded with 0.9.1, and both pipelines. Existing tests that describe the screening without an upper limit pass iv_upper_threshold=None explicitly; their assertions are unchanged.

test_empty_stage_fallback.py (new, 44 tests): the settings and their validation; the x + z frame on all four screening paths (['z'], x with iv_above_upper, z with fallback_kept, both warnings), with keep_all (['x', 'z']), without the limit (['x']) and with "raise"; x alone (EmptyStageError, warning before the raise); the top 5 of eight soft drops by iv, in order, on a frame where iv_raw would rank differently; an empty group-stability gate of FeatureValidationPipeline, an empty backward elimination and a failed screening of CreditModelPipeline, and the CSV batch mode; one leak warning per leak-gate drop and both warnings for a class-pure leak; the results of 7e601de where the fallback does not run (recorded numbers on all four paths) and the -999 frame. Nine tests whose synthetic features are near-separators above the limit passed only because the old fallback revived them; they now pass iv_upper_threshold=None (their assertions are unchanged). Two tests of the old fallback now check the new one: the pipeline test of the default fallback expects the top-k warning and the fallback_kept rows, and the group-stability test of insufficient_group_policy="drop" expects the rescued rows as fallback_kept with their original metric (eligible_groups) and threshold (the original reason stays under on_empty_stage="keep_all").