跳转至

v0.6.6

Patch release landing the evaluation-governance switches for CreditModelPipeline (audit items G13/G14/G15/G16): one frozen score evaluated against multiple labels, with trustworthy bins, a consistent direction, and explicit evaluation weights. All new fields default to legacy behavior — default-config outputs are byte-identical to 0.6.5 (regression-asserted).

Multi-Label Evaluation (G13)

  • eval_target_cols — extra labels evaluated against the same frozen model scores; the effective label set is [target_col] + eval_target_cols (deduplicated). Perf tables concatenate per-label rows with a tgt_name column and figures are emitted per label (native PerformanceEvaluator multi-label support, now exposed). The pred_{model} score is computed once per dataset and reused across labels. When unset, output shape is unchanged.
  • All evaluation labels are validated up front against the input data and every extra_eval_datasets frame.

All-Missing Score Override (G14)

  • all_missing_score_value — rows whose raw model features are ALL missing receive this score (e.g. -1). The membership test is shared with Core.scoring's all_missing_spec_value (Pipeline/_common.all_missing_mask), so pipeline evaluation and production scoring flag identical rows — parity-tested end to end.
  • The rule is defined on raw features: WOE-source models map their feature set back to raw columns. Saved model metadata carries all_missing_score_value and all_missing_raw_features; production alignment is one call: Core.scoring(..., all_missing_spec_value=metadata_value).

Special Score Bins (G15)

  • special_score_values — sentinel scores (e.g. [-1]) get their own evaluation bin: excluded from quantile edges, reported as dedicated special:<value> gains rows (ranking columns NaN), and excluded from AUC/KS/avgScore.
  • The weighted evaluation path gained full special-value support (previously none): special rows are split out before rank_bins, and weighted summaries report N_SPECIAL / N_SPECIAL_RAW. Non-special bins are numerically identical to a clean-frame run.
  • Combined with G14, -1 overrides flow automatically into their own special bin.

Gains Direction & Evaluation Weights (G16)

  • gains_ascending — None (default) keeps every underlying path's legacy default (summary descending, some figures ascending, weighted descending); an explicit bool threads one direction through summary/gains and the weighted path uniformly. rank_bins gained an ascending parameter (previously hard-locked to descending — bin 1 was always the high-score bucket) and TRUE_BAD_SHIFT follows the direction.
  • eval_weight_col — "inherit" (default) reuses weight_col as before; None evaluates unweighted even when training is weighted (training weights no longer leak into evaluation implicitly); any other string names the evaluation weight column. The same resolution drives lr-search/optuna eval_weight_col and backward's validation_weight_col.

Behavior Notes

  • Byte-identical outputs under default configs (seeded regression assertion).
  • Known limitation: gains_ascending does not yet control evaluate_model.evaluate_performance's distribution figures (legacy fixed-signature plotting); special rows ARE pre-filtered from those figures. Slated for the default-flip release.
  • Announced default flips (next minor, 0.7.0): gains_ascending None→True — joining the already-announced synthesize_missing_oot None→False, search_eval_splits None→["oos"], and backward_report_splits None→[].

Validation

  • Full local pytest suite: 721 passed, 0 skipped (11 new evaluation-governance regressions; test_eval_weighted_* safety net unchanged).
  • Source package verification and wheel/sdist build passed (tasks.py verify).
  • Documentation passed mkdocs build --strict.