Skip to content

v0.9.0

Not published

0.9.0 was prepared but never tagged or uploaded to PyPI. Its changes, including the three defaults below, ship in 0.9.1; the upgrade guide lists the settings that reproduce 0.8.2.

A minor release. Every text is English, every public docstring documents all of its parameters, and the defects found while verifying the documentation and while auditing CreditModelPipeline and FeatureValidationPipeline end to end are fixed, each with regression tests. Three defaults change.

Behavior changes: three defaults flip

Setting 0.9.0 default Keep the 0.8.2 numbers with Effect of a refit with the new default
MonotoneWOEBinner(unseen_special_policy=...), also in monotone_woe_params "neutral" "normal_bin" A declared special value that has no fit row gets an empty placeholder bin and scores missing_woe instead of the WOE of the bin its number falls in. In the pipelines the -999999 sentinel, declared when any feature holds it, gets such a row in every other monotone feature
MonotoneWOEBinner(sv_total_basis=...), also in monotone_woe_params (new) "all" "ordinary" The WOE of the ordinary bins (one constant per feature) and the IV of every feature with special or missing values; selections that use the IV can change
CreditModelPipelineConfig(warm_start_score_scope=...) (new) "full" "train" Warm-started LightGBM and XGBoost models stop early on the combined score, the Optuna table scores the combined model, and the Owen explanation has a warm_start_prior group

unseen_special_policy was announced in 0.8.2. sv_total_basis and warm_start_score_scope are new in this release and take the consistent behavior as their default at once, an exception to the release policy below that the audit findings justify; each keeps the old numbers with one setting. Saved scorecards, binners and models keep the numbers they were built with until they are refitted, and a pickled binner keeps its own settings.

The other results that change are listed in the "Results that change" boxes of sections 4 and 5.

1. All Text Is English

Every docstring, comment, and user-visible string is English, and the four repositories contain no Chinese text. The code is unchanged: the translation touches comments, docstrings, and string contents only.

Code that matches on message text

Log, warning, and exception messages, Excel sheet names, and report headings changed language. Scripts that filter logs, use pytest.raises(match=...), or read workbook sheets by name must use the new wording.

Area Now
Docstrings and comments English. The API Reference renders the English text
Log, warning, and exception messages English. For example the binner logs [MonotoneWOEBinner] Fitting 3 features ..., and the feature_screen fallback warning reads [feature_screen] stage 'psi' eliminated all 40 variables; keeping all of them (on_empty_stage='keep_all_warn')
MonotoneWOEBinner.export_woe_report The sheets are named WOE Bin Details and WOE Bin Charts, with English headers
Report templates English headings, for example Multi-Model Evaluation (Untuned), Final Model Evaluation, and Feature Importance Evaluation
Plot labels English, for example the axes of plot_var_reduction
Pipeline GUI schema FieldMeta.label, description, and group, the registry card text, and the validate_pipeline_config messages are English; field names, option values, and defaults are unchanged

2. Fixes Found While Verifying the Documentation

The guides and the docstrings were checked against the installed package, and eleven defects surfaced. Each has a regression test in test_doc_audit_regressions.py that fails without the fix.

Defect Fix
Several Report.Report_Tool functions used pd and os without importing them, so they raised NameError when called The imports are in place
The SampleBalancer methods nearmiss, tomek, and enn raised TypeError with imbalanced-learn 0.14 and later, which dropped random_state from those samplers random_state is passed only to samplers that still accept it
get_gains_table, get_gains_table_by_cust_metrics, and get_perf_summary iterated over the unfiltered group list after dropping groups below min_data_size, so a small group made the next group appear twice and the last one disappear They iterate over the filtered groups
upload_score, h2o_apply_regex, and get_dtypes_file(ck_format=True) raised NameError, and the all-missing log call in scoring made logging report a formatting error The helpers import what they use, ck_format=True raises NotImplementedError with an explanation, and the log call has its placeholder
calibrate of LightGBMModel, XGBoostModel, CatBoostModel, and GradientBoostingModel defaulted to cv='prefit', which scikit-learn 1.8 removed, so every call raised InvalidParameterError The fitted estimator is wrapped in FrozenEstimator when it exists; older scikit-learn keeps cv='prefit'
BackwardEliminationAnalyzer.get_perf_trend returned None for every value, because it only read dictionary summaries while BackwardVariableEliminator stores DataFrames It reads the stored DataFrames
GradientBoostingModel.param_search turned integer parameters into floats (num_leaves=7.0, which LightGBM rejects), and refit=True kept fitting with the pre-search parameters best_params_ keeps native types and the model uses the merged parameters
TextEncryptor.decrypt raised ValueError for any text with a non-ASCII character, because encrypt stored the plaintext length in characters and decrypt compared it with the length in bytes The length prefix is the byte length. Ciphertexts of ASCII text are unchanged, so existing ones keep decrypting
ExcelFormat.base_filepath used str.strip, which removes characters and not a prefix (out2/report.xlsx gave ut2/) It is the directory part of the path, including the trailing separator
compute_overfitting_shift told the caller that unequal column counts "are the the same" The message says "are not the same"
ModelExplainer.lime_explain_instance and lime_global_importance raised AttributeError: 'NoneType' object has no attribute 'copy' when neither X_train nor background_data was available, because the intended ValueError was checked too late The check runs first, so the message is LIME requires X_train or background_data

3. Complete Docstrings and Parameter Tables

Every public class, function, and method of Modeling_Tool, ExcelMaster, and Report now documents exactly the parameters of its signature, in signature order, in the NumPy layout that the API Reference renders. The Config dataclasses of the Pipelines list every field with its type, default, and behavior, and the parameter tables on the Top-Level Pipelines page have a row for every field (34 were missing). The pytest test test_public_docstring_parameters.py keeps the docstrings in step with the signatures.

4. Further Fixes

Twelve more defects came out of documenting the functions that the first pass had described as they behaved. Each has regression tests in test_doc_audit_regressions.py that fail without the fix.

Defect Fix
ExcelMaster.add_worksheet(auto_fit=True) raised AttributeError, because it called worksheet.auto_fit() and xlsxwriter only has autofit() close_workbook fits the columns of those sheets with autofit() once the data is written. A column is only widened, never narrowed below the width the sheet already has
write_duo_chart without y2_list raised TypeError, because it built the second chart from None Only the first chart is built and inserted. With retChart=True the result is (chart1, None), and write_combined_chart accepts chart2=None
multi_subset_wrapper({}) raised UnboundLocalError, because the final return used a loop variable of a loop that never ran It returns an empty DataFrame that has only the column subset_var_name
plot_boxplot and write_boxplot multiplied the values by 100 whatever y_percentage said, so y_percentage=False only dropped the percent labels y_percentage=True multiplies by 100 and formats the axis as percent; False draws the values as they are
cross_risk_weighted_mean kept the weight of a NaN row in the denominator while adding nothing to the numerator, so NaN values pulled the mean towards 0 A NaN row is skipped. A cell whose values are all NaN is NaN
The weighted get_gains_table divided the regular bins by the weight of the non-special rows and the special rows by the weight of all rows, so PROP added up to more than 1 with spec_values PROP is the share of the weight of all rows in every row, so it adds up to 1
tie_breaker (feature screening and lr_elimination_params) and iv_equal_freq were accepted with any value although nothing read them: ties were always broken by ascending feature name (screening) or column order (LR elimination), and the IV binning has no equal-frequency switch, so tie_breaker="random" or iv_equal_freq=False looked effective. The feature-validation pipeline also took iv_equal_freq from psi_params["equal_freq"] Only the value that matches the real behavior is accepted: tie_breaker "name" (or None) for screening and "pvalue" (or None) for the LR elimination, and iv_equal_freq=True. Any other value raises ValueError when the config is built (screening) or when run() starts (LR elimination). iv_equal_freq no longer follows psi_params["equal_freq"]
CorrelationFilter(spec_values=...) stored the special values but never passed them to the VarExtractionInsights that computes the IV and KS deciding each correlated group, so on the default path the argument was silently ignored; it worked only with woe_binner or woe_engine="monotone" The special values reach the IV and KS calculation on the default path too, in CorrelationFilter and in Feature_Insights.CorrelationFilter. The default spec_values=[] gives the same result as before
ModelExplainer.explain_owen kept the first SHAP PartitionExplainer unless rebuild=True or model_output changed, so a new prior_groups, coalition_structure, or background_data updated coalition_structure_ while the Owen values were still computed on the old partition tree (a switch between two groupings gave values that differed from a fresh explainer by up to 0.019 in my check) The cached explainer is reused only while the coalition structure (features and linkage), the background values, and model_output are unchanged; any change rebuilds it. rebuild=True still forces a new one
WOE_Master gave a bin with only goods or only bads a WOE and IV of -inf or +inf, so a feature with such a bin made LR and XGBoost training fail (ValueError and XGBoostError) while LightGBM and CatBoost silently took the infinite value. The default woe_engine="equal_freq" is affected Such a bin gets a finite WOE with the eps of MonotoneWOEBinner: ln((bad_pct + eps) / (good_pct + eps)) with eps=1e-06. Every other WOE and IV value is unchanged, and calc_woe, calc_iv, and the Gains tables still return infinity for a zero share
CreditModelPipelineConfig.random_state never reached the LightGBM and XGBoost models, because their built-in parameters fixed random_state at 42; the Optuna candidates and the backward-elimination proxy were fixed at 42 too The built-in LightGBM and XGBoost parameters carry no seed. random_state seeds every GBM model the pipeline builds (final models, Optuna candidates, backward proxy), and a random_state in model_params[name] still wins. The default seed 42 gives the same models as before
CreditModelPipelineConfig.backward_model ran the XGBoost elimination for every value except the exact lower-case "lgb", so "LGB", "lr", and "cat" silently ran XGBoost, and the GUI schema offered "lr" and "cat" run() accepts "lgb" and "xgb" in any case and raises ValueError for anything else while backward_enabled is on. The GUI schema offers only "lgb" and "xgb", and validate_pipeline_config reports other values

Results that change

A direct call of plot_boxplot or write_boxplot with the default y_percentage=False used to draw the values times 100 and now draws them as they are; the hyperparameter box plots of the reports pass y_percentage=True and do not change. cross_risk_weighted_mean returns a higher mean for every cell that has NaN values. PROP of the weighted get_gains_table is smaller in the regular bins when spec_values matches rows. A backward_model other than "lgb" or "xgb" now raises ValueError when run() starts, where it used to run XGBoost; "LGB" now runs LightGBM. A credit-model run with a random_state other than 42 now trains different LightGBM and XGBoost models (and Optuna candidates and backward proxies) than before; runs with the default seed are unchanged. A feature that has a bin with only goods or only bads now has a finite WOE and IV (it was infinite), so its IV can rank and screen differently; LR and XGBoost models that used to fail now train, LightGBM models on such features change slightly, and CatBoost models are unchanged. Tables with no such bin give identical results. explain_owen called again with another prior_groups, coalition_structure, or background_data now returns the Owen values of the new grouping or background; they used to come from the first grouping. Calls with unchanged inputs still reuse the cached explainer. A CorrelationFilter called with non-empty spec_values now computes its IV and KS with those values in rows of their own, so the IV of a variable whose signal sits in its special values drops and the survivor of a correlated group can change (in my test data the kept variable switched from a to b); calls with the default spec_values=[] are unchanged. A config that sets iv_equal_freq=False or a tie_breaker other than "name" (screening) or "pvalue" (LR elimination) now raises ValueError where it used to run with no effect; a PSI equal_freq=False no longer reaches iv_equal_freq, so it keeps working.

5. Fixes From the End-to-End Audit of the Two Pipelines

CreditModelPipeline and FeatureValidationPipeline were run end to end on synthetic data with known properties (a strong feature, a near-duplicate, a constant, a drifting feature, a high-missing feature, special values, weights, an OOT sample), and each stage was compared with an independent recomputation. Each defect below has regression tests in test_pipeline_e2e_audit.py that fail without the fix.

Defect Fix
A constant feature crashed get_decision_tree_binning_edges (it returned a tuple instead of a list of edges), so the WOE plots of the screening failed for it. CreditModelPipeline swallowed the exception: the whole screening was discarded and every feature, the constant one and the noise included, went on to the models A feature with one distinct value gets the same list of edges as a tree that makes no split
The random INS/OOS split of a frame with a duplicated index (for example after pd.concat) duplicated rows and put training rows into OOS: SampleSplitter.split_df selected by index label It selects by position. The partition of a frame with a unique index is unchanged
WOE_Master(missing_ref_value=...) with a value other than -999999 (for example -99999) fitted the missing rows at -999999 but filled them with missing_ref_value when it scored, so they got the WOE of an unrelated bin (-1.32 instead of -0.03 in the test) fit fills with missing_ref_value (or fillna) too. The default -999999 is unchanged
A feature with few distinct values is binned at its own values, so a larger value in new data fell in no bin and got a NaN WOE. An LR model then failed, a tree model read it as missing A value above the last fitted bin (or below the first) takes the WOE of the nearest bin, and the log reports the count
The Excel report allocated all 1,048,576 rows of every worksheet: about 0.5 GB and 8 s per sheet, so the 20 sheets of the default CreditModelPipeline report needed about 10 GB and were killed by the kernel after everything else had been written ExcelMaster.set_cell_size sets the default row height of the sheet. The layout is the same, a 3-sheet workbook takes 0.1 s and 200 MB instead of 16 s and 1.7 GB
Rows with a missing or unknown sample label ("inss", "train") belong to no split, which is documented, but they vanished without a trace UserWarning with the count and the labels, in both pipelines
psi_compare_splits was case-sensitive and a bare string was exploded into characters, so ["OOT"] or "oot" switched the PSI stage off silently, and a split without rows was skipped silently Case and spaces are ignored, a bare string is one split, any other name raises ValueError, and a requested split without rows gives a UserWarning
The psi column of feature_selection_summary['psi'] was the INS-vs-OOS value even when the OOT value decided It is psi_max, the value compared with the threshold
Every exception of the screening stage of CreditModelPipeline became feature_selection_summary['error'] with all features kept and no warning, and on_empty_stage='raise' could never raise. FeatureValidationPipeline did not forward on_empty_stage A RuntimeWarning announces the failure; on_empty_stage='raise' raises EmptyStageError (a ValueError) through both pipelines
woe_fit_query was checked on the first 100 rows only: a query that failed on the full data or selected nothing fell back to all rows without a word, and valid expressions that call a method (x.notna(), channel.isin([...])) were rejected as unknown columns The query is run on the INS rows when the pipeline starts, a failing or empty query raises ValueError, and method calls, string literals and backtick names are read correctly
The weighted screening left the missing bin out of the IV, so a feature whose signal is its missingness was dropped with weights of 1.0 and kept without weights The missing bin counts, as in the unweighted run
The weighted correlation dedup was a pairwise greedy pass: the result depended on the column order (a feature was lost because of a partner that a later pair removed), and it ranked by an IV of 0 for every feature when the IV stage was off Features are processed by descending IV (ties by name) and each one drops the weaker features correlated with it; the IV of the ranking is computed when the IV stage is off
corr_params['method'] and ['base_metric'] of the feature validation only reached the correlation report: a pair that Spearman flags was reported as "remove" and then kept by the selection FeatureScreenConfig has corr_method and corr_base_metric, and the feature validation sets them from corr_params. Weighted runs with non-constant weights support Pearson only and raise ValueError otherwise
The LR grid search read its best row as one float Series, so an integer parameter such as max_iter=100 came out as 100.0 and the final fit raised InvalidParameterError Each parameter is read from its own column and keeps its type
The default CatBoost parameters carried random_seed=42 and the pipeline then added random_state, which the wrapper turns into random_seed and lets win, so a random_seed in model_params['cat'] was silently replaced The default carries no seed (the config random_state supplies 42), and a random_seed of the user is kept
The backward-elimination proxy took the default LightGBM parameters of the final model: n_estimators and early_stopping_rounds overrode num_boost_round and early_stopping_rounds of backward_params['run'] (300 rounds instead of the documented 200, a patience of 50 instead of 20), the pipeline seed was passed as random_state while the preset seed=42 won, and eval_weight_col=None left the validation set weighted with weight_col. The XGBoost proxy fitted a squared-error regression to the 0/1 target The keys that duplicate the run's arguments are dropped, random_state becomes the proxy seed (a seed in backward_params['run'] still wins), eval_weight_col=None gives an unweighted validation set, and the XGBoost proxy uses binary:logistic
warm_start_score_type='probability' clipped log-odds given by mistake to 1e-6 without a word, and the GBMs trained on a constant offset Values outside [0, 1] raise ValueError that names warm_start_score_type='log_odds'
An extra_eval_datasets frame without the evaluation weight column failed with KeyError after screening, WOE, searches, training and Optuna; explain_params sizes below 1 gave an all-NaN importance table or an IndexError; nrows in csv_read_kwargs raised an obscure TypeError All three are checked when the pipeline starts, with a message that names the setting
The target was never validated: a 1/2 coding gave a KS of 600 and an IV of 0, a single-class target an IV of 13 for every feature (noise included), a missing label crashed a model late. A mistyped target_cols name switched the WOE, IV/KS, correlation detail and selection stages of FeatureValidationPipeline off without a word Both pipelines raise ValueError for labels other than 0/1 and for a single class (CreditModelPipeline for missing labels too; FeatureValidationPipeline keeps rows with a missing label, as documented), and FeatureValidationPipeline warns about target_cols names that are not columns
FeatureValidationPipeline with weight_col and INS rows whose target is missing: the weighted bad/good totals were NaN, every IV came out 0.0, and the "no feature passed" fallback kept every feature, pure noise included (the unweighted screening drops those rows) Rows with a missing target are dropped before the weighted screening, as in the unweighted one
The direction gates (direction_consistency_min, monthly_iv_min) compared class means of the raw values, so a -999999 missing sentinel dominated the means and flipped the sign of a feature that is monotone in every month The sentinel (missing_ref_value) is left out, like a missing value
A time_dims or population_dims column that the data does not have is documented as skipped, but the distribution summary emitted a table named after it that held the global statistics, and the IV/KS summary a second block of rows labelled global per absent column. A group_specs entry on an absent column did the same Absent time and population columns are skipped; an absent group_specs column is skipped with a UserWarning
new_feature_cols=None inferred weight_col, the other target_cols and the grouping columns as features (the weight was WOE-binned, IV-ranked and could be selected). The CSV batch mode resolved the columns by another rule and did not read the columns of group_specs, ivks_group_dims and psi_group_dims One rule for the DataFrame and the CSV path (see the parameter table of new_feature_cols); the batch mode reads every grouping column
woe_params['woe_suffix'] other than _woe crashed the PSI and IV/KS stages of FeatureValidationPipeline with KeyError, and a reused WOE_Master engine with its own suffix crashed CreditModelPipeline The adapters take the suffix of the engine
psi_reference_dataset='oot' without OOT rows compared every feature with an empty reference and reported PSI values around 0.7 for features that did not move A UserWarning and no PSI
apply_time_col holding 20240131 integers was read as nanoseconds since 1970 (every row in 1970-01), and a column that mixes dates and date-times lost half of its rows to NaT YYYYMMDD integers, epoch seconds and milliseconds, and mixed date strings are read; a UserWarning reports the values that still cannot be parsed
FeatureValidationPipeline and the screening engine did not declare the -999999 sentinel as a special value (CreditModelPipeline did), so the sentinel shared a bin with the lowest real values, and an artifact handed to CreditModelPipeline scored the same data differently than a self-fit (LR coefficients 0.65 vs 0.57 in the test) All three declare it the same way (when a fitted numeric feature holds it, unless special_values is set)
A reused screening engine that did not bin some selected feature (for example the missing-rate gate left it out) left the raw-feature GBMs training on it while the LR did not, and feature_selection_summary['final_features'] claimed it was used. A validation run without WOE kept no engine, and an artifact without one (a CSV batch run) made CreditModelPipeline refit with another engine silently The unbinned features are left out of every model with a RuntimeWarning (dropped_without_woe in the summary); the engine that the screening fitted is kept when woe_enabled=False; a RuntimeWarning says when no engine can be reused
A credit-model run whose INS/OOS split differs from the split of the artifact's validation run put the validation's INS rows into its OOS sample: in the test (a pure-noise target) the OOS AUC of the LR was 0.58 instead of 0.51 The artifact records the split settings (config_snapshot['split']) and CreditModelPipeline warns when they differ. A selected feature that is not a column of the data now raises KeyError that names it, and run_modeling_from_validation warns when it replaces a target_col or weight_col that you set in cm_config
CSV batch mode: rows with an unknown sample label were promoted to INS; the WOE bins of the incumbent features came out once per batch (the IV summed over the table was multiplied by the number of batches); after a failed batch the keys of batch_results and psi_details were renumbered, the summary still counted its features, and the error of an all-failed run said to inspect a table that is not returned; result.splits carried an internal row id Rows without a label stay out of every split; incumbent bins are listed once; the keys keep the batch ids, n_failed_features is reported with a RuntimeWarning, and the all-failed error quotes the first causes; the row id is removed
woe_artifacts['woe_table'] (also woe_table.csv and the Excel sheet) of CreditModelPipeline was recomputed from the raw counts, so with special-value policies, smoothing, include_missing=False or a low precision its WOE differed from the WOE the models received It is the mapping the transform applies
With woe_params={'woe_suffix': ''} the WOE columns replace the raw ones, and the all_missing_score_value rule, defined on raw features, never fired The rule reads the raw frames
The monotone binner's merge guard let one merge too many through, so a feature could end with min_n_bins - 1 bins The guard stops at min_n_bins
A feature in tiny units (1e-6) collapsed to one or two bins in the equal-frequency engine and the screening, which round to 5 decimals, got an IV of about 0 and was dropped (or entered the model as a constant) without a message A UserWarning names the features whose distinct values the rounding halves; the rounding itself is unchanged
An object feature that is not in categorical_features failed to bin and the run crashed later with an unrelated KeyError; so did an external PSI reference without a feature column; and one categorical feature turned the PSI without WOE bins of every feature into a single error row ValueError before any work for the first two, with the names; the numeric PSI skips non-numeric features with a UserWarning
The missing-rate gate that runs before the WOE fit left no trace in the selection summary and, when every feature exceeded the threshold, listed as dropped the features it had kept; the config snapshot held only the keys the user overrode, not the thresholds used selection_summary['missing_gate_dropped'] (also in the artifact), the gate lists only real drops, and the snapshot records psi_threshold, psi_compare_splits, iv_threshold, corr_threshold, corr_method, corr_base_metric, on_empty_stage, missing_rate_ref
SHAP and Owen explained the unscaled features of an LR with standardize=True (additivity error 1.6 on the scored logit, ranks of three features wrong); CatBoost was explained with the model-agnostic explainer in probability units (and not reproducibly) while the other models are in log-odds; a loaded monotone engine ignored its own woe_suffix The explainer receives the scaled features; CatBoost uses the tree explainer (log-odds, additive to the margin, reproducible); MonotoneBinnerAdapter.transform defaults to its own suffix
The saved model metadata recorded only the user's overrides in model_params (an LR search result contradicted it), the pipeline seed instead of the seed the model used, warm_start_enabled=True for CatBoost and unknown names that were not warm-started, no warm_start_score_type, and no woe_suffix The metadata holds the parameters the model was built with, its seed, the real warm-start state, the score type and the suffix
The Feature_Selection sheet of the Excel report showed each table as DataFrame(r, c) (with write_outputs=False the tables existed nowhere on disk) and cut texts at the 32,767-character limit of a cell One FS_<table> sheet per table; long texts continue on extra rows
MonotoneWOEBinner measured ordinary bins against the totals of the ordinary rows and special-value or [Missing] bins against the totals of all rows, so a riskier special bin could get a lower WOE than a safer ordinary bin and the IV was overstated New sv_total_basis, default 'all' (every bin on the totals of all rows); 'ordinary' reproduces the legacy numbers, and a binner pickled before the setting keeps it on refit. The by-group charts of plot_woe_graph measure their in-group IV and WOE lines on the fitted basis. Accepted by the binner, monotone_woe_params of both pipelines and feature_screen
With a warm start the prior score was added to the training and to the final evaluation only: early stopping, the AUC_* of the Optuna table and the Owen explanations saw the increment alone New warm_start_score_scope, default 'full' (the prior is also added to the early-stopping validation set, to the Optuna scoring and, as the group warm_start_prior, to the Owen explanation); 'train' reproduces the legacy models. GradientBoostingModel.fit takes eval_init_score, param_search takes eval_init_scores
Cosmetic outputs: quantile columns were not all named Q<percent> (0.29 gave 29%, the median stayed 50%, 0.025 stayed 2.5%); the PSI summary called the sample dimension _smf_split; CreditModelPipeline created empty figs/woe, figs/mono_woe and figs/perf folders whenever an output switch was on and drew each WOE chart twice (f_x.png and f_x__smf_plot_group.png); a model that is not explained left an empty entry in explain_outputs; model_paths, artifact_paths, report_path, explain_paths and the saved metadata held paths relative to the working directory Quantile columns are Q<percent> (Q29, Q2.5, Q50); the PSI column and group_col value are sample; chart folders are created when a chart is written and each WOE chart once; the empty entry is gone; the recorded paths are absolute
CSV batch mode with selection_enabled: each batch selected on its own and the result was the union of their choices, so correlated features of different batches were both kept, max_selected_features held per batch (three features with a cap of one in the test), a batch whose features all failed a stage kept them (or failed with on_empty_stage='raise'), and the screening artifact held no WOE engine, so CreditModelPipeline fitted other bins Two passes: each batch applies the stages that judge a feature on its own (missing rate, PSI, IV, group and target gates); the candidates of all batches are then read together and the full selection runs once on them, as without batches. selection_summary gains batch_candidates and batch_screen_summary; the artifact carries the engine of that pass
synthesize_missing_oot=True pooled the OOS copy that stands in for the OOT with the real rows: n_rows, the distribution, the global and grouped IV/KS, the global PSI and the correlation counted the OOS rows twice (2567 rows reported for 2000) The stand-in rows feed only the per-split outputs (the oot group of the PSI by sample, the OOT checks of the selection); result.splits['oot'].attrs['smf_stand_in_oot'] marks them
ivks_summary on WOE bins counted the -999999 sentinel as a value (min -999999, mean -99,999.9, missing_rate 0) while the report without WOE bins counted it as missing Both paths count ivks_params['missing_rate_ref'] (default woe_params['missing_ref_value'], -999999) as missing in n, missing_rate, min, mean and max
With spec_values / special_score_values the unweighted evaluation summary left the sentinel rows out of N and avgTrue (2700 rows and 0.255 for 3000 rows and 0.288 in the test) while the weighted one counted them Both count every row and report N_SPECIAL / N_SPECIAL_RAW; AUC, KS, the Top/Btm rates and lifts and avgScore still use the ranked rows only
The cross-batch rows of correlated_detail (batch_corr_mode='block_pairwise') computed IV/KS/lift on raw values (0.896) while the in-batch rows and ivks_summary use the WOE engine (0.951), so the keep/remove choice of a cross-batch pair used another metric They take the global ivks_summary metrics when both use the engine; metric_source says which basis a row uses
perf_min_bin_prop (min_bin_prop of PerformanceEvaluator) was a target only: the unweighted Gains table kept at least 5 bins (20% each when 25% was asked) and the weighted one ignored it (10 bins), so the two reported different IV and LIFT Both use perf_pct_bins bins capped at 1 / perf_min_bin_prop, so every bin holds about that share or more
A run into a reused output_dir left the files of the stages that no longer run (models/model_xgb.pkl, perf/perf_xgb.csv, a former feature_batches layout) Each run lists its files in a hidden manifest (.smf_manifest_<pipeline>.json); the new opt-in clean_output_dir=True of both pipelines removes, after a successful run, the files of the previous manifest that the run did not write again, and nothing else
On WOE bins the weighted screening (weight_col) left a bin that holds only goods or only bads out of the IV, while the unweighted screening and the validation tables count it with the 1e-6 floor of the WOE engine: a feature whose lowest fifth held no bad row had an IV of 3.08 without weights and 0.09 with weights of 1.0 The weighted screening on WOE bins uses the convention of the engine; with unit weights it gives the unweighted IV

Results that change

A feature screening with weight_col now counts the missing bin in the IV, and its correlation stage can keep another feature of a correlated group; screening results with weights therefore differ from before whenever a feature has missing values or a correlated group has three or more members. A WOE_Master with a missing_ref_value other than -999999 now bins the missing rows at that value. A woe_fit_query that failed, or selected no row, now raises where it used to fall back to every INS row. psi_compare_splits with a name other than oos or oot raises. Rows with an unknown sample label are unchanged but now warn. A backward elimination with the default settings now trains 200 rounds with a patience of 20 and the seed random_state (it ran 300 rounds, patience 50, seed 42), so the features it keeps can differ; the XGBoost proxy is now a logistic model. A CatBoost model with a random_seed in model_params now uses it. A target with labels other than 0/1, or with a single class, now raises where it used to run. A weighted selection with unlabelled INS rows keeps different features (they used to zero the IV of every feature). The direction gates ignore the sentinel, so a feature that was dropped because of it can now be kept. new_feature_cols=None drops weight_col, the other targets and the grouping columns from the inferred features, and the DataFrame and CSV paths agree. apply_time_col values that are integers, epochs or mixed strings now fall in their real months (they fell in 1970-01 or in NaT). FeatureValidationPipeline and the screening engine now give the -999999 sentinel its own bin, so their WOE, IV and PSI change for data that holds it (they agree with CreditModelPipeline now). woe_table of CreditModelPipeline shows the applied WOE, which differs from the recomputed one when special-value policies, smoothing, include_missing=False or precision are set. The monotone binner keeps at least min_n_bins bins (it could end one below). SHAP/Owen values of an LR with standardize=True are those of the scaled model, and CatBoost's mean_abs_shap is now in log-odds (about 7 times larger than the probability-unit value in the test). A model's saved model_params, random_state and warm_start_enabled metadata change as described. A reused engine that lacks a selected feature drops that feature from the raw-feature GBMs. A batch run that used to merge unlabelled rows into INS, or to repeat the incumbent bins, now gives the non-batch numbers. A batch run with selection_enabled selects what a run without batches selects, usually fewer features than the old union; the selected_features of each batch folder are now the candidates of the first pass. With synthesize_missing_oot=True the validation pipeline's pooled tables (n_rows, distribution, global IV/KS, global PSI, correlation) no longer count the OOS rows twice. The IV/KS report on WOE bins now counts the sentinel as missing (n, missing_rate, min, mean, max change for features that hold it). The unweighted evaluation summary with spec_values reports N and avgTrue over every row and adds N_SPECIAL / N_SPECIAL_RAW. Cross-batch rows of correlated_detail report the IV of ivks_summary and have a new metric_source column, and their keep/remove can change. Two defaults changed: MonotoneWOEBinner(sv_total_basis='all') puts every bin on the totals of all rows, so a refit moves the WOE of the ordinary bins (by one constant per feature) and the IV of every feature that has special or missing values, in both pipelines and the screening; pass sv_total_basis='ordinary' (in monotone_woe_params for the pipelines) to reproduce a scorecard fitted before. warm_start_score_scope='full' makes a warm-started LightGBM or XGBoost model stop early on the combined score, scores the Optuna candidates with the prior and adds the group warm_start_prior to its Owen explanation, so the trained trees, the search table and the explanation change; pass warm_start_score_scope='train' to reproduce a model trained before. Saved scorecards, binners and models keep their numbers until they are refitted. A perf_min_bin_prop (or min_bin_prop of PerformanceEvaluator) above 1 / perf_pct_bins now gives fewer, larger Gains bins, and the weighted evaluation follows it, so IV, LIFT, KS_IN_GAINS and N_BINS change for such settings (the defaults are unaffected). Both pipelines write a hidden .smf_manifest_<pipeline>.json into output_dir when they write files. A weighted screening on WOE bins counts the bins that hold one class only, so a feature with such bins gets a higher IV, can now pass iv_threshold and ranks higher in the cap and the VIF tie-break. Code that reads the old names must change: the 50%, 2.5% and other percent-style quantile columns are Q50, Q2.5, ..., and the sample dimension of psi_summary is sample (column and group_col value), no longer _smf_split; the __smf_plot_group chart files are no longer written.

6. Known Issues

The audit found these behaviors and decided to document them instead of changing the numbers. None of them is silent any more: each is stated in the docstring and the guide of the parameter it concerns.

  • Without WOE bins the weighted and the unweighted screening bin differently. With iv_use_woe_bins=False and no engine, the unweighted IV comes from the decision-tree bins of the gains table and the weighted IV from weighted equal-frequency bins, and each leaves out the bins that hold one class only; the two IVs differ even with weights of 1.0 (1.53 against 1.41 for the same feature in a test). On WOE bins, the default of the feature validation, both use the convention of the WOE engine.
  • correlated_detail covers the pairs of the anchor group of each correlated feature, not every pair of high_corr_pairs. With batch_corr_mode='block_pairwise' the merged corr_matrix is block diagonal (the cross-batch pairs are in high_corr_pairs).
  • A stricter threshold can return a looser selection. With the default on_empty_stage='keep_all_warn', a threshold that would eliminate every feature keeps them all and warns; set on_empty_stage='raise' (in selection_params, feature_selection or the screening config) to stop instead. Changed in 0.9.3: from the IV stage on, this fallback keeps only the fallback_top_k (5) features with the highest screening IV and never one that the IV upper limit dropped (it raises EmptyStageError when only those are left); the missing-rate and PSI stages still keep every feature, and on_empty_stage='keep_all' restores the behavior described here. See v0.9.3.

Should a new issue be confirmed, the FAQ will list it with its workaround.