Modeling_Tool.Pipeline

One-click pipelines. Each pipeline is a class that takes a config dataclass, runs with Pipeline(config).run(data) (ScoreConsistencyUATPipeline can also read SQL files, and MockSamplePipeline needs no data), and returns a result dataclass. This page is the generated reference. The Top-Level Pipelines guide explains what each pipeline does and how to configure it.

All names below are also importable from Modeling_Tool, for example from Modeling_Tool import CreditModelPipeline, CreditModelPipelineConfig.

Credit Model Pipeline: credit_model

credit_model

CreditModelPipelineConfig dataclass

Configuration of :class:CreditModelPipeline.

Parameters:

Name	Type	Description	Default
output_dir	str	Root of all outputs: the CSV files, the figs charts (var_analysis, woe, mono_woe, perf), explain, artifacts, models (unless model_output_dir is set) and the Excel report. Directories are created as needed when write_outputs, write_excel or save_models is on. A run overwrites the files it writes and leaves the others alone, so a second run into the same directory with fewer stages keeps the files of the stages that no longer run (for example backward_summary.csv or models/model_xgb.pkl) unless clean_output_dir is on. Every run that writes files lists them in output_dir/.smf_manifest_credit_model.json.	"output"
clean_output_dir	bool	After a successful run, remove the files that the previous manifest of this pipeline lists and this run did not write again (the outputs of stages that no longer run), and the directories they leave empty. Only files of the manifest are removed: files written by hand, by another pipeline, by a version without the manifest, or under model_output_dir outside output_dir stay.	False
target_col	str	Binary target column (1 = bad). It must be a column of the input data.	"badflag"
feature_cols	list of str or None	Raw model input features. None (or an empty list) infers them as the numeric columns of the input data except target_col, sample_col, split_col, oot_col and weight_col. Other helper columns, such as eval_target_cols, warm_start_score_col or a separate eval_weight_col, are not excluded, so list the features explicitly when the data has such numeric columns. A listed column that is missing raises KeyError.	None
split_col	str or None	Column with the sample labels ins / oos / oot (case and surrounding spaces ignored; a row with a missing label belongs to no split, which a UserWarning reports with the count). When set it replaces sample_col; it must exist (KeyError), may hold only those labels and must contain non-empty INS and OOS samples (ValueError).	None
sample_col	str	Legacy sample label column with the same labels, used when split_col is None. Rows with a missing or unknown label belong to no split and a UserWarning reports them. If it is absent or has no INS or no OOS rows, the pipeline silently falls back to oot_col and a random INS/OOS split.	"sample_ind"
oot_col	str or None	OOT flag column, used only when the sample labels do not define INS and OOS: rows with a non-zero flag (missing counts as 0) become OOT and the others are split randomly into INS and OOS (see split_config). The column must be convertible to numbers, otherwise TypeError is raised; an absent column is ignored.	"oot_flag"
weight_col	str or None	Sample-weight column of the input data (finite, non-negative, with a positive sum; KeyError if missing). It weights feature screening, LR and GBM training, the LR and Optuna searches, backward elimination and, through eval_weight_col, the evaluation. The WOE fit is unweighted. None runs unweighted.	None
random_state	int	Seed of the random INS/OOS split (unless split_config sets its own), the Optuna searches, the explanation sampling, and the LightGBM, XGBoost and CatBoost models: the final models, their Optuna candidates and the backward-elimination proxy. A random_state in model_params for a model takes precedence for that model (for cat, a random_seed too).	42
write_outputs	bool	Whether to write the CSV files and the explanation files into output_dir, for the results that exist: psi_result.csv, iv_report.csv, lr_pvalue_elimination.csv, woe_table_ins.csv, backward_summary.csv, lr_param_search.csv, warm_start_summary.csv, model_feature_sources.csv, model_paths.csv, <model>_optuna_search.csv, perf/perf_<model>.csv and explain/. Charts also need plot_outputs.	True
write_excel	bool	Whether to write SMF_Model_Report.xlsx into output_dir (its path is result.report_path), with the sheets Feature_Selection, WOE_Table, Backward, LR_Param_Search, Warm_Start, Model_Feature_Source, Model_Paths, Perf_<MODEL> and the Explain_* sheets, plus one FS_<table> sheet per table of the feature selection summary (FS_psi, FS_iv, ...). A text longer than an Excel cell holds is split over several rows.	True
plot_outputs	bool	Whether to draw the charts: the IV and WOE analysis plots of the screening (figs/var_analysis), the WOE bin plots (figs/woe or figs/mono_woe), the performance figure of each model (figs/perf/perf_<model>.png) and the SHAP summary (explain/<model>/shap_summary.png). They are written only when write_outputs is also True.	True
save_models	bool	Whether to save every trained model as model_<name>.pkl in the model directory and to record the paths in result.model_paths and result.artifact_paths. model_paths.csv is then written to output_dir even when write_outputs is False. No models directory is created otherwise.	False
model_output_dir	str or None	Directory of the saved models. None uses <output_dir>/models.	None
model_include_metadata	bool	True saves each model and the WOE engine inside the SMF artifact envelope together with metadata (pipeline, target, features, split governance, scoring options); False saves the bare object. Used only with save_models.	True
save_woe_artifacts	bool	With save_models, also save the WOE table (artifacts/woe_table.csv) and the WOE engine (artifacts/woe_engine.pkl) under output_dir so that the models can be reproduced. No effect without save_models.	True
split_config	dict	Settings of the random INS/OOS split, which runs only when the sample labels do not define INS and OOS: test_size (OOS share, default 0.3), stratify (stratify by the target, default True) and random_state (default: the random_state field). Other keys are ignored.	{'test_size': 0.3, 'stratify': True}
feature_selection	dict	Settings of the PSI, IV and correlation screening, keyed like the fields of FeatureScreenConfig; they are read by screen_config_from_mapping, which supplies the defaults of missing keys (iv_nbins is an alias of iv_bins). The WOE settings of the screening come from woe_engine, woe_fit_query, woe_params and monotone_woe_params of this config, not from the dict. Any exception raised by the screening (for example an invalid setting, or a G03 or G04 gate setting, which needs evidence that this pipeline does not provide) is caught: it is recorded in feature_selection_summary['error'], announced by a RuntimeWarning, and every feature is kept, except the leak-gate drops that the screening found before it failed (listed in feature_selection_summary['hard_dropped']; since 0.9.3). EmptyStageError (a ValueError) is not caught: it is raised when a stage would drop every feature and on_empty_stage="raise", or when every feature the stage would drop is a leak-gate drop. on_empty_stage (default "keep_all_warn") and fallback_top_k (default 5) set the empty-stage fallback, which never brings back a leak-gate drop (see FeatureScreenConfig.on_empty_stage; "keep_all" restores the fallback up to 0.9.3 for comparisons with old results). The backward elimination falls back to the screened features only, so it cannot bring a leak-gate drop back either. psi_compare_splits ignores case and spaces and accepts a bare string, corr_method and corr_base_metric set the correlation coefficient and the metric that decides between two correlated features. iv_upper_threshold (the leakage limit, 2.0 since 0.9.3; also the value taken when the key is missing) drops features above it, recorded in feature_selection_summary['dropped_detail'] with a warning per feature; pass None to switch it off. iv_gap_ratio_warn and iv_gap_abs_warn set the leak warning of the screening on WOE bins (see FeatureScreenConfig).	{'psi_enabled': True, 'psi_threshold': 0.2, 'psi_compare_splits': ['oos'], 'iv_enabled': True, 'iv_threshold': 0.02, 'iv_upper_threshold': 2.0, 'corr_enabled': True, 'corr_threshold': 0.75, 'corr_max_iterations': 10}
woe_engine	str	WOE binning engine: "monotone" (case-insensitive) uses MonotoneWOEBinner with monotone_woe_params, any other value uses WOE_Master with woe_params. It is also the engine that the screening fits when feature_selection sets a *_use_woe_bins flag.	"equal_freq"
woe_fit_query	str or None	pandas query expression that selects the INS rows used to fit the WOE binning (and the screening engine); the transform, training and evaluation still use all rows. It is checked when run starts: a referenced column that is missing from the input data raises KeyError, and an invalid expression, one that fails on the INS rows or one that selects none of them raises ValueError.	None
extra_eval_datasets	dict of str to pandas.DataFrame or None	Evaluation-only frames {name: DataFrame}. They are WOE-transformed with the fitted engine and evaluated like the splits (their rows appear under name in each model's table of perf_results), but take no part in screening, WOE fit, training, backward elimination, Optuna or the explanations, and evaluation_splits does not limit them. A name must not be ins, oos or oot (ValueError); each frame must hold the evaluation targets (target_col and eval_target_cols) and, with warm start, warm_start_score_col (KeyError).	None
woe_params	dict	Parameters of the WOE_Master engine. The keys woe_suffix (default "_woe", which also names the WOE features of the monotone engine) and missing_ref_value (default -999999) go to the WOE_Master constructor, all other keys to WOE_Master.fit. The default dict holds nbins, equal_freq and min_bin_prop plus the four special-value bin settings sv_*, which reproduce the behavior before 0.8.0. A dict you pass replaces the default; omitted keys take the fit defaults.	{'nbins': 10, 'equal_freq': True, 'min_bin_prop': 0.05, 'sv_min_bin_size': 0.0, 'sv_small_policy': 'keep', 'sv_woe_smoothing': 'none', 'sv_smoothing_alpha': 0.0}
monotone_woe_params	dict	Parameters of the MonotoneWOEBinner engine: its constructor keys, and the fit-only keys chi2_binning (default False), chi2_p, chi2_init_size and n_jobs, which are passed to fit. Without a special_values key, the sentinel -999999 is declared only when it occurs in the fit sample. A dict you pass replaces the default.	{'n_init_bins': 20, 'min_bin_size': 0.03, 'min_n_bins': 2, 'sv_min_bin_size': 0.0, 'sv_small_policy': 'keep', 'sv_woe_smoothing': 'none', 'sv_smoothing_alpha': 0.0, 'unseen_special_policy': 'neutral'}
train_models	list of str	Models to train, any of "lr" (LRMaster) and "lgb", "xgb", "cat" (GradientBoostingModel), case-insensitive; another name raises ValueError when training starts. INS is the training set and OOS the validation set.	['lr', 'lgb', 'xgb', 'cat']
model_params	dict of str to dict	Per-model parameter overrides {model_name: {parameter: value}}, merged over the built-in defaults of that model (for example n_estimators=300 and learning_rate=0.05 for lgb). For lr the key standardize (bool, default False) is passed to LRMaster and the other keys are its sklearn parameters.	{}
gbm_feature_source	str or dict of str to str	Input features of the GBM models: "woe" (WOE-encoded) or "raw", for all three models, or a dict keyed by lgb, xgb and cat (a missing key means "woe"). LR always uses the WOE features. Other keys or values raise ValueError when the pipeline is created.	"woe"
lr_search_enabled	bool	Whether to run a hyper-parameter grid search for the LR model (LRMaster.grid_search_params) before training. It needs "lr" in train_models; the table is returned as result.lr_search_results.	False
lr_search_param_grid	dict of str to list	Parameter grid of the LR search, searched as a Cartesian product.	{'C': [0.01, 0.1, 1.0, 10.0]}
lr_search_params	dict	Overrides of the LR search settings. The allowed keys are objective, primary_set, gap_ref_sets, metric, refit and verbose; any other key raises ValueError (the search is a holdout search and does not accept cv). The defaults are objective="oot_gap_penalized", primary_set="oos" and gap_ref_sets=["oot"] when OOT is among search_eval_splits, otherwise objective=search_objective_when_no_oot with primary_set="oos" and no gap reference, and metric="auc".	{}
use_lr_search_params	bool	Whether the best parameters of the LR search are merged over model_params["lr"] for the final LR model. No effect without the search.	True
lr_elimination_mode	str or None	Backward elimination of the final LR model: None keeps every feature, "pvalue" refits the LR without its feature of highest coefficient p-value until every p-value is at most pvalue_threshold (see lr_elimination_params). Any other value raises ValueError. The dropped features are recorded in feature_selection_summary["lr_elimination"]; the model's final features are result.models["lr"][2], while result.selected_features, selected_woe_features and model_feature_sets still show the list from before the elimination.	None
lr_elimination_params	dict	Settings of the p-value elimination: pvalue_threshold (default 0.05), min_features (default 1; the elimination stops when this many features remain) and max_iterations (default 20). tie_breaker may only be "pvalue" (or None), which is what happens anyway: equal p-values are resolved by column order. Any other tie_breaker value and any other key raise ValueError.	{}
warm_start_enabled	bool	Whether to start the GBM models from a prior score (LightGBM init_score, XGBoost base margin), in training and in evaluation, so that a model's probability combines the prior score with its increment. It needs warm_start_score_col and is supported for lgb and xgb only. By default (see warm_start_score_scope) the prior is also seen by early stopping, the Optuna search (AUC_* of its table) and the Owen explanations, so every stage judges the combined model.	False
warm_start_score_col	str or None	Column of the input data that holds the prior score. It is required when warm_start_enabled is on (ValueError), must exist (KeyError) and must have no missing values in any evaluated frame. It is copied by position onto the WOE-transformed splits, with a length check (ValueError).	None
warm_start_score_type	('probability', 'log_odds')	"probability" scores are clipped to [1e-6, 1 - 1e-6] and converted to log-odds, "log_odds" scores are used as the init score as they are. Any other value raises ValueError when warm start is enabled.	"probability"
warm_start_models	list of str	GBM models that use the warm start, compared case-insensitively. "cat" is not supported (see warm_start_on_unsupported). Names that are not in train_models appear in warm_start_summary with the status "not_in_train_models".	['lgb', 'xgb']
warm_start_on_unsupported	('skip', 'raise')	What to do when "cat" is requested for warm start: "skip" trains it without the prior score and records "skipped_unsupported" in warm_start_summary, "raise" raises NotImplementedError. Any other value raises ValueError when warm start is enabled.	"skip"
warm_start_apply_to_optuna	bool	Whether to pass the prior score as init_score to the Optuna search of the warm-start models (lgb and xgb).	False
warm_start_score_scope	('full', 'train')	Where the prior score is seen. "full" (default) adds it to the training, to the validation set of the early stopping (so the models stop when the combined model stops improving), to the scoring of the Optuna candidates (with warm_start_apply_to_optuna), to the final evaluation and to the Owen explanation, where the prior enters as a group of its own, warm_start_prior, so that the Owen values add up to the scored probability. "train" (the behavior of 0.8.2 and earlier) adds it to the training and to the final evaluation only: early stopping, the AUC_* of the Optuna search table and the Owen explanations then see the increment alone; pass it to reproduce models trained before the change. Any other value raises ValueError when warm start is enabled. SHAP values of the trees are unaffected by the prior (it is an additive offset in log-odds), so explain_models gives the same feature importance in both scopes.	"full"
backward_enabled	bool	Whether to run backward variable elimination on the WOE features before the models are trained.	True
backward_model	str	Proxy model of the backward elimination: "lgb" runs the LightGBM elimination and "xgb" the XGBoost one (case and surrounding spaces are ignored). Any other value raises ValueError when run starts, if backward_enabled is on.	"lgb"
backward_params	dict	{"init": {...}, "run": {...}}: init overrides the arguments of BackwardVariableEliminator and run those of its run call (defaults n_rounds=3, stopping_metric="auc", num_boost_round=200, early_stopping_rounds=20, cum_importance_threshold=0.99, min_vars=max(3, n_features // 2) and ret_perf=True). A split named in init["test_data_dict"] must not be forbidden. Any failure of the backward stage is caught: backward_summary then holds a one-row frame with the columns step and error and all WOE features are kept.	{}
use_backward_features	bool	True trains the models on the features that survive the backward elimination (WOE names; a raw GBM gets them mapped back to raw names), False on all screened features, so that the elimination only reports.	True
candidate_mode	bool	True forbids any consumption of OOT in the candidate stage: "oot" is added to forbidden_splits and the OOT frame is removed from the working splits. Explicit settings that request OOT (synthesize_missing_oot=True, "oot" in evaluation_splits, search_eval_splits or backward_report_splits, or backward_validation_split="oot") raise ValueError when the pipeline is created.	False
synthesize_missing_oot	bool or None	When the data has no real OOT rows: True copies the OOS rows in as a stand-in OOT (with a UserWarning; its metrics are OOS metrics), False or None leaves OOT out, so the evaluation runs on INS and OOS only.	False
evaluation_splits	list of str or None	Splits that the performance evaluation, the charts and the Excel report use. None means ['ins', 'oos'], so a real OOT must be listed explicitly. Names are case-insensitive and must be ins, oos or oot (ValueError); a listed split that does not exist in the run is silently skipped. extra_eval_datasets are not limited by it.	None
forbidden_splits	list of str	Splits that no stage may consume, such as ['oot']. They are removed from the working splits. A split list that names one raises ValueError when the pipeline is created; one named in the nested optuna_params or backward_params settings is rejected when that stage runs (in the backward stage the error is caught and shown in backward_summary).	[]
search_eval_splits	list of str or None	Splits that the LR search and the Optuna search evaluate on. None means ['oos']. An explicitly listed split that is absent from the run raises ValueError, whereas an absent split of the default is silently dropped.	None
search_objective_when_no_oot	str	Objective of the LR and Optuna searches when oot is not among the search eval splits (with OOT it is "oot_gap_penalized"). "max_primary" maximizes the AUC on oos; "oot_gap_penalized" needs a gap reference split and therefore raises ValueError here.	"max_primary"
backward_validation_split	str	Split used as the validation data of the backward elimination: "ins", "oos" or "oot".	"oos"
backward_report_splits	list of str or None	Splits on which the backward elimination reports performance after each round. None means none ([]), so OOT is not read. An explicitly listed split that is absent from the run makes the backward stage fail (caught and reported in backward_summary).	None
optuna_models	list of str	Models to tune with Optuna; [] turns the search off. Only trained models that have a search space run (see optuna_params). The search only reports: result.optuna_results holds its tables, and the models in result.models are not retrained with the tuned parameters.	['lgb', 'xgb', 'cat']
optuna_n_trials	int	Number of Optuna trials per model.	5
optuna_params	dict	Optuna settings: search_spaces ({model: {parameter: space}}, which replaces the built-in spaces, so a model missing from it is skipped), common (overrides of the param_search arguments such as objective, primary_set, gap_ref_sets, metric, refit and eval_sets; named splits must not be forbidden) and fit_kwargs (extra fit arguments). A search that fails is caught and its table holds the single column error.	{}
explain_models	list of str	Trained models explained with SHAP: feature_importance and, with charts, shap_summary.png. [] together with owen_enabled=False skips the explanations altogether.	['lr', 'lgb', 'cat']
explain_params	dict	Explanation settings: sample_n (OOS rows sampled for the explanations, capped at the OOS size), background_n (INS rows used as background, capped at the INS size) and the Owen options owen_threshold (0.35), owen_method ("complete"), owen_corr_method ("spearman"), owen_min_group_size (1), owen_intra_dist (0.01), owen_inter_dist (0.99) and owen_model_output ("probability"). Omitted keys take the values given here.	{'sample_n': 500, 'background_n': 200}
owen_enabled	bool	Whether to compute Owen values (Shapley values over groups of related features) for every trained model except xgb. While it is True, the explanations run for all trained models, also those that are not in explain_models (Owen values only).	True
business_prior_groups	dict of str to list of str or None	Business groups {group: [feature, ...]} for the Owen coalition structure, named like the model's input features (for example income_woe). Features that the model does not use are dropped from the groups, and the remaining features are clustered automatically. None uses a built-in example grouping that only matches features with those exact names.	None
perf_pct_bins	int	Number of percentile bins of the performance evaluation.	10
perf_min_bin_prop	float	Minimum share of a bin in the Gains tables of the performance evaluation (IV, LIFT, KS_IN_GAINS, N_BINS, ... of perf_results), weighted or not: they use perf_pct_bins bins capped at 1 / perf_min_bin_prop, so each bin holds about that share or more (ties in the scores can move a few rows). The Top/Btm percentile bands keep perf_pct_bins.	0.03
eval_target_cols	list of str or None	Extra label columns evaluated against the same model scores in addition to target_col (duplicates removed; the results are stacked with a tgt_name column). They must exist in the input data and in every extra_eval_datasets frame, are not used for training, and are not excluded from inferred feature_cols.	None
all_missing_score_value	float or None	Score given in the evaluation to rows whose raw model features are all missing (for example -1), the rule of the scoring API; None applies no override. It is stored in the saved model metadata. The raw features must be present in every evaluated frame (KeyError otherwise).	None
special_score_values	list of float or None	Sentinel scores (for example [-1]) that get their own evaluation bin and are left out of the quantile edges and the ranking metrics (AUC, KS, the Top/Btm rates and lifts). N and avgTrue of the summary count every row, weighted or not, and N_SPECIAL / N_SPECIAL_RAW report the sentinel part.	None
gains_ascending	bool or None	Score direction of the evaluation summary, the gains tables and the figures: True ascending (bin 1 holds the lowest scores, the lowest risk), False descending, None keeps the historical direction of each code path.	True
eval_weight_col	str or None	Weight column of the evaluation, of the validation weights of the backward elimination and of the LR and Optuna searches. "inherit" reuses weight_col, None evaluates unweighted even when the training is weighted, and any other string names a separate column that must exist in the input data (KeyError).	"inherit"
screening_artifact	FeatureScreeningArtifact or None	Output of FeatureValidationPipeline. When given, the pipeline does not screen itself: it uses the artifact's selected features (all feature_cols if it selected none) after validate_for_cm has checked target_col and weight_col, and reuses its fitted WOE engine when reuse_screening_woe is True. It forces feature_selection_mode to "from_artifact".	None
feature_validation_result	FeatureValidationPipelineResult or None	Convenience field: the result of FeatureValidationPipeline, converted internally to a FeatureScreeningArtifact with this config's target_col and weight_col. Ignored when screening_artifact is given.	None
feature_selection_mode	('run', 'from_artifact', 'skip')	"run" screens with feature_selection, "from_artifact" takes the selection from the artifact (ValueError without one) and "skip" keeps all feature_cols. Whenever an artifact is available the mode becomes "from_artifact", and any other value behaves like "run".	"run"
reuse_screening_woe	bool	With an artifact, reuse the WOE engine it carries for the target instead of fitting again, provided it covers at least one selected feature (features it did not fit are left out of every model, with a RuntimeWarning); otherwise the engine is fitted as usual, with a RuntimeWarning when an artifact is given. A reused engine keeps its own kind, bins and parameters: woe_engine, woe_params, monotone_woe_params and woe_fit_query of this config do not apply to it. The artifact also records the INS/OOS split settings of the validation run; a RuntimeWarning reports a split that differs from this run's, because the validation fitted the bins and chose the features on rows that this run would then score as OOS. No effect without an artifact.	True
screen_iv_convention	('guarded', 'raw')	How the screening IV on the bins of a WOE engine (a *_use_woe_bins flag of feature_selection, or a weighted run with an engine) treats a class-pure bin, whose WOE comes from the 1e-6 of the formula alone: "guarded" leaves it out of the IV sum, "raw" counts every bin (0.9.1). See FeatureScreenConfig.screen_iv_convention. feature_selection["screen_iv_convention"] is accepted as well; a value there that differs from this field raises ValueError when run starts, as does any other value. It does not change the WOE of the bins or the scoring.	"guarded"

Source code in Modeling_Tool/Pipeline/credit_model.py

@dataclass
class CreditModelPipelineConfig:
    """Configuration of :class:`CreditModelPipeline`.

    Parameters
    ----------
    output_dir : str, default "output"
        Root of all outputs: the CSV files, the ``figs`` charts (``var_analysis``, ``woe``, ``mono_woe``, ``perf``),
        ``explain``, ``artifacts``, ``models`` (unless ``model_output_dir`` is set) and the Excel report. Directories are
        created as needed when ``write_outputs``, ``write_excel`` or ``save_models`` is on. A run overwrites the files it
        writes and leaves the others alone, so a second run into the same directory with fewer stages keeps the files of
        the stages that no longer run (for example ``backward_summary.csv`` or ``models/model_xgb.pkl``) unless
        ``clean_output_dir`` is on. Every run that writes files lists them in ``output_dir/.smf_manifest_credit_model.json``.
    clean_output_dir : bool, default False
        After a successful run, remove the files that the previous manifest of this pipeline lists and this run did not
        write again (the outputs of stages that no longer run), and the directories they leave empty. Only files of the
        manifest are removed: files written by hand, by another pipeline, by a version without the manifest, or under
        ``model_output_dir`` outside ``output_dir`` stay.
    target_col : str, default "badflag"
        Binary target column (1 = bad). It must be a column of the input data.
    feature_cols : list of str or None, default None
        Raw model input features. ``None`` (or an empty list) infers them as the numeric columns of the input data except
        ``target_col``, ``sample_col``, ``split_col``, ``oot_col`` and ``weight_col``. Other helper columns, such as
        ``eval_target_cols``, ``warm_start_score_col`` or a separate ``eval_weight_col``, are not excluded, so list the
        features explicitly when the data has such numeric columns. A listed column that is missing raises ``KeyError``.
    split_col : str or None, default None
        Column with the sample labels ``ins`` / ``oos`` / ``oot`` (case and surrounding spaces ignored; a row with a
        missing label belongs to no split, which a ``UserWarning`` reports with the count). When set it replaces ``sample_col``; it must exist (``KeyError``), may hold
        only those labels and must contain non-empty INS and OOS samples (``ValueError``).
    sample_col : str, default "sample_ind"
        Legacy sample label column with the same labels, used when ``split_col`` is ``None``. Rows with a missing or
        unknown label belong to no split and a ``UserWarning`` reports them. If it is absent or has no
        INS or no OOS rows, the pipeline silently falls back to ``oot_col`` and a random INS/OOS split.
    oot_col : str or None, default "oot_flag"
        OOT flag column, used only when the sample labels do not define INS and OOS: rows with a non-zero flag (missing
        counts as 0) become OOT and the others are split randomly into INS and OOS (see ``split_config``). The column
        must be convertible to numbers, otherwise ``TypeError`` is raised; an absent column is ignored.
    weight_col : str or None, default None
        Sample-weight column of the input data (finite, non-negative, with a positive sum; ``KeyError`` if missing). It
        weights feature screening, LR and GBM training, the LR and Optuna searches, backward elimination and, through
        ``eval_weight_col``, the evaluation. The WOE fit is unweighted. ``None`` runs unweighted.
    random_state : int, default 42
        Seed of the random INS/OOS split (unless ``split_config`` sets its own), the Optuna searches, the explanation
        sampling, and the LightGBM, XGBoost and CatBoost models: the final models, their Optuna candidates and the
        backward-elimination proxy. A ``random_state`` in ``model_params`` for a model takes precedence for that model (for ``cat``, a ``random_seed`` too).
    write_outputs : bool, default True
        Whether to write the CSV files and the explanation files into ``output_dir``, for the results that exist:
        ``psi_result.csv``, ``iv_report.csv``, ``lr_pvalue_elimination.csv``, ``woe_table_ins.csv``,
        ``backward_summary.csv``, ``lr_param_search.csv``, ``warm_start_summary.csv``, ``model_feature_sources.csv``,
        ``model_paths.csv``, ``<model>_optuna_search.csv``, ``perf/perf_<model>.csv`` and ``explain/``. Charts also need
        ``plot_outputs``.
    write_excel : bool, default True
        Whether to write ``SMF_Model_Report.xlsx`` into ``output_dir`` (its path is ``result.report_path``), with the
        sheets Feature_Selection, WOE_Table, Backward, LR_Param_Search, Warm_Start, Model_Feature_Source, Model_Paths,
        ``Perf_<MODEL>`` and the ``Explain_*`` sheets, plus one ``FS_<table>`` sheet per table of the feature selection
        summary (``FS_psi``, ``FS_iv``, ...). A text longer than an Excel cell holds is split over several rows.
    plot_outputs : bool, default True
        Whether to draw the charts: the IV and WOE analysis plots of the screening (``figs/var_analysis``), the WOE bin
        plots (``figs/woe`` or ``figs/mono_woe``), the performance figure of each model (``figs/perf/perf_<model>.png``)
        and the SHAP summary (``explain/<model>/shap_summary.png``). They are written only when ``write_outputs`` is also
        True.
    save_models : bool, default False
        Whether to save every trained model as ``model_<name>.pkl`` in the model directory and to record the paths in
        ``result.model_paths`` and ``result.artifact_paths``. ``model_paths.csv`` is then written to ``output_dir`` even
        when ``write_outputs`` is False. No ``models`` directory is created otherwise.
    model_output_dir : str or None, default None
        Directory of the saved models. ``None`` uses ``<output_dir>/models``.
    model_include_metadata : bool, default True
        ``True`` saves each model and the WOE engine inside the SMF artifact envelope together with metadata (pipeline,
        target, features, split governance, scoring options); ``False`` saves the bare object. Used only with
        ``save_models``.
    save_woe_artifacts : bool, default True
        With ``save_models``, also save the WOE table (``artifacts/woe_table.csv``) and the WOE engine
        (``artifacts/woe_engine.pkl``) under ``output_dir`` so that the models can be reproduced. No effect without
        ``save_models``.
    split_config : dict, default {'test_size': 0.3, 'stratify': True}
        Settings of the random INS/OOS split, which runs only when the sample labels do not define INS and OOS:
        ``test_size`` (OOS share, default 0.3), ``stratify`` (stratify by the target, default True) and ``random_state``
        (default: the ``random_state`` field). Other keys are ignored.
    feature_selection : dict, default {'psi_enabled': True, 'psi_threshold': 0.2, 'psi_compare_splits': ['oos'], 'iv_enabled': True, 'iv_threshold': 0.02, 'iv_upper_threshold': 2.0, 'corr_enabled': True, 'corr_threshold': 0.75, 'corr_max_iterations': 10}
        Settings of the PSI, IV and correlation screening, keyed like the fields of ``FeatureScreenConfig``; they are read
        by ``screen_config_from_mapping``, which supplies the defaults of missing keys (``iv_nbins`` is an alias of
        ``iv_bins``). The WOE settings of the screening come from ``woe_engine``, ``woe_fit_query``, ``woe_params`` and
        ``monotone_woe_params`` of this config, not from the dict. Any exception raised by the screening (for example an
        invalid setting, or a G03 or G04 gate setting, which needs evidence that this pipeline does not provide) is
        caught: it is recorded in ``feature_selection_summary['error']``, announced by a ``RuntimeWarning``, and every
        feature is kept, except the leak-gate drops that the screening found before it failed (listed in
        ``feature_selection_summary['hard_dropped']``; since 0.9.3). ``EmptyStageError`` (a ``ValueError``) is not
        caught: it is raised when a stage would drop every feature and ``on_empty_stage="raise"``, or when every feature
        the stage would drop is a leak-gate drop. ``on_empty_stage`` (default ``"keep_all_warn"``) and
        ``fallback_top_k`` (default 5) set the empty-stage fallback, which never brings back a leak-gate drop (see
        ``FeatureScreenConfig.on_empty_stage``; ``"keep_all"`` restores the fallback up to 0.9.3 for comparisons with
        old results). The backward elimination falls back to the screened features only, so it cannot bring a
        leak-gate drop back either. ``psi_compare_splits`` ignores case and spaces and accepts
        a bare string, ``corr_method`` and ``corr_base_metric`` set the correlation coefficient and the metric that
        decides between two correlated features. ``iv_upper_threshold`` (the leakage limit, 2.0 since 0.9.3; also the
        value taken when the key is missing) drops features above it, recorded in
        ``feature_selection_summary['dropped_detail']`` with a warning per feature; pass ``None`` to switch it off.
        ``iv_gap_ratio_warn`` and ``iv_gap_abs_warn`` set the leak warning of the screening on WOE bins (see
        ``FeatureScreenConfig``).
    woe_engine : str, default "equal_freq"
        WOE binning engine: ``"monotone"`` (case-insensitive) uses ``MonotoneWOEBinner`` with ``monotone_woe_params``,
        any other value uses ``WOE_Master`` with ``woe_params``. It is also the engine that the screening fits when
        ``feature_selection`` sets a ``*_use_woe_bins`` flag.
    woe_fit_query : str or None, default None
        pandas ``query`` expression that selects the INS rows used to fit the WOE binning (and the screening engine);
        the transform, training and evaluation still use all rows. It is checked when ``run`` starts: a referenced column
        that is missing from the input data raises ``KeyError``, and an invalid expression, one that fails on the INS rows
        or one that selects none of them raises ``ValueError``.
    extra_eval_datasets : dict of str to pandas.DataFrame or None, default None
        Evaluation-only frames ``{name: DataFrame}``. They are WOE-transformed with the fitted engine and evaluated like
        the splits (their rows appear under ``name`` in each model's table of ``perf_results``), but take no part in
        screening, WOE fit, training, backward elimination, Optuna or the explanations, and ``evaluation_splits`` does
        not limit them. A name must not be ``ins``, ``oos`` or ``oot`` (``ValueError``); each frame must hold the
        evaluation targets (``target_col`` and ``eval_target_cols``) and, with warm start, ``warm_start_score_col``
        (``KeyError``).
    woe_params : dict, default {'nbins': 10, 'equal_freq': True, 'min_bin_prop': 0.05, 'sv_min_bin_size': 0.0, 'sv_small_policy': 'keep', 'sv_woe_smoothing': 'none', 'sv_smoothing_alpha': 0.0}
        Parameters of the ``WOE_Master`` engine. The keys ``woe_suffix`` (default ``"_woe"``, which also names the WOE
        features of the monotone engine) and ``missing_ref_value`` (default -999999) go to the ``WOE_Master``
        constructor, all other keys to ``WOE_Master.fit``. The default dict holds ``nbins``, ``equal_freq`` and
        ``min_bin_prop`` plus the four special-value bin settings ``sv_*``, which reproduce the behavior before 0.8.0. A
        dict you pass replaces the default; omitted keys take the ``fit`` defaults.
    monotone_woe_params : dict, default {'n_init_bins': 20, 'min_bin_size': 0.03, 'min_n_bins': 2, 'sv_min_bin_size': 0.0, 'sv_small_policy': 'keep', 'sv_woe_smoothing': 'none', 'sv_smoothing_alpha': 0.0, 'unseen_special_policy': 'neutral'}
        Parameters of the ``MonotoneWOEBinner`` engine: its constructor keys, and the fit-only keys ``chi2_binning``
        (default False), ``chi2_p``, ``chi2_init_size`` and ``n_jobs``, which are passed to ``fit``. Without a
        ``special_values`` key, the sentinel -999999 is declared only when it occurs in the fit sample. A dict you pass
        replaces the default.
    train_models : list of str, default ['lr', 'lgb', 'xgb', 'cat']
        Models to train, any of ``"lr"`` (``LRMaster``) and ``"lgb"``, ``"xgb"``, ``"cat"`` (``GradientBoostingModel``),
        case-insensitive; another name raises ``ValueError`` when training starts. INS is the training set and OOS the
        validation set.
    model_params : dict of str to dict, default {}
        Per-model parameter overrides ``{model_name: {parameter: value}}``, merged over the built-in defaults of that
        model (for example ``n_estimators=300`` and ``learning_rate=0.05`` for ``lgb``). For ``lr`` the key
        ``standardize`` (bool, default False) is passed to ``LRMaster`` and the other keys are its sklearn parameters.
    gbm_feature_source : str or dict of str to str, default "woe"
        Input features of the GBM models: ``"woe"`` (WOE-encoded) or ``"raw"``, for all three models, or a dict keyed by
        ``lgb``, ``xgb`` and ``cat`` (a missing key means ``"woe"``). LR always uses the WOE features. Other keys or
        values raise ``ValueError`` when the pipeline is created.
    lr_search_enabled : bool, default False
        Whether to run a hyper-parameter grid search for the LR model (``LRMaster.grid_search_params``) before training.
        It needs ``"lr"`` in ``train_models``; the table is returned as ``result.lr_search_results``.
    lr_search_param_grid : dict of str to list, default {'C': [0.01, 0.1, 1.0, 10.0]}
        Parameter grid of the LR search, searched as a Cartesian product.
    lr_search_params : dict, default {}
        Overrides of the LR search settings. The allowed keys are ``objective``, ``primary_set``, ``gap_ref_sets``,
        ``metric``, ``refit`` and ``verbose``; any other key raises ``ValueError`` (the search is a holdout search and
        does not accept ``cv``). The defaults are ``objective="oot_gap_penalized"``, ``primary_set="oos"`` and
        ``gap_ref_sets=["oot"]`` when OOT is among ``search_eval_splits``, otherwise
        ``objective=search_objective_when_no_oot`` with ``primary_set="oos"`` and no gap reference, and
        ``metric="auc"``.
    use_lr_search_params : bool, default True
        Whether the best parameters of the LR search are merged over ``model_params["lr"]`` for the final LR model. No
        effect without the search.
    lr_elimination_mode : str or None, default None
        Backward elimination of the final LR model: ``None`` keeps every feature, ``"pvalue"`` refits the LR without its
        feature of highest coefficient p-value until every p-value is at most ``pvalue_threshold`` (see
        ``lr_elimination_params``). Any other value raises ``ValueError``. The dropped features are recorded in
        ``feature_selection_summary["lr_elimination"]``; the model's final features are ``result.models["lr"][2]``, while
        ``result.selected_features``, ``selected_woe_features`` and ``model_feature_sets`` still show the list from
        before the elimination.
    lr_elimination_params : dict, default {}
        Settings of the p-value elimination: ``pvalue_threshold`` (default 0.05), ``min_features`` (default 1; the
        elimination stops when this many features remain) and ``max_iterations`` (default 20). ``tie_breaker`` may only
        be ``"pvalue"`` (or None), which is what happens anyway: equal p-values are resolved by column order. Any other
        ``tie_breaker`` value and any other key raise ``ValueError``.
    warm_start_enabled : bool, default False
        Whether to start the GBM models from a prior score (LightGBM ``init_score``, XGBoost base margin), in training
        and in evaluation, so that a model's probability combines the prior score with its increment. It needs
        ``warm_start_score_col`` and is supported for ``lgb`` and ``xgb`` only. By default (see
        ``warm_start_score_scope``) the prior is also seen by early stopping, the Optuna search (``AUC_*`` of its table)
        and the Owen explanations, so every stage judges the combined model.
    warm_start_score_col : str or None, default None
        Column of the input data that holds the prior score. It is required when ``warm_start_enabled`` is on
        (``ValueError``), must exist (``KeyError``) and must have no missing values in any evaluated frame. It is copied
        by position onto the WOE-transformed splits, with a length check (``ValueError``).
    warm_start_score_type : {"probability", "log_odds"}, default "probability"
        ``"probability"`` scores are clipped to [1e-6, 1 - 1e-6] and converted to log-odds, ``"log_odds"`` scores are used
        as the init score as they are. Any other value raises ``ValueError`` when warm start is enabled.
    warm_start_models : list of str, default ['lgb', 'xgb']
        GBM models that use the warm start, compared case-insensitively. ``"cat"`` is not supported (see
        ``warm_start_on_unsupported``). Names that are not in ``train_models`` appear in ``warm_start_summary`` with the
        status ``"not_in_train_models"``.
    warm_start_on_unsupported : {"skip", "raise"}, default "skip"
        What to do when ``"cat"`` is requested for warm start: ``"skip"`` trains it without the prior score and records
        ``"skipped_unsupported"`` in ``warm_start_summary``, ``"raise"`` raises ``NotImplementedError``. Any other value
        raises ``ValueError`` when warm start is enabled.
    warm_start_apply_to_optuna : bool, default False
        Whether to pass the prior score as ``init_score`` to the Optuna search of the warm-start models (``lgb`` and
        ``xgb``).
    warm_start_score_scope : {"full", "train"}, default "full"
        Where the prior score is seen. ``"full"`` (default) adds it to the training, to the validation set of the early
        stopping (so the models stop when the combined model stops improving), to the scoring of the Optuna candidates
        (with ``warm_start_apply_to_optuna``), to the final evaluation and to the Owen explanation, where the prior enters
        as a group of its own, ``warm_start_prior``, so that the Owen values add up to the scored probability.
        ``"train"`` (the behavior of 0.8.2 and earlier) adds it to the training and to the final
        evaluation only: early stopping, the ``AUC_*`` of the Optuna search table and the Owen explanations then see the
        increment alone; pass it to reproduce models trained before the change. Any other value raises ``ValueError``
        when warm start is enabled. SHAP values of the trees are unaffected by the prior (it is an
        additive offset in log-odds), so ``explain_models`` gives the same feature importance in both scopes.
    backward_enabled : bool, default True
        Whether to run backward variable elimination on the WOE features before the models are trained.
    backward_model : str, default "lgb"
        Proxy model of the backward elimination: ``"lgb"`` runs the LightGBM elimination and ``"xgb"`` the XGBoost one
        (case and surrounding spaces are ignored). Any other value raises ``ValueError`` when ``run`` starts, if
        ``backward_enabled`` is on.
    backward_params : dict, default {}
        ``{"init": {...}, "run": {...}}``: ``init`` overrides the arguments of ``BackwardVariableEliminator`` and ``run``
        those of its ``run`` call (defaults ``n_rounds=3``, ``stopping_metric="auc"``, ``num_boost_round=200``,
        ``early_stopping_rounds=20``, ``cum_importance_threshold=0.99``, ``min_vars=max(3, n_features // 2)`` and
        ``ret_perf=True``). A split named in ``init["test_data_dict"]`` must not be forbidden. Any failure of the
        backward stage is caught: ``backward_summary`` then holds a one-row frame with the columns ``step`` and ``error``
        and all WOE features are kept.
    use_backward_features : bool, default True
        ``True`` trains the models on the features that survive the backward elimination (WOE names; a raw GBM gets them
        mapped back to raw names), ``False`` on all screened features, so that the elimination only reports.
    candidate_mode : bool, default False
        ``True`` forbids any consumption of OOT in the candidate stage: ``"oot"`` is added to ``forbidden_splits`` and the
        OOT frame is removed from the working splits. Explicit settings that request OOT
        (``synthesize_missing_oot=True``, ``"oot"`` in ``evaluation_splits``, ``search_eval_splits`` or
        ``backward_report_splits``, or ``backward_validation_split="oot"``) raise ``ValueError`` when the pipeline is
        created.
    synthesize_missing_oot : bool or None, default False
        When the data has no real OOT rows: ``True`` copies the OOS rows in as a stand-in OOT (with a ``UserWarning``;
        its metrics are OOS metrics), ``False`` or ``None`` leaves OOT out, so the evaluation runs on INS and OOS only.
    evaluation_splits : list of str or None, default None
        Splits that the performance evaluation, the charts and the Excel report use. ``None`` means ``['ins', 'oos']``,
        so a real OOT must be listed explicitly. Names are case-insensitive and must be ``ins``, ``oos`` or ``oot``
        (``ValueError``); a listed split that does not exist in the run is silently skipped. ``extra_eval_datasets``
        are not limited by it.
    forbidden_splits : list of str, default []
        Splits that no stage may consume, such as ``['oot']``. They are removed from the working splits. A split list that
        names one raises ``ValueError`` when the pipeline is created; one named in the nested ``optuna_params`` or
        ``backward_params`` settings is rejected when that stage runs (in the backward stage the error is caught and
        shown in ``backward_summary``).
    search_eval_splits : list of str or None, default None
        Splits that the LR search and the Optuna search evaluate on. ``None`` means ``['oos']``. An explicitly listed
        split that is absent from the run raises ``ValueError``, whereas an absent split of the default is silently
        dropped.
    search_objective_when_no_oot : str, default "max_primary"
        Objective of the LR and Optuna searches when ``oot`` is not among the search eval splits (with OOT it is
        ``"oot_gap_penalized"``). ``"max_primary"`` maximizes the AUC on ``oos``; ``"oot_gap_penalized"`` needs a gap
        reference split and therefore raises ``ValueError`` here.
    backward_validation_split : str, default "oos"
        Split used as the validation data of the backward elimination: ``"ins"``, ``"oos"`` or ``"oot"``.
    backward_report_splits : list of str or None, default None
        Splits on which the backward elimination reports performance after each round. ``None`` means none (``[]``), so
        OOT is not read. An explicitly listed split that is absent from the run makes the backward stage fail (caught
        and reported in ``backward_summary``).
    optuna_models : list of str, default ['lgb', 'xgb', 'cat']
        Models to tune with Optuna; ``[]`` turns the search off. Only trained models that have a search space run (see
        ``optuna_params``). The search only reports: ``result.optuna_results`` holds its tables, and the models in
        ``result.models`` are not retrained with the tuned parameters.
    optuna_n_trials : int, default 5
        Number of Optuna trials per model.
    optuna_params : dict, default {}
        Optuna settings: ``search_spaces`` (``{model: {parameter: space}}``, which replaces the built-in spaces, so a
        model missing from it is skipped), ``common`` (overrides of the ``param_search`` arguments such as
        ``objective``, ``primary_set``, ``gap_ref_sets``, ``metric``, ``refit`` and ``eval_sets``; named splits must
        not be forbidden) and ``fit_kwargs`` (extra ``fit`` arguments). A search that fails is caught and its table holds
        the single column ``error``.
    explain_models : list of str, default ['lr', 'lgb', 'cat']
        Trained models explained with SHAP: ``feature_importance`` and, with charts, ``shap_summary.png``. ``[]``
        together with ``owen_enabled=False`` skips the explanations altogether.
    explain_params : dict, default {'sample_n': 500, 'background_n': 200}
        Explanation settings: ``sample_n`` (OOS rows sampled for the explanations, capped at the OOS size),
        ``background_n`` (INS rows used as background, capped at the INS size) and the Owen options ``owen_threshold``
        (0.35), ``owen_method`` (``"complete"``), ``owen_corr_method`` (``"spearman"``), ``owen_min_group_size`` (1),
        ``owen_intra_dist`` (0.01), ``owen_inter_dist`` (0.99) and ``owen_model_output`` (``"probability"``). Omitted
        keys take the values given here.
    owen_enabled : bool, default True
        Whether to compute Owen values (Shapley values over groups of related features) for every trained model except
        ``xgb``. While it is True, the explanations run for all trained models, also those that are not in
        ``explain_models`` (Owen values only).
    business_prior_groups : dict of str to list of str or None, default None
        Business groups ``{group: [feature, ...]}`` for the Owen coalition structure, named like the model's input
        features (for example ``income_woe``). Features that the model does not use are dropped from the groups, and the
        remaining features are clustered automatically. ``None`` uses a built-in example grouping that only matches
        features with those exact names.
    perf_pct_bins : int, default 10
        Number of percentile bins of the performance evaluation.
    perf_min_bin_prop : float, default 0.03
        Minimum share of a bin in the Gains tables of the performance evaluation (``IV``, ``LIFT``, ``KS_IN_GAINS``,
        ``N_BINS``, ... of ``perf_results``), weighted or not: they use ``perf_pct_bins`` bins capped at
        ``1 / perf_min_bin_prop``, so each bin holds about that share or more (ties in the scores can move a few rows).
        The Top/Btm percentile bands keep ``perf_pct_bins``.
    eval_target_cols : list of str or None, default None
        Extra label columns evaluated against the same model scores in addition to ``target_col`` (duplicates removed;
        the results are stacked with a ``tgt_name`` column). They must exist in the input data and in every
        ``extra_eval_datasets`` frame, are not used for training, and are not excluded from inferred ``feature_cols``.
    all_missing_score_value : float or None, default None
        Score given in the evaluation to rows whose raw model features are all missing (for example -1), the rule of the
        scoring API; ``None`` applies no override. It is stored in the saved model metadata. The raw features must be
        present in every evaluated frame (``KeyError`` otherwise).
    special_score_values : list of float or None, default None
        Sentinel scores (for example ``[-1]``) that get their own evaluation bin and are left out of the quantile edges
        and the ranking metrics (``AUC``, ``KS``, the Top/Btm rates and lifts). ``N`` and ``avgTrue`` of the summary count
        every row, weighted or not, and ``N_SPECIAL`` / ``N_SPECIAL_RAW`` report the sentinel part.
    gains_ascending : bool or None, default True
        Score direction of the evaluation summary, the gains tables and the figures: ``True`` ascending (bin 1 holds the
        lowest scores, the lowest risk), ``False`` descending, ``None`` keeps the historical direction of each code path.
    eval_weight_col : str or None, default "inherit"
        Weight column of the evaluation, of the validation weights of the backward elimination and of the LR and Optuna
        searches. ``"inherit"`` reuses ``weight_col``, ``None`` evaluates unweighted even when the training is weighted,
        and any other string names a separate column that must exist in the input data (``KeyError``).
    screening_artifact : FeatureScreeningArtifact or None, default None
        Output of ``FeatureValidationPipeline``. When given, the pipeline does not screen itself: it uses the artifact's
        selected features (all ``feature_cols`` if it selected none) after ``validate_for_cm`` has checked
        ``target_col`` and ``weight_col``, and reuses its fitted WOE engine when ``reuse_screening_woe`` is True. It
        forces ``feature_selection_mode`` to ``"from_artifact"``.
    feature_validation_result : FeatureValidationPipelineResult or None, default None
        Convenience field: the result of ``FeatureValidationPipeline``, converted internally to a
        ``FeatureScreeningArtifact`` with this config's ``target_col`` and ``weight_col``. Ignored when
        ``screening_artifact`` is given.
    feature_selection_mode : {"run", "from_artifact", "skip"}, default "run"
        ``"run"`` screens with ``feature_selection``, ``"from_artifact"`` takes the selection from the artifact
        (``ValueError`` without one) and ``"skip"`` keeps all ``feature_cols``. Whenever an artifact is available the mode
        becomes ``"from_artifact"``, and any other value behaves like ``"run"``.
    reuse_screening_woe : bool, default True
        With an artifact, reuse the WOE engine it carries for the target instead of fitting again, provided it covers
        at least one selected feature (features it did not fit are left out of every model, with a ``RuntimeWarning``);
        otherwise the engine is fitted as usual, with a ``RuntimeWarning`` when an artifact is given. A reused engine
        keeps its own kind, bins and parameters: ``woe_engine``, ``woe_params``, ``monotone_woe_params`` and
        ``woe_fit_query`` of this config do not apply to it. The artifact also records the INS/OOS split settings of the
        validation run; a ``RuntimeWarning`` reports a split that differs from this run's, because the validation fitted the
        bins and chose the features on rows that this run would then score as OOS. No effect without an artifact.
    screen_iv_convention : {"guarded", "raw"}, default "guarded"
        How the screening IV on the bins of a WOE engine (a ``*_use_woe_bins`` flag of ``feature_selection``, or a
        weighted run with an engine) treats a class-pure bin, whose WOE comes from the ``1e-6`` of the formula alone:
        ``"guarded"`` leaves it out of the IV sum, ``"raw"`` counts every bin (0.9.1). See
        ``FeatureScreenConfig.screen_iv_convention``. ``feature_selection["screen_iv_convention"]`` is accepted as
        well; a value there that differs from this field raises ``ValueError`` when ``run`` starts, as does any other
        value. It does not change the WOE of the bins or the scoring.
    """

    output_dir: str = "output"
    clean_output_dir: bool = False
    target_col: str = "badflag"
    feature_cols: list[str] | None = None
    split_col: str | None = None
    sample_col: str = "sample_ind"
    oot_col: str | None = "oot_flag"
    weight_col: str | None = None
    random_state: int = 42
    write_outputs: bool = True
    write_excel: bool = True
    plot_outputs: bool = True
    save_models: bool = False
    model_output_dir: str | None = None
    model_include_metadata: bool = True
    save_woe_artifacts: bool = True

    split_config: dict[str, Any] = field(default_factory=lambda: {"test_size": 0.3, "stratify": True})
    feature_selection: dict[str, Any] = field(
        default_factory=lambda: {
            "psi_enabled": True,
            "psi_threshold": 0.2,
            "psi_compare_splits": ["oos"],
            "iv_enabled": True,
            "iv_threshold": 0.02,
            "iv_upper_threshold": 2.0,
            "corr_enabled": True,
            "corr_threshold": 0.75,
            "corr_max_iterations": 10,
        }
    )
    woe_engine: str = "equal_freq"
    woe_fit_query: str | None = None
    extra_eval_datasets: dict[str, pd.DataFrame] | None = None
    # sv_* govern special-value (SV) WOE bins only; the defaults below reproduce
    # pre-0.8.0 behavior bit-for-bit. They reach WOE_Master via fit(**woe_params)
    # and MonotoneWOEBinner via __init__(**monotone_woe_params).
    woe_params: dict[str, Any] = field(
        default_factory=lambda: {
            "nbins": 10,
            "equal_freq": True,
            "min_bin_prop": 0.05,
            "sv_min_bin_size": 0.0,
            "sv_small_policy": "keep",
            "sv_woe_smoothing": "none",
            "sv_smoothing_alpha": 0.0,
        }
    )
    # unseen_special_policy (monotone only): declared special values absent from
    # the fit sample — "neutral" placeholder bins (default since 0.9.0) or "normal_bin" (legacy).
    # Without an explicit "special_values" key the monotone self-fit declares the
    # legacy -999999 sentinel only when the WOE fit sample contains it.
    monotone_woe_params: dict[str, Any] = field(
        default_factory=lambda: {
            "n_init_bins": 20,
            "min_bin_size": 0.03,
            "min_n_bins": 2,
            "sv_min_bin_size": 0.0,
            "sv_small_policy": "keep",
            "sv_woe_smoothing": "none",
            "sv_smoothing_alpha": 0.0,
            "unseen_special_policy": "neutral",
        }
    )

    train_models: list[str] = field(default_factory=lambda: ["lr", "lgb", "xgb", "cat"])
    model_params: dict[str, dict[str, Any]] = field(default_factory=dict)
    gbm_feature_source: str | dict[str, str] = "woe"
    lr_search_enabled: bool = False
    lr_search_param_grid: dict[str, list[Any]] = field(
        default_factory=lambda: {"C": [0.01, 0.1, 1.0, 10.0]}
    )
    lr_search_params: dict[str, Any] = field(default_factory=dict)
    use_lr_search_params: bool = True
    # G07: post-fit LR backward elimination. None (legacy) keeps every
    # feature; "pvalue" iteratively refits without the worst-p feature until
    # all coefficient p-values clear pvalue_threshold. Params:
    # pvalue_threshold=0.05, min_features=1, max_iterations=20,
    # tie_breaker="pvalue" (float ties resolve by column order).
    lr_elimination_mode: str | None = None
    lr_elimination_params: dict[str, Any] = field(default_factory=dict)

    warm_start_enabled: bool = False
    warm_start_score_col: str | None = None
    warm_start_score_type: Literal["probability", "log_odds"] = "probability"
    warm_start_models: list[str] = field(default_factory=lambda: ["lgb", "xgb"])
    warm_start_on_unsupported: Literal["skip", "raise"] = "skip"
    warm_start_apply_to_optuna: bool = False
    warm_start_score_scope: Literal["train", "full"] = "full"

    backward_enabled: bool = True
    backward_model: str = "lgb"
    backward_params: dict[str, Any] = field(default_factory=dict)
    use_backward_features: bool = True

    # --- OOT governance (G10/G11/G12) -----------------------------------
    # None means "not set explicitly" for split lists: the effective value is
    # resolved at run() start. Explicit values conflicting with
    # candidate_mode=True raise at validation time.
    # By default, missing OOT is not synthesized; model evaluation runs on
    # INS/OOS, hyperparameter search evaluates OOS, and backward reports omit
    # OOT. Set synthesize_missing_oot=True to retain the legacy OOS-copy OOT
    # stand-in.
    candidate_mode: bool = False
    synthesize_missing_oot: bool | None = False
    evaluation_splits: list[str] | None = None
    forbidden_splits: list[str] = field(default_factory=list)
    search_eval_splits: list[str] | None = None
    search_objective_when_no_oot: str = "max_primary"
    backward_validation_split: str = "oos"
    backward_report_splits: list[str] | None = None

    optuna_models: list[str] = field(default_factory=lambda: ["lgb", "xgb", "cat"])
    optuna_n_trials: int = 5
    optuna_params: dict[str, Any] = field(default_factory=dict)

    explain_models: list[str] = field(default_factory=lambda: ["lr", "lgb", "cat"])
    explain_params: dict[str, Any] = field(default_factory=lambda: {"sample_n": 500, "background_n": 200})
    owen_enabled: bool = True
    business_prior_groups: dict[str, list[str]] | None = None

    perf_pct_bins: int = 10
    perf_min_bin_prop: float = 0.03

    # --- Evaluation governance (G13/G14/G15/G16) -------------------------
    # eval_target_cols: extra labels evaluated against the SAME frozen model
    # scores; effective label set = [target_col] + eval_target_cols (deduped).
    # all_missing_score_value: rows whose raw model features are ALL missing
    # get this score (e.g. -1); same test as Core.scoring's
    # all_missing_spec_value, persisted in the model artifact metadata.
    # special_score_values: sentinel scores (e.g. [-1]) get their own
    # evaluation bin, excluded from quantile edges and ranking metrics.
    # gains_ascending=True uses score ascending (bin 1 = low risk) across
    # summary, gains tables, and figures. None retains each legacy path's
    # historical direction.
    # eval_weight_col: "inherit" (default) reuses weight_col for evaluation
    # and search/backward eval weights; None evaluates unweighted even when
    # training is weighted; any other string names the evaluation weight
    # column explicitly.
    eval_target_cols: list[str] | None = None
    all_missing_score_value: float | None = None
    special_score_values: list[float] | None = None
    gains_ascending: bool | None = True
    eval_weight_col: str | None = "inherit"

    screening_artifact: Any | None = None
    feature_validation_result: Any | None = None
    feature_selection_mode: Literal["run", "from_artifact", "skip"] = "run"
    reuse_screening_woe: bool = True
    # 0.9.2: class-pure bins of the screening IV on WOE bins ("guarded" leaves them out of the sum; "raw" is 0.9.1)
    screen_iv_convention: Literal["guarded", "raw"] = "guarded"

CreditModelPipelineResult dataclass

Result returned by :meth:CreditModelPipeline.run.

Parameters:

Name	Type	Description	Default
splits	dict of str to pandas.DataFrame	The raw (not WOE-transformed) working splits: "ins" and "oos", plus "oot" when the data has a real OOT (or one was synthesized with synthesize_missing_oot=True) and it is not forbidden.	required
feature_selection_summary	dict	Outcome of the feature selection: initial_features and final_features; the tables psi, iv and corr_dropped (when they exist), corr_features and screen_summary; dropped_detail (the leak-gate drops, the gate drops and the fallback_kept rows); error when the screening failed (all features were then kept, except the leak-gate drops it had found, listed in hard_dropped); skipped or from_artifact and artifact_source for those modes; and lr_elimination (the dropped-feature trace) when the LR p-value elimination ran.	required
woe_artifacts	dict	The WOE step: engine (a WOE_Master or an adapter of the monotone binner), engine_name, features, woe_features, woe_suffix, splits (the WOE-transformed frames by split name), extra_eval (the transformed extra_eval_datasets), woe_table (the mapping the transform applies, one row per bin, with the counts of the rows the engine was fitted on) and, when the screening engine was reused, reused_from_screening.	required
models	dict of str to tuple	{model_name: (wrapper, raw_model, feature_cols)} for the trained models. feature_cols is the final feature list of that model, after the LR p-value elimination for lr.	required
selected_features	list of str	The WOE features chosen before the models were trained (after the backward elimination when use_backward_features is on); the same as selected_woe_features. The LR p-value elimination is not reflected here.	required
backward_summary	DataFrame or None	Summary of the backward elimination; a one-row frame with the columns step and error when that stage failed. None when backward_enabled is off.	None
optuna_results	dict of str to pandas.DataFrame	Optuna search table per model; a single-column error frame for a search that failed. These searches do not change the trained models.	{}
perf_results	dict of str to pandas.DataFrame	Performance summary table per model (PerformanceEvaluator.evaluate), with the metrics of every evaluated dataset: the splits listed in evaluation_splits and the extra_eval_datasets.	{}
explain_outputs	dict	Explanation outputs per model: feature_importance (DataFrame), shap_summary (chart path), plot_error, owen (feature_importance and group_importance frames, or error) or error; a top-level import_error entry when the explainer could not be imported.	{}
explain_paths	dict of str to dict of str to str	Index of the files written under explain/ per model (importance tables, SHAP chart, error files), with the manifest under the key _manifest; empty when write_outputs is off.	{}
report_path	str or None	Path of the Excel report. None when write_excel is off.	None
lr_search_results	DataFrame or None	Table of the LR grid search. None when the search did not run.	None
warm_start_summary	DataFrame or None	One row per requested warm-start model (model, status, score_col, score_type, missing_rate_ins, apply_to_optuna, n_features). None when warm_start_enabled is off.	None
model_feature_sources	dict of str to str	{model_name: "woe" or "raw"}: the kind of features each model was given.	{}
model_feature_sets	dict of str to list of str	The features each model was given for training, evaluation and explanation; the LR p-value elimination is not reflected here (see models).	{}
selected_raw_features	list of str	The final feature set under the raw column names (what a raw GBM uses).	[]
selected_woe_features	list of str	The final feature set under the WOE column names (what LR and WOE-based GBMs use).	[]
model_paths	dict of str to str	{model_name: path} of the saved model files; empty unless save_models is on.	{}
artifact_paths	dict of str to str	Paths of the saved woe_table and woe_engine; empty unless save_models and save_woe_artifacts are on.	{}
split_governance	dict	The effective OOT-governance settings of the run: candidate_mode, synthesize_missing_oot, evaluation_splits, forbidden_splits, search_eval_splits, search_eval_splits_explicit, backward_validation_split, backward_report_splits, backward_report_splits_explicit, oot_synthesized and oot_withheld.	{}

Source code in Modeling_Tool/Pipeline/credit_model.py

@dataclass
class CreditModelPipelineResult:
    """Result returned by :meth:`CreditModelPipeline.run`.

    Parameters
    ----------
    splits : dict of str to pandas.DataFrame
        The raw (not WOE-transformed) working splits: ``"ins"`` and ``"oos"``, plus ``"oot"`` when the data has a real OOT
        (or one was synthesized with ``synthesize_missing_oot=True``) and it is not forbidden.
    feature_selection_summary : dict
        Outcome of the feature selection: ``initial_features`` and ``final_features``; the tables ``psi``, ``iv`` and
        ``corr_dropped`` (when they exist), ``corr_features`` and ``screen_summary``; ``dropped_detail`` (the leak-gate
        drops, the gate drops and the ``fallback_kept`` rows); ``error`` when the screening failed (all features were then
        kept, except the leak-gate drops it had found, listed in ``hard_dropped``); ``skipped`` or ``from_artifact`` and
        ``artifact_source`` for those modes; and
        ``lr_elimination`` (the dropped-feature trace) when the LR p-value elimination ran.
    woe_artifacts : dict
        The WOE step: ``engine`` (a ``WOE_Master`` or an adapter of the monotone binner), ``engine_name``, ``features``,
        ``woe_features``, ``woe_suffix``, ``splits`` (the WOE-transformed frames by split name), ``extra_eval`` (the
        transformed ``extra_eval_datasets``), ``woe_table`` (the mapping the transform applies, one row per bin, with the
        counts of the rows the engine was fitted on) and, when the screening engine was reused, ``reused_from_screening``.
    models : dict of str to tuple
        ``{model_name: (wrapper, raw_model, feature_cols)}`` for the trained models. ``feature_cols`` is the final feature
        list of that model, after the LR p-value elimination for ``lr``.
    selected_features : list of str
        The WOE features chosen before the models were trained (after the backward elimination when
        ``use_backward_features`` is on); the same as ``selected_woe_features``. The LR p-value elimination is not
        reflected here.
    backward_summary : pandas.DataFrame or None, default None
        Summary of the backward elimination; a one-row frame with the columns ``step`` and ``error`` when that stage
        failed. ``None`` when ``backward_enabled`` is off.
    optuna_results : dict of str to pandas.DataFrame, default {}
        Optuna search table per model; a single-column ``error`` frame for a search that failed. These searches do not
        change the trained models.
    perf_results : dict of str to pandas.DataFrame, default {}
        Performance summary table per model (``PerformanceEvaluator.evaluate``), with the metrics of every evaluated
        dataset: the splits listed in ``evaluation_splits`` and the ``extra_eval_datasets``.
    explain_outputs : dict, default {}
        Explanation outputs per model: ``feature_importance`` (DataFrame), ``shap_summary`` (chart path), ``plot_error``,
        ``owen`` (``feature_importance`` and ``group_importance`` frames, or ``error``) or ``error``; a top-level
        ``import_error`` entry when the explainer could not be imported.
    explain_paths : dict of str to dict of str to str, default {}
        Index of the files written under ``explain/`` per model (importance tables, SHAP chart, error files), with the
        manifest under the key ``_manifest``; empty when ``write_outputs`` is off.
    report_path : str or None, default None
        Path of the Excel report. ``None`` when ``write_excel`` is off.
    lr_search_results : pandas.DataFrame or None, default None
        Table of the LR grid search. ``None`` when the search did not run.
    warm_start_summary : pandas.DataFrame or None, default None
        One row per requested warm-start model (``model``, ``status``, ``score_col``, ``score_type``,
        ``missing_rate_ins``, ``apply_to_optuna``, ``n_features``). ``None`` when ``warm_start_enabled`` is off.
    model_feature_sources : dict of str to str, default {}
        ``{model_name: "woe" or "raw"}``: the kind of features each model was given.
    model_feature_sets : dict of str to list of str, default {}
        The features each model was given for training, evaluation and explanation; the LR p-value elimination is not
        reflected here (see ``models``).
    selected_raw_features : list of str, default []
        The final feature set under the raw column names (what a raw GBM uses).
    selected_woe_features : list of str, default []
        The final feature set under the WOE column names (what LR and WOE-based GBMs use).
    model_paths : dict of str to str, default {}
        ``{model_name: path}`` of the saved model files; empty unless ``save_models`` is on.
    artifact_paths : dict of str to str, default {}
        Paths of the saved ``woe_table`` and ``woe_engine``; empty unless ``save_models`` and ``save_woe_artifacts`` are
        on.
    split_governance : dict, default {}
        The effective OOT-governance settings of the run: ``candidate_mode``, ``synthesize_missing_oot``,
        ``evaluation_splits``, ``forbidden_splits``, ``search_eval_splits``, ``search_eval_splits_explicit``,
        ``backward_validation_split``, ``backward_report_splits``, ``backward_report_splits_explicit``,
        ``oot_synthesized`` and ``oot_withheld``.
    """

    splits: dict[str, pd.DataFrame]
    feature_selection_summary: dict[str, Any]
    woe_artifacts: dict[str, Any]
    models: dict[str, tuple[Any, Any, list[str]]]
    selected_features: list[str]
    backward_summary: pd.DataFrame | None = None
    optuna_results: dict[str, pd.DataFrame] = field(default_factory=dict)
    perf_results: dict[str, pd.DataFrame] = field(default_factory=dict)
    explain_outputs: dict[str, Any] = field(default_factory=dict)
    explain_paths: dict[str, dict[str, str]] = field(default_factory=dict)
    report_path: str | None = None
    lr_search_results: pd.DataFrame | None = None
    warm_start_summary: pd.DataFrame | None = None
    model_feature_sources: dict[str, str] = field(default_factory=dict)
    model_feature_sets: dict[str, list[str]] = field(default_factory=dict)
    selected_raw_features: list[str] = field(default_factory=list)
    selected_woe_features: list[str] = field(default_factory=list)
    model_paths: dict[str, str] = field(default_factory=dict)
    artifact_paths: dict[str, str] = field(default_factory=dict)
    split_governance: dict[str, Any] = field(default_factory=dict)

CreditModelPipeline

Reusable credit modeling workflow: split, feature selection, WOE, models, evaluation.

Parameters:

Name	Type	Description	Default
config	CreditModelPipelineConfig or None	Pipeline settings. None uses CreditModelPipelineConfig(). The object is stored as config and is not modified. Creating the pipeline raises ValueError for an invalid gbm_feature_source and for contradictory or unknown OOT-governance settings (candidate_mode, forbidden_splits and the split lists).	None

Attributes:

Name	Type	Description
config	CreditModelPipelineConfig	The settings in use.
predict_positive_nan_stats	dict of str to dict of str to int	Filled by run during the evaluation: for each model, the number of NaN or infinite predictions in each evaluated dataset.

Notes

Several stages catch their own failures instead of stopping the run, and record the error in the result: the feature selection (feature_selection_summary["error"], every feature is kept except the leak-gate drops found before the failure), the backward elimination (backward_summary with the columns step and error), each Optuna search (a table with the column error) and the explanations (an error entry per model).

Source code in Modeling_Tool/Pipeline/credit_model.py

class CreditModelPipeline:
    """Reusable credit modeling workflow: split, feature selection, WOE, models, evaluation.

    Parameters
    ----------
    config : CreditModelPipelineConfig or None, default None
        Pipeline settings. ``None`` uses ``CreditModelPipelineConfig()``. The object is stored as ``config`` and is not
        modified. Creating the pipeline raises ``ValueError`` for an invalid ``gbm_feature_source`` and for contradictory
        or unknown OOT-governance settings (``candidate_mode``, ``forbidden_splits`` and the split lists).

    Attributes
    ----------
    config : CreditModelPipelineConfig
        The settings in use.
    predict_positive_nan_stats : dict of str to dict of str to int
        Filled by ``run`` during the evaluation: for each model, the number of NaN or infinite predictions in each
        evaluated dataset.

    Notes
    -----
    Several stages catch their own failures instead of stopping the run, and record the error in the result: the
    feature selection (``feature_selection_summary["error"]``, every feature is kept except the leak-gate drops found
    before the failure), the backward elimination
    (``backward_summary`` with the columns ``step`` and ``error``), each Optuna search (a table with the column
    ``error``) and the explanations (an ``error`` entry per model).
    """

    _DEFAULT_MODEL_PARAMS = {
        "lgb": {
            "n_estimators": 300,
            "learning_rate": 0.05,
            "num_leaves": 31,
            "max_depth": -1,
            "min_child_samples": 50,
            "subsample": 0.8,
            "colsample_bytree": 0.8,
            "reg_alpha": 0.1,
            "reg_lambda": 1.0,
            "n_jobs": -1,
            "verbose": -1,
            "early_stopping_rounds": 50,
            "eval_metric": "auc",
        },
        "xgb": {
            "n_estimators": 300,
            "learning_rate": 0.05,
            "max_depth": 4,
            "min_child_weight": 10,
            "subsample": 0.8,
            "colsample_bytree": 0.8,
            "reg_alpha": 0.1,
            "reg_lambda": 1.0,
            "n_jobs": -1,
            "eval_metric": "auc",
        },
        "cat": {
            "iterations": 300,
            "learning_rate": 0.05,
            "depth": 4,
            "l2_leaf_reg": 3,
            "verbose": 0,
            "eval_metric": "AUC",
        },
        "lr": {},
    }

    def __init__(self, config: CreditModelPipelineConfig | None = None):
        self.config = config or CreditModelPipelineConfig()
        self.predict_positive_nan_stats: dict[str, dict[str, int]] = {}
        self._validate_gbm_feature_source_config()
        # Resolved again at run() start; kept here so helpers (_split_data,
        # _run_lr_search, ...) work when exercised directly in tests.
        self._governance = self._resolve_split_governance()

    def run(self, data: pd.DataFrame) -> CreditModelPipelineResult:
        """Run the whole workflow on one dataset and return everything it produced.

        The steps are the INS/OOS/OOT split, the feature screening, the WOE fit and transform, the backward
        elimination, the LR search, the model training, the Optuna search, the evaluation, the explanations, the model
        saving, the CSV output and the Excel report; the settings of ``CreditModelPipelineConfig`` switch them on or off.

        Parameters
        ----------
        data : pandas.DataFrame
            Modeling data with ``target_col``, the feature columns (``feature_cols``, or every numeric column when it is
            ``None``) and, when configured, the split, OOT flag, weight, warm-start score and evaluation columns. It is
            not modified.

        Returns
        -------
        CreditModelPipelineResult
            The splits, the feature selection summary, the WOE artifacts, the trained models, the selected features and
            the optional search, performance, explanation and path outputs.

        Raises
        ------
        KeyError
            If a required column is missing from ``data``: the target, a feature, the weight or evaluation-weight column,
            the split column, the warm-start score, an evaluation target, or a column used by ``woe_fit_query``.
        ValueError
            If the data or the settings are invalid: weights, sample labels, ``woe_fit_query``, ``lr_elimination_mode``,
            the keys of ``lr_search_params`` or ``lr_elimination_params``, unsupported model names, an unusable search
            or evaluation split, or a request for a forbidden split.
        TypeError
            If ``oot_col`` is used for the split and is not numeric.
        NotImplementedError
            If ``"cat"`` is requested for warm start and ``warm_start_on_unsupported`` is ``"raise"``.

        Notes
        -----
        The failures that the feature selection, the backward elimination, each Optuna search and the explanations catch
        are not raised; they are recorded in the result (see the class notes).
        """
        cfg = self.config
        manifest = None
        if not getattr(self, "_nested_run", False) and (cfg.write_outputs or cfg.write_excel or cfg.save_models):
            manifest = OutputManifest.begin(cfg.output_dir, "credit_model", clean=cfg.clean_output_dir)
        try:
            result = self._run(data)
        except BaseException:
            if manifest is not None:
                try:
                    manifest.finish(clean=False)  # list what the failed run wrote; remove nothing
                except Exception:
                    pass
            raise
        if manifest is not None:
            removed = manifest.finish()
            if removed:
                _logger.info("CreditModelPipeline: clean_output_dir removed %d stale file(s): %s", len(removed), removed)
        return result

    def _run(self, data: pd.DataFrame) -> CreditModelPipelineResult:
        cfg = self.config
        feature_cols = self._resolve_feature_cols(data)
        self._validate_input(data, feature_cols)
        self._governance = self._resolve_split_governance()

        output_dir = Path(cfg.output_dir)
        if cfg.write_outputs or cfg.write_excel:
            # the chart folders (figs/woe, figs/mono_woe, figs/perf) are created when a chart is written; creating them
            # here left empty folders behind whenever plots were off or the other WOE engine was used
            dirs = [output_dir]
            if cfg.write_outputs and self._will_run_explainability():
                dirs.append(output_dir / "explain")
            make_dirs(*dirs)
        if cfg.save_models:
            make_dirs(self._model_output_dir(), output_dir / "artifacts")

        splits = self._apply_split_governance(self._split_data(data))
        self._raw_splits = splits
        self._data_columns = list(data.columns)
        check_woe_fit_query_rows(splits["ins"], cfg.woe_fit_query, context="INS")
        if cfg.woe_engine.lower() != "monotone":
            warn_precision_collapse(
                splits["ins"], feature_cols, int(cfg.woe_params.get("precision", 5)), "CreditModelPipeline"
            )
        elif cfg.feature_selection_mode == "run" and cfg.feature_selection:
            warn_precision_collapse(splits["ins"], feature_cols, 5, "CreditModelPipeline")
        fs_summary, final_features, screening_artifact = self._resolve_feature_selection(
            splits,
            feature_cols,
        )
        prefit_woe = screening_artifact.woe_artifacts if screening_artifact and cfg.reuse_screening_woe else None
        absent_features = [col for col in final_features if col not in data.columns]
        if absent_features:
            raise KeyError(
                f"The selected features {absent_features[:10]} (from the screening artifact) are not columns of the "
                "modeling data."
            )
        woe_artifacts = self._fit_woe(splits, final_features, prefit_woe_artifacts=prefit_woe)
        if screening_artifact is not None and cfg.reuse_screening_woe and not woe_artifacts.get("reused_from_screening"):
            warnings.warn(
                "CreditModelPipeline: reuse_screening_woe=True, but the screening artifact holds no usable WOE engine for "
                f"target {cfg.target_col!r}, so the WOE is fitted again with woe_engine={cfg.woe_engine!r} and its bins "
                "differ from the ones the screening used. Fit the engine in the validation run (woe_enabled=True) to hand "
                "its bins over.",
                RuntimeWarning,
                stacklevel=2,
            )
        fitted_raw = set(woe_artifacts.get("features") or final_features)
        unfitted = [col for col in final_features if col not in fitted_raw]
        if unfitted:
            # the reused engine did not bin these features (for example the missing-rate gate left them out of the fit);
            # the raw-feature models used to train on them while the WOE models did not, and the summary claimed all
            warnings.warn(
                f"CreditModelPipeline: the WOE engine has no bins for {len(unfitted)} of the {len(final_features)} "
                f"selected features ({unfitted[:10]}), so they are left out of every model.",
                RuntimeWarning,
                stacklevel=2,
            )
            final_features = [col for col in final_features if col in fitted_raw]
            fs_summary["final_features"] = list(final_features)
            fs_summary["dropped_without_woe"] = unfitted
        woe_features = woe_artifacts["woe_features"]
        woe_splits = woe_artifacts["splits"]
        woe_suffix = woe_artifacts.get("woe_suffix", cfg.woe_params.get("woe_suffix", "_woe"))

        backward_summary = None
        selected_woe_features = list(woe_features)
        if cfg.backward_enabled:
            backward_summary, selected_woe_features = self._run_backward(woe_splits, woe_features)
            if not selected_woe_features:
                selected_woe_features = list(woe_features)

        feature_set = selected_woe_features if cfg.use_backward_features else woe_features
        selected_woe_features = list(feature_set)
        selected_raw_features = (
            self._woe_to_raw_features(selected_woe_features, woe_suffix)
            if cfg.use_backward_features
            else list(final_features)
        )
        model_inputs = self._build_model_inputs(splits, woe_splits, selected_raw_features, selected_woe_features)
        model_feature_sources, model_feature_sets = self._summarize_model_inputs(model_inputs)
        model_feature_source_summary = self._model_feature_source_frame(model_feature_sources, model_feature_sets)

        lr_search_results = self._run_lr_search(woe_splits, selected_woe_features)
        warm_start_summary = self._build_warm_start_summary(model_inputs)
        models = self._train_models(model_inputs)
        lr_elimination_trace = getattr(self, "_lr_elimination_trace", None)
        if lr_elimination_trace is not None:
            fs_summary["lr_elimination"] = lr_elimination_trace

        optuna_results = self._run_optuna(model_inputs)
        perf_results = self._evaluate_models(
            model_inputs,
            models,
            woe_artifacts.get("extra_eval"),
        )
        explain_outputs = self._run_explainability(model_inputs, models)
        explain_paths: dict[str, dict[str, str]] = {}
        if cfg.write_outputs and explain_outputs:
            explain_paths = {
                model: {key: str(Path(path).resolve()) for key, path in files.items()}
                for model, files in persist_explain_outputs(explain_outputs, output_dir / "explain").items()
            }
        model_paths: dict[str, str] = {}
        artifact_paths: dict[str, str] = {}
        model_paths_frame = None
        if cfg.save_models:
            model_paths, artifact_paths = self._save_models_and_artifacts(
                models=models,
                woe_artifacts=woe_artifacts,
                perf_results=perf_results,
                model_feature_sources=model_feature_sources,
                model_feature_sets=model_feature_sets,
            )
            model_paths_frame = self._paths_to_frame(model_paths, artifact_paths)
            safe_to_csv(model_paths_frame, output_dir / "model_paths.csv", index=False)

        if cfg.write_outputs:
            self._write_outputs(
                output_dir,
                fs_summary,
                woe_artifacts,
                backward_summary,
                optuna_results,
                perf_results,
                lr_search_results,
                warm_start_summary,
                model_feature_source_summary,
                model_paths_frame,
            )

        report_path = None
        if cfg.write_excel:
            report_path = str((output_dir / "SMF_Model_Report.xlsx").resolve())
            sheets = {
                "Feature_Selection": self._summary_to_frame(fs_summary),
                **self._summary_tables(fs_summary),
                "WOE_Table": woe_artifacts.get("woe_table"),
                "Backward": backward_summary,
                "LR_Param_Search": lr_search_results,
                "Warm_Start": warm_start_summary,
                "Model_Feature_Source": self._split_long_cells(model_feature_source_summary, "features"),
                "Model_Paths": model_paths_frame,
            }
            for name, perf in perf_results.items():
                sheets[f"Perf_{name.upper()}"] = perf
            sheets.update(self._explain_excel_sheets(explain_outputs))
            write_basic_excel(report_path, sheets, title="SuperModelingFactory Credit Model Pipeline Report")

        return CreditModelPipelineResult(
            splits=splits,
            feature_selection_summary=fs_summary,
            woe_artifacts=woe_artifacts,
            models=models,
            selected_features=list(selected_woe_features),
            backward_summary=backward_summary,
            optuna_results=optuna_results,
            perf_results=perf_results,
            explain_outputs=explain_outputs,
            explain_paths=explain_paths,
            report_path=report_path,
            lr_search_results=lr_search_results,
            warm_start_summary=warm_start_summary,
            model_feature_sources=model_feature_sources,
            model_feature_sets=model_feature_sets,
            selected_raw_features=list(selected_raw_features),
            selected_woe_features=list(selected_woe_features),
            model_paths=model_paths,
            artifact_paths=artifact_paths,
            split_governance=dict(self._governance),
        )

    def _resolve_feature_cols(self, data: pd.DataFrame) -> list[str]:
        cfg = self.config
        if cfg.feature_cols:
            return list(cfg.feature_cols)
        excluded = {cfg.target_col, cfg.sample_col}
        if cfg.split_col:
            excluded.add(cfg.split_col)
        if cfg.oot_col:
            excluded.add(cfg.oot_col)
        if cfg.weight_col:
            excluded.add(cfg.weight_col)
        numeric_cols = data.select_dtypes(include=[np.number]).columns
        return [col for col in numeric_cols if col not in excluded]

    def _model_params(self, name: str) -> dict[str, Any]:
        """Parameters of a built-in model: the defaults, then ``model_params[name]``.

        The GBM models (``lgb``, ``xgb``, ``cat``) get ``random_state=config.random_state`` unless ``model_params`` sets
        one (for ``cat``, ``random_seed`` counts as well); the built-in defaults carry no seed of their own.
        """
        params = merge_dict(self._DEFAULT_MODEL_PARAMS.get(name, {}), self.config.model_params.get(name, {}))
        if name in {"lgb", "xgb", "cat"} and not (name == "cat" and "random_seed" in params):
            # CatBoost's own name for the seed is ``random_seed``: the wrapper turns ``random_state`` into it and lets it
            # win, so adding the config seed next to a ``random_seed`` of the user used to override the user's value.
            params.setdefault("random_state", self.config.random_state)
        return params

    def _effective_seed(self, name: str) -> Any:
        """The seed the final model uses: the one in ``model_params`` when the user set it, else ``random_state``."""
        params = self._model_params(name)
        for key in ("random_state", "random_seed", "seed"):
            if key in params:
                return params[key]
        return self.config.random_state

    def _effective_model_params(self, name: str) -> dict[str, Any]:
        """The parameters the final model is built with: the defaults, ``model_params[name]``, the seed, and for ``lr``
        the best row of the LR search when ``use_lr_search_params`` is on."""
        params = self._model_params(name)
        if name == "lr" and self.config.use_lr_search_params and hasattr(self, "_lr_best_params"):
            params = merge_dict(params, getattr(self, "_lr_best_params", {}))
        return params

    # Settings of the final models that must not reach the backward proxy: ``n_estimators`` and
    # ``early_stopping_rounds`` are aliases of LightGBM's ``num_iterations`` / ``early_stopping_round`` and override the
    # ``num_boost_round`` / ``early_stopping_rounds`` of the backward run, ``eval_metric`` duplicates its
    # ``stopping_metric`` and ``n_jobs`` its thread setting.
    _BACKWARD_PROXY_DROPPED_KEYS = {
        "lgb": ("n_estimators", "early_stopping_rounds", "eval_metric", "n_jobs"),
        "xgb": ("n_estimators", "early_stopping_rounds", "eval_metric"),
    }

    def _model_params_for_backward(self) -> dict[str, Any]:
        """Default parameters of the backward-elimination proxy model, seeded with ``config.random_state``.

        The tree settings of the final model are kept (learning rate, depth, leaves, regularisation), the keys that
        would override the arguments of the backward run are dropped, and the seed is set as ``seed``: the eliminator
        presets ``seed=42`` and that preset beat the ``random_state`` that was passed before.
        """
        name = self._backward_model_name()
        params = dict(self._DEFAULT_MODEL_PARAMS.get(name, {}))
        for key in self._BACKWARD_PROXY_DROPPED_KEYS.get(name, ()):
            params.pop(key, None)
        params["seed"] = self.config.random_state
        if name == "xgb":
            # ``backward_xgbm`` presets no objective, so XGBoost would fit a squared-error regression to the 0/1 target
            params["objective"] = "binary:logistic"
        return params

    def _backward_model_name(self) -> str:
        """Normalized ``backward_model``: stripped and lower-case (``"None"`` for a missing value)."""
        return str(self.config.backward_model).strip().lower()

    def _validate_input(self, data: pd.DataFrame, feature_cols: list[str]) -> None:
        cfg = self.config
        missing = [cfg.target_col] + [col for col in feature_cols if col not in data.columns]
        missing = [col for col in missing if col not in data.columns]
        if missing:
            raise KeyError(f"Missing required columns: {missing}")
        if cfg.weight_col:
            if cfg.weight_col not in data.columns:
                raise KeyError(f"Missing weight_col {cfg.weight_col!r}")
            from Modeling_Tool.Core.sample_weight_utils import resolve_sample_weight

            resolve_sample_weight(data=data, weight_col=cfg.weight_col, expected_len=len(data))
        if cfg.backward_enabled and self._backward_model_name() not in {"lgb", "xgb"}:
            raise ValueError(f"backward_model must be 'lgb' or 'xgb', got {cfg.backward_model!r}")
        self._screen_iv_convention()
        if cfg.warm_start_enabled:
            if not cfg.warm_start_score_col:
                raise ValueError("warm_start_score_col is required when warm_start_enabled=True")
            if cfg.warm_start_score_col not in data.columns:
                raise KeyError(f"Missing warm_start_score_col {cfg.warm_start_score_col!r}")
            if cfg.warm_start_score_type not in {"probability", "log_odds"}:
                raise ValueError("warm_start_score_type must be 'probability' or 'log_odds'")
            if cfg.warm_start_on_unsupported not in {"skip", "raise"}:
                raise ValueError("warm_start_on_unsupported must be 'skip' or 'raise'")
            if cfg.warm_start_score_scope not in {"train", "full"}:
                raise ValueError("warm_start_score_scope must be 'train' or 'full'")
        if cfg.lr_elimination_mode is not None:
            if cfg.lr_elimination_mode != "pvalue":
                raise ValueError(
                    f"lr_elimination_mode must be None or 'pvalue'; got {cfg.lr_elimination_mode!r}"
                )
            allowed = {"pvalue_threshold", "min_features", "max_iterations", "tie_breaker"}
            unknown = set(cfg.lr_elimination_params or {}) - allowed
            if unknown:
                raise ValueError(
                    f"Unknown lr_elimination_params keys {sorted(unknown)}; allowed: {sorted(allowed)}"
                )
            tie_breaker = (cfg.lr_elimination_params or {}).get("tie_breaker")
            if tie_breaker is not None and str(tie_breaker).strip().lower() != "pvalue":
                raise ValueError(
                    f"lr_elimination_params['tie_breaker'] must be 'pvalue' or None; got {tie_breaker!r} "
                    "(equal p-values are always resolved by column order)"
                )
        if cfg.woe_fit_query:
            validate_woe_fit_query_columns(cfg.woe_fit_query, data.columns, context="input data")
            validate_woe_fit_query_syntax(data, cfg.woe_fit_query)
        eval_targets = self._effective_eval_targets()
        missing_targets = [t for t in eval_targets if t not in data.columns]
        if missing_targets:
            raise KeyError(
                f"Missing evaluation target column(s) {missing_targets} "
                f"(target_col + eval_target_cols must all exist in the input data)"
            )
        validate_binary_target(data, eval_targets, owner="CreditModelPipeline")
        eval_weight = self._resolve_eval_weight_col()
        if eval_weight and eval_weight != cfg.weight_col and eval_weight not in data.columns:
            raise KeyError(f"Missing eval_weight_col {eval_weight!r}")
        if self._will_run_explainability():
            for key in ("sample_n", "background_n"):
                if key in cfg.explain_params:
                    try:
                        valid = int(cfg.explain_params[key]) >= 1
                    except (TypeError, ValueError):
                        valid = False
                    if not valid:
                        raise ValueError(f"explain_params[{key!r}] must be a positive integer; got {cfg.explain_params[key]!r}")
        if cfg.all_missing_score_value is not None:
            float(cfg.all_missing_score_value)
        if cfg.special_score_values is not None:
            [float(v) for v in cfg.special_score_values]
        if cfg.extra_eval_datasets:
            reserved = {"ins", "oos", "oot"}
            conflicts = sorted(set(cfg.extra_eval_datasets) & reserved)
            if conflicts:
                raise ValueError(
                    f"extra_eval_datasets names cannot conflict with split names {sorted(reserved)}: {conflicts}"
                )
            for name, extra_df in cfg.extra_eval_datasets.items():
                extra_missing = [t for t in eval_targets if t not in extra_df.columns]
                if extra_missing:
                    raise KeyError(
                        f"extra_eval_datasets[{name!r}] missing evaluation target column(s) {extra_missing}"
                    )
                if eval_weight and eval_weight not in extra_df.columns:
                    # without it the evaluation failed after screening, WOE, searches, training and Optuna
                    raise KeyError(f"extra_eval_datasets[{name!r}] missing the evaluation weight column {eval_weight!r}")
                if cfg.warm_start_enabled and cfg.warm_start_score_col and cfg.warm_start_score_col not in extra_df.columns:
                    raise KeyError(
                        f"extra_eval_datasets[{name!r}] missing warm_start_score_col {cfg.warm_start_score_col!r}"
                    )

    _KNOWN_SPLITS = ("ins", "oos", "oot")

    def _resolve_split_governance(self) -> dict[str, Any]:
        """Resolve effective OOT-governance settings (G10/G11/G12).

        Split-list config fields default to None ("not set explicitly") and
        resolve to the package governance defaults. Explicit values that
        contradict candidate_mode=True raise instead of being silently
        overridden, and the user's config object is never mutated.
        """
        cfg = self.config

        def _norm(values: Any, field_name: str) -> list[str]:
            names = [str(v).strip().lower() for v in as_list(values)]
            unknown = sorted(set(names) - set(self._KNOWN_SPLITS))
            if unknown:
                raise ValueError(
                    f"{field_name} only supports splits {list(self._KNOWN_SPLITS)}; got {unknown}"
                )
            return names

        forbidden = set(_norm(cfg.forbidden_splits, "forbidden_splits"))
        if cfg.candidate_mode:
            conflicts = []
            if cfg.synthesize_missing_oot is True:
                conflicts.append("synthesize_missing_oot=True")
            if cfg.evaluation_splits is not None and "oot" in _norm(cfg.evaluation_splits, "evaluation_splits"):
                conflicts.append("evaluation_splits contains 'oot'")
            if cfg.search_eval_splits is not None and "oot" in _norm(cfg.search_eval_splits, "search_eval_splits"):
                conflicts.append("search_eval_splits contains 'oot'")
            if cfg.backward_report_splits is not None and "oot" in _norm(cfg.backward_report_splits, "backward_report_splits"):
                conflicts.append("backward_report_splits contains 'oot'")
            if str(cfg.backward_validation_split).strip().lower() == "oot":
                conflicts.append("backward_validation_split='oot'")
            if conflicts:
                raise ValueError(
                    f"candidate_mode=True forbids any OOT consumption, but the config "
                    f"explicitly requests it: {conflicts}. Drop candidate_mode or the "
                    f"conflicting settings."
                )
            forbidden |= {"oot"}

        synthesize = False if cfg.synthesize_missing_oot is None else bool(cfg.synthesize_missing_oot)
        evaluation_splits = ["ins", "oos"]
        if cfg.evaluation_splits is not None:
            evaluation_splits = _norm(cfg.evaluation_splits, "evaluation_splits")
        if cfg.search_eval_splits is not None:
            search_eval = _norm(cfg.search_eval_splits, "search_eval_splits")
        else:
            search_eval = ["oos"]
        if cfg.backward_report_splits is not None:
            backward_report = _norm(cfg.backward_report_splits, "backward_report_splits")
        else:
            backward_report = []
        backward_validation = _norm([cfg.backward_validation_split], "backward_validation_split")[0]

        # Fail fast on explicit forbidden-split consumption.
        if evaluation_splits is not None:
            for name in evaluation_splits:
                assert_split_allowed(name, forbidden, "evaluation_splits")
        for name in search_eval:
            assert_split_allowed(name, forbidden, "search (lr_search/optuna) eval_sets")
        for name in backward_report:
            assert_split_allowed(name, forbidden, "backward test_data_dict")
        assert_split_allowed(backward_validation, forbidden, "backward validation_data")

        return {
            "candidate_mode": bool(cfg.candidate_mode),
            "synthesize_missing_oot": bool(synthesize),
            "evaluation_splits": evaluation_splits,
            "forbidden_splits": sorted(forbidden),
            "search_eval_splits": search_eval,
            "search_eval_splits_explicit": cfg.search_eval_splits is not None,
            "backward_validation_split": backward_validation,
            "backward_report_splits": backward_report,
            "backward_report_splits_explicit": cfg.backward_report_splits is not None,
            "oot_synthesized": False,
            "oot_withheld": False,
        }

    def _resolve_eval_weight_col(self) -> str | None:
        """Resolve the evaluation weight column (G16).

        "inherit" (default) reuses the training weight_col — the legacy
        behavior; None evaluates unweighted even when training is weighted;
        any other string names the evaluation weight column explicitly.
        """
        value = self.config.eval_weight_col
        if value == "inherit":
            return self.config.weight_col
        return value

    def _effective_eval_targets(self) -> list[str]:
        """[target_col] + eval_target_cols, deduplicated, order preserved."""
        cfg = self.config
        targets = [cfg.target_col]
        for name in as_list(cfg.eval_target_cols):
            label = str(name)
            if label not in targets:
                targets.append(label)
        return targets

    def _apply_split_governance(self, splits: dict[str, pd.DataFrame]) -> dict[str, pd.DataFrame]:
        """Remove forbidden splits from the working splits dict entirely, so
        no downstream stage can consume them even by accident."""
        governed = dict(splits)
        for name in self._governance["forbidden_splits"]:
            if name in governed:
                governed.pop(name)
                if name == "oot":
                    self._governance["oot_withheld"] = True
        return governed

    def _gbm_weight_kwargs(self, train: pd.DataFrame, val: pd.DataFrame) -> dict[str, Any]:
        cfg = self.config
        if not cfg.weight_col:
            return {}
        from Modeling_Tool.Core.sample_weight_utils import resolve_sample_weight

        kwargs: dict[str, Any] = {}
        train_sw = resolve_sample_weight(data=train, weight_col=cfg.weight_col, expected_len=len(train))
        if train_sw is not None:
            kwargs["sample_weight"] = train_sw
        eval_sw = resolve_sample_weight(data=val, weight_col=cfg.weight_col, expected_len=len(val))
        if eval_sw is not None:
            kwargs["eval_sample_weight"] = eval_sw
        return kwargs

    def _validate_gbm_feature_source_config(self) -> None:
        source_cfg = self.config.gbm_feature_source
        if isinstance(source_cfg, dict):
            invalid_keys = set(source_cfg) - {"lgb", "xgb", "cat"}
            if invalid_keys:
                raise ValueError(f"gbm_feature_source only supports lgb/xgb/cat keys: {sorted(invalid_keys)}")
            invalid_values = {
                key: value
                for key, value in source_cfg.items()
                if str(value).lower() not in {"woe", "raw"}
            }
            if invalid_values:
                raise ValueError(
                    "gbm_feature_source dict values must be 'woe' or 'raw': "
                    f"{invalid_values}"
                )
        elif str(source_cfg).lower() not in {"woe", "raw"}:
            raise ValueError("gbm_feature_source must be 'woe', 'raw', or a dict with those values.")

    def _split_data(self, data: pd.DataFrame) -> dict[str, pd.DataFrame]:
        from Modeling_Tool import SampleSplitter

        cfg = self.config
        work = data.copy()
        self._split_mode = "random"
        sample_col = cfg.split_col or cfg.sample_col
        if cfg.split_col and cfg.split_col not in work.columns:
            raise KeyError(f"Missing split_col {cfg.split_col!r}")
        if sample_col in work.columns:
            raw_split = work[sample_col]
            lower = raw_split.astype(str).str.strip().str.lower()
            if cfg.split_col:
                invalid = sorted(set(raw_split.dropna().astype(str).str.strip().str.lower()) - {"ins", "oos", "oot"})
                if invalid:
                    raise ValueError(f"split_col {cfg.split_col!r} only supports ins/oos/oot values, got {invalid}")
            ins = work[lower == "ins"].copy()
            oos = work[lower == "oos"].copy()
            oot = work[lower == "oot"].copy()
            if len(ins) and len(oos):
                self._split_mode = "label"
                warn_rows_without_split(raw_split, sample_col, "CreditModelPipeline")
                if not len(oot):
                    synthesized = resolve_missing_oot(
                        oos,
                        self._governance["synthesize_missing_oot"],
                        "CreditModelPipeline",
                    )
                    if synthesized is None:
                        return {"ins": ins, "oos": oos}
                    self._governance["oot_synthesized"] = True
                    oot = synthesized
                return {"ins": ins, "oos": oos, "oot": oot}
            if cfg.split_col:
                raise ValueError(f"split_col {cfg.split_col!r} must contain non-empty ins and oos samples")

        if cfg.oot_col and cfg.oot_col in work.columns:
            ins_oos, oot = split_oot_by_flag(work, cfg.oot_col)
        else:
            ins_oos = work
            oot = pd.DataFrame(columns=work.columns)

        splitter = SampleSplitter(
            test_size=float(cfg.split_config.get("test_size", 0.3)),
            random_state=int(cfg.split_config.get("random_state", cfg.random_state)),
            stratify=bool(cfg.split_config.get("stratify", True)),
        )
        ins, oos = splitter.split_df(ins_oos, target=cfg.target_col)
        if len(oot) == 0:
            synthesized = resolve_missing_oot(
                oos,
                self._governance["synthesize_missing_oot"],
                "CreditModelPipeline",
            )
            if synthesized is None:
                return {"ins": ins.copy(), "oos": oos.copy()}
            self._governance["oot_synthesized"] = True
            oot = synthesized
        return {"ins": ins.copy(), "oos": oos.copy(), "oot": oot.copy()}

    def _resolve_screening_artifact(self) -> Any | None:
        cfg = self.config
        if cfg.screening_artifact is not None:
            return cfg.screening_artifact
        if cfg.feature_validation_result is not None:
            from .screening_artifact import FeatureScreeningArtifact

            return FeatureScreeningArtifact.from_fvp_result(
                cfg.feature_validation_result,
                target_col=cfg.target_col,
                weight_col=cfg.weight_col,
            )
        return None

    def _resolve_feature_selection(
        self,
        splits: dict[str, pd.DataFrame],
        feature_cols: list[str],
    ) -> tuple[dict[str, Any], list[str], Any | None]:
        cfg = self.config
        artifact = self._resolve_screening_artifact()
        mode = cfg.feature_selection_mode
        if artifact is not None:
            mode = "from_artifact"
        if mode == "from_artifact":
            if artifact is None:
                raise ValueError("feature_selection_mode='from_artifact' requires screening_artifact.")
            artifact.validate_for_cm(target_col=cfg.target_col, weight_col=cfg.weight_col)
            self._warn_split_differs_from_artifact(artifact)
            summary = dict(artifact.selection_summary or {})
            summary["from_artifact"] = True
            summary["artifact_source"] = artifact.source
            selected = list(artifact.selected_features) or list(feature_cols)
            return summary, selected, artifact
        if mode == "skip":
            summary = {
                "skipped": True,
                "initial_features": list(feature_cols),
                "final_features": list(feature_cols),
            }
            return summary, list(feature_cols), None
        summary, selected = self._feature_selection(splits, feature_cols)
        return summary, selected, None

    def _warn_split_differs_from_artifact(self, artifact: Any) -> None:
        """Warn when the artifact was built on another INS/OOS split than this run uses.

        The validation fitted the WOE bins and chose the features on its INS sample. If this run's OOS sample holds those
        rows, the OOS metrics are no longer out-of-sample (a pure-noise target showed an OOS AUC of 0.58 instead of 0.51).
        """
        recorded = (getattr(artifact, "config_snapshot", None) or {}).get("split")
        if not recorded:
            return
        own = split_settings(self.config, getattr(self, "_data_columns", None))
        keys = ("split_col", "sample_col") if getattr(self, "_split_mode", "random") == "label" else (
            "oot_col", "test_size", "stratify", "random_state"
        )
        differing = {key: (recorded[key], own[key]) for key in keys if key in recorded and recorded[key] != own[key]}
        if differing:
            details = ", ".join(f"{key}: artifact {old!r} vs {new!r} here" for key, (old, new) in differing.items())
            warnings.warn(
                "CreditModelPipeline: the screening artifact was built on another INS/OOS split than this run uses "
                f"({details}). Rows that the validation used to fit the WOE bins and to select features can fall into this "
                "run's OOS sample, so its OOS metrics are optimistic. Use the same split_col / sample_col, or the same "
                "split_config and random_state, in both runs.",
                RuntimeWarning,
                stacklevel=3,
            )

    def _screen_iv_convention(self) -> str:
        """``screen_iv_convention`` of the config, checked against ``feature_selection['screen_iv_convention']``."""
        from Modeling_Tool._utils.screen_iv import check_screen_iv_convention

        cfg = self.config
        value = check_screen_iv_convention(getattr(cfg, "screen_iv_convention", "guarded"))
        in_mapping = (cfg.feature_selection or {}).get("screen_iv_convention")
        if in_mapping is not None and in_mapping != value:
            raise ValueError(
                f"feature_selection['screen_iv_convention']={in_mapping!r} differs from "
                f"screen_iv_convention={value!r}; set one of them, or the same value in both"
            )
        return value

    def _feature_selection(
        self,
        splits: dict[str, pd.DataFrame],
        feature_cols: list[str],
    ) -> tuple[dict[str, Any], list[str]]:
        from Modeling_Tool.Feature.Feature_Screen import feature_screen, screen_config_from_mapping
        from Modeling_Tool.Feature.Weighted_Screen import EmptyStageError

        cfg = self.config
        fs_cfg = cfg.feature_selection

        plot_path = None
        if fs_cfg.get("iv_enabled", True):
            plot_path = str(Path(cfg.output_dir) / "figs" / "var_analysis")
            if cfg.write_outputs and cfg.plot_outputs:
                make_dirs(plot_path, Path(plot_path) / "overall")

        screen_config = screen_config_from_mapping(
            fs_cfg,
            woe_engine=cfg.woe_engine,
            woe_fit_query=cfg.woe_fit_query,
            woe_params=cfg.woe_params,
            monotone_woe_params=cfg.monotone_woe_params,
            plot_path=plot_path,
            plot_outputs=bool(cfg.write_outputs and cfg.plot_outputs),
            screen_iv_convention=self._screen_iv_convention(),
        )

        summary: dict[str, Any] = {"initial_features": list(feature_cols)}
        try:
            # ``feature_screen`` historically receives all three canonical
            # split keys. Under the 0.7.0 no-synthetic-OOT default, keep that
            # internal shape with an empty frame rather than fabricating OOS
            # rows or re-exposing an ``oot`` split to pipeline consumers.
            screen_splits = splits
            if "oot" not in screen_splits:
                screen_splits = dict(splits)
                screen_splits["oot"] = splits["ins"].iloc[0:0].copy()
            result = feature_screen(
                screen_splits,
                feature_cols,
                cfg.target_col,
                weight_col=cfg.weight_col,
                config=screen_config,
            )
            summary = self._screen_result_to_summary(result, feature_cols)
            return summary, list(result.selected_features)
        except EmptyStageError:
            # ``on_empty_stage='raise'``, or only leak-gate drops left (0.9.3): a request to stop, which must not be
            # turned into "keep everything"
            raise
        except Exception as exc:
            # the leak-gate drops that the screening found before it failed stay dropped (0.9.3), except under the
            # explicit legacy ``on_empty_stage='keep_all'``
            hard_dropped = dict(getattr(exc, "smf_hard_dropped", None) or {})
            if screen_config.on_empty_stage == "keep_all":
                hard_dropped = {}
            kept = [col for col in feature_cols if col not in hard_dropped]
            summary["error"] = repr(exc)
            summary["final_features"] = list(kept)
            if hard_dropped:
                summary["hard_dropped"] = dict(hard_dropped)
            held = (
                f" except the {len(hard_dropped)} leak-gate drop(s) {sorted(hard_dropped)}, which stay dropped"
                if hard_dropped else ""
            )
            warnings.warn(
                f"CreditModelPipeline: the feature screening failed ({exc!r}), so every one of the "
                f"{len(feature_cols)} features is kept{held}. The error is in feature_selection_summary['error'].",
                RuntimeWarning,
                stacklevel=2,
            )
            return summary, list(kept)

    def _screen_result_to_summary(
        self,
        result: Any,
        initial_features: list[str],
    ) -> dict[str, Any]:
        summary: dict[str, Any] = {"initial_features": list(initial_features)}
        if not result.psi_table.empty:
            psi = result.psi_table.copy()
            # ``psi_max`` is the value the screening compared with the threshold (the largest PSI over the compared
            # splits); the INS-vs-OOS column alone hid the decisive OOT value when OOT was compared.
            if "psi_max" in psi.columns:
                psi["psi"] = psi["psi_max"]
            elif "psi_ins_oos" in psi.columns:
                psi["psi"] = psi["psi_ins_oos"]
            summary["psi"] = psi
        if not result.iv_table.empty:
            from .screening_artifact import iv_table_for_summary

            summary["iv"] = iv_table_for_summary(result.iv_table)
        sv_pure_bins = (getattr(result, "stage_tables", None) or {}).get("iv_sv_pure_bins")
        if isinstance(sv_pure_bins, pd.DataFrame) and not sv_pure_bins.empty:
            summary["iv_sv_pure_bins"] = sv_pure_bins
        # the reasons of the IV upper limit (on by default since 0.9.3) and of the gates; iv_gap_flag is in "iv"
        dropped_detail = getattr(result, "dropped_detail", None)
        if isinstance(dropped_detail, pd.DataFrame) and not dropped_detail.empty:
            summary["dropped_detail"] = dropped_detail
        if not result.corr_dropped.empty:
            summary["corr_dropped"] = result.corr_dropped
        summary["corr_features"] = list(result.selected_features)
        summary["screen_summary"] = result.summary
        summary["final_features"] = list(result.selected_features)
        return summary

    def _transform_extra_eval_datasets(
        self,
        transform_fn: Any,
        feature_cols: list[str],
        *,
        woe_suffix: str | None = None,
    ) -> dict[str, pd.DataFrame]:
        cfg = self.config
        if not cfg.extra_eval_datasets:
            return {}
        transformed: dict[str, pd.DataFrame] = {}
        for name, df in cfg.extra_eval_datasets.items():
            if woe_suffix is not None:
                transformed[name] = transform_fn(df, varlist=feature_cols, suffix=woe_suffix)
            else:
                transformed[name] = transform_fn(df)
        return transformed

    def _reuse_screening_woe(
        self,
        splits: dict[str, pd.DataFrame],
        feature_cols: list[str],
        prefit_woe_artifacts: dict[str, Any],
    ) -> dict[str, Any] | None:
        from Modeling_Tool.WOE.WOE_Adapter import WOEMasterAdapter, as_woe_engine

        cfg = self.config
        by_target = prefit_woe_artifacts.get("by_target", {}) if prefit_woe_artifacts else {}
        item = by_target.get(cfg.target_col)
        if not item:
            return None

        adapter = item.get("adapter")
        if adapter is None and item.get("engine") is not None:
            engine = item["engine"]
            adapter = as_woe_engine(
                engine, woe_suffix=getattr(engine, "woe_suffix", None) or cfg.woe_params.get("woe_suffix", "_woe")
            )
        if adapter is None:
            return None

        fitted = set(item.get("features") or [])
        usable = [col for col in feature_cols if col in fitted]
        if not usable:
            return None

        woe_suffix = cfg.woe_params.get("woe_suffix", "_woe")
        if isinstance(adapter, WOEMasterAdapter):
            # a WOE_Master names its columns with its own suffix and the adapter ignores the one passed to transform
            woe_suffix = getattr(adapter.engine, "woe_suffix", None) or adapter.woe_suffix
        woe_features = [f"{col}{woe_suffix}" for col in usable]
        woe_splits = {
            name: adapter.transform(df, varlist=usable, suffix=woe_suffix)
            for name, df in splits.items()
        }
        extra_eval = self._transform_extra_eval_datasets(
            adapter.transform,
            usable,
            woe_suffix=woe_suffix,
        )
        if cfg.warm_start_enabled and cfg.warm_start_score_col:
            for name, df in woe_splits.items():
                if cfg.warm_start_score_col not in df.columns and cfg.warm_start_score_col in splits[name].columns:
                    copy_column_length_checked(
                        df,
                        splits[name],
                        cfg.warm_start_score_col,
                        dst_name=f"woe_splits[{name!r}]",
                        src_name=f"splits[{name!r}]",
                    )
            for name, df in extra_eval.items():
                if cfg.warm_start_score_col not in df.columns and cfg.warm_start_score_col in cfg.extra_eval_datasets[name].columns:
                    copy_column_length_checked(
                        df,
                        cfg.extra_eval_datasets[name],
                        cfg.warm_start_score_col,
                        dst_name=f"extra_eval[{name!r}]",
                        src_name=f"extra_eval_datasets[{name!r}]",
                    )

        return {
            "engine": adapter,
            # a WOE_Master adapter calls itself "master"; the self-fit path records the config name "equal_freq"
            "engine_name": (
                {"master": "equal_freq"}.get(adapter.get_engine_name(), adapter.get_engine_name())
                if hasattr(adapter, "get_engine_name")
                else cfg.woe_engine
            ),
            "features": list(usable),
            "woe_features": woe_features,
            "woe_suffix": woe_suffix,
            "splits": woe_splits,
            "extra_eval": extra_eval,
            "woe_table": adapter.get_woe_table(varlist=usable),
            "reused_from_screening": True,
        }

    def _fit_woe(
        self,
        splits: dict[str, pd.DataFrame],
        feature_cols: list[str],
        *,
        prefit_woe_artifacts: dict[str, Any] | None = None,
    ) -> dict[str, Any]:
        from Modeling_Tool import MonotoneWOEBinner, WOE_Master
        from Modeling_Tool.WOE.WOE_Adapter import as_woe_engine

        cfg = self.config
        if prefit_woe_artifacts and cfg.reuse_screening_woe:
            reused = self._reuse_screening_woe(splits, feature_cols, prefit_woe_artifacts)
            if reused is not None:
                return reused

        woe_suffix = cfg.woe_params.get("woe_suffix", "_woe")
        graph_dir = str(Path(cfg.output_dir) / "figs" / "woe")
        woe_features = [f"{col}{woe_suffix}" for col in feature_cols]
        fit_ins, _ = apply_woe_fit_query(
            splits["ins"],
            cfg.woe_fit_query,
            target=cfg.target_col,
        )

        if cfg.woe_engine.lower() == "monotone":
            defaults = {"feature_cols": feature_cols, "target_col": cfg.target_col}
            # the sentinel -999999 is declared only if the fit sample holds it (shared with the other monotone fits)
            params = merge_dict(
                defaults, with_default_special_values(cfg.monotone_woe_params, fit_ins, feature_cols)
            )
            # fit()-only kwargs must not reach MonotoneWOEBinner.__init__ —
            # n_jobs / chi2_p / chi2_init_size in monotone_woe_params used to
            # raise TypeError on this self-fit path (the screening-side
            # fit_screening_woe_engine already splits init vs fit keys).
            fit_kwargs = {
                key: params.pop(key)
                for key in ("chi2_binning", "chi2_p", "chi2_init_size", "n_jobs")
                if key in params
            }
            fit_kwargs["chi2_binning"] = bool(fit_kwargs.get("chi2_binning", False))
            binner = MonotoneWOEBinner(**params)
            with quiet_default_sentinel(cfg.monotone_woe_params):
                binner.fit(fit_ins, **fit_kwargs)
            if cfg.write_outputs and cfg.plot_outputs:
                make_dirs(Path(cfg.output_dir) / "figs" / "mono_woe")
                binner.plot_woe_graph(graph_path=str(Path(cfg.output_dir) / "figs" / "mono_woe"))
            adapter = as_woe_engine(binner, woe_suffix=woe_suffix)
            woe_splits = {
                name: adapter.transform(df, varlist=feature_cols, suffix=woe_suffix)
                for name, df in splits.items()
            }
            woe_table = adapter.get_woe_table(varlist=feature_cols)
            engine = adapter
            extra_eval = self._transform_extra_eval_datasets(
                adapter.transform,
                feature_cols,
                woe_suffix=woe_suffix,
            )
        else:
            master = WOE_Master(
                train_data=fit_ins,
                varlist=feature_cols,
                dep=cfg.target_col,
                graph_save_dir=graph_dir,
                woe_suffix=woe_suffix,
                missing_ref_value=cfg.woe_params.get("missing_ref_value", -999999),
            )
            fit_params = {k: v for k, v in cfg.woe_params.items() if k not in {"woe_suffix", "missing_ref_value"}}
            master.fit(**fit_params)
            woe_splits = {name: master.transform(df) for name, df in splits.items()}
            if cfg.write_outputs and cfg.plot_outputs:
                # no group: a group of one value drew each chart a second time (<var>__smf_plot_group.png)
                master.plot_bivar_graph(
                    woe_splits["ins"].copy(),
                    group=None,
                    dirname="overall",
                    varlist=feature_cols,
                )
            woe_table = self._applied_woe_table(master, feature_cols)
            engine = master
            extra_eval = self._transform_extra_eval_datasets(master.transform, feature_cols)

        if cfg.warm_start_enabled and cfg.warm_start_score_col:
            for name, df in woe_splits.items():
                if cfg.warm_start_score_col not in df.columns:
                    copy_column_length_checked(
                        df,
                        splits[name],
                        cfg.warm_start_score_col,
                        dst_name=f"woe_splits[{name!r}]",
                        src_name=f"splits[{name!r}]",
                    )
            for name, df in extra_eval.items():
                if cfg.warm_start_score_col not in df.columns:
                    source = cfg.extra_eval_datasets[name]
                    copy_column_length_checked(
                        df,
                        source,
                        cfg.warm_start_score_col,
                        dst_name=f"extra_eval[{name!r}]",
                        src_name=f"extra_eval_datasets[{name!r}]",
                    )

        return {
            "engine": engine,
            "engine_name": cfg.woe_engine,
            "features": list(feature_cols),
            "woe_features": woe_features,
            "woe_suffix": woe_suffix,
            "splits": woe_splits,
            "extra_eval": extra_eval,
            "woe_table": woe_table,
        }

    @staticmethod
    def _applied_woe_table(master: Any, feature_cols: list[str]) -> pd.DataFrame:
        """The mapping table that ``master.transform`` applies, one row per bin of ``feature_cols``.

        The table used to be recomputed from the raw counts (``get_overall_woe_table``). That recomputation ignores the
        special-value policies and smoothing, ``include_missing=False`` and ``precision``, so with those settings the
        reported WOE differed from the WOE the models received.
        """
        mapping = master.get_mapping_table()
        columns = [
            "VAR", "BIN_NUM", "BIN_RANGE", "MIN", "MAX", "N", "AVG_BAD", "WOE", "IV", "N_BAD", "N_GOOD",
            "BAD_PCT_PER_BIN", "GOOD_PCT_PER_BIN", "LIFT",
        ]
        order = {col: pos for pos, col in enumerate(feature_cols)}
        mapping = mapping[mapping["VAR"].isin(order)]
        mapping = mapping.assign(_order=mapping["VAR"].map(order)).sort_values("_order", kind="stable")
        return mapping[[col for col in columns if col in mapping.columns]].reset_index(drop=True)

    def _resolve_gbm_feature_source(self, model_name: str) -> str:
        cfg = self.config
        source_cfg = cfg.gbm_feature_source
        if isinstance(source_cfg, dict):
            source = str(source_cfg.get(model_name, "woe")).lower()
        else:
            source = str(source_cfg).lower()
        return source

    @staticmethod
    def _woe_to_raw_features(woe_features: list[str], woe_suffix: str) -> list[str]:
        if not woe_suffix:
            return list(woe_features)
        return [
            feature[: -len(woe_suffix)] if feature.endswith(woe_suffix) else feature
            for feature in woe_features
        ]

    @staticmethod
    def _raw_to_woe_features(raw_features: list[str], woe_suffix: str) -> list[str]:
        return [f"{feature}{woe_suffix}" for feature in raw_features]

    def _build_model_inputs(
        self,
        raw_splits: dict[str, pd.DataFrame],
        woe_splits: dict[str, pd.DataFrame],
        selected_raw_features: list[str],
        selected_woe_features: list[str],
    ) -> dict[str, dict[str, Any]]:
        inputs: dict[str, dict[str, Any]] = {}
        for raw_name in as_list(self.config.train_models):
            name = str(raw_name).lower()
            if name == "lr":
                source = "woe"
                splits = woe_splits
                features = list(selected_woe_features)
            elif name in {"lgb", "xgb", "cat"}:
                source = self._resolve_gbm_feature_source(name)
                splits = raw_splits if source == "raw" else woe_splits
                features = list(selected_raw_features if source == "raw" else selected_woe_features)
            else:
                continue
            inputs[name] = {
                "source": source,
                "splits": splits,
                "features": features,
                # Raw-column view of the model's inputs, used by the
                # all-missing score override (G14): the business rule is
                # defined on raw features regardless of what representation
                # the model consumed.
                "raw_features": list(selected_raw_features if source == "woe" else features),
            }
        return inputs

    @staticmethod
    def _summarize_model_inputs(
        model_inputs: dict[str, dict[str, Any]]
    ) -> tuple[dict[str, str], dict[str, list[str]]]:
        sources = {name: str(item["source"]) for name, item in model_inputs.items()}
        feature_sets = {name: list(item["features"]) for name, item in model_inputs.items()}
        return sources, feature_sets

    @staticmethod
    def _model_feature_source_frame(
        model_feature_sources: dict[str, str],
        model_feature_sets: dict[str, list[str]],
    ) -> pd.DataFrame:
        return pd.DataFrame(
            [
                {
                    "model": name,
                    "feature_source": model_feature_sources[name],
                    "n_features": len(model_feature_sets.get(name, [])),
                    "features": ",".join(model_feature_sets.get(name, [])),
                }
                for name in sorted(model_feature_sources)
            ]
        )

    def _train_models(
        self,
        model_inputs: dict[str, dict[str, Any]],
    ) -> dict[str, tuple[Any, Any, list[str]]]:
        from Modeling_Tool import GradientBoostingModel, LRMaster

        cfg = self.config
        models: dict[str, tuple[Any, Any, list[str]]] = {}
        self._lr_elimination_trace = None
        for raw_name in as_list(cfg.train_models):
            name = str(raw_name).lower()
            if name not in {"lr", "lgb", "xgb", "cat"}:
                raise ValueError(f"Unsupported model type: {raw_name!r}")
            input_info = model_inputs.get(name)
            if input_info is None:
                raise ValueError(f"No model input prepared for model type: {raw_name!r}")
            splits = input_info["splits"]
            feature_cols = list(input_info["features"])
            if not feature_cols:
                raise ValueError(f"No training features available for model type: {raw_name!r}")
            train, val = splits["ins"], splits["oos"]
            params = self._effective_model_params(name)
            if name == "lr":
                lr_params = dict(params) if params else {}
                standardize = bool(lr_params.pop("standardize", False))
                lr = LRMaster(params=lr_params or None, standardize=standardize)
                lr.fit(
                    data=train,
                    varlist=feature_cols,
                    tgt_name=cfg.target_col,
                    val_data=val,
                    val_varlist=feature_cols,
                    val_tgt_name=cfg.target_col,
                    weight_col=cfg.weight_col,
                )
                if cfg.lr_elimination_mode == "pvalue":
                    lr, feature_cols = self._eliminate_lr_pvalue(
                        lr, train, val, feature_cols, lr_params, standardize,
                    )
                models[name] = (lr, getattr(lr, "model", lr), list(feature_cols))
            elif name in {"lgb", "xgb", "cat"}:
                if self._warm_start_requested_for(name) and name == "cat":
                    if cfg.warm_start_on_unsupported == "raise":
                        raise NotImplementedError("CatBoost does not support warm-start init_score")
                gbm = GradientBoostingModel(name, params)
                init_score = self._get_warm_start_init_score(name, train)
                extra_fit: dict[str, Any] = {}
                if init_score is not None and cfg.warm_start_score_scope == "full":
                    # early stopping measures the combined model (prior plus trees), not the trees alone
                    extra_fit["eval_init_score"] = self._get_warm_start_init_score(name, val)
                gbm.fit(
                    x=train[feature_cols],
                    y=train[cfg.target_col].astype(int),
                    valx=val[feature_cols],
                    valy=val[cfg.target_col].astype(int),
                    init_score=init_score,
                    **extra_fit,
                    **self._gbm_weight_kwargs(train, val),
                )
                raw = gbm._model.model if hasattr(gbm, "_model") else gbm
                models[name] = (gbm, raw, list(feature_cols))
        return models

    def _eliminate_lr_pvalue(
        self,
        lr: Any,
        train: pd.DataFrame,
        val: pd.DataFrame,
        feature_cols: list[str],
        lr_params: dict[str, Any],
        standardize: bool,
    ) -> tuple[Any, list[str]]:
        """G07: backward-eliminate LR features by coefficient p-value, refitting
        after each drop. P-values come from the scipy Fisher-information summary,
        so they describe exactly the sklearn model that ships."""
        from Modeling_Tool import LRMaster
        from Modeling_Tool.Model.LRM_Tool import fast_lr_pvalues

        cfg = self.config
        elim = dict(cfg.lr_elimination_params or {})
        threshold = float(elim.get("pvalue_threshold", 0.05))
        min_features = int(elim.get("min_features", 1))
        max_iterations = int(elim.get("max_iterations", 20))
        current = list(feature_cols)
        trace_rows: list[dict] = []
        for iteration in range(max_iterations):
            pvals = fast_lr_pvalues(
                lr.model, lr._apply_standardizer(train[current]), current,
            ).drop("Intercept")
            worst = str(pvals.idxmax())
            worst_p = float(pvals.max())
            if not np.isfinite(worst_p) or worst_p <= threshold or len(current) <= min_features:
                break
            current.remove(worst)
            trace_rows.append({
                "iteration": iteration,
                "dropped_feature": worst,
                "p_value": worst_p,
                "n_remaining": len(current),
            })
            lr = LRMaster(params=lr_params or None, standardize=standardize)
            lr.fit(
                data=train,
                varlist=current,
                tgt_name=cfg.target_col,
                val_data=val,
                val_varlist=current,
                val_tgt_name=cfg.target_col,
                weight_col=cfg.weight_col,
            )
        self._lr_elimination_trace = pd.DataFrame(
            trace_rows, columns=["iteration", "dropped_feature", "p_value", "n_remaining"],
        )
        return lr, current

    def _run_backward(
        self,
        splits: dict[str, pd.DataFrame],
        feature_cols: list[str],
    ) -> tuple[pd.DataFrame | None, list[str]]:
        cfg = self.config
        try:
            from Modeling_Tool import BackwardVariableEliminator

            gov = self._governance
            validation_split = gov["backward_validation_split"]
            if validation_split not in splits:
                raise ValueError(
                    f"backward_validation_split {validation_split!r} is not available "
                    f"in this run (available: {sorted(splits)})."
                )
            test_data_dict: dict[str, pd.DataFrame] = {}
            for split_name in gov["backward_report_splits"]:
                if split_name not in splits:
                    if gov["backward_report_splits_explicit"]:
                        raise ValueError(
                            f"backward_report_splits contains {split_name!r}, which is "
                            f"not available in this run (available: {sorted(splits)})."
                        )
                    continue
                test_data_dict[split_name] = splits[split_name]
            user_init = dict(cfg.backward_params.get("init", {}))
            for split_name in (user_init.get("test_data_dict") or {}):
                assert_split_allowed(
                    str(split_name).lower(),
                    gov["forbidden_splits"],
                    "backward test_data_dict (backward_params['init'])",
                )
            validation_data = splits[validation_split]
            validation_weight_col = self._resolve_eval_weight_col()
            if validation_weight_col is None and cfg.weight_col:
                # ``eval_weight_col=None`` means an unweighted validation, but the eliminator falls back to the
                # training weights when it gets no validation weight column, so give it a column of ones.
                unit = "_smf_unit_weight"
                validation_data = validation_data.assign(**{unit: 1.0})
                test_data_dict = {name: frame.assign(**{unit: 1.0}) for name, frame in test_data_dict.items()}
                validation_weight_col = unit
            params = merge_dict(
                {
                    "train_data": splits["ins"],
                    "varlist": feature_cols,
                    "dep": cfg.target_col,
                    "model_type": f"{self._backward_model_name()}m",
                    "validation_data": validation_data,
                    "test_data_dict": test_data_dict,
                    "weight_col": cfg.weight_col,
                    "validation_weight_col": validation_weight_col,
                },
                user_init,
            )
            bwd = BackwardVariableEliminator(**params)
            run_params = merge_dict(
                {
                    "n_rounds": 3,
                    "varreduct_params": self._model_params_for_backward(),
                    "stopping_metric": "auc",
                    "num_boost_round": 200,
                    "early_stopping_rounds": 20,
                    "cum_importance_threshold": 0.99,
                    "min_vars": max(3, len(feature_cols) // 2),
                    "ret_perf": True,
                },
                cfg.backward_params.get("run", {}),
            )
            user_run = cfg.backward_params.get("run", {})
            if "seed" in user_run and "seed" not in (user_run.get("varreduct_params") or {}):
                # the eliminator only uses its ``seed`` argument when the parameters carry none, and ours carry one
                run_params["varreduct_params"] = {**run_params["varreduct_params"], "seed": user_run["seed"]}
            if hasattr(bwd, "run"):
                bwd.run(**run_params)
                selected = list(bwd.get_final_vars()) if hasattr(bwd, "get_final_vars") else list(feature_cols)
                summary = bwd.get_summary() if hasattr(bwd, "get_summary") else None
            else:
                bwd.fit(feature_cols)
                selected = list(bwd.get_result().get("final_vars", feature_cols)) if hasattr(bwd, "get_result") else list(feature_cols)
                summary = bwd.get_backward_summary() if hasattr(bwd, "get_backward_summary") else None
            return summary, selected
        except Exception as exc:
            return pd.DataFrame({"step": ["backward"], "error": [repr(exc)]}), list(feature_cols)

    def _build_search_eval_sets(
        self,
        splits: dict[str, pd.DataFrame],
        *,
        component: str,
    ) -> dict[str, pd.DataFrame]:
        """Build hyperparameter-search eval_sets from search_eval_splits.

        Every requested split passes the forbidden_splits hard gate. A split
        missing from the run (e.g. no real OOT with synthesis disabled) is an
        error when search_eval_splits was set explicitly; when the field was
        left unset, unavailable splits are silently dropped from the resolved
        legacy default instead.
        """
        gov = self._governance
        eval_sets: dict[str, pd.DataFrame] = {}
        for name in gov["search_eval_splits"]:
            assert_split_allowed(name, gov["forbidden_splits"], component)
            if name not in splits:
                if gov["search_eval_splits_explicit"]:
                    raise ValueError(
                        f"{component}: search eval split {name!r} is not available in "
                        f"this run (available: {sorted(splits)}). Drop it from "
                        f"search_eval_splits or provide the split."
                    )
                continue
            eval_sets[name] = splits[name]
        if not eval_sets:
            raise ValueError(
                f"{component}: no usable search eval splits "
                f"(requested {gov['search_eval_splits']}, available {sorted(splits)})."
            )
        return eval_sets

    def _default_search_objective_params(self, eval_sets: dict[str, pd.DataFrame]) -> dict[str, Any]:
        has_oot = "oot" in eval_sets
        return {
            "objective": "oot_gap_penalized" if has_oot else str(self.config.search_objective_when_no_oot),
            "primary_set": "oos",
            "gap_ref_sets": ["oot"] if has_oot else [],
            "metric": "auc",
        }

    def _validate_search_objective_params(
        self,
        params: dict[str, Any],
        eval_sets: dict[str, pd.DataFrame],
        *,
        component: str,
    ) -> None:
        primary = str(params.get("primary_set"))
        if primary not in eval_sets:
            raise ValueError(
                f"{component}: primary_set {primary!r} is not among the search eval "
                f"sets {sorted(eval_sets)}."
            )
        gap_refs = [str(x) for x in as_list(params.get("gap_ref_sets"))]
        missing = [name for name in gap_refs if name not in eval_sets]
        if missing:
            raise ValueError(
                f"{component}: gap_ref_sets {missing} are not among the search eval "
                f"sets {sorted(eval_sets)} — the split may be absent from this run "
                f"or listed in forbidden_splits."
            )
        objective = params.get("objective")
        if not callable(objective) and str(objective) == "oot_gap_penalized" and not gap_refs:
            raise ValueError(
                f"{component}: objective='oot_gap_penalized' requires a non-empty "
                f"gap_ref_sets, but no gap reference split is available in this run. "
                f"Use objective='max_primary' (see search_objective_when_no_oot) or "
                f"provide a real OOT split."
            )

    def _run_lr_search(
        self,
        splits: dict[str, pd.DataFrame],
        feature_cols: list[str],
    ) -> pd.DataFrame | None:
        from Modeling_Tool import LRMaster

        cfg = self.config
        self._lr_best_params = {}
        if not cfg.lr_search_enabled or "lr" not in {str(x).lower() for x in as_list(cfg.train_models)}:
            return None
        base_params = dict(cfg.model_params.get("lr", {}))
        standardize = bool(base_params.pop("standardize", False))
        lr = LRMaster(params=base_params or None, standardize=standardize)
        allowed_search_params = {
            "objective",
            "primary_set",
            "gap_ref_sets",
            "metric",
            "refit",
            "verbose",
        }
        unknown_search_params = sorted(set(cfg.lr_search_params) - allowed_search_params)
        if unknown_search_params:
            raise ValueError(
                f"Unsupported lr_search_params keys: {unknown_search_params}. "
                f"Allowed keys are {sorted(allowed_search_params)}. "
                "LRMaster.grid_search_params uses holdout eval_sets and does not accept cv."
            )
        eval_sets = self._build_search_eval_sets(splits, component="lr_search")
        params = merge_dict(
            self._default_search_objective_params(eval_sets),
            cfg.lr_search_params,
        )
        self._validate_search_objective_params(params, eval_sets, component="lr_search")
        results = lr.grid_search_params(
            data=splits["ins"],
            varlist=feature_cols,
            tgt_name=cfg.target_col,
            eval_sets=eval_sets,
            param_grid=cfg.lr_search_param_grid,
            weight_col=cfg.weight_col,
            eval_weight_col=self._resolve_eval_weight_col(),
            **params,
        )
        self._lr_best_params = dict(getattr(lr, "best_params_", {}) or {})
        return results

    def _warm_start_requested_for(self, model_name: str) -> bool:
        cfg = self.config
        return bool(
            cfg.warm_start_enabled
            and cfg.warm_start_score_col
            and model_name in {str(x).lower() for x in as_list(cfg.warm_start_models)}
        )

    def _get_warm_start_init_score(self, model_name: str, data: pd.DataFrame) -> np.ndarray | None:
        if not self._warm_start_requested_for(model_name):
            return None
        if model_name == "cat":
            return None
        cfg = self.config
        score = data[cfg.warm_start_score_col]
        if score.isna().any():
            raise ValueError(f"warm_start_score_col {cfg.warm_start_score_col!r} contains missing values")
        arr = score.to_numpy(dtype=float)
        if cfg.warm_start_score_type == "probability":
            if ((arr < 0) | (arr > 1)).any():
                raise ValueError(
                    f"warm_start_score_col {cfg.warm_start_score_col!r} holds values outside [0, 1], but "
                    "warm_start_score_type is 'probability'; use warm_start_score_type='log_odds' for log-odds scores"
                )
            arr = np.clip(arr, 1e-6, 1 - 1e-6)
            return np.log(arr / (1 - arr))
        return arr

    def _build_warm_start_summary(
        self,
        model_inputs: dict[str, dict[str, Any]],
    ) -> pd.DataFrame | None:
        cfg = self.config
        if not cfg.warm_start_enabled:
            return None
        rows = []
        train_models = {str(x).lower() for x in as_list(cfg.train_models)}
        requested = {str(x).lower() for x in as_list(cfg.warm_start_models)}
        for model_name in sorted(requested):
            if model_name not in train_models:
                status = "not_in_train_models"
            elif model_name == "cat":
                status = "skipped_unsupported"
                if cfg.warm_start_on_unsupported == "raise":
                    raise NotImplementedError("CatBoost does not support warm-start init_score")
            elif model_name in {"lgb", "xgb"}:
                status = "enabled"
            else:
                status = "skipped_unknown_model"
            missing_rate = np.nan
            n_features = 0
            input_info = model_inputs.get(model_name)
            if input_info is not None:
                split_ins = input_info["splits"]["ins"]
                n_features = len(input_info["features"])
                if cfg.warm_start_score_col and cfg.warm_start_score_col in split_ins.columns:
                    missing_rate = float(split_ins[cfg.warm_start_score_col].isna().mean())
            rows.append(
                {
                    "model": model_name,
                    "status": status,
                    "score_col": cfg.warm_start_score_col,
                    "score_type": cfg.warm_start_score_type,
                    "missing_rate_ins": missing_rate,
                    "apply_to_optuna": bool(cfg.warm_start_apply_to_optuna and model_name in {"lgb", "xgb"}),
                    "n_features": n_features,
                }
            )
        return pd.DataFrame(rows)

    def _run_optuna(
        self,
        model_inputs: dict[str, dict[str, Any]],
    ) -> dict[str, pd.DataFrame]:
        from Modeling_Tool import GradientBoostingModel

        cfg = self.config
        results = {}
        if not as_list(cfg.optuna_models):
            return results
        user_search_spaces = cfg.optuna_params.get("search_spaces")
        search_spaces = self._default_search_spaces() if user_search_spaces is None else user_search_spaces
        for raw_name in as_list(cfg.optuna_models):
            name = str(raw_name).lower()
            if name not in {"lgb", "xgb", "cat"} or name not in search_spaces:
                continue
            input_info = model_inputs.get(name)
            if input_info is None:
                continue
            splits = input_info["splits"]
            feature_cols = list(input_info["features"])
            user_common = dict(cfg.optuna_params.get("common", {}))
            user_eval_sets = user_common.get("eval_sets")
            if user_eval_sets is not None:
                # Legacy override kept, but every named split still passes the
                # governance hard gate — forbidden splits cannot sneak back in
                # through optuna_params.
                for split_name in user_eval_sets:
                    assert_split_allowed(
                        str(split_name).lower(),
                        self._governance["forbidden_splits"],
                        "optuna_search eval_sets (optuna_params['common'])",
                    )
                eval_sets = dict(user_eval_sets)
            else:
                eval_sets = self._build_search_eval_sets(splits, component="optuna_search")
            common = merge_dict(
                {
                    "varlist": feature_cols,
                    "tgt_name": cfg.target_col,
                    "eval_sets": eval_sets,
                    "engine": "optuna",
                    **self._default_search_objective_params(eval_sets),
                    "n_trials": cfg.optuna_n_trials,
                    "refit": True,
                    "verbose": False,
                    "random_state": cfg.random_state,
                },
                user_common,
            )
            self._validate_search_objective_params(common, eval_sets, component="optuna_search")
            if self._warm_start_requested_for(name) and name == "cat":
                if cfg.warm_start_on_unsupported == "raise":
                    raise NotImplementedError("CatBoost does not support warm-start init_score")
            try:
                searcher = GradientBoostingModel(name, self._model_params(name))
                fit_kwargs = dict(cfg.optuna_params.get("fit_kwargs", {}))
                search_extra: dict[str, Any] = {}
                if cfg.warm_start_apply_to_optuna and self._warm_start_requested_for(name):
                    fit_kwargs["init_score"] = self._get_warm_start_init_score(name, splits["ins"])
                    if cfg.warm_start_score_scope == "full":
                        # candidates are early-stopped and scored as the combined model (prior plus trees)
                        search_extra["eval_init_scores"] = {
                            set_name: self._get_warm_start_init_score(name, frame)
                            for set_name, frame in common["eval_sets"].items()
                        }
                results[name] = searcher.param_search(
                    data=splits["ins"],
                    search_space=search_spaces[name],
                    fit_kwargs=fit_kwargs or None,
                    weight_col=cfg.weight_col,
                    eval_weight_col=self._resolve_eval_weight_col(),
                    **search_extra,
                    **common,
                )
            except Exception as exc:
                results[name] = pd.DataFrame({"error": [repr(exc)]})
        return results

    def _evaluate_models(
        self,
        model_inputs: dict[str, dict[str, Any]],
        models: dict[str, tuple[Any, Any, list[str]]],
        extra_eval_splits: dict[str, pd.DataFrame] | None = None,
    ) -> dict[str, pd.DataFrame]:
        from Modeling_Tool import PerformanceEvaluator

        cfg = self.config
        results = {}
        extra_eval_splits = extra_eval_splits or {}
        for name, (wrapper, _, feature_cols) in models.items():
            splits = model_inputs[name]["splits"]
            source = str(model_inputs[name].get("source", "woe")).lower()
            if source == "raw" and cfg.extra_eval_datasets:
                model_extra = {key: df.copy() for key, df in cfg.extra_eval_datasets.items()}
            else:
                model_extra = extra_eval_splits
            eval_targets = self._effective_eval_targets()
            eval_weight = self._resolve_eval_weight_col()
            evaluator = PerformanceEvaluator(
                tgt_name=eval_targets if len(eval_targets) > 1 else cfg.target_col,
                scr_name=f"pred_{name}",
                pct_bins=cfg.perf_pct_bins,
                min_bin_prop=cfg.perf_min_bin_prop,
                equal_freq=True,
                spec_values=cfg.special_score_values,
                ascending=cfg.gains_ascending,
            )
            allowed = self._governance.get("evaluation_splits")
            if allowed is not None:
                splits = {name: df for name, df in splits.items() if name in set(allowed)}
            eval_splits = {**splits, **model_extra}
            # the raw frames: with woe_suffix='' the WOE columns REPLACE the raw ones, and the rule is defined on raw values
            raw_frames = {**getattr(self, "_raw_splits", {}), **(cfg.extra_eval_datasets or {})}
            nan_stats: dict[str, int] = {}
            for ds_name, df in eval_splits.items():
                scored = df.copy()
                scored[f"pred_{name}"] = self._predict_model_positive(name, wrapper, scored, feature_cols)
                if cfg.all_missing_score_value is not None:
                    raw_frame = raw_frames.get(ds_name)
                    if raw_frame is None or len(raw_frame) != len(scored):
                        raw_frame = scored
                    override = all_missing_mask(raw_frame, model_inputs[name].get("raw_features") or [])
                    if override.any():
                        scored.loc[override, f"pred_{name}"] = float(cfg.all_missing_score_value)
                nan_stats[str(ds_name)] = int((~np.isfinite(scored[f"pred_{name}"].to_numpy(dtype=float))).sum())
                add_dataset_with_optional_weight(evaluator, ds_name, scored, weight_col=eval_weight)
            self.predict_positive_nan_stats[str(name)] = nan_stats
            total_bad = sum(nan_stats.values())
            if total_bad:
                total_rows = sum(len(df) for df in eval_splits.values())
                detail = ", ".join(f"{k}={v}" for k, v in nan_stats.items() if v)
                warnings.warn(
                    f"{name}: NaN/Inf predictions detected across evaluation datasets: "
                    f"{detail}; total {total_bad}/{total_rows}.",
                    RuntimeWarning,
                    stacklevel=2,
                )
            fig_save_path = None
            if cfg.write_outputs and cfg.plot_outputs:
                make_dirs(Path(cfg.output_dir) / "figs" / "perf")
                fig_save_path = str(Path(cfg.output_dir) / "figs" / "perf" / f"perf_{name}.png")
            results[name] = evaluator.evaluate(to_show=False, display=False, fig_save_path=fig_save_path)
        return results

    def _raw_features_for_saved_model(
        self,
        model_name: str,
        model_feature_sources: dict[str, str],
        model_feature_sets: dict[str, list[str]],
        woe_artifacts: dict[str, Any],
    ) -> list[str]:
        """Raw-column view of a saved model's inputs, for the all-missing
        override metadata (the business rule is defined on raw features)."""
        features = list(model_feature_sets.get(model_name, []))
        if str(model_feature_sources.get(model_name, "woe")).lower() == "raw":
            return features
        suffix = woe_artifacts.get("woe_suffix", self.config.woe_params.get("woe_suffix", "_woe"))
        return self._woe_to_raw_features(features, suffix)

    def _predict_model_positive(
        self,
        model_name: str,
        wrapper: Any,
        data: pd.DataFrame,
        feature_cols: list[str],
    ) -> np.ndarray:
        if self._warm_start_requested_for(model_name) and model_name in {"lgb", "xgb"}:
            init_score = self._get_warm_start_init_score(model_name, data)
            return wrapper.predict_with_base_margin(data[feature_cols], init_score, return_prob=True)
        return predict_positive(wrapper, data, feature_cols, warn_nan=False)

    def _will_run_explainability(self) -> bool:
        cfg = self.config
        return bool(as_list(cfg.explain_models)) or bool(cfg.owen_enabled)

    def _explain_excel_sheets(self, explain_outputs: dict[str, Any]) -> dict[str, pd.DataFrame]:
        sheets: dict[str, pd.DataFrame] = {}
        for model_name, payload in explain_outputs.items():
            if model_name == "import_error" or not isinstance(payload, dict):
                continue
            fi = payload.get("feature_importance")
            if isinstance(fi, pd.DataFrame) and not fi.empty:
                sheets[f"Explain_{str(model_name).upper()}_FI"] = fi
            owen = payload.get("owen")
            if isinstance(owen, dict):
                owen_fi = owen.get("feature_importance")
                if isinstance(owen_fi, pd.DataFrame) and not owen_fi.empty:
                    sheets[f"Explain_{str(model_name).upper()}_Owen_FI"] = owen_fi
                owen_grp = owen.get("group_importance")
                if isinstance(owen_grp, pd.DataFrame) and not owen_grp.empty:
                    sheets[f"Explain_{str(model_name).upper()}_Owen_Group"] = owen_grp
        return sheets

    def _run_explainability(
        self,
        model_inputs: dict[str, dict[str, Any]],
        models: dict[str, tuple[Any, Any, list[str]]],
    ) -> dict[str, Any]:
        cfg = self.config
        explain_models = set(str(x).lower() for x in as_list(cfg.explain_models))
        if not explain_models and not cfg.owen_enabled:
            return {}
        outputs: dict[str, Any] = {}
        explain_dir = Path(cfg.output_dir) / "explain"
        try:
            from Modeling_Tool import ModelExplainer
        except Exception as exc:
            return {"import_error": repr(exc)}

        for name, (wrapper, _, feature_cols) in models.items():
            if name not in explain_models and not cfg.owen_enabled:
                continue
            splits = model_inputs[name]["splits"]
            try:
                n_eval = min(int(cfg.explain_params.get("sample_n", 500)), len(splits["oos"]))
                n_bg = min(int(cfg.explain_params.get("background_n", 200)), len(splits["ins"]))
                eval_rows = splits["oos"].sample(n_eval, random_state=cfg.random_state)
                background_rows = splits["ins"].sample(n_bg, random_state=cfg.random_state)
                eval_x = eval_rows[feature_cols]
                background = background_rows[feature_cols]
                if getattr(wrapper, "standardizer", None) is not None:
                    # LRMaster(standardize=True) scales the features before its sklearn model; the explainer unwraps that
                    # model, so it has to see the scaled values or it explains a model that was never scored
                    eval_x = wrapper._apply_standardizer(eval_x)
                    background = wrapper._apply_standardizer(background)
                exp = ModelExplainer(model=wrapper, feature_names=feature_cols, background_data=background)
                item: dict[str, Any] = {}
                if name in explain_models:
                    item["feature_importance"] = exp.feature_importance(X=eval_x, normalize=True)
                    if cfg.write_outputs and cfg.plot_outputs:
                        plot_path = explain_dir / name / "shap_summary.png"
                        plot_path.parent.mkdir(parents=True, exist_ok=True)
                        try:
                            exp.summary_plot(X=eval_x, show=False, save_path=str(plot_path))
                            item["shap_summary"] = str(plot_path)
                        except Exception as plot_exc:
                            item["plot_error"] = repr(plot_exc)
                if cfg.owen_enabled and name != "xgb":
                    owen_exp, owen_x, owen_cols, owen_groups = exp, eval_x, feature_cols, None
                    if (
                        cfg.warm_start_score_scope == "full"
                        and name in {"lgb", "xgb"}
                        and self._warm_start_requested_for(name)
                    ):
                        # explain the scored probability: the prior is one more input, with a group of its own
                        prior_col = _WARM_START_PRIOR_COL
                        owen_x = eval_x.assign(**{prior_col: self._get_warm_start_init_score(name, eval_rows)})
                        owen_bg = background.assign(**{prior_col: self._get_warm_start_init_score(name, background_rows)})
                        owen_cols = list(feature_cols) + [prior_col]
                        owen_exp = ModelExplainer(
                            model=_WarmStartScoredModel(wrapper, feature_cols, prior_col),
                            feature_names=owen_cols,
                            background_data=owen_bg,
                        )
                        owen_groups = {prior_col: [prior_col]}
                    item["owen"] = self._run_owen(owen_exp, owen_x, owen_cols, extra_groups=owen_groups)
                if item:
                    # a model that is neither explained nor Owen-eligible (xgb) used to leave an empty entry
                    outputs[name] = item
            except Exception as exc:
                outputs[name] = {"error": repr(exc)}
        return outputs

    def _run_owen(
        self,
        explainer: Any,
        eval_x: pd.DataFrame,
        feature_cols: list[str],
        extra_groups: dict[str, list[str]] | None = None,
    ) -> dict[str, Any]:
        cfg = self.config
        try:
            from Modeling_Tool.Explainability.Coalition_Structure import build_coalition_structure

            prior_groups = self._filtered_prior_groups(feature_cols)
            if extra_groups:
                prior_groups = {**(prior_groups or {}), **extra_groups}
            coalition_structure = build_coalition_structure(
                eval_x,
                prior_groups=prior_groups,
                threshold=float(cfg.explain_params.get("owen_threshold", 0.35)),
                method=str(cfg.explain_params.get("owen_method", "complete")),
                corr_method=str(cfg.explain_params.get("owen_corr_method", "spearman")),
                min_group_size=int(cfg.explain_params.get("owen_min_group_size", 1)),
                intra_dist=float(cfg.explain_params.get("owen_intra_dist", 0.01)),
                inter_dist=float(cfg.explain_params.get("owen_inter_dist", 0.99)),
            )
            explainer.explain_owen(
                X=eval_x,
                coalition_structure=coalition_structure,
                model_output=str(cfg.explain_params.get("owen_model_output", "probability")),
            )
            return {
                "feature_importance": explainer.owen_feature_importance(normalize=True),
                "group_importance": explainer.owen_group_importance(normalize=True),
            }
        except Exception as exc:
            return {"error": repr(exc)}

    def _filtered_prior_groups(self, feature_cols: list[str]) -> dict[str, list[str]] | None:
        groups = self.config.business_prior_groups or {
            "repayment_capacity": ["income_woe", "employment_months_woe", "loan_amount_woe"],
            "credit_behavior": ["score_b_woe", "overdue_days_max_woe", "mob_on_book_woe"],
            "leverage_risk": ["util_rate_woe", "num_credits_woe"],
            "demographics": ["age_woe", "city_tier_woe"],
        }
        feat_set = set(feature_cols)
        filtered = {group: [feat for feat in feats if feat in feat_set] for group, feats in groups.items()}
        filtered = {group: feats for group, feats in filtered.items() if feats}
        return filtered or None

    def _default_search_spaces(self) -> dict[str, dict[str, dict[str, Any]]]:
        return {
            "lgb": {
                "num_leaves": {"type": "int", "low": 16, "high": 64},
                "max_depth": {"type": "int", "low": 3, "high": 8},
                "learning_rate": {"type": "float", "low": 0.01, "high": 0.1, "log": True},
                "min_child_samples": {"type": "int", "low": 20, "high": 100},
                "subsample": {"type": "float", "low": 0.6, "high": 1.0},
                "colsample_bytree": {"type": "float", "low": 0.6, "high": 1.0},
                "reg_alpha": {"type": "float", "low": 1e-4, "high": 1.0, "log": True},
                "reg_lambda": {"type": "float", "low": 1e-4, "high": 5.0, "log": True},
            },
            "xgb": {
                "max_depth": {"type": "int", "low": 3, "high": 7},
                "learning_rate": {"type": "float", "low": 0.01, "high": 0.1, "log": True},
                "min_child_weight": {"type": "int", "low": 5, "high": 50},
                "subsample": {"type": "float", "low": 0.6, "high": 1.0},
                "colsample_bytree": {"type": "float", "low": 0.6, "high": 1.0},
                "reg_alpha": {"type": "float", "low": 1e-4, "high": 1.0, "log": True},
                "reg_lambda": {"type": "float", "low": 1e-4, "high": 5.0, "log": True},
            },
            "cat": {
                "depth": {"type": "int", "low": 3, "high": 6},
                "learning_rate": {"type": "float", "low": 0.01, "high": 0.1, "log": True},
                "l2_leaf_reg": {"type": "float", "low": 1.0, "high": 10.0},
            },
        }

    def _model_output_dir(self) -> Path:
        cfg = self.config
        return Path(cfg.model_output_dir) if cfg.model_output_dir else Path(cfg.output_dir) / "models"

    def _save_models_and_artifacts(
        self,
        models: dict[str, tuple[Any, Any, list[str]]],
        woe_artifacts: dict[str, Any],
        perf_results: dict[str, pd.DataFrame],
        model_feature_sources: dict[str, str],
        model_feature_sets: dict[str, list[str]],
    ) -> tuple[dict[str, str], dict[str, str]]:
        from Modeling_Tool import save_model

        cfg = self.config
        model_dir = self._model_output_dir()
        artifact_dir = Path(cfg.output_dir) / "artifacts"
        make_dirs(model_dir)
        model_paths: dict[str, str] = {}
        artifact_paths: dict[str, str] = {}

        woe_table_path = None
        if cfg.save_woe_artifacts:
            make_dirs(artifact_dir)
            woe_table = woe_artifacts.get("woe_table")
            if isinstance(woe_table, pd.DataFrame):
                woe_table_path = artifact_dir / "woe_table.csv"
                safe_to_csv(woe_table, woe_table_path, index=False)
                artifact_paths["woe_table"] = str(woe_table_path.resolve())
            engine = woe_artifacts.get("engine")
            if engine is not None:
                engine_path = artifact_dir / "woe_engine.pkl"
                save_model(
                    engine,
                    engine_path,
                    metadata={
                        "pipeline": "CreditModelPipeline",
                        "artifact_role": "woe_engine",
                        "target_col": cfg.target_col,
                        "raw_features": list(woe_artifacts.get("features") or []),
                        "woe_features": list(woe_artifacts.get("woe_features") or []),
                        "woe_engine": woe_artifacts.get("engine_name"),
                        "woe_suffix": woe_artifacts.get("woe_suffix"),
                        "random_state": cfg.random_state,
                    },
                    feature_cols=woe_artifacts.get("features"),
                    include_metadata=cfg.model_include_metadata,
                )
                artifact_paths["woe_engine"] = str(engine_path.resolve())

        warm_start_requested = {str(x).lower() for x in as_list(cfg.warm_start_models)}
        for name, (wrapper, _, feature_cols) in models.items():
            path = model_dir / f"model_{name}.pkl"
            metadata = {
                "pipeline": "CreditModelPipeline",
                "model_name": name,
                "target_col": cfg.target_col,
                "feature_cols": list(feature_cols),
                "feature_source": model_feature_sources.get(name),
                "model_feature_set": model_feature_sets.get(name, list(feature_cols)),
                # what the model was built with, not only the overrides of the user: the defaults, the best row of the LR
                # search and the pipeline seed used to be missing, and a search result contradicted the saved value
                "model_params": self._effective_model_params(name),
                # CatBoost cannot take an init score and an unknown name is skipped: neither is warm-started
                "warm_start_enabled": bool(
                    cfg.warm_start_enabled and name in warm_start_requested and name in {"lgb", "xgb"}
                ),
                "warm_start_score_col": cfg.warm_start_score_col,
                "warm_start_score_type": cfg.warm_start_score_type,
                "random_state": self._effective_seed(name),
                "woe_suffix": woe_artifacts.get("woe_suffix"),
                "candidate_mode": self._governance["candidate_mode"],
                "oot_synthesized": self._governance["oot_synthesized"],
                "oot_withheld": self._governance["oot_withheld"],
                "synthesize_missing_oot": self._governance["synthesize_missing_oot"],
                "evaluation_splits": self._governance["evaluation_splits"],
                "forbidden_splits": self._governance["forbidden_splits"],
                "search_eval_splits": self._governance["search_eval_splits"],
                "backward_validation_split": self._governance["backward_validation_split"],
                "backward_report_splits": self._governance["backward_report_splits"],
                "eval_target_cols": list(cfg.eval_target_cols) if cfg.eval_target_cols else None,
                "all_missing_score_value": cfg.all_missing_score_value,
                "all_missing_raw_features": (
                    self._raw_features_for_saved_model(name, model_feature_sources, model_feature_sets, woe_artifacts)
                    if cfg.all_missing_score_value is not None
                    else None
                ),
                "special_score_values": list(cfg.special_score_values) if cfg.special_score_values else None,
                "gains_ascending": cfg.gains_ascending,
            }
            metrics = None
            if isinstance(perf_results.get(name), pd.DataFrame):
                metrics = {"perf_results": perf_results[name].to_dict("records")}
            save_model(
                wrapper,
                path,
                metadata=metadata,
                feature_cols=feature_cols,
                woe_mapping_path=str(woe_table_path.resolve()) if woe_table_path else None,
                metrics=metrics,
                model_name=name,
                include_metadata=cfg.model_include_metadata,
            )
            model_paths[name] = str(path.resolve())
        return model_paths, artifact_paths

    @staticmethod
    def _paths_to_frame(model_paths: dict[str, str], artifact_paths: dict[str, str]) -> pd.DataFrame:
        rows = [{"name": name, "path": path, "kind": "model"} for name, path in model_paths.items()]
        rows.extend({"name": name, "path": path, "kind": "artifact"} for name, path in artifact_paths.items())
        return pd.DataFrame(rows)

    def _write_outputs(
        self,
        output_dir: Path,
        fs_summary: dict[str, Any],
        woe_artifacts: dict[str, Any],
        backward_summary: pd.DataFrame | None,
        optuna_results: dict[str, pd.DataFrame],
        perf_results: dict[str, pd.DataFrame],
        lr_search_results: pd.DataFrame | None,
        warm_start_summary: pd.DataFrame | None,
        model_feature_source_summary: pd.DataFrame | None,
        model_paths_frame: pd.DataFrame | None,
    ) -> None:
        if isinstance(fs_summary.get("psi"), pd.DataFrame):
            safe_to_csv(fs_summary["psi"], output_dir / "psi_result.csv", index=False)
        if isinstance(fs_summary.get("iv"), pd.DataFrame):
            safe_to_csv(fs_summary["iv"], output_dir / "iv_report.csv", index=False)
        if isinstance(fs_summary.get("iv_sv_pure_bins"), pd.DataFrame):
            safe_to_csv(fs_summary["iv_sv_pure_bins"], output_dir / "iv_sv_pure_bins.csv", index=False)
        if isinstance(fs_summary.get("dropped_detail"), pd.DataFrame):
            safe_to_csv(fs_summary["dropped_detail"], output_dir / "feature_selection_dropped.csv", index=False)
        if isinstance(fs_summary.get("lr_elimination"), pd.DataFrame):
            safe_to_csv(fs_summary["lr_elimination"], output_dir / "lr_pvalue_elimination.csv", index=False)
        safe_to_csv(woe_artifacts.get("woe_table"), output_dir / "woe_table_ins.csv", index=False)
        safe_to_csv(backward_summary, output_dir / "backward_summary.csv", index=False)
        safe_to_csv(lr_search_results, output_dir / "lr_param_search.csv", index=False)
        safe_to_csv(warm_start_summary, output_dir / "warm_start_summary.csv", index=False)
        safe_to_csv(model_feature_source_summary, output_dir / "model_feature_sources.csv", index=False)
        safe_to_csv(model_paths_frame, output_dir / "model_paths.csv", index=False)
        for name, df in optuna_results.items():
            safe_to_csv(df, output_dir / f"{name}_optuna_search.csv", index=False)
        for name, df in perf_results.items():
            safe_to_csv(df, output_dir / "perf" / f"perf_{name}.csv", index=False)

    # an Excel cell holds 32,767 characters; a longer text was cut off without a message
    _EXCEL_CELL_CHARS = 30000

    def _summary_to_frame(self, summary: dict[str, Any]) -> pd.DataFrame:
        rows = []
        for key, value in summary.items():
            if isinstance(value, pd.DataFrame):
                rows.append({"item": key, "value": f"DataFrame{value.shape} (sheet FS_{key})"})
                continue
            text = str(value)
            if len(text) <= self._EXCEL_CELL_CHARS:
                rows.append({"item": key, "value": text})
                continue
            parts = [text[i : i + self._EXCEL_CELL_CHARS] for i in range(0, len(text), self._EXCEL_CELL_CHARS)]
            rows.extend({"item": f"{key} (part {n}/{len(parts)})", "value": part} for n, part in enumerate(parts, 1))
        return pd.DataFrame(rows)

    @classmethod
    def _split_long_cells(cls, frame: pd.DataFrame | None, column: str) -> pd.DataFrame | None:
        """Copy of ``frame`` in which a ``column`` text longer than an Excel cell is continued on extra rows."""
        if frame is None or column not in frame.columns:
            return frame
        limit = cls._EXCEL_CELL_CHARS
        rows = []
        for record in frame.to_dict("records"):
            text = str(record[column])
            if len(text) <= limit:
                rows.append(record)
                continue
            parts = [text[i : i + limit] for i in range(0, len(text), limit)]
            rows.append({**record, column: parts[0]})
            rows.extend({key: ("" if key != column else part) for key in record} for part in parts[1:])
        return pd.DataFrame(rows, columns=list(frame.columns))

    @staticmethod
    def _summary_tables(summary: dict[str, Any]) -> dict[str, pd.DataFrame]:
        """The tables of the feature selection summary, one report sheet each (they used to show only as their shape)."""
        return {f"FS_{key}": value for key, value in summary.items() if isinstance(value, pd.DataFrame)}

run

run(data: DataFrame) -> CreditModelPipelineResult

Run the whole workflow on one dataset and return everything it produced.

The steps are the INS/OOS/OOT split, the feature screening, the WOE fit and transform, the backward elimination, the LR search, the model training, the Optuna search, the evaluation, the explanations, the model saving, the CSV output and the Excel report; the settings of CreditModelPipelineConfig switch them on or off.

Parameters:

Name	Type	Description	Default
data	DataFrame	Modeling data with target_col, the feature columns (feature_cols, or every numeric column when it is None) and, when configured, the split, OOT flag, weight, warm-start score and evaluation columns. It is not modified.	required

Returns:

Type	Description
CreditModelPipelineResult	The splits, the feature selection summary, the WOE artifacts, the trained models, the selected features and the optional search, performance, explanation and path outputs.

Raises:

Type	Description
KeyError	If a required column is missing from data: the target, a feature, the weight or evaluation-weight column, the split column, the warm-start score, an evaluation target, or a column used by woe_fit_query.
ValueError	If the data or the settings are invalid: weights, sample labels, woe_fit_query, lr_elimination_mode, the keys of lr_search_params or lr_elimination_params, unsupported model names, an unusable search or evaluation split, or a request for a forbidden split.
TypeError	If oot_col is used for the split and is not numeric.
NotImplementedError	If "cat" is requested for warm start and warm_start_on_unsupported is "raise".

Notes

The failures that the feature selection, the backward elimination, each Optuna search and the explanations catch are not raised; they are recorded in the result (see the class notes).

Source code in Modeling_Tool/Pipeline/credit_model.py

def run(self, data: pd.DataFrame) -> CreditModelPipelineResult:
    """Run the whole workflow on one dataset and return everything it produced.

    The steps are the INS/OOS/OOT split, the feature screening, the WOE fit and transform, the backward
    elimination, the LR search, the model training, the Optuna search, the evaluation, the explanations, the model
    saving, the CSV output and the Excel report; the settings of ``CreditModelPipelineConfig`` switch them on or off.

    Parameters
    ----------
    data : pandas.DataFrame
        Modeling data with ``target_col``, the feature columns (``feature_cols``, or every numeric column when it is
        ``None``) and, when configured, the split, OOT flag, weight, warm-start score and evaluation columns. It is
        not modified.

    Returns
    -------
    CreditModelPipelineResult
        The splits, the feature selection summary, the WOE artifacts, the trained models, the selected features and
        the optional search, performance, explanation and path outputs.

    Raises
    ------
    KeyError
        If a required column is missing from ``data``: the target, a feature, the weight or evaluation-weight column,
        the split column, the warm-start score, an evaluation target, or a column used by ``woe_fit_query``.
    ValueError
        If the data or the settings are invalid: weights, sample labels, ``woe_fit_query``, ``lr_elimination_mode``,
        the keys of ``lr_search_params`` or ``lr_elimination_params``, unsupported model names, an unusable search
        or evaluation split, or a request for a forbidden split.
    TypeError
        If ``oot_col`` is used for the split and is not numeric.
    NotImplementedError
        If ``"cat"`` is requested for warm start and ``warm_start_on_unsupported`` is ``"raise"``.

    Notes
    -----
    The failures that the feature selection, the backward elimination, each Optuna search and the explanations catch
    are not raised; they are recorded in the result (see the class notes).
    """
    cfg = self.config
    manifest = None
    if not getattr(self, "_nested_run", False) and (cfg.write_outputs or cfg.write_excel or cfg.save_models):
        manifest = OutputManifest.begin(cfg.output_dir, "credit_model", clean=cfg.clean_output_dir)
    try:
        result = self._run(data)
    except BaseException:
        if manifest is not None:
            try:
                manifest.finish(clean=False)  # list what the failed run wrote; remove nothing
            except Exception:
                pass
        raise
    if manifest is not None:
        removed = manifest.finish()
        if removed:
            _logger.info("CreditModelPipeline: clean_output_dir removed %d stale file(s): %s", len(removed), removed)
    return result

Feature Validation Pipeline: feature_validation

feature_validation

FeatureValidationPipelineConfig dataclass

Configuration of :class:FeatureValidationPipeline.

Parameters:

Name	Type	Description	Default
output_dir	str	Output directory. CSV tables and the Excel report are written directly into it, WOE plots to figs/woe/<target> and batch results to <batch_output_subdir>/batch_NNN. A run overwrites the files it writes and leaves the others alone; every run that writes files lists them in output_dir/.smf_manifest_feature_validation.json.	'output/feature_validation'
clean_output_dir	bool	After a successful run, remove the files that the previous manifest of this pipeline lists and this run did not write again (tables of stages that no longer run, batch folders of a former batch layout), and the directories they leave empty. Only files of the manifest are removed: files written by hand, by another pipeline or by a version without the manifest stay.	False
id_col	str	Unique row identifier column. It must exist in the data (KeyError) and is never treated as a feature.	'flow_id'
apply_time_col	str	Application time column; it must exist in the data. When time_dims contains apply_week, apply_month or apply_quarter and that column is absent, it is derived from this column (as period strings).	'apply_time'
target_cols	list of str or None	Target columns (1 = bad). Names that are not columns of the data are ignored silently. WOE, IV/KS, correlation detail and selection need at least one target; the first target drives the stratified INS/OOS split and the selection.	None
new_feature_cols	list of str or None	Features to validate. None uses every numeric column except the id, application time, sample_col, split_col, oot_col, target and incumbent columns. The list must not be empty (ValueError) and every name must exist in the data (KeyError).	None
incumbent_feature_cols	list of str or None	Existing features to compare with the new ones in the correlation analysis. Names that are also new features are ignored. They must exist in the data.	None
input_type	('auto', 'dataframe', 'csv')	How run interprets its argument. 'auto' treats a DataFrame as data and anything else as a CSV path; 'dataframe' requires a DataFrame (TypeError otherwise); 'csv' reads a file path. Any other value raises ValueError.	'auto'
csv_read_kwargs	dict	Extra keyword arguments for pandas.read_csv in CSV mode. usecols and chunksize are controlled by the pipeline and raise ValueError.	{}
enable_batch	bool	Process a CSV in batches of feature columns to limit memory. It requires CSV input, and feature_batch_size or feature_batches (ValueError otherwise). If batch settings are given while this is False, the CSV is read in full and a RuntimeWarning is issued. With selection_enabled the selection runs in two passes: each batch applies only the stages that judge a feature on its own (missing rate, PSI, IV and the group and target gates), then the candidates of all batches are read together and the full selection (WOE fit, correlation across batches, VIF, max_selected_features / min_selected_features) runs once on them, as in a run without batches. Only the candidates are held in memory at once in that pass; selection_summary['batch_candidates'] lists them. batch_corr_mode='off' also switches off the correlation stage of the selection unless selection_params['corr_enabled'] is set. A batch that fails is recorded in batch_metadata with its error and warned about; its features are left out of every output and counted in n_failed_features.	False
feature_batch_size	int or None	Number of new features per batch; must be positive.	None
feature_batches	list of list of str or None	Explicit feature batches; they take precedence over feature_batch_size. Unknown features raise ValueError and features that are not listed are collected in an extra last batch.	None
batch_base_cols	list of str or None	Extra columns read with every batch. The id, application time, split, sample, OOT, target, time, population, categorical, WOE-plot-group and incumbent columns that exist in the CSV are always read.	None
batch_output_subdir	str	Sub-directory of output_dir for the per-batch outputs (batch_000, batch_001, ...).	'feature_batches'
batch_keep_intermediate	bool	Whether each batch writes its own CSV tables and plots (subject to write_outputs). The merged report is not affected, and batches never write an Excel report.	True
batch_corr_mode	('within_batch', 'block_pairwise', 'off')	Correlation report in batch mode. 'within_batch' correlates features only inside each batch, 'block_pairwise' also reads pairs of batches to find cross-batch pairs above corr_params['corr_cutpoint'] (Pearson or Spearman only), 'off' skips correlation. The correlation stage of the selection compares the candidates of all batches in either of the first two modes and is skipped with 'off' (an explicit selection_params['corr_enabled'] wins). Other values raise ValueError.	'within_batch'
batch_corr_pair_chunk_size	int or None	With 'block_pairwise', maximum number of feature columns per chunk when pairs of batches are read, to cap memory; must be positive. None reads a whole batch at once.	None
split_col	str or None	Column with sample labels (case and surrounding spaces ignored). It must contain non-empty ins and oos values (ValueError); oot is optional and any other label becomes an additional named split. It takes precedence over sample_col.	None
sample_col	str	Legacy label column, used when split_col is None and the column exists with non-empty ins and oos values; otherwise the data are split with oot_col and split_config. Never treated as a feature.	'sample_ind'
oot_col	str or None	Numeric OOT flag column of the fallback split: rows with 0 or a missing value are INS/OOS candidates and non-zero rows are OOT. A non-numeric flag raises TypeError. Not used when labels define the splits.	'oot_flag'
split_config	dict	Settings of the random INS/OOS split: test_size (OOS share), stratify (stratify on the first target; rows with a missing target then go to INS) and random_state (defaults to random_state).	{'test_size': 0.3, 'stratify': True}
random_state	int	Seed of the random INS/OOS split unless split_config contains its own random_state.	42
time_dims	list of str	Time columns for the grouped analyses (distribution, PSI, IV/KS). apply_week, apply_month and apply_quarter are derived from apply_time_col when missing; columns that are still absent are skipped.	['apply_month']
population_dims	list of str	Population columns (for example channel) for the grouped analyses; columns absent from the data are skipped.	[]
group_specs	dict or list or None	Custom groups for the distribution tables only: {name: [columns]} or a list of column lists or dicts with columns and name. None uses global, time, population and time x population groups. PSI and IV/KS use psi_group_dims and ivks_group_dims instead.	None
min_group_size	int	Minimum row count. A target needs at least max(10, min_group_size) INS rows for the WOE fit and at least min_group_size observed rows for IV/KS and the correlation detail, and smaller IV/KS groups are skipped. It is also the default min_group_n of the selection gates.	100
distribution_enabled	bool	Whether to build distribution_summary.	True
distribution_params	dict	Keys: q (quantiles of the numeric summary), spec_missing_value (value counted as missing, default None) and feature_block_size (columns per block, default 128).	{'q': [0.05, 0.15, 0.25, 0.5, 0.75, 0.95, 0.99]}
woe_enabled	bool	Whether to fit WOE binning per target on the INS rows with an observed target. It needs a target. Without it PSI, IV/KS and correlation use their own binning.	True
woe_fit_query	str or None	pandas query expression that selects the INS rows used to fit the WOE bins; PSI, IV/KS and the transforms still use the full splits. It is validated up front (KeyError for unknown columns, ValueError for invalid syntax) and the filtering is recorded in woe_artifacts['refine_summary'].	None
woe_engine	str	'monotone' (MonotoneWOEBinner) or 'equal_freq' (WOE_Master), case-insensitive. Anything else raises ValueError.	'monotone'
woe_params	dict	Used with woe_engine='equal_freq': woe_suffix ('_woe') and missing_ref_value (-999999) configure WOE_Master and every other key is passed to WOE_Master.fit. missing_ref_value is also read as the missing-value sentinel of the missing-rate gate and the selection stage, and the whole dict is forwarded to the selection stage.	{'nbins': 10, 'equal_freq': True, 'min_bin_prop': 0.05, 'sv_min_bin_size': 0.0, 'sv_small_policy': 'keep', 'sv_woe_smoothing': 'none', 'sv_smoothing_alpha': 0.0}
monotone_woe_params	dict	Used with woe_engine='monotone': constructor keys of MonotoneWOEBinner and the fit keys chi2_binning, chi2_p, chi2_init_size and n_jobs. Keys outside the built-in allowlist are dropped silently. Without a special_values key the legacy sentinel -999999 is declared a special value when a numeric feature holds it in the fit sample (CreditModelPipeline does the same, so a screening artifact and a self-fit bin the same data the same way).	{'n_init_bins': 20, 'min_bin_size': 0.03, 'min_n_bins': 2, 'sv_min_bin_size': 0.0, 'sv_small_policy': 'keep', 'sv_woe_smoothing': 'none', 'sv_smoothing_alpha': 0.0, 'unseen_special_policy': 'neutral'}
categorical_features	list of str or None	Features binned as categorical by the monotone engine (cate_feats); also passed to the selection stage.	None
monotone_refine_cate_enabled	bool	Whether to call refine_cate on the fitted monotone binner (monotone engine only).	False
monotone_refine_cate_params	dict	Keyword arguments of refine_cate; features defaults to the categorical features.	{}
monotone_refine_dtree_enabled	bool	Whether to call refine_dtree on the fitted monotone binner (monotone engine only).	False
monotone_refine_dtree_params	dict	Keyword arguments of refine_dtree; features defaults to the numeric features.	{}
monotone_refine_chi2_enabled	bool	Whether to call refine_chi2 on the fitted monotone binner (monotone engine only).	False
monotone_refine_chi2_params	dict	Keyword arguments of refine_chi2; features defaults to the numeric features.	{}
woe_plot_groups	list of str	Group columns for extra WOE plots by group (figs/woe/<target>/by_<group>). Plots are written only while write_outputs and plot_outputs are both True.	[]
psi_enabled	bool	Whether to compute psi_summary and psi_details.	True
psi_reference_dataset	str	Reference sample of the PSI: 'ins', 'oos', 'oot' or 'external' (ValueError otherwise). The PSI compares all splits combined with it.	'ins'
psi_reference_data	DataFrame or None	Reference data for psi_reference_dataset='external' (required then, otherwise ValueError).	None
psi_group_dims	list of str	Grouping of the PSI: 'sample' is the split, 'time' and 'population' expand to time_dims and population_dims, other names are column names. An overall PSI is computed only when no group column remains ('global' adds nothing by itself).	['sample', 'time', 'population']
psi_use_woe_bins	bool	Whether to bin with the WOE engine of each target when available. Otherwise PSICalculator bins the data with psi_params; that numeric PSI skips the non-numeric features with a UserWarning.	True
psi_params	dict	Keyword arguments of PSICalculator.	{'buckets': 10, 'equal_freq': True, 'min_bin_prop': 0.05}
ivks_enabled	bool	Whether to compute ivks_summary. It needs a target.	True
ivks_group_dims	list of str	Groupings of the IV/KS report: 'global' for all rows, 'time' and 'population' for each column of time_dims and population_dims, other names as single group columns.	['global', 'time', 'population']
ivks_use_woe_bins	bool	Whether to compute IV/KS on the bins of the WOE engine of each target when available.	True
ivks_params	dict	iv_cut (minimum IV of the reported features), feature_block_size (columns per block with WOE bins, default 64), missing_rate_ref (the sentinel counted as missing, default woe_params['missing_ref_value']) and, without WOE bins, other keyword arguments of VarExtractionInsights.	{'iv_cut': 0.0}
corr_enabled	bool	Whether to compute corr_matrix, high_corr_pairs and correlated_detail.	True
corr_include_incumbent	bool	Whether incumbent features are included in the correlation analysis (and, with corr_use_woe_bins, in the WOE fit).	True
corr_use_woe_bins	bool	Whether CorrelationFilter ranks correlated features with IV from the WOE bins of the target.	True
corr_params	dict	corr_cutpoint (pairs with a larger absolute correlation are flagged), method ('pearson', 'spearman' or 'kendall'), max_iterations of the removal loop and other CorrelationFilter arguments such as base_metric.	{'corr_cutpoint': 0.75, 'method': 'pearson', 'max_iterations': 10, 'base_metric': 'iv'}
missing_rate_threshold	float or None	Maximum missing rate of a feature on INS (NaN or the missing_ref_value sentinel). With woe_fit_scope='post_missing_gate' features above it are removed before the WOE fit and from all later stages (they are listed in woe_artifacts['missing_gate_dropped']). It is also the default threshold of the selection stage. None disables the gate.	None
selection_enabled	bool	Whether to run the feature screening (missing rate, PSI, IV and correlation stages and optional gates) on the first target. It fills selected_features, selection_summary and screening_artifact and needs a target.	False
selection_params	dict	Screening settings read by the pipeline; other keys are ignored. Thresholds: psi_threshold (0.2), psi_compare_splits (['oos']), iv_threshold (0.02), iv_upper_threshold (2.0 since 0.9.3; None switches the leakage limit off), iv_gap_ratio_warn (2.0) and iv_gap_abs_warn (0.5) of the leak warning, corr_threshold (or corr_cutpoint), missing_rate_threshold, max_selected_features, min_selected_features. Stability gates: monthly_iv_min, monthly_iv_cv_max, direction_consistency_min, min_group_n, insufficient_group_policy, target_rules, min_pass_count, per_target_iv_range, direction_reference_target. VIF: vif_enabled, vif_threshold, vif_min_features, vif_tie_break_metric, vif_use_woe_bins. Others: ranking_metric, tie_breaker, missing_rate_ref, on_empty_stage ('keep_all_warn', 'keep_all' or 'raise'; see FeatureScreenConfig.on_empty_stage: since 0.9.3 the default never brings back a leak-gate drop), fallback_top_k (5), corr_method and corr_base_metric (default to corr_params['method'] and ['base_metric']) and the stage switches and binning options derived from the other config fields (for example psi_enabled, iv_nbins, corr_max_iterations). psi_compare_splits ignores case and spaces and accepts a bare string.	{}
selection_group_dims	list of str or None	Columns of the group-stability gates (monthly_iv_min, monthly_iv_cv_max, direction_consistency_min). They are required when one of those gates is set (ValueError) and must exist in the INS split (KeyError).	None
weight_col	str or None	Sample-weight column of the weighted selection and its IV/direction evidence; it must exist in the data. The WOE bins are still fitted without weights.	None
synthesize_missing_oot	bool or None	When no OOT rows exist, True copies the OOS rows in as a stand-in OOT (with a UserWarning); False keeps OOT empty. None counts as False. The stand-in rows feed only what is reported per split (the oot group of the PSI by sample, the OOT comparisons of the selection); the tables that pool every split (the distribution, n_rows of the summary, the global and grouped PSI and IV/KS, the correlation) count each row once. The stand-in frame in result.splits['oot'] carries attrs['smf_stand_in_oot'] = True.	False
woe_fit_scope	('all', 'post_missing_gate')	'all' fits the WOE on every new feature. 'post_missing_gate' first drops the features above missing_rate_threshold (a no-op when it is None or there is no target) and continues with the remaining ones.	'all'
write_outputs	bool	Whether to write the result tables as CSV files in output_dir; it is also the master switch for the WOE plots.	True
write_excel	bool	Whether to write Feature_Validation_Report.xlsx with the non-empty tables.	True
plot_outputs	bool	Whether to write the WOE plots to figs/woe; they are written only when write_outputs is True as well.	True
screen_iv_convention	('guarded', 'raw')	How the IV on the WOE bins treats a class-pure bin (no bad or no good row), whose WOE comes from the 1e-6 of ln((bad_share + 1e-6) / (good_share + 1e-6)) alone: "guarded" leaves it out of the IV sum (its rows stay in the totals, so the other bins keep their WOE and IV), "raw" counts every bin (0.9.1). It applies to ivks_summary (per group, on the group's counts), to the selection stage (see FeatureScreenConfig.screen_iv_convention), to its group and per-target evidence, and to the IV that CorrelationFilter ranks correlated features by. ivks_summary reports iv with iv_raw, iv_sv_pure and n_sv_pure; ivks_sv_pure_bins of the result lists the class-pure special-value bins. selection_params['screen_iv_convention'] and corr_params['screen_iv_convention'] are accepted as well; a value there that differs from this field raises ValueError when run starts, as does any other value. The WOE of the bins is not changed.	"guarded"

Source code in Modeling_Tool/Pipeline/feature_validation.py

@dataclass
class FeatureValidationPipelineConfig:
    """Configuration of :class:`FeatureValidationPipeline`.

    Parameters
    ----------
    output_dir : str, default 'output/feature_validation'
        Output directory. CSV tables and the Excel report are written directly into it, WOE plots to
        ``figs/woe/<target>`` and batch results to ``<batch_output_subdir>/batch_NNN``. A run overwrites the files it
        writes and leaves the others alone; every run that writes files lists them in
        ``output_dir/.smf_manifest_feature_validation.json``.
    clean_output_dir : bool, default False
        After a successful run, remove the files that the previous manifest of this pipeline lists and this run did not
        write again (tables of stages that no longer run, batch folders of a former batch layout), and the directories
        they leave empty. Only files of the manifest are removed: files written by hand, by another pipeline or by a
        version without the manifest stay.
    id_col : str, default 'flow_id'
        Unique row identifier column. It must exist in the data (``KeyError``) and is never treated as a feature.
    apply_time_col : str, default 'apply_time'
        Application time column; it must exist in the data. When ``time_dims`` contains ``apply_week``, ``apply_month``
        or ``apply_quarter`` and that column is absent, it is derived from this column (as period strings).
    target_cols : list of str or None, default None
        Target columns (1 = bad). Names that are not columns of the data are ignored silently. WOE, IV/KS, correlation
        detail and selection need at least one target; the first target drives the stratified INS/OOS split and the
        selection.
    new_feature_cols : list of str or None, default None
        Features to validate. ``None`` uses every numeric column except the id, application time, ``sample_col``,
        ``split_col``, ``oot_col``, target and incumbent columns. The list must not be empty (``ValueError``) and every
        name must exist in the data (``KeyError``).
    incumbent_feature_cols : list of str or None, default None
        Existing features to compare with the new ones in the correlation analysis. Names that are also new features are
        ignored. They must exist in the data.
    input_type : {'auto', 'dataframe', 'csv'}, default 'auto'
        How ``run`` interprets its argument. ``'auto'`` treats a DataFrame as data and anything else as a CSV path;
        ``'dataframe'`` requires a DataFrame (``TypeError`` otherwise); ``'csv'`` reads a file path. Any other value
        raises ``ValueError``.
    csv_read_kwargs : dict, default {}
        Extra keyword arguments for ``pandas.read_csv`` in CSV mode. ``usecols`` and ``chunksize`` are controlled by the
        pipeline and raise ``ValueError``.
    enable_batch : bool, default False
        Process a CSV in batches of feature columns to limit memory. It requires CSV input, and ``feature_batch_size`` or
        ``feature_batches`` (``ValueError`` otherwise). If batch settings are given while this is False, the CSV is read
        in full and a ``RuntimeWarning`` is issued. With ``selection_enabled`` the selection runs in two passes: each
        batch applies only the stages that judge a feature on its own (missing rate, PSI, IV and the group and target
        gates), then the candidates of all batches are read together and the full selection (WOE fit, correlation across
        batches, VIF, ``max_selected_features`` / ``min_selected_features``) runs once on them, as in a run without
        batches. Only the candidates are held in memory at once in that pass; ``selection_summary['batch_candidates']``
        lists them. ``batch_corr_mode='off'`` also switches off the correlation stage of the selection unless
        ``selection_params['corr_enabled']`` is set. A batch that fails is recorded in ``batch_metadata`` with its error
        and warned about; its features are left out of every output and counted in ``n_failed_features``.
    feature_batch_size : int or None, default None
        Number of new features per batch; must be positive.
    feature_batches : list of list of str or None, default None
        Explicit feature batches; they take precedence over ``feature_batch_size``. Unknown features raise
        ``ValueError`` and features that are not listed are collected in an extra last batch.
    batch_base_cols : list of str or None, default None
        Extra columns read with every batch. The id, application time, split, sample, OOT, target, time, population,
        categorical, WOE-plot-group and incumbent columns that exist in the CSV are always read.
    batch_output_subdir : str, default 'feature_batches'
        Sub-directory of ``output_dir`` for the per-batch outputs (``batch_000``, ``batch_001``, ...).
    batch_keep_intermediate : bool, default True
        Whether each batch writes its own CSV tables and plots (subject to ``write_outputs``). The merged report is not
        affected, and batches never write an Excel report.
    batch_corr_mode : {'within_batch', 'block_pairwise', 'off'}, default 'within_batch'
        Correlation report in batch mode. ``'within_batch'`` correlates features only inside each batch,
        ``'block_pairwise'`` also reads pairs of batches to find cross-batch pairs above
        ``corr_params['corr_cutpoint']`` (Pearson or Spearman only), ``'off'`` skips correlation. The correlation stage
        of the selection compares the candidates of all batches in either of the first two modes and is skipped with
        ``'off'`` (an explicit ``selection_params['corr_enabled']`` wins). Other values raise ``ValueError``.
    batch_corr_pair_chunk_size : int or None, default None
        With ``'block_pairwise'``, maximum number of feature columns per chunk when pairs of batches are read, to cap
        memory; must be positive. ``None`` reads a whole batch at once.
    split_col : str or None, default None
        Column with sample labels (case and surrounding spaces ignored). It must contain non-empty ``ins`` and ``oos``
        values (``ValueError``); ``oot`` is optional and any other label becomes an additional named split. It takes
        precedence over ``sample_col``.
    sample_col : str, default 'sample_ind'
        Legacy label column, used when ``split_col`` is None and the column exists with non-empty ``ins`` and ``oos``
        values; otherwise the data are split with ``oot_col`` and ``split_config``. Never treated as a feature.
    oot_col : str or None, default 'oot_flag'
        Numeric OOT flag column of the fallback split: rows with 0 or a missing value are INS/OOS candidates and non-zero
        rows are OOT. A non-numeric flag raises ``TypeError``. Not used when labels define the splits.
    split_config : dict, default {'test_size': 0.3, 'stratify': True}
        Settings of the random INS/OOS split: ``test_size`` (OOS share), ``stratify`` (stratify on the first target;
        rows with a missing target then go to INS) and ``random_state`` (defaults to ``random_state``).
    random_state : int, default 42
        Seed of the random INS/OOS split unless ``split_config`` contains its own ``random_state``.
    time_dims : list of str, default ['apply_month']
        Time columns for the grouped analyses (distribution, PSI, IV/KS). ``apply_week``, ``apply_month`` and
        ``apply_quarter`` are derived from ``apply_time_col`` when missing; columns that are still absent are skipped.
    population_dims : list of str, default []
        Population columns (for example channel) for the grouped analyses; columns absent from the data are skipped.
    group_specs : dict or list or None, default None
        Custom groups for the distribution tables only: ``{name: [columns]}`` or a list of column lists or dicts with
        ``columns`` and ``name``. ``None`` uses global, time, population and time x population groups. PSI and IV/KS use
        ``psi_group_dims`` and ``ivks_group_dims`` instead.
    min_group_size : int, default 100
        Minimum row count. A target needs at least ``max(10, min_group_size)`` INS rows for the WOE fit and at least
        ``min_group_size`` observed rows for IV/KS and the correlation detail, and smaller IV/KS groups are skipped. It
        is also the default ``min_group_n`` of the selection gates.
    distribution_enabled : bool, default True
        Whether to build ``distribution_summary``.
    distribution_params : dict, default {'q': [0.05, 0.15, 0.25, 0.5, 0.75, 0.95, 0.99]}
        Keys: ``q`` (quantiles of the numeric summary), ``spec_missing_value`` (value counted as missing, default None)
        and ``feature_block_size`` (columns per block, default 128).
    woe_enabled : bool, default True
        Whether to fit WOE binning per target on the INS rows with an observed target. It needs a target. Without it
        PSI, IV/KS and correlation use their own binning.
    woe_fit_query : str or None, default None
        pandas ``query`` expression that selects the INS rows used to fit the WOE bins; PSI, IV/KS and the transforms
        still use the full splits. It is validated up front (``KeyError`` for unknown columns, ``ValueError`` for invalid
        syntax) and the filtering is recorded in ``woe_artifacts['refine_summary']``.
    woe_engine : str, default 'monotone'
        ``'monotone'`` (``MonotoneWOEBinner``) or ``'equal_freq'`` (``WOE_Master``), case-insensitive. Anything else
        raises ``ValueError``.
    woe_params : dict, default {'nbins': 10, 'equal_freq': True, 'min_bin_prop': 0.05, 'sv_min_bin_size': 0.0, 'sv_small_policy': 'keep', 'sv_woe_smoothing': 'none', 'sv_smoothing_alpha': 0.0}
        Used with ``woe_engine='equal_freq'``: ``woe_suffix`` ('_woe') and ``missing_ref_value`` (-999999) configure
        ``WOE_Master`` and every other key is passed to ``WOE_Master.fit``. ``missing_ref_value`` is also read as the
        missing-value sentinel of the missing-rate gate and the selection stage, and the whole dict is forwarded to the
        selection stage.
    monotone_woe_params : dict, default {'n_init_bins': 20, 'min_bin_size': 0.03, 'min_n_bins': 2, 'sv_min_bin_size': 0.0, 'sv_small_policy': 'keep', 'sv_woe_smoothing': 'none', 'sv_smoothing_alpha': 0.0, 'unseen_special_policy': 'neutral'}
        Used with ``woe_engine='monotone'``: constructor keys of ``MonotoneWOEBinner`` and the ``fit`` keys
        ``chi2_binning``, ``chi2_p``, ``chi2_init_size`` and ``n_jobs``. Keys outside the built-in allowlist are dropped
        silently. Without a ``special_values`` key the legacy sentinel ``-999999`` is declared a special value when a
        numeric feature holds it in the fit sample (``CreditModelPipeline`` does the same, so a screening artifact
        and a self-fit bin the same data the same way).
    categorical_features : list of str or None, default None
        Features binned as categorical by the monotone engine (``cate_feats``); also passed to the selection stage.
    monotone_refine_cate_enabled : bool, default False
        Whether to call ``refine_cate`` on the fitted monotone binner (monotone engine only).
    monotone_refine_cate_params : dict, default {}
        Keyword arguments of ``refine_cate``; ``features`` defaults to the categorical features.
    monotone_refine_dtree_enabled : bool, default False
        Whether to call ``refine_dtree`` on the fitted monotone binner (monotone engine only).
    monotone_refine_dtree_params : dict, default {}
        Keyword arguments of ``refine_dtree``; ``features`` defaults to the numeric features.
    monotone_refine_chi2_enabled : bool, default False
        Whether to call ``refine_chi2`` on the fitted monotone binner (monotone engine only).
    monotone_refine_chi2_params : dict, default {}
        Keyword arguments of ``refine_chi2``; ``features`` defaults to the numeric features.
    woe_plot_groups : list of str, default []
        Group columns for extra WOE plots by group (``figs/woe/<target>/by_<group>``). Plots are written only while
        ``write_outputs`` and ``plot_outputs`` are both True.
    psi_enabled : bool, default True
        Whether to compute ``psi_summary`` and ``psi_details``.
    psi_reference_dataset : str, default 'ins'
        Reference sample of the PSI: ``'ins'``, ``'oos'``, ``'oot'`` or ``'external'`` (``ValueError`` otherwise). The PSI
        compares all splits combined with it.
    psi_reference_data : pandas.DataFrame or None, default None
        Reference data for ``psi_reference_dataset='external'`` (required then, otherwise ``ValueError``).
    psi_group_dims : list of str, default ['sample', 'time', 'population']
        Grouping of the PSI: ``'sample'`` is the split, ``'time'`` and ``'population'`` expand to ``time_dims`` and
        ``population_dims``, other names are column names. An overall PSI is computed only when no group column remains
        (``'global'`` adds nothing by itself).
    psi_use_woe_bins : bool, default True
        Whether to bin with the WOE engine of each target when available. Otherwise ``PSICalculator`` bins the data with
        ``psi_params``; that numeric PSI skips the non-numeric features with a ``UserWarning``.
    psi_params : dict, default {'buckets': 10, 'equal_freq': True, 'min_bin_prop': 0.05}
        Keyword arguments of ``PSICalculator``.
    ivks_enabled : bool, default True
        Whether to compute ``ivks_summary``. It needs a target.
    ivks_group_dims : list of str, default ['global', 'time', 'population']
        Groupings of the IV/KS report: ``'global'`` for all rows, ``'time'`` and ``'population'`` for each column of
        ``time_dims`` and ``population_dims``, other names as single group columns.
    ivks_use_woe_bins : bool, default True
        Whether to compute IV/KS on the bins of the WOE engine of each target when available.
    ivks_params : dict, default {'iv_cut': 0.0}
        ``iv_cut`` (minimum IV of the reported features), ``feature_block_size`` (columns per block with WOE bins,
        default 64), ``missing_rate_ref`` (the sentinel counted as missing, default ``woe_params['missing_ref_value']``)
        and, without WOE bins, other keyword arguments of ``VarExtractionInsights``.
    corr_enabled : bool, default True
        Whether to compute ``corr_matrix``, ``high_corr_pairs`` and ``correlated_detail``.
    corr_include_incumbent : bool, default True
        Whether incumbent features are included in the correlation analysis (and, with ``corr_use_woe_bins``, in the WOE
        fit).
    corr_use_woe_bins : bool, default True
        Whether ``CorrelationFilter`` ranks correlated features with IV from the WOE bins of the target.
    corr_params : dict, default {'corr_cutpoint': 0.75, 'method': 'pearson', 'max_iterations': 10, 'base_metric': 'iv'}
        ``corr_cutpoint`` (pairs with a larger absolute correlation are flagged), ``method`` (``'pearson'``,
        ``'spearman'`` or ``'kendall'``), ``max_iterations`` of the removal loop and other ``CorrelationFilter``
        arguments such as ``base_metric``.
    missing_rate_threshold : float or None, default None
        Maximum missing rate of a feature on INS (NaN or the ``missing_ref_value`` sentinel). With
        ``woe_fit_scope='post_missing_gate'`` features above it are removed before the WOE fit and from all later stages
        (they are listed in ``woe_artifacts['missing_gate_dropped']``). It is also the default threshold of the
        selection stage. ``None`` disables the gate.
    selection_enabled : bool, default False
        Whether to run the feature screening (missing rate, PSI, IV and correlation stages and optional gates) on the
        first target. It fills ``selected_features``, ``selection_summary`` and ``screening_artifact`` and needs a
        target.
    selection_params : dict, default {}
        Screening settings read by the pipeline; other keys are ignored. Thresholds: ``psi_threshold`` (0.2),
        ``psi_compare_splits`` (['oos']), ``iv_threshold`` (0.02), ``iv_upper_threshold`` (2.0 since 0.9.3; ``None``
        switches the leakage limit off), ``iv_gap_ratio_warn`` (2.0) and ``iv_gap_abs_warn`` (0.5) of the leak
        warning, ``corr_threshold`` (or
        ``corr_cutpoint``), ``missing_rate_threshold``, ``max_selected_features``, ``min_selected_features``. Stability
        gates: ``monthly_iv_min``, ``monthly_iv_cv_max``, ``direction_consistency_min``, ``min_group_n``,
        ``insufficient_group_policy``, ``target_rules``, ``min_pass_count``, ``per_target_iv_range``,
        ``direction_reference_target``. VIF: ``vif_enabled``, ``vif_threshold``, ``vif_min_features``,
        ``vif_tie_break_metric``, ``vif_use_woe_bins``. Others: ``ranking_metric``, ``tie_breaker``,
        ``missing_rate_ref``, ``on_empty_stage`` (``'keep_all_warn'``, ``'keep_all'`` or ``'raise'``; see
        ``FeatureScreenConfig.on_empty_stage``: since 0.9.3 the default never brings back a leak-gate drop),
        ``fallback_top_k`` (5), ``corr_method`` and
        ``corr_base_metric`` (default to ``corr_params['method']`` and ``['base_metric']``) and the stage switches and
        binning options derived from the other config fields (for example ``psi_enabled``, ``iv_nbins``,
        ``corr_max_iterations``). ``psi_compare_splits`` ignores case and spaces and accepts a bare string.
    selection_group_dims : list of str or None, default None
        Columns of the group-stability gates (``monthly_iv_min``, ``monthly_iv_cv_max``,
        ``direction_consistency_min``). They are required when one of those gates is set (``ValueError``) and must exist
        in the INS split (``KeyError``).
    weight_col : str or None, default None
        Sample-weight column of the weighted selection and its IV/direction evidence; it must exist in the data. The WOE
        bins are still fitted without weights.
    synthesize_missing_oot : bool or None, default False
        When no OOT rows exist, True copies the OOS rows in as a stand-in OOT (with a ``UserWarning``); False keeps OOT
        empty. ``None`` counts as False. The stand-in rows feed only what is reported per split (the ``oot`` group of the
        PSI by ``sample``, the OOT comparisons of the selection); the tables that pool every split (the distribution,
        ``n_rows`` of the summary, the global and grouped PSI and IV/KS, the correlation) count each row once. The
        stand-in frame in ``result.splits['oot']`` carries ``attrs['smf_stand_in_oot'] = True``.
    woe_fit_scope : {'all', 'post_missing_gate'}, default 'post_missing_gate'
        ``'all'`` fits the WOE on every new feature. ``'post_missing_gate'`` first drops the features above
        ``missing_rate_threshold`` (a no-op when it is None or there is no target) and continues with the remaining ones.
    write_outputs : bool, default True
        Whether to write the result tables as CSV files in ``output_dir``; it is also the master switch for the WOE
        plots.
    write_excel : bool, default True
        Whether to write ``Feature_Validation_Report.xlsx`` with the non-empty tables.
    plot_outputs : bool, default True
        Whether to write the WOE plots to ``figs/woe``; they are written only when ``write_outputs`` is True as well.
    screen_iv_convention : {"guarded", "raw"}, default "guarded"
        How the IV on the WOE bins treats a class-pure bin (no bad or no good row), whose WOE comes from the ``1e-6`` of
        ``ln((bad_share + 1e-6) / (good_share + 1e-6))`` alone: ``"guarded"`` leaves it out of the IV sum (its rows stay
        in the totals, so the other bins keep their WOE and IV), ``"raw"`` counts every bin (0.9.1). It applies to
        ``ivks_summary`` (per group, on the group's counts), to the selection stage (see
        ``FeatureScreenConfig.screen_iv_convention``), to its group and per-target evidence, and to the IV that
        ``CorrelationFilter`` ranks correlated features by. ``ivks_summary`` reports ``iv`` with ``iv_raw``,
        ``iv_sv_pure`` and ``n_sv_pure``; ``ivks_sv_pure_bins`` of the result lists the class-pure special-value bins.
        ``selection_params['screen_iv_convention']`` and ``corr_params['screen_iv_convention']`` are accepted as well;
        a value there that differs from this field raises ``ValueError`` when ``run`` starts, as does any other value.
        The WOE of the bins is not changed.
    """

    output_dir: str = "output/feature_validation"
    clean_output_dir: bool = False
    id_col: str = "flow_id"
    apply_time_col: str = "apply_time"
    target_cols: list[str] | None = None
    new_feature_cols: list[str] | None = None
    incumbent_feature_cols: list[str] | None = None

    input_type: Literal["auto", "dataframe", "csv"] = "auto"
    csv_read_kwargs: dict[str, Any] = field(default_factory=dict)
    enable_batch: bool = False
    feature_batch_size: int | None = None
    feature_batches: list[list[str]] | None = None
    batch_base_cols: list[str] | None = None
    batch_output_subdir: str = "feature_batches"
    batch_keep_intermediate: bool = True
    batch_corr_mode: Literal["within_batch", "block_pairwise", "off"] = "within_batch"
    batch_corr_pair_chunk_size: int | None = None

    split_col: str | None = None
    sample_col: str = "sample_ind"
    oot_col: str | None = "oot_flag"
    split_config: dict[str, Any] = field(default_factory=lambda: {"test_size": 0.3, "stratify": True})
    random_state: int = 42

    time_dims: list[str] = field(default_factory=lambda: ["apply_month"])
    population_dims: list[str] = field(default_factory=list)
    group_specs: dict[str, list[str]] | list[Any] | None = None
    min_group_size: int = 100

    distribution_enabled: bool = True
    distribution_params: dict[str, Any] = field(
        default_factory=lambda: {"q": [0.05, 0.15, 0.25, 0.5, 0.75, 0.95, 0.99]}
    )

    woe_enabled: bool = True
    woe_fit_query: str | None = None
    woe_engine: str = "monotone"
    # sv_* govern special-value (SV) WOE bins only; the defaults below reproduce
    # pre-0.8.0 behavior bit-for-bit. Monotone-side sv_* keys only survive because
    # they are listed in _MONOTONE_INIT_KEYS below — unlisted keys are dropped
    # silently, so the two lists must be kept in sync.
    woe_params: dict[str, Any] = field(
        default_factory=lambda: {
            "nbins": 10,
            "equal_freq": True,
            "min_bin_prop": 0.05,
            "sv_min_bin_size": 0.0,
            "sv_small_policy": "keep",
            "sv_woe_smoothing": "none",
            "sv_smoothing_alpha": 0.0,
        }
    )
    # unseen_special_policy (monotone only): declared special values absent from
    # the fit sample — "neutral" placeholder bins (default since 0.9.0) or "normal_bin" (legacy).
    monotone_woe_params: dict[str, Any] = field(
        default_factory=lambda: {
            "n_init_bins": 20,
            "min_bin_size": 0.03,
            "min_n_bins": 2,
            "sv_min_bin_size": 0.0,
            "sv_small_policy": "keep",
            "sv_woe_smoothing": "none",
            "sv_smoothing_alpha": 0.0,
            "unseen_special_policy": "neutral",
        }
    )
    categorical_features: list[str] | None = None
    monotone_refine_cate_enabled: bool = False
    monotone_refine_cate_params: dict[str, Any] = field(default_factory=dict)
    monotone_refine_dtree_enabled: bool = False
    monotone_refine_dtree_params: dict[str, Any] = field(default_factory=dict)
    monotone_refine_chi2_enabled: bool = False
    monotone_refine_chi2_params: dict[str, Any] = field(default_factory=dict)
    woe_plot_groups: list[str] = field(default_factory=list)

    psi_enabled: bool = True
    psi_reference_dataset: str = "ins"
    psi_reference_data: pd.DataFrame | None = None
    psi_group_dims: list[str] = field(default_factory=lambda: ["sample", "time", "population"])
    psi_use_woe_bins: bool = True
    psi_params: dict[str, Any] = field(
        default_factory=lambda: {"buckets": 10, "equal_freq": True, "min_bin_prop": 0.05}
    )

    ivks_enabled: bool = True
    ivks_group_dims: list[str] = field(default_factory=lambda: ["global", "time", "population"])
    ivks_use_woe_bins: bool = True
    ivks_params: dict[str, Any] = field(default_factory=lambda: {"iv_cut": 0.0})

    corr_enabled: bool = True
    corr_include_incumbent: bool = True
    corr_use_woe_bins: bool = True
    corr_params: dict[str, Any] = field(
        default_factory=lambda: {
            "corr_cutpoint": 0.75,
            "method": "pearson",
            "max_iterations": 10,
            "base_metric": "iv",
        }
    )

    missing_rate_threshold: float | None = None
    selection_enabled: bool = False
    selection_params: dict[str, Any] = field(default_factory=dict)
    # G03/G04: group dims (e.g. ["apply_month"]) the selection gates use for
    # per-group IV/direction evidence. None keeps the gates evidence-free —
    # setting group-stability thresholds without dims raises loudly.
    selection_group_dims: list[str] | None = None
    weight_col: str | None = None
    # OOT governance (G10): missing OOT stays empty by default. Set True to
    # retain the legacy OOS-copy stand-in (with its UserWarning).
    synthesize_missing_oot: bool | None = False
    # G00: "all" fits the top-level WOE on every new feature;
    # "post_missing_gate" runs the selection-grade missing-rate gate
    # (missing_rate_threshold) BEFORE the WOE fit so high-missing features
    # cannot break a target's WOE artifacts.
    woe_fit_scope: Literal["all", "post_missing_gate"] = "post_missing_gate"

    write_outputs: bool = True
    write_excel: bool = True
    plot_outputs: bool = True
    # 0.9.2: class-pure bins of the IV on WOE bins ("guarded" leaves them out of the sum; "raw" is 0.9.1)
    screen_iv_convention: Literal["guarded", "raw"] = "guarded"

FeatureValidationPipelineResult dataclass

Result returned by :meth:FeatureValidationPipeline.run.

Parameters:

Name	Type	Description	Default
splits	dict of str to pandas.DataFrame	Sample splits: ins, oos and oot (an empty frame when there is no OOT) plus any extra label of split_col. In CSV batch mode only the base columns are held, without the batch features.	required
distribution_summary	dict of str to pandas.DataFrame	Distribution tables keyed numeric_<group> and categorical_<group>; empty when distribution_enabled is False.	required
woe_artifacts	dict	WOE results: by_target (per target the fitted engine, its adapter, woe_splits and the fitted features; empty in batch mode), woe_table, refine_summary, the *_stats_by_target audit entries and, when the missing gate dropped features, missing_gate_dropped. Empty when WOE is disabled or there is no target.	required
psi_summary	DataFrame	PSI of each feature against the reference sample by group, with the columns target, group_col and group_value; groups that failed carry an error message. Empty when PSI is disabled.	required
psi_details	dict	Detailed PSI tables keyed <target>:<group_col> (prefixed batch_NNN: in batch mode).	required
ivks_summary	DataFrame	IV, KS and lift of each feature per target and group (target, group_spec, var, n, iv, ks_in_gains, lift_in_gains, missing_rate, n_bins, ...), limited to iv >= iv_cut. n, missing_rate, min, mean and max count ivks_params['missing_rate_ref'] (default woe_params['missing_ref_value'], -999999) as missing, with or without WOE bins.	required
corr_matrix	DataFrame	Correlation matrix of the numeric features (and incumbents when included); empty with fewer than two features or when correlation is disabled.	required
high_corr_pairs	DataFrame	Feature pairs whose absolute correlation exceeds corr_cutpoint: var1, var2, corr, abs_corr and pair_type (new_new, new_incumbent, incumbent_incumbent or new_new_cross_batch).	required
correlated_detail	DataFrame	Per correlated pair and target the metrics of both features (iv, ks_in_gains, lift_in_gains) and a recommended_action (keep or remove) from CorrelationFilter. The cross-batch rows of batch_corr_mode='block_pairwise' take the metrics of the global ivks_summary rows when both use the WOE engine (metric_source='ivks_summary'), else the raw values ('raw_values').	required
validation_summary	DataFrame	Two columns metric and value with row, feature and target counts, table sizes and, with selection, the number of selected features.	required
output_paths	dict of str to str	Absolute paths of the files written: one CSV per table name plus excel_report. Empty when nothing was written.	{}
report_path	str or None	Absolute path of the Excel report; None when write_excel is False.	None
batch_metadata	DataFrame or None	CSV batch mode only: one row per batch (batch_id, features, n_features, read_cols, output_dir, status and error for failed batches). None otherwise.	None
batch_results	dict	CSV batch mode only: {"batch_NNN": {"output_paths": ..., "report_path": ...}}.	{}
selected_features	list of str	Features kept by the selection stage; empty when selection_enabled is False.	[]
selection_summary	dict	Selection audit: initial_features, final_features, target_col, config_snapshot and the stage tables of the screening (for example missing_rate, psi, iv, corr_dropped, screen_summary). In CSV batch mode the per-feature tables (missing_rate, psi, iv) cover every feature, batch_candidates lists the features that passed the per-batch stages, batch_screen_summary holds the stage counts of each batch (batch_id) and screen_summary those of the global pass over the candidates.	{}
screening_artifact	FeatureScreeningArtifact or None	Artifact for CreditModelPipeline (selected features, summary and WOE artifacts). None unless the selection stage ran; in CSV batch mode it is also None when no feature passed the per-batch stages. In batch mode its WOE artifacts are those of the global pass (fitted on the candidates).	None
config_snapshot	dict	Effective settings of the run (targets, features, selection, WOE and batch-mode flags).	{}
ivks_sv_pure_bins	DataFrame	The class-pure special-value bins found by the global rows of ivks_summary on WOE bins (no bad or no good row; a bin whose rows are all missing counts as one): target, var, special_value, bin_label, n, bad, good, bad_rate, woe (the WOE the binning gives the bin), iv (its contribution to iv_raw) and low_count (n < 30). Their WOE comes from the 1e-6 of the formula alone; resolve them (a separate rule, sv_small_policy='neutral' or 'merge_missing') before training a new model. The list is for review, for example as candidate rules; nothing acts on it. Written as the ivks_sv_pure_bins table when not empty.	empty frame

Source code in Modeling_Tool/Pipeline/feature_validation.py

@dataclass
class FeatureValidationPipelineResult:
    """Result returned by :meth:`FeatureValidationPipeline.run`.

    Parameters
    ----------
    splits : dict of str to pandas.DataFrame
        Sample splits: ``ins``, ``oos`` and ``oot`` (an empty frame when there is no OOT) plus any extra label of
        ``split_col``. In CSV batch mode only the base columns are held, without the batch features.
    distribution_summary : dict of str to pandas.DataFrame
        Distribution tables keyed ``numeric_<group>`` and ``categorical_<group>``; empty when ``distribution_enabled`` is
        False.
    woe_artifacts : dict
        WOE results: ``by_target`` (per target the fitted ``engine``, its ``adapter``, ``woe_splits`` and the fitted
        ``features``; empty in batch mode), ``woe_table``, ``refine_summary``, the ``*_stats_by_target`` audit entries and,
        when the missing gate dropped features, ``missing_gate_dropped``. Empty when WOE is disabled or there is no
        target.
    psi_summary : pandas.DataFrame
        PSI of each feature against the reference sample by group, with the columns ``target``, ``group_col`` and
        ``group_value``; groups that failed carry an ``error`` message. Empty when PSI is disabled.
    psi_details : dict
        Detailed PSI tables keyed ``<target>:<group_col>`` (prefixed ``batch_NNN:`` in batch mode).
    ivks_summary : pandas.DataFrame
        IV, KS and lift of each feature per target and group (``target``, ``group_spec``, ``var``, ``n``, ``iv``,
        ``ks_in_gains``, ``lift_in_gains``, ``missing_rate``, ``n_bins``, ...), limited to ``iv >= iv_cut``. ``n``,
        ``missing_rate``, ``min``, ``mean`` and ``max`` count ``ivks_params['missing_rate_ref']`` (default
        ``woe_params['missing_ref_value']``, -999999) as missing, with or without WOE bins.
    corr_matrix : pandas.DataFrame
        Correlation matrix of the numeric features (and incumbents when included); empty with fewer than two features or
        when correlation is disabled.
    high_corr_pairs : pandas.DataFrame
        Feature pairs whose absolute correlation exceeds ``corr_cutpoint``: ``var1``, ``var2``, ``corr``, ``abs_corr`` and
        ``pair_type`` (``new_new``, ``new_incumbent``, ``incumbent_incumbent`` or ``new_new_cross_batch``).
    correlated_detail : pandas.DataFrame
        Per correlated pair and target the metrics of both features (``iv``, ``ks_in_gains``, ``lift_in_gains``) and a
        ``recommended_action`` (``keep`` or ``remove``) from ``CorrelationFilter``. The cross-batch rows of
        ``batch_corr_mode='block_pairwise'`` take the metrics of the global ``ivks_summary`` rows when both use the WOE
        engine (``metric_source='ivks_summary'``), else the raw values (``'raw_values'``).
    validation_summary : pandas.DataFrame
        Two columns ``metric`` and ``value`` with row, feature and target counts, table sizes and, with selection, the
        number of selected features.
    output_paths : dict of str to str, default {}
        Absolute paths of the files written: one CSV per table name plus ``excel_report``. Empty when nothing was written.
    report_path : str or None, default None
        Absolute path of the Excel report; None when ``write_excel`` is False.
    batch_metadata : pandas.DataFrame or None, default None
        CSV batch mode only: one row per batch (``batch_id``, ``features``, ``n_features``, ``read_cols``, ``output_dir``,
        ``status`` and ``error`` for failed batches). None otherwise.
    batch_results : dict, default {}
        CSV batch mode only: ``{"batch_NNN": {"output_paths": ..., "report_path": ...}}``.
    selected_features : list of str, default []
        Features kept by the selection stage; empty when ``selection_enabled`` is False.
    selection_summary : dict, default {}
        Selection audit: ``initial_features``, ``final_features``, ``target_col``, ``config_snapshot`` and the stage
        tables of the screening (for example ``missing_rate``, ``psi``, ``iv``, ``corr_dropped``, ``screen_summary``).
        In CSV batch mode the per-feature tables (``missing_rate``, ``psi``, ``iv``) cover every feature,
        ``batch_candidates`` lists the features that passed the per-batch stages, ``batch_screen_summary`` holds the
        stage counts of each batch (``batch_id``) and ``screen_summary`` those of the global pass over the candidates.
    screening_artifact : FeatureScreeningArtifact or None, default None
        Artifact for ``CreditModelPipeline`` (selected features, summary and WOE artifacts). None unless the selection
        stage ran; in CSV batch mode it is also None when no feature passed the per-batch stages. In batch mode its WOE
        artifacts are those of the global pass (fitted on the candidates).
    config_snapshot : dict, default {}
        Effective settings of the run (targets, features, selection, WOE and batch-mode flags).
    ivks_sv_pure_bins : pandas.DataFrame, default empty frame
        The class-pure special-value bins found by the global rows of ``ivks_summary`` on WOE bins (no bad or no good
        row; a bin whose rows are all missing counts as one): ``target``, ``var``, ``special_value``, ``bin_label``,
        ``n``, ``bad``, ``good``, ``bad_rate``, ``woe`` (the WOE the binning gives the bin), ``iv`` (its contribution
        to ``iv_raw``) and ``low_count`` (``n < 30``). Their WOE comes from the ``1e-6`` of the formula alone; resolve
        them (a separate rule, ``sv_small_policy='neutral'`` or ``'merge_missing'``) before training a new model. The
        list is for review, for example as candidate rules; nothing acts on it. Written as the ``ivks_sv_pure_bins``
        table when not empty.
    """

    splits: dict[str, pd.DataFrame]
    distribution_summary: dict[str, pd.DataFrame]
    woe_artifacts: dict[str, Any]
    psi_summary: pd.DataFrame
    psi_details: dict[str, Any]
    ivks_summary: pd.DataFrame
    corr_matrix: pd.DataFrame
    high_corr_pairs: pd.DataFrame
    correlated_detail: pd.DataFrame
    validation_summary: pd.DataFrame
    output_paths: dict[str, str] = field(default_factory=dict)
    report_path: str | None = None
    batch_metadata: pd.DataFrame | None = None
    batch_results: dict[str, Any] = field(default_factory=dict)
    selected_features: list[str] = field(default_factory=list)
    selection_summary: dict[str, Any] = field(default_factory=dict)
    screening_artifact: Any | None = None
    config_snapshot: dict[str, Any] = field(default_factory=dict)
    ivks_sv_pure_bins: pd.DataFrame = field(default_factory=lambda: _empty_sv_pure_bins())

FeatureValidationPipeline

Feature validation workflow for new wide-table feature feeds.

The pipeline splits the data into INS/OOS/OOT, then builds distribution tables, fits WOE bins per target and computes PSI, IV/KS and correlation reports (optionally feature selection). It can read a CSV file directly and process very wide files in batches of feature columns.

Parameters:

Name	Type	Description	Default
config	FeatureValidationPipelineConfig or None	Pipeline configuration. None (or any falsy value) uses FeatureValidationPipelineConfig() with its defaults.	None

Attributes:

Name	Type	Description
config	FeatureValidationPipelineConfig	The configuration in use.

Source code in Modeling_Tool/Pipeline/feature_validation.py

class FeatureValidationPipeline:
    """Feature validation workflow for new wide-table feature feeds.

    The pipeline splits the data into INS/OOS/OOT, then builds distribution tables, fits WOE bins per target and
    computes PSI, IV/KS and correlation reports (optionally feature selection). It can read a CSV file directly and
    process very wide files in batches of feature columns.

    Parameters
    ----------
    config : FeatureValidationPipelineConfig or None, default None
        Pipeline configuration. ``None`` (or any falsy value) uses ``FeatureValidationPipelineConfig()`` with its
        defaults.

    Attributes
    ----------
    config : FeatureValidationPipelineConfig
        The configuration in use.
    """

    _MONOTONE_INIT_KEYS = {
        "n_init_bins",
        "min_bin_size",
        "min_n_bins",
        "eps",
        "missing_woe",
        "special_values",
        "bin_label_decimals",
        "min_bad_count",
        "min_good_count",
        "small_bin_policy",
        "monotone_direction",
        "reference_target",
        "direction_conflict_policy",
        "missing_bin_strategy",
        "refine_min_n_bins_policy",
        "sv_min_bin_size",
        "sv_small_policy",
        "sv_woe_smoothing",
        "sv_smoothing_alpha",
        "sv_total_basis",
        "unseen_special_policy",
    }
    _MONOTONE_FIT_KEYS = {"chi2_binning", "chi2_p", "chi2_init_size", "n_jobs"}

    def __init__(self, config: FeatureValidationPipelineConfig | None = None):
        self.config = config or FeatureValidationPipelineConfig()

    def run(self, data: pd.DataFrame | str | Path) -> FeatureValidationPipelineResult:
        """Validate the new features in ``data`` and write the configured reports.

        Parameters
        ----------
        data : pandas.DataFrame or str or pathlib.Path
            The feature table, or the path of a CSV file (see ``input_type``). It must contain ``id_col``,
            ``apply_time_col``, the new and incumbent feature columns, ``weight_col`` and ``split_col`` when configured.
            A DataFrame is not modified.

        Returns
        -------
        FeatureValidationPipelineResult
            Splits, distribution, WOE, PSI, IV/KS, correlation and selection results, and the paths of the files written.

        Raises
        ------
        KeyError
            If a required column is missing from the data.
        ValueError
            If a configuration value is invalid (``input_type``, ``woe_engine``, ``psi_reference_dataset``, batch
            settings, ``woe_fit_query``), if there are no new features, if ``split_col`` lacks ``ins`` or ``oos`` rows, or
            if every batch fails.
        TypeError
            If ``input_type='dataframe'`` and ``data`` is not a DataFrame, or ``oot_col`` is not numeric.
        FileNotFoundError
            If a CSV path does not exist.

        Notes
        -----
        In CSV batch mode a failing batch is recorded in ``batch_metadata`` (``status='error'``) and the run continues
        with the remaining batches.
        """
        cfg = self.config
        manifest = None
        if not getattr(self, "_nested_run", False) and (cfg.write_outputs or cfg.write_excel):
            manifest = OutputManifest.begin(cfg.output_dir, "feature_validation", clean=cfg.clean_output_dir)
        try:
            result = self._run(data)
        except BaseException:
            if manifest is not None:
                try:
                    manifest.finish(clean=False)  # list what the failed run wrote; remove nothing
                except Exception:
                    pass
            raise
        if manifest is not None:
            removed = manifest.finish()
            if removed:
                _logger.info(
                    "FeatureValidationPipeline: clean_output_dir removed %d stale file(s): %s", len(removed), removed
                )
        return result

    def _screen_iv_convention(self) -> str:
        """``screen_iv_convention`` of the config, checked against ``selection_params['screen_iv_convention']``."""
        from Modeling_Tool._utils.screen_iv import check_screen_iv_convention

        cfg = self.config
        value = check_screen_iv_convention(getattr(cfg, "screen_iv_convention", "guarded"))
        for name in ("selection_params", "corr_params"):
            in_params = (getattr(cfg, name, None) or {}).get("screen_iv_convention")
            if in_params is not None and in_params != value:
                raise ValueError(
                    f"{name}['screen_iv_convention']={in_params!r} differs from "
                    f"screen_iv_convention={value!r}; set one of them, or the same value in both"
                )
        return value

    def _run(self, data: pd.DataFrame | str | Path) -> FeatureValidationPipelineResult:
        self._screen_iv_convention()
        input_type = self._resolve_input_type(data)
        if self.config.enable_batch and input_type != "csv":
            raise ValueError("enable_batch=True currently requires CSV input.")
        if input_type == "csv":
            csv_path = Path(data)
            if self._is_batch_mode():
                return self._run_csv_batches(csv_path)
            if self._has_batch_config():
                warnings.warn(
                    "feature_batch_size/feature_batches are configured but enable_batch=False; "
                    "FeatureValidationPipeline will read the CSV in full and ignore batch settings.",
                    RuntimeWarning,
                    stacklevel=2,
                )
            return self._run_dataframe(self._read_csv(csv_path))
        if not isinstance(data, pd.DataFrame):
            raise TypeError("data must be a pandas DataFrame when input_type='dataframe'.")
        return self._run_dataframe(data)

    def _run_dataframe(self, data: pd.DataFrame) -> FeatureValidationPipelineResult:
        cfg = self.config
        work = data.copy()
        self._add_time_columns(work)

        new_features = self._resolve_new_features(work)
        incumbent_features = self._resolve_incumbent_features(work, new_features)
        target_cols = [col for col in as_list(cfg.target_cols) if col in work.columns]
        self._warn_unknown_targets(work.columns)
        self._validate_input(work, new_features, incumbent_features, target_cols)
        config_snapshot = self._build_config_snapshot(
            new_features,
            incumbent_features,
            target_cols,
            batch_mode=False,
            columns=work.columns,
        )

        output_dir = Path(cfg.output_dir)
        if cfg.write_outputs or cfg.write_excel:
            make_dirs(output_dir)
        if cfg.write_outputs and cfg.plot_outputs:
            make_dirs(output_dir / "figs" / "woe")

        splits = self._split_data(work, target_cols[0] if target_cols else None)
        combined = self._combine_splits(splits)
        feature_sources = self._feature_source_frame(new_features, incumbent_features)

        distribution_summary = (
            self._run_distribution(combined, new_features)
            if cfg.distribution_enabled
            else {}
        )
        # G00: with woe_fit_scope="post_missing_gate", the selection-grade
        # missing-rate gate runs BEFORE the top-level WOE fit, so one
        # high-missing feature can no longer take down a whole target's WOE
        # artifacts through the coarse per-target try/except in _fit_woe.
        woe_fit_features = None
        missing_gate_dropped = pd.DataFrame()
        if (
            cfg.woe_fit_scope == "post_missing_gate"
            and cfg.missing_rate_threshold is not None
            and target_cols
        ):
            from Modeling_Tool.Feature.Weighted_Screen import _apply_missing_rate_stage

            gate_rows: list[dict] = []
            woe_fit_features, _gate_table, missing_gate_dropped = _apply_missing_rate_stage(
                splits["ins"],
                list(new_features),
                gate_rows,
                missing_rate_threshold=cfg.missing_rate_threshold,
                missing_rate_ref=cfg.woe_params.get("missing_ref_value", -999999),
            )
            if len(missing_gate_dropped):
                # when every feature exceeds the threshold the gate keeps them all (and warns): they were not dropped
                missing_gate_dropped = missing_gate_dropped[~missing_gate_dropped["var"].isin(woe_fit_features)]
        if cfg.woe_enabled and target_cols:
            check_woe_fit_query_rows(splits["ins"], cfg.woe_fit_query, context="INS")
            if cfg.woe_engine.lower() != "monotone":
                warn_precision_collapse(
                    splits["ins"], new_features, int(cfg.woe_params.get("precision", 5)), "FeatureValidationPipeline"
                )
        woe_artifacts = (
            self._fit_woe(
                splits,
                new_features,
                incumbent_features,
                target_cols,
                fit_features_override=woe_fit_features,
            )
            if cfg.woe_enabled and target_cols
            else {}
        )
        if woe_artifacts and len(missing_gate_dropped):
            woe_artifacts["missing_gate_dropped"] = missing_gate_dropped
            refine = woe_artifacts.get("refine_summary")
            gate_row = pd.DataFrame([{
                "target": "*", "step": "missing_gate", "status": "ok",
                "n_in": len(new_features),
                "n_out": len(woe_fit_features if woe_fit_features is not None else new_features),
                "threshold": cfg.missing_rate_threshold,
            }])
            woe_artifacts["refine_summary"] = (
                pd.concat([refine, gate_row], ignore_index=True)
                if isinstance(refine, pd.DataFrame) and len(refine) else gate_row
            )
        # G00: once the missing gate has run, every WOE-consuming downstream
        # stage (PSI/IVKS/corr/selection) operates on the gated feature set —
        # gated-out features keep their distribution diagnostics and appear in
        # missing_gate_dropped, but never touch the WOE ecosystem.
        effective_features = (
            list(woe_fit_features) if woe_fit_features is not None else new_features
        )
        psi_summary, psi_details = (
            self._run_psi(combined, splits, effective_features, target_cols, woe_artifacts)
            if cfg.psi_enabled
            else (pd.DataFrame(), {})
        )
        ivks_summary = (
            self._run_ivks(combined, effective_features, target_cols, woe_artifacts)
            if cfg.ivks_enabled and target_cols
            else pd.DataFrame()
        )
        ivks_sv_pure_bins = (
            pd.DataFrame(self._ivks_sv_pure_rows, columns=_empty_sv_pure_bins().columns)
            if cfg.ivks_enabled and target_cols and getattr(self, "_ivks_sv_pure_rows", None)
            else _empty_sv_pure_bins()
        )
        corr_matrix, high_corr_pairs, correlated_detail = (
            self._run_correlation(combined, effective_features, incumbent_features, target_cols, woe_artifacts)
            if cfg.corr_enabled
            else (pd.DataFrame(), pd.DataFrame(), pd.DataFrame())
        )
        selected_features: list[str] = []
        selection_summary: dict[str, Any] = {}
        screening_artifact = None
        if cfg.selection_enabled and target_cols:
            selected_features, selection_summary, screening_artifact = self._run_selection(
                splits=splits,
                new_features=effective_features,
                incumbent_features=incumbent_features,
                target_cols=target_cols,
                woe_artifacts=woe_artifacts,
                config_snapshot=config_snapshot,
            )
        validation_summary = self._build_validation_summary(
            data=combined,
            new_features=new_features,
            incumbent_features=incumbent_features,
            target_cols=target_cols,
            feature_sources=feature_sources,
            distribution_summary=distribution_summary,
            psi_summary=psi_summary,
            ivks_summary=ivks_summary,
            high_corr_pairs=high_corr_pairs,
            woe_artifacts=woe_artifacts,
            selected_features=selected_features,
        )

        tables = self._collect_tables(
            feature_sources,
            distribution_summary,
            woe_artifacts,
            psi_summary,
            ivks_summary,
            corr_matrix,
            high_corr_pairs,
            correlated_detail,
            validation_summary,
            selected_features=selected_features,
            selection_summary=selection_summary,
            ivks_sv_pure_bins=ivks_sv_pure_bins,
        )
        output_paths, report_path = self._write_outputs(tables)

        return FeatureValidationPipelineResult(
            splits=splits,
            distribution_summary=distribution_summary,
            woe_artifacts=woe_artifacts,
            psi_summary=psi_summary,
            psi_details=psi_details,
            ivks_summary=ivks_summary,
            corr_matrix=corr_matrix,
            high_corr_pairs=high_corr_pairs,
            correlated_detail=correlated_detail,
            validation_summary=validation_summary,
            output_paths=output_paths,
            report_path=report_path,
            selected_features=selected_features,
            selection_summary=selection_summary,
            screening_artifact=screening_artifact,
            config_snapshot=config_snapshot,
            ivks_sv_pure_bins=ivks_sv_pure_bins,
        )

    def _warn_unknown_targets(self, columns: Any) -> None:
        """Warn about ``target_cols`` that are not columns of the data: they are ignored, and with no valid target left the
        WOE, IV/KS, correlation detail and selection stages are skipped without any other sign."""
        available = set(columns)
        unknown = [col for col in as_list(self.config.target_cols) if col not in available]
        if not unknown:
            return
        none_left = len(unknown) == len(as_list(self.config.target_cols))
        warnings.warn(
            f"FeatureValidationPipeline: target_cols {unknown} are not columns of the data and are ignored"
            + ("; with no target left the WOE, IV/KS, correlation detail and selection stages are skipped." if none_left else "."),
            UserWarning,
            stacklevel=3,
        )

    def _resolve_input_type(self, data: pd.DataFrame | str | Path) -> str:
        cfg = self.config
        if cfg.input_type not in {"auto", "dataframe", "csv"}:
            raise ValueError("input_type must be one of auto/dataframe/csv")
        if cfg.input_type != "auto":
            return cfg.input_type
        return "dataframe" if isinstance(data, pd.DataFrame) else "csv"

    def _has_batch_config(self) -> bool:
        cfg = self.config
        return bool(cfg.feature_batches) or cfg.feature_batch_size is not None

    def _is_batch_mode(self) -> bool:
        cfg = self.config
        if not cfg.enable_batch:
            return False
        if not self._has_batch_config():
            raise ValueError("enable_batch=True requires feature_batch_size or feature_batches.")
        return True

    def _read_csv(self, csv_path: Path, usecols: list[str] | None = None, nrows: int | None = None) -> pd.DataFrame:
        if not csv_path.exists():
            raise FileNotFoundError(f"CSV file not found: {csv_path}")
        kwargs = dict(self.config.csv_read_kwargs or {})
        if "usecols" in kwargs:
            raise ValueError("csv_read_kwargs cannot include usecols; FeatureValidationPipeline controls usecols.")
        if "chunksize" in kwargs:
            raise ValueError("csv_read_kwargs cannot include chunksize; CSV batch mode splits by feature columns.")
        if "nrows" in kwargs:
            raise ValueError("csv_read_kwargs cannot include nrows; FeatureValidationPipeline controls the rows read.")
        return pd.read_csv(csv_path, usecols=usecols, nrows=nrows, **kwargs)

    def _run_csv_batches(self, csv_path: Path) -> FeatureValidationPipelineResult:
        cfg = self.config
        header = list(self._read_csv(csv_path, nrows=0).columns)
        header_set = set(header)
        new_features = self._resolve_csv_new_features(
            header, None if cfg.new_feature_cols else self._read_csv(csv_path, nrows=1000)
        )
        if not new_features:
            raise ValueError("new_feature_cols cannot be empty")
        incumbent_features = [col for col in as_list(cfg.incumbent_feature_cols) if col in header_set and col not in set(new_features)]
        target_cols = [col for col in as_list(cfg.target_cols) if col in header_set]
        self._validate_batch_config(new_features)
        feature_batches = self._make_feature_batches(new_features)
        base_cols = self._resolve_batch_base_cols(header, incumbent_features, target_cols)
        if cfg.woe_fit_query:
            validate_woe_fit_query_columns(cfg.woe_fit_query, base_cols, context="CSV batch base columns")

        base_df = self._read_csv(csv_path, usecols=base_cols).copy()
        base_df["_smf_batch_row_id"] = np.arange(len(base_df))
        self._add_time_columns(base_df)
        base_splits = self._split_data(base_df, target_cols[0] if target_cols else None)
        split_labels = self._stable_split_labels(base_df, base_splits)

        batch_results: list[FeatureValidationPipelineResult] = []
        batch_rows: list[dict[str, Any]] = []
        for idx, batch_features in enumerate(feature_batches):
            read_cols = self._dedupe([col for col in base_cols + batch_features + incumbent_features if col in header_set])
            batch_dir = Path(cfg.output_dir) / cfg.batch_output_subdir / f"batch_{idx:03d}"
            row: dict[str, Any] = {
                "batch_id": idx,
                "features": ",".join(batch_features),
                "n_features": len(batch_features),
                "read_cols": ",".join(read_cols),
                "output_dir": str(batch_dir),
                "status": "ok",
            }
            batch_df: pd.DataFrame | None = None
            batch_result: FeatureValidationPipelineResult | None = None
            try:
                batch_df = self._read_csv(csv_path, usecols=read_cols).copy()
                batch_df["_smf_batch_split"] = split_labels.to_numpy()
                batch_cfg = replace(
                    cfg,
                    output_dir=str(batch_dir),
                    input_type="dataframe",
                    enable_batch=False,
                    feature_batch_size=None,
                    feature_batches=None,
                    new_feature_cols=list(batch_features),
                    incumbent_feature_cols=list(incumbent_features),
                    split_col="_smf_batch_split",
                    corr_enabled=cfg.corr_enabled and cfg.batch_corr_mode != "off",
                    write_outputs=cfg.write_outputs and cfg.batch_keep_intermediate,
                    write_excel=False,
                    selection_params=self._batch_candidate_selection_params(),
                )
                with warnings.catch_warnings():
                    # the split of the whole file already warned about rows without a label
                    warnings.filterwarnings("ignore", message=r".*missing or unknown label in '_smf_batch_split'.*")
                    # a batch whose features all fail a stage keeps them as candidates; the global pass decides
                    warnings.filterwarnings(
                        "ignore", message=r".*keeping all of them.*\(on_empty_stage='keep_all'\).*"
                    )
                    batch_pipeline = FeatureValidationPipeline(batch_cfg)
                    batch_pipeline._nested_run = True
                    batch_result = batch_pipeline.run(batch_df)
                batch_results.append(self._slim_batch_result(batch_result))
                row["n_rows"] = len(batch_df)
            except Exception as exc:
                row["status"] = "error"
                row["error"] = repr(exc)
            finally:
                del batch_result
                del batch_df
                gc.collect()
            batch_rows.append(row)

        batch_metadata = pd.DataFrame(batch_rows)
        if not batch_results:
            errors = list(dict.fromkeys(str(err) for err in batch_metadata.get("error", pd.Series(dtype=object)).dropna()))
            raise ValueError(
                f"All {len(batch_metadata)} feature validation batches failed. First error(s): {'; '.join(errors[:3])}"
            )

        global_selection = None
        if cfg.selection_enabled and target_cols:
            candidates, hard_dropped = self._batch_candidates(batch_results)
            if candidates:
                global_selection = self._run_batch_global_selection(
                    csv_path, header, target_cols, split_labels, candidates
                )
            elif hard_dropped:
                from Modeling_Tool.Feature.Weighted_Screen import EmptyStageError

                listed = ", ".join(f"{var} ({reason})" for var, reason in hard_dropped.items())
                raise EmptyStageError(
                    "feature validation batches: every candidate of the selection was dropped by the leak gate, which "
                    f"no fallback may revive: {listed}. Check these fields for target leakage (time point and source) "
                    "before training; on_empty_stage='keep_all' restores the fallback up to 0.9.3 and is only meant "
                    "for comparing with old results"
                )
        result = self._merge_batch_results(
            batch_results=batch_results,
            batch_metadata=batch_metadata,
            base_splits=base_splits,
            new_features=new_features,
            incumbent_features=incumbent_features,
            target_cols=target_cols,
            n_rows=len(base_df),
            csv_path=csv_path,
            feature_batches=feature_batches,
            global_selection=global_selection,
        )
        return result

    def _batch_candidate_selection_params(self) -> dict[str, Any]:
        """Selection settings of the first pass inside a batch: only the stages that judge each feature on its own
        (missing rate, PSI, IV and the group and target gates) run there. The stages that compare features (correlation,
        VIF) and the caps wait for the global pass over the candidates of every batch."""
        params = dict(self.config.selection_params or {})
        params.update({
            "corr_enabled": False,
            "vif_enabled": False,
            "max_selected_features": None,
            "min_selected_features": None,
            # a batch whose features all fail a stage keeps them as candidates: the global pass decides (and applies
            # the user's on_empty_stage there). The leak-gate drops that this keeps are taken out of the candidates by
            # ``_batch_candidates``, so they never reach the global pass (0.9.3)
            "on_empty_stage": "keep_all",
        })
        return params

    def _batch_candidates(
        self, batch_results: list[FeatureValidationPipelineResult]
    ) -> tuple[list[str], dict[str, str]]:
        """The candidates of the global selection pass and the hard drops of the batch passes (feature -> reason).

        A batch pass keeps every feature of a stage that would drop them all, leak-gate drops included, so those drops
        (recorded in its ``dropped_detail``) are taken out of the candidates here: no fallback may bring them back
        (0.9.3). Under ``on_empty_stage='keep_all'`` they stay, as up to 0.9.3.
        """
        from Modeling_Tool.Feature.Weighted_Screen import hard_dropped_features

        candidates = self._dedupe([feat for res in batch_results for feat in as_list(res.selected_features)])
        hard_dropped: dict[str, str] = {}
        for res in batch_results:
            hard_dropped.update(hard_dropped_features((res.selection_summary or {}).get("dropped_detail")))
        if (self.config.selection_params or {}).get("on_empty_stage", "keep_all_warn") == "keep_all":
            return candidates, hard_dropped
        return [feat for feat in candidates if feat not in hard_dropped], hard_dropped

    def _run_batch_global_selection(
        self,
        csv_path: Path,
        header: list[str],
        target_cols: list[str],
        split_labels: pd.Series,
        candidates: list[str],
    ) -> FeatureValidationPipelineResult:
        """Second pass of the batch selection: read the candidates of every batch together and run the selection once
        on them as a run without batches does (WOE fit, correlation across batches, VIF, caps). The report stages are
        switched off; the selection stages keep the values they take in a non-batch run."""
        cfg = self.config
        params = dict(cfg.selection_params or {})
        params.setdefault("psi_enabled", cfg.psi_enabled)
        params.setdefault("iv_enabled", cfg.ivks_enabled)
        params.setdefault("corr_enabled", cfg.corr_enabled and cfg.batch_corr_mode != "off")
        base_cols = self._resolve_batch_base_cols(header, [], target_cols)
        data = self._read_csv(csv_path, usecols=self._dedupe(base_cols + list(candidates))).copy()
        data["_smf_batch_split"] = split_labels.to_numpy()
        selection_cfg = replace(
            cfg,
            output_dir=str(Path(cfg.output_dir) / cfg.batch_output_subdir / "selection"),
            input_type="dataframe",
            enable_batch=False,
            feature_batch_size=None,
            feature_batches=None,
            new_feature_cols=list(candidates),
            incumbent_feature_cols=[],
            split_col="_smf_batch_split",
            selection_params=params,
            distribution_enabled=False,
            psi_enabled=False,
            ivks_enabled=False,
            corr_enabled=False,
            write_outputs=False,
            write_excel=False,
        )
        with warnings.catch_warnings():
            warnings.filterwarnings("ignore", message=r".*missing or unknown label in '_smf_batch_split'.*")
            selection_pipeline = FeatureValidationPipeline(selection_cfg)
            selection_pipeline._nested_run = True
            return selection_pipeline.run(data)

    @staticmethod
    def _slim_batch_result(result: FeatureValidationPipelineResult) -> FeatureValidationPipelineResult:
        """Drop per-batch raw/WOE dataframes that are not needed for final merge."""
        woe_artifacts = dict(result.woe_artifacts or {})
        woe_artifacts["by_target"] = {}
        for key in (
            "categorical_transform_stats_by_target",
            "unseen_category_stats_by_target",
            "unseen_special_stats_by_target",
        ):
            woe_artifacts[key] = copy.deepcopy(woe_artifacts.get(key, {}))
        return FeatureValidationPipelineResult(
            splits={},
            distribution_summary=result.distribution_summary,
            woe_artifacts=woe_artifacts,
            psi_summary=result.psi_summary,
            psi_details=result.psi_details,
            ivks_summary=result.ivks_summary,
            corr_matrix=result.corr_matrix,
            high_corr_pairs=result.high_corr_pairs,
            correlated_detail=result.correlated_detail,
            validation_summary=result.validation_summary,
            output_paths=result.output_paths,
            report_path=result.report_path,
            batch_metadata=result.batch_metadata,
            batch_results=result.batch_results,
            selected_features=list(result.selected_features),
            selection_summary=dict(result.selection_summary or {}),
            screening_artifact=None,
            config_snapshot=dict(result.config_snapshot or {}),
            ivks_sv_pure_bins=result.ivks_sv_pure_bins,
        )

    def _validate_batch_config(self, new_features: list[str]) -> None:
        cfg = self.config
        if cfg.feature_batch_size is not None and cfg.feature_batch_size <= 0:
            raise ValueError("feature_batch_size must be a positive integer.")
        if cfg.batch_corr_pair_chunk_size is not None and cfg.batch_corr_pair_chunk_size <= 0:
            raise ValueError("batch_corr_pair_chunk_size must be a positive integer.")
        if cfg.batch_corr_mode not in {"within_batch", "block_pairwise", "off"}:
            raise ValueError("batch_corr_mode must be one of within_batch/block_pairwise/off")
        method = str(cfg.corr_params.get("method", "pearson")).lower()
        if cfg.batch_corr_mode == "block_pairwise" and method == "kendall":
            raise ValueError("batch_corr_mode='block_pairwise' does not support corr method 'kendall'.")
        if cfg.batch_corr_mode == "block_pairwise" and method not in {"pearson", "spearman"}:
            raise ValueError("block_pairwise correlation supports only pearson or spearman.")
        if cfg.feature_batches:
            unknown = sorted(set(sum((list(batch) for batch in cfg.feature_batches), [])) - set(new_features))
            if unknown:
                raise ValueError(f"feature_batches contains unknown features: {unknown}")

    def _resolve_csv_new_features(self, header: list[str], sample: pd.DataFrame | None = None) -> list[str]:
        cfg = self.config
        header_set = set(header)
        if cfg.new_feature_cols:
            missing = [col for col in cfg.new_feature_cols if col not in header_set]
            if missing:
                raise KeyError(f"Missing new_feature_cols in CSV: {missing}")
            return list(dict.fromkeys(cfg.new_feature_cols))
        excluded = self._auto_excluded_columns()
        if cfg.batch_base_cols:
            excluded.update(cfg.batch_base_cols)
        candidates = [col for col in header if col not in excluded]
        if sample is not None:
            # the DataFrame path takes numeric columns only; the sample of the first rows gives the CSV path the dtypes
            candidates = [col for col in candidates if col in sample.columns and pd.api.types.is_numeric_dtype(sample[col])]
        return candidates

    def _make_feature_batches(self, new_features: list[str]) -> list[list[str]]:
        cfg = self.config
        if cfg.feature_batches:
            seen = set()
            batches = []
            for batch in cfg.feature_batches:
                item = [col for col in dict.fromkeys(batch) if col in new_features and col not in seen]
                seen.update(item)
                if item:
                    batches.append(item)
            remaining = [col for col in new_features if col not in seen]
            if remaining:
                batches.append(remaining)
            return batches
        size = int(cfg.feature_batch_size or len(new_features))
        return [new_features[i : i + size] for i in range(0, len(new_features), size)]

    def _resolve_batch_base_cols(
        self,
        header: list[str],
        incumbent_features: list[str],
        target_cols: list[str],
    ) -> list[str]:
        cfg = self.config
        candidates = []
        if cfg.batch_base_cols:
            candidates.extend(cfg.batch_base_cols)
        candidates.extend(
            [
                cfg.id_col,
                cfg.apply_time_col,
                cfg.split_col,
                cfg.sample_col,
                cfg.oot_col,
                *target_cols,
                *cfg.time_dims,
                *cfg.population_dims,
                *as_list(cfg.categorical_features),
                *as_list(cfg.woe_plot_groups),
                *self._grouping_columns(),
                *incumbent_features,
            ]
        )
        header_set = set(header)
        return self._dedupe([col for col in candidates if col and col in header_set])

    def _stable_split_labels(self, base_df: pd.DataFrame, splits: dict[str, pd.DataFrame]) -> pd.Series:
        cfg = self.config
        if cfg.split_col and cfg.split_col in base_df.columns:
            return normalize_split_values(base_df[cfg.split_col])

        # a row that belongs to no split (missing or unknown label) keeps a missing label: starting every row as "ins"
        # used to promote those rows to the training sample of every batch while the non-batch run leaves them out
        labels = pd.Series(pd.NA, index=base_df.index, dtype=object)
        ins_ids = set(splits.get("ins", pd.DataFrame()).get("_smf_batch_row_id", pd.Series(dtype=int)).tolist())
        oos_ids = set(splits.get("oos", pd.DataFrame()).get("_smf_batch_row_id", pd.Series(dtype=int)).tolist())
        oot_ids = set(splits.get("oot", pd.DataFrame()).get("_smf_batch_row_id", pd.Series(dtype=int)).tolist())
        unique_oot_ids = oot_ids - oos_ids
        row_ids = base_df["_smf_batch_row_id"]
        labels.loc[row_ids.isin(ins_ids)] = "ins"
        labels.loc[row_ids.isin(oos_ids)] = "oos"
        labels.loc[row_ids.isin(unique_oot_ids)] = "oot"
        for name, frame in splits.items():
            if name in {"ins", "oos", "oot"}:
                continue
            ids = set(frame.get("_smf_batch_row_id", pd.Series(dtype=int)).tolist())
            labels.loc[row_ids.isin(ids)] = name
        return labels

    def _merge_batch_results(
        self,
        batch_results: list[FeatureValidationPipelineResult],
        batch_metadata: pd.DataFrame,
        base_splits: dict[str, pd.DataFrame],
        new_features: list[str],
        incumbent_features: list[str],
        target_cols: list[str],
        n_rows: int,
        csv_path: Path,
        feature_batches: list[list[str]],
        global_selection: FeatureValidationPipelineResult | None = None,
    ) -> FeatureValidationPipelineResult:
        # a failed batch has no result: carry the id of the batch with each result so that the keys of the merged outputs
        # still match batch_metadata and the folders on disk
        status = batch_metadata["status"] if "status" in batch_metadata else pd.Series(dtype=object)
        batch_ids = [int(idx) for idx in batch_metadata.loc[status.eq("ok"), "batch_id"]]
        failed_rows = batch_metadata.loc[status.ne("ok")]
        if len(failed_rows):
            failed_features = {
                col for text in failed_rows["features"].astype(str) for col in text.split(",") if col
            }
            analysed_features = [col for col in new_features if col not in failed_features]
            feature_batches = [batch for idx, batch in enumerate(feature_batches) if idx in set(batch_ids)]
            warnings.warn(
                f"FeatureValidationPipeline: {len(failed_rows)} of {len(batch_metadata)} batches failed "
                f"(batch_id {failed_rows['batch_id'].tolist()}); their {len(failed_features)} features are missing from "
                f"every output. First error: {failed_rows['error'].iloc[0]}",
                RuntimeWarning,
                stacklevel=3,
            )
            new_features = analysed_features
        else:
            failed_features = set()
        distribution_summary = self._merge_distribution_summaries([res.distribution_summary for res in batch_results])
        woe_artifacts = self._merge_woe_artifacts(
            [res.woe_artifacts for res in batch_results], batch_metadata, incumbent_features
        )
        psi_summary = self._concat_frames([res.psi_summary for res in batch_results])
        psi_details = self._merge_psi_details(batch_results, batch_ids)
        ivks_summary = self._concat_frames([res.ivks_summary for res in batch_results])
        ivks_sv_pure_bins = self._concat_frames([res.ivks_sv_pure_bins for res in batch_results])
        if ivks_sv_pure_bins.empty:
            ivks_sv_pure_bins = _empty_sv_pure_bins()
        corr_matrix = self._merge_corr_matrices([res.corr_matrix for res in batch_results])
        high_corr_pairs = self._concat_frames([res.high_corr_pairs for res in batch_results])
        correlated_detail = self._concat_frames([res.correlated_detail for res in batch_results])
        if self.config.batch_corr_mode == "block_pairwise" and self.config.corr_enabled:
            cross_pairs = self._run_block_pairwise_correlation(csv_path, feature_batches)
            high_corr_pairs = self._concat_frames([high_corr_pairs, cross_pairs])
        high_corr_pairs = self._dedupe_high_corr_pairs(high_corr_pairs)
        if self.config.batch_corr_mode == "block_pairwise" and self.config.corr_enabled:
            cross_pairs_for_detail = (
                high_corr_pairs[high_corr_pairs["pair_type"].eq("new_new_cross_batch")].copy()
                if "pair_type" in high_corr_pairs.columns
                else pd.DataFrame()
            )
            cross_detail = self._cross_batch_correlated_detail(
                csv_path, cross_pairs_for_detail, target_cols, ivks_summary=ivks_summary
            )
            correlated_detail = self._concat_frames([correlated_detail, cross_detail])

        feature_sources = self._feature_source_frame(new_features, incumbent_features)
        validation_summary = self._build_batch_validation_summary(
            n_rows=n_rows,
            new_features=new_features,
            incumbent_features=incumbent_features,
            target_cols=target_cols,
            distribution_summary=distribution_summary,
            psi_summary=psi_summary,
            ivks_summary=ivks_summary,
            high_corr_pairs=high_corr_pairs,
            woe_artifacts=woe_artifacts,
            batch_metadata=batch_metadata,
            n_failed_features=len(failed_features),
        )
        config_snapshot = self._build_config_snapshot(
            new_features,
            incumbent_features,
            target_cols,
            batch_mode=True,
            columns=base_splits["ins"].columns,
        )
        selected_features: list[str] = []
        selection_summary: dict[str, Any] = {}
        screening_artifact = None
        if self.config.selection_enabled and target_cols:
            selected_features, selection_summary, screening_artifact = self._merge_batch_selection(
                batch_results=batch_results,
                batch_ids=batch_ids,
                global_selection=global_selection,
                new_features=new_features,
                incumbent_features=incumbent_features,
                target_cols=target_cols,
                config_snapshot=config_snapshot,
                failed_features=failed_features,
            )
        tables = self._collect_tables(
            feature_sources,
            distribution_summary,
            woe_artifacts,
            psi_summary,
            ivks_summary,
            corr_matrix,
            high_corr_pairs,
            correlated_detail,
            validation_summary,
            selected_features=selected_features,
            selection_summary=selection_summary,
            ivks_sv_pure_bins=ivks_sv_pure_bins,
        )
        tables["batch_metadata"] = batch_metadata
        output_paths, report_path = self._write_outputs(tables)
        return FeatureValidationPipelineResult(
            # the row id that the batch run added is internal
            splits={name: frame.drop(columns=["_smf_batch_row_id"], errors="ignore") for name, frame in base_splits.items()},
            distribution_summary=distribution_summary,
            woe_artifacts=woe_artifacts,
            psi_summary=psi_summary,
            psi_details=psi_details,
            ivks_summary=ivks_summary,
            corr_matrix=corr_matrix,
            high_corr_pairs=high_corr_pairs,
            correlated_detail=correlated_detail,
            validation_summary=validation_summary,
            output_paths=output_paths,
            report_path=report_path,
            batch_metadata=batch_metadata,
            batch_results={
                f"batch_{idx:03d}": {
                    "output_paths": res.output_paths,
                    "report_path": res.report_path,
                }
                for idx, res in zip(batch_ids, batch_results)
            },
            selected_features=selected_features,
            selection_summary=selection_summary,
            screening_artifact=screening_artifact,
            config_snapshot=config_snapshot,
            ivks_sv_pure_bins=ivks_sv_pure_bins,
        )

    def _merge_batch_selection(
        self,
        batch_results: list[FeatureValidationPipelineResult],
        batch_ids: list[int],
        global_selection: FeatureValidationPipelineResult | None,
        new_features: list[str],
        incumbent_features: list[str],
        target_cols: list[str],
        config_snapshot: dict[str, Any],
        failed_features: set[str],
    ) -> tuple[list[str], dict[str, Any], Any | None]:
        """Selection outputs of a batch run: the features, stage tables and artifact of the global pass, with the
        per-feature tables of every batch (they cover the features that never became candidates)."""
        candidates, _ = self._batch_candidates(batch_results)
        global_summary = dict(global_selection.selection_summary or {}) if global_selection is not None else {}
        selected = list(global_selection.selected_features) if global_selection is not None else []
        snapshot = self._selection_config_snapshot(config_snapshot, self._build_selection_config(), incumbent_features)
        summary: dict[str, Any] = dict(global_summary)
        summary.update({
            "initial_features": list(new_features),
            "batch_candidates": candidates,
            "final_features": selected,
            "target_col": target_cols[0],
            "config_snapshot": snapshot,
        })
        batch_summaries = [dict(res.selection_summary or {}) for res in batch_results]
        for key in ("missing_rate", "missing_rate_dropped", "psi", "iv"):
            merged = self._concat_frames([item.get(key) for item in batch_summaries])
            if not merged.empty:
                summary[key] = merged
        dropped = self._concat_frames(
            [item.get("dropped_detail") for item in batch_summaries] + [global_summary.get("dropped_detail")]
        )
        if not dropped.empty:
            summary["dropped_detail"] = (
                dropped.drop_duplicates(subset=["var", "stage"], keep="last").reset_index(drop=True)
                if {"var", "stage"} <= set(dropped.columns)
                else dropped
            )
        batch_screens = self._concat_frames([
            item["screen_summary"].assign(batch_id=batch_id)
            for batch_id, item in zip(batch_ids, batch_summaries)
            if isinstance(item.get("screen_summary"), pd.DataFrame)
        ])
        if not batch_screens.empty:
            summary["batch_screen_summary"] = batch_screens
        gate_dropped = self._dedupe(
            [var for item in batch_summaries for var in as_list(item.get("missing_gate_dropped"))]
        )
        if gate_dropped:
            summary["missing_gate_dropped"] = gate_dropped
        if failed_features:
            summary["failed_features"] = sorted(failed_features)
        artifact = None
        if global_selection is not None and global_selection.screening_artifact is not None:
            artifact = replace(
                global_selection.screening_artifact,
                selection_summary=summary,
                config_snapshot=snapshot,
            )
        return selected, summary, artifact

    @staticmethod
    def _dedupe(values: list[str]) -> list[str]:
        return list(dict.fromkeys(values))

    def _merge_distribution_summaries(self, summaries: list[dict[str, pd.DataFrame]]) -> dict[str, pd.DataFrame]:
        keys = sorted({key for summary in summaries for key in summary})
        merged: dict[str, pd.DataFrame] = {}
        for key in keys:
            merged[key] = self._concat_frames([summary.get(key, pd.DataFrame()) for summary in summaries])
        return merged

    def _merge_woe_artifacts(
        self,
        artifacts: list[dict[str, Any]],
        batch_metadata: pd.DataFrame,
        incumbent_features: list[str] | None = None,
    ) -> dict[str, Any]:
        valid = [item for item in artifacts if item]
        woe_table = self._concat_frames([item.get("woe_table", pd.DataFrame()) for item in valid])
        var_col = next((col for col in ("VAR", "var") if col in woe_table.columns), None)
        if var_col is not None and incumbent_features:
            # every batch fits the incumbent features again (for the correlation), so their bins came out once per batch
            # and the IV summed over the table was multiplied by the number of batches
            is_incumbent = woe_table[var_col].isin(list(incumbent_features))
            woe_table = (
                pd.concat([woe_table[~is_incumbent], woe_table[is_incumbent].drop_duplicates()])
                .sort_index()
                .reset_index(drop=True)
            )
        return {
            "by_target": {},
            "woe_table": woe_table,
            "refine_summary": self._concat_frames([item.get("refine_summary", pd.DataFrame()) for item in valid]),
            "categorical_transform_stats_by_target": self._merge_transform_stats(
                valid, "categorical_transform_stats_by_target"
            ),
            "unseen_category_stats_by_target": self._merge_transform_stats(
                valid, "unseen_category_stats_by_target"
            ),
            "unseen_special_stats_by_target": self._merge_transform_stats(
                valid, "unseen_special_stats_by_target"
            ),
            "batch_metadata": batch_metadata,
        }

    @staticmethod
    def _merge_transform_stats(
        artifacts: list[dict[str, Any]], key: str
    ) -> dict[str, Any]:
        """Deep-merge target/split/feature audit stats without silent overwrite."""
        merged: dict[str, Any] = {}
        for artifact in artifacts:
            payload = artifact.get(key, {}) or {}
            for target, split_stats in payload.items():
                target_out = merged.setdefault(target, {})
                for split, feature_stats in split_stats.items():
                    split_out = target_out.setdefault(split, {})
                    for feature, stats in feature_stats.items():
                        value = copy.deepcopy(stats)
                        if feature in split_out and split_out[feature] != value:
                            raise ValueError(
                                f"conflicting {key} stats for target={target!r}, "
                                f"split={split!r}, feature={feature!r}"
                            )
                        split_out[feature] = value
        return merged

    @staticmethod
    def _concat_frames(frames: list[pd.DataFrame]) -> pd.DataFrame:
        valid = [df for df in frames if isinstance(df, pd.DataFrame) and not df.empty]
        return pd.concat(valid, ignore_index=True) if valid else pd.DataFrame()

    @staticmethod
    def _merge_psi_details(
        batch_results: list[FeatureValidationPipelineResult], batch_ids: list[int] | None = None
    ) -> dict[str, Any]:
        details: dict[str, Any] = {}
        for idx, result in zip(batch_ids if batch_ids is not None else range(len(batch_results)), batch_results):
            for key, value in (result.psi_details or {}).items():
                details[f"batch_{idx:03d}:{key}"] = value
        return details

    def _merge_corr_matrices(self, matrices: list[pd.DataFrame]) -> pd.DataFrame:
        matrices = [matrix for matrix in matrices if isinstance(matrix, pd.DataFrame) and not matrix.empty]
        if not matrices:
            return pd.DataFrame()
        all_cols = self._dedupe([col for matrix in matrices for col in matrix.columns])
        merged = pd.DataFrame(np.nan, index=all_cols, columns=all_cols)
        for matrix in matrices:
            merged.loc[matrix.index, matrix.columns] = matrix.to_numpy()
        return merged

    def _run_block_pairwise_correlation(self, csv_path: Path, feature_batches: list[list[str]]) -> pd.DataFrame:
        cfg = self.config
        method = str(cfg.corr_params.get("method", "pearson")).lower()
        threshold = float(cfg.corr_params.get("corr_cutpoint", 0.75))
        rows = []
        for left_idx in range(len(feature_batches)):
            for right_idx in range(left_idx + 1, len(feature_batches)):
                for left_cols in self._corr_subchunks(feature_batches[left_idx]):
                    for right_cols in self._corr_subchunks(feature_batches[right_idx]):
                        cols = self._dedupe(left_cols + right_cols)
                        data = self._read_csv(csv_path, usecols=cols)
                        numeric_cols = [col for col in cols if pd.api.types.is_numeric_dtype(data[col])]
                        left_numeric = [col for col in left_cols if col in numeric_cols]
                        right_numeric = [col for col in right_cols if col in numeric_cols]
                        if not left_numeric or not right_numeric:
                            continue
                        corr_block = data[numeric_cols].corr(method=method).loc[
                            left_numeric,
                            right_numeric,
                        ]
                        values = corr_block.to_numpy(dtype=float)
                        row_idx, col_idx = np.nonzero(
                            np.isfinite(values) & (np.abs(values) > threshold)
                        )
                        if len(row_idx):
                            left_names = np.asarray(left_numeric, dtype=object)
                            right_names = np.asarray(right_numeric, dtype=object)
                            selected = values[row_idx, col_idx]
                            rows.extend(
                                pd.DataFrame(
                                    {
                                        "var1": left_names[row_idx],
                                        "var2": right_names[col_idx],
                                        "corr": selected,
                                        "abs_corr": np.abs(selected),
                                        "pair_type": "new_new_cross_batch",
                                        "corr_method": method,
                                        "batch_left": left_idx,
                                        "batch_right": right_idx,
                                    }
                                ).to_dict("records")
                            )
        return pd.DataFrame(rows)

    def _cross_batch_correlated_detail(
        self,
        csv_path: Path,
        cross_pairs: pd.DataFrame,
        target_cols: list[str],
        ivks_summary: pd.DataFrame | None = None,
    ) -> pd.DataFrame:
        """Detail rows of the pairs that ``block_pairwise`` found across batches. The engines of the batches are not
        kept, so the metrics come from the global rows of the merged ``ivks_summary`` when the rows of a batch use the
        WOE engine too (``corr_use_woe_bins`` and ``ivks_use_woe_bins``); otherwise, or for a feature that the summary
        lacks, they are computed on the raw values as the correlation filter does. ``metric_source`` says which."""
        if cross_pairs.empty or not target_cols:
            return pd.DataFrame()
        required_cols = {"var1", "var2"}
        if not required_cols.issubset(cross_pairs.columns):
            return pd.DataFrame()

        from Modeling_Tool import CorrelationFilter

        cfg = self.config
        pair_features = self._dedupe(
            cross_pairs["var1"].dropna().astype(str).tolist()
            + cross_pairs["var2"].dropna().astype(str).tolist()
        )
        read_cols = self._dedupe(pair_features + target_cols)
        data = self._read_csv(csv_path, usecols=read_cols)

        rows = []
        for target in target_cols:
            if target not in data.columns:
                continue
            observed = data[data[target].notna()].copy()
            if len(observed) < cfg.min_group_size or observed[target].nunique() < 2:
                continue
            params = dict(cfg.corr_params or {})
            params.pop("max_iterations", None)
            corr_method = params.get("method", "pearson")
            params["screen_iv_convention"] = self._screen_iv_convention()
            filt = CorrelationFilter(data=observed, dep=target, **params)
            gains = filt._metric_summary(pair_features)
            metric_cols = [
                col
                for col in ["var", "iv", "ks_in_gains", "lift_in_gains"]
                if col in gains.columns
            ]
            metric_map = (
                gains[metric_cols].set_index("var").to_dict("index")
                if "var" in gains.columns
                else {}
            )
            metric_source = {var: "raw_values" for var in metric_map}
            engine_map = self._ivks_engine_metrics(ivks_summary, target)
            for var, metrics in engine_map.items():
                metric_map[var] = metrics
                metric_source[var] = "ivks_summary"
            for pair in cross_pairs.to_dict("records"):
                var1 = str(pair.get("var1"))
                var2 = str(pair.get("var2"))
                if var1 not in observed.columns or var2 not in observed.columns:
                    continue
                metric_name = (
                    "ks_in_gains"
                    if str(params.get("base_metric", "iv")).lower() == "ks"
                    else "iv"
                )
                metric1 = metric_map.get(var1, {}).get(metric_name)
                metric2 = metric_map.get(var2, {}).get(metric_name)
                keep_var = var1
                if pd.notna(metric2) and (pd.isna(metric1) or float(metric2) > float(metric1)):
                    keep_var = var2
                row_corr_method = pair.get("corr_method", corr_method)
                for var in (var1, var2):
                    metrics = metric_map.get(var, {})
                    rows.append(
                        {
                            "target": target,
                            "anchor_var": var1,
                            "var": var,
                            "corr_var1": var1,
                            "corr_var2": var2,
                            "corr": pair.get("corr"),
                            "recommended_action": "keep" if var == keep_var else "remove",
                            "iv": metrics.get("iv"),
                            "ks_in_gains": metrics.get("ks_in_gains"),
                            "lift_in_gains": metrics.get("lift_in_gains"),
                            **self._corr_detail_fields(
                                var1,
                                var2,
                                var,
                                row_corr_method,
                                "cross_batch",
                            ),
                            "pair_type": "new_new_cross_batch",
                            "batch_left": pair.get("batch_left"),
                            "batch_right": pair.get("batch_right"),
                            "metric_source": metric_source.get(var, "raw_values"),
                        }
                    )
        return pd.DataFrame(rows)

    def _ivks_engine_metrics(self, ivks_summary: pd.DataFrame | None, target: str) -> dict[str, dict[str, Any]]:
        """``iv``, ``ks_in_gains`` and ``lift_in_gains`` per feature from the global rows of ``ivks_summary`` for
        ``target``, when those rows and the in-batch correlation detail both come from the WOE engine; else empty."""
        cfg = self.config
        if not (cfg.woe_enabled and cfg.corr_use_woe_bins and cfg.ivks_use_woe_bins):
            return {}
        if not isinstance(ivks_summary, pd.DataFrame) or ivks_summary.empty:
            return {}
        if not {"target", "group_spec", "var"} <= set(ivks_summary.columns):
            return {}
        rows = ivks_summary[ivks_summary["target"].eq(target) & ivks_summary["group_spec"].eq("global")]
        metric_cols = [col for col in ("iv", "ks_in_gains", "lift_in_gains") if col in rows.columns]
        if rows.empty or not metric_cols:
            return {}
        return rows.drop_duplicates("var").set_index("var")[metric_cols].to_dict("index")

    def _corr_subchunks(self, columns: list[str]) -> list[list[str]]:
        size = self.config.batch_corr_pair_chunk_size
        if not size:
            return [columns]
        return [columns[i : i + size] for i in range(0, len(columns), size)]

    def _dedupe_high_corr_pairs(self, high_corr_pairs: pd.DataFrame) -> pd.DataFrame:
        if high_corr_pairs.empty or not {"var1", "var2"}.issubset(high_corr_pairs.columns):
            return high_corr_pairs
        result = high_corr_pairs.copy()
        var1 = result["var1"].astype(str).to_numpy()
        var2 = result["var2"].astype(str).to_numpy()
        # NumPy 1.x keeps ``np.where`` string results as object arrays and
        # rejects mixing them with the Unicode separator in ``np.char.add``.
        # Normalize both sides to the same string dtype before composing keys.
        left = np.asarray(np.where(var1 <= var2, var1, var2), dtype=np.str_)
        right = np.asarray(np.where(var1 <= var2, var2, var1), dtype=np.str_)
        result["_pair_key"] = np.char.add(
            np.char.add(left, "||"),
            right,
        )
        result = result.sort_values("abs_corr", ascending=False, na_position="last")
        result = result.drop_duplicates("_pair_key").drop(columns="_pair_key")
        return result.reset_index(drop=True)

    def _build_batch_validation_summary(
        self,
        n_rows: int,
        new_features: list[str],
        incumbent_features: list[str],
        target_cols: list[str],
        distribution_summary: dict[str, pd.DataFrame],
        psi_summary: pd.DataFrame,
        ivks_summary: pd.DataFrame,
        high_corr_pairs: pd.DataFrame,
        woe_artifacts: dict[str, Any],
        batch_metadata: pd.DataFrame,
        n_failed_features: int = 0,
    ) -> pd.DataFrame:
        rows = [
            {"metric": "n_rows", "value": n_rows},
            {"metric": "n_new_features", "value": len(new_features)},
            {"metric": "n_failed_features", "value": n_failed_features},
            {"metric": "n_incumbent_features", "value": len(incumbent_features)},
            {"metric": "n_targets", "value": len(target_cols)},
            {"metric": "woe_engine", "value": self.config.woe_engine},
            {"metric": "distribution_tables", "value": len(distribution_summary)},
            {"metric": "psi_rows", "value": len(psi_summary)},
            {"metric": "ivks_rows", "value": len(ivks_summary)},
            {"metric": "high_corr_pairs", "value": len(high_corr_pairs)},
            {"metric": "woe_targets", "value": int(woe_artifacts.get("woe_table", pd.DataFrame()).get("target", pd.Series(dtype=object)).nunique()) if woe_artifacts else 0},
            {"metric": "n_batches", "value": len(batch_metadata)},
            {"metric": "batch_status", "value": batch_metadata["status"].value_counts().to_dict() if "status" in batch_metadata else {}},
        ]
        return pd.DataFrame(rows)

    def _build_config_snapshot(
        self,
        new_features: list[str],
        incumbent_features: list[str],
        target_cols: list[str],
        *,
        batch_mode: bool,
        columns: Iterable[str] | None = None,
    ) -> dict[str, Any]:
        cfg = self.config
        return {
            "selection_enabled": bool(cfg.selection_enabled),
            "selection_params": dict(cfg.selection_params or {}),
            "weight_col": cfg.weight_col,
            "target_col": target_cols[0] if target_cols else None,
            "target_cols": list(target_cols),
            "new_feature_cols": list(new_features),
            "incumbent_feature_cols": list(incumbent_features),
            "woe_engine": cfg.woe_engine,
            "woe_fit_query": cfg.woe_fit_query,
            "woe_params": dict(cfg.woe_params or {}),
            "monotone_woe_params": dict(cfg.monotone_woe_params or {}),
            "synthesize_missing_oot": self._synthesize_missing_oot(),
            "woe_fit_scope": cfg.woe_fit_scope,
            "missing_rate_threshold": cfg.missing_rate_threshold,
            "refine_min_n_bins_policy": dict(cfg.monotone_woe_params or {}).get(
                "refine_min_n_bins_policy", "warn"
            ),
            "batch_mode": bool(batch_mode),
            "split": split_settings(cfg, columns),
        }

    def _validate_input(
        self,
        data: pd.DataFrame,
        new_features: list[str],
        incumbent_features: list[str],
        target_cols: list[str],
    ) -> None:
        cfg = self.config
        missing = [col for col in [cfg.id_col, cfg.apply_time_col] if col and col not in data.columns]
        missing += [col for col in new_features + incumbent_features + target_cols if col not in data.columns]
        if cfg.weight_col and cfg.weight_col not in data.columns:
            missing.append(cfg.weight_col)
        if missing:
            raise KeyError(f"Missing required columns: {sorted(set(missing))}")
        if not new_features:
            raise ValueError("new_feature_cols cannot be empty")
        validate_binary_target(data, target_cols, owner="FeatureValidationPipeline", allow_missing=True)
        if cfg.woe_engine.lower() not in {"monotone", "equal_freq"}:
            raise ValueError("woe_engine must be 'monotone' or 'equal_freq'")
        if cfg.woe_enabled and cfg.woe_engine.lower() == "monotone":
            declared = set(as_list(cfg.categorical_features))
            undeclared = [
                col for col in new_features + incumbent_features
                if col not in declared and not pd.api.types.is_numeric_dtype(data[col])
            ]
            if undeclared:
                # the numeric fit failed on them, the message of the failure was lost in the console and the run crashed
                # later with an unrelated KeyError ("WOE column ... was not produced")
                raise ValueError(
                    f"Non-numeric feature(s) {undeclared[:8]} are not listed in categorical_features; the monotone WOE "
                    "engine bins them only as categorical. Declare them in categorical_features, encode them, or drop them."
                )
        if cfg.psi_reference_dataset not in {"ins", "oos", "oot", "external"}:
            raise ValueError("psi_reference_dataset must be one of ins/oos/oot/external")
        if cfg.psi_reference_dataset == "external" and cfg.psi_reference_data is None:
            raise ValueError("psi_reference_data is required when psi_reference_dataset='external'")
        if cfg.woe_fit_query:
            validate_woe_fit_query_columns(cfg.woe_fit_query, data.columns, context="input data")
            validate_woe_fit_query_syntax(data, cfg.woe_fit_query)

    def _grouping_columns(self) -> list[str]:
        """Columns that the grouped analyses and the weighted selection read besides the features: the time and
        population dimensions, the plot, selection, PSI and IV/KS group columns, the columns of ``group_specs`` and the
        weight column."""
        cfg = self.config
        columns = [
            *as_list(cfg.time_dims),
            *as_list(cfg.population_dims),
            *as_list(cfg.woe_plot_groups),
            *as_list(cfg.selection_group_dims),
            *self._resolve_group_dim_columns(list(as_list(cfg.psi_group_dims))),
            *[dim for dim in as_list(cfg.ivks_group_dims) if dim not in {"global", "time", "population"}],
        ]
        if cfg.group_specs is not None:
            for spec in normalize_group_specs(cfg.group_specs):
                columns.extend(spec["columns"])
        if cfg.weight_col:
            columns.append(cfg.weight_col)
        return self._dedupe([str(col) for col in columns if col])

    def _auto_excluded_columns(self) -> set[str]:
        """Columns that are never inferred as new features: the roles (id, time, labels, targets, incumbents, weight and
        grouping columns). The same rule serves the DataFrame and the CSV path."""
        cfg = self.config
        excluded = {cfg.id_col, cfg.apply_time_col, cfg.sample_col}
        if cfg.split_col:
            excluded.add(cfg.split_col)
        if cfg.oot_col:
            excluded.add(cfg.oot_col)
        excluded.update(as_list(cfg.target_cols))
        excluded.update(as_list(cfg.incumbent_feature_cols))
        excluded.update(self._grouping_columns())
        return excluded

    def _resolve_new_features(self, data: pd.DataFrame) -> list[str]:
        cfg = self.config
        if cfg.new_feature_cols:
            return list(dict.fromkeys(cfg.new_feature_cols))
        excluded = self._auto_excluded_columns()
        return [col for col in data.columns if col not in excluded and pd.api.types.is_numeric_dtype(data[col])]

    def _resolve_incumbent_features(self, data: pd.DataFrame, new_features: list[str]) -> list[str]:
        if not self.config.incumbent_feature_cols:
            return []
        new_set = set(new_features)
        return [col for col in dict.fromkeys(self.config.incumbent_feature_cols) if col not in new_set]

    def _add_time_columns(self, data: pd.DataFrame) -> None:
        cfg = self.config
        if cfg.apply_time_col not in data.columns:
            return
        needed = [name for name in ("apply_week", "apply_month", "apply_quarter") if name in cfg.time_dims and name not in data.columns]
        if not needed:
            return
        dt = self._parse_apply_time(data[cfg.apply_time_col])
        if "apply_week" in cfg.time_dims and "apply_week" not in data.columns:
            data["apply_week"] = dt.dt.to_period("W").astype(str)
        if "apply_month" in cfg.time_dims and "apply_month" not in data.columns:
            data["apply_month"] = dt.dt.to_period("M").astype(str)
        if "apply_quarter" in cfg.time_dims and "apply_quarter" not in data.columns:
            data["apply_quarter"] = dt.dt.to_period("Q").astype(str)

    def _parse_apply_time(self, values: pd.Series) -> pd.Series:
        """Parse the application time. Numbers are read as YYYYMMDD integers or Unix epochs (seconds or milliseconds),
        strings of mixed formats (date and date-time) are parsed one by one; ``to_datetime(errors="coerce")`` alone
        read an integer date as nanoseconds since 1970 (every row in '1970-01') and turned half of a mixed column into
        NaT. A warning reports the values that still cannot be parsed."""
        if pd.api.types.is_datetime64_any_dtype(values):
            return pd.to_datetime(values, errors="coerce")
        if pd.api.types.is_numeric_dtype(values) and not pd.api.types.is_bool_dtype(values):
            numbers = pd.to_numeric(values, errors="coerce")
            observed = numbers.dropna()
            if len(observed) and observed.between(19000101, 22001231).all():
                parsed = pd.to_datetime(numbers.astype("Int64").astype("string"), format="%Y%m%d", errors="coerce")
            elif len(observed) and observed.abs().max() >= 1e11:
                parsed = pd.to_datetime(numbers, unit="ms", errors="coerce")
            elif len(observed) and observed.abs().max() >= 1e8:
                parsed = pd.to_datetime(numbers, unit="s", errors="coerce")
            else:
                parsed = pd.to_datetime(numbers, errors="coerce")
        else:
            try:
                parsed = pd.to_datetime(values, errors="coerce", format="mixed")
            except (TypeError, ValueError):
                parsed = pd.to_datetime(values, errors="coerce")
        lost = int((parsed.isna() & values.notna()).sum())
        if lost:
            warnings.warn(
                f"FeatureValidationPipeline: {lost} of {len(values)} values of {self.config.apply_time_col!r} could not be "
                "parsed as dates; their rows have no week, month or quarter.",
                UserWarning,
                stacklevel=3,
            )
        return parsed

    def _synthesize_missing_oot(self) -> bool:
        """Resolve missing-OOT governance without mutating the user config."""
        value = self.config.synthesize_missing_oot
        return False if value is None else bool(value)

    def _split_data(self, data: pd.DataFrame, target_col: str | None) -> dict[str, pd.DataFrame]:
        cfg = self.config
        work = data.copy()
        sample_col = cfg.split_col or cfg.sample_col
        if cfg.split_col and cfg.split_col not in work.columns:
            raise KeyError(f"Missing split_col {cfg.split_col!r}")
        if sample_col in work.columns:
            raw_split = work[sample_col]
            normalized = normalize_split_values(raw_split)
            ins = work[normalized.eq("ins").fillna(False)].copy()
            oos = work[normalized.eq("oos").fillna(False)].copy()
            oot = work[normalized.eq("oot").fillna(False)].copy()
            if len(ins) and len(oos):
                warn_rows_without_split(
                    raw_split, sample_col, "FeatureValidationPipeline", allowed=None if cfg.split_col else ("ins", "oos", "oot")
                )
                if not len(oot):
                    synthesized = resolve_missing_oot(
                        oos,
                        self._synthesize_missing_oot(),
                        "FeatureValidationPipeline",
                    )
                    # FVP keeps the empty-OOT representation: downstream
                    # stages already guard every OOT consumer with len(oot).
                    oot = self._mark_stand_in_oot(synthesized) if synthesized is not None else oot
                splits = {"ins": ins, "oos": oos, "oot": oot}
                if cfg.split_col:
                    reserved = set(splits)
                    for label in normalized.dropna().drop_duplicates().tolist():
                        if label not in reserved:
                            splits[str(label)] = work[normalized.eq(label).fillna(False)].copy()
                return splits
            if cfg.split_col:
                raise ValueError(f"split_col {cfg.split_col!r} must contain non-empty ins and oos samples")

        if cfg.oot_col and cfg.oot_col in work.columns:
            ins_oos, oot = split_oot_by_flag(work, cfg.oot_col)
        else:
            ins_oos = work.copy()
            oot = pd.DataFrame(columns=work.columns)

        if len(ins_oos) == 0:
            return {"ins": ins_oos.copy(), "oos": ins_oos.copy(), "oot": oot.copy()}

        synthesized = None

        if target_col and bool(cfg.split_config.get("stratify", True)) and target_col in ins_oos.columns:
            observed = ins_oos[ins_oos[target_col].notna()].copy()
            missing = ins_oos[ins_oos[target_col].isna()].copy()
            ins, oos = self._split_frame(observed, target_col)
            if len(missing):
                ins = pd.concat([ins, missing], ignore_index=True)
        else:
            ins, oos = self._split_frame(ins_oos, None)
        if len(oot) == 0:
            synthesized = resolve_missing_oot(
                oos,
                self._synthesize_missing_oot(),
                "FeatureValidationPipeline",
            )
            if synthesized is not None:
                oot = synthesized
        splits = {"ins": ins.copy(), "oos": oos.copy(), "oot": oot.copy()}
        if synthesized is not None:
            self._mark_stand_in_oot(splits["oot"])
        return splits

    def _split_frame(self, data: pd.DataFrame, target_col: str | None) -> tuple[pd.DataFrame, pd.DataFrame]:
        cfg = self.config
        test_size = float(cfg.split_config.get("test_size", 0.3))
        random_state = int(cfg.split_config.get("random_state", cfg.random_state))
        if target_col and data[target_col].nunique(dropna=True) > 1:
            try:
                from Modeling_Tool import SampleSplitter

                splitter = SampleSplitter(test_size=test_size, random_state=random_state, stratify=True)
                return splitter.split_df(data, target=target_col)
            except Exception as exc:
                _logger.warning(
                    "SampleSplitter stratified split failed; falling back to naive random split. "
                    "target=%s test_size=%s random_state=%s error=%r",
                    target_col,
                    test_size,
                    random_state,
                    exc,
                )
        oos = data.sample(frac=test_size, random_state=random_state)
        ins = data.drop(index=oos.index)
        return ins.reset_index(drop=True), oos.reset_index(drop=True)

    @staticmethod
    def _mark_stand_in_oot(frame: pd.DataFrame) -> pd.DataFrame:
        """Tag the OOS copy that stands in for a missing OOT, so that the pooled tables can leave it out."""
        frame.attrs["smf_stand_in_oot"] = True
        return frame

    @staticmethod
    def _has_stand_in_oot(splits: dict[str, pd.DataFrame]) -> bool:
        return any(bool(df.attrs.get("smf_stand_in_oot")) for df in splits.values())

    def _combine_splits(self, splits: dict[str, pd.DataFrame], include_stand_in: bool = False) -> pd.DataFrame:
        """Pool the splits, tagged with ``_smf_split``. The stand-in OOT (a copy of OOS) is left out unless
        ``include_stand_in``: pooled tables would count those rows twice."""
        frames = []
        for name, df in splits.items():
            if not include_stand_in and df.attrs.get("smf_stand_in_oot"):
                continue
            item = df.copy()
            item["_smf_split"] = name
            frames.append(item)
        # The empty OOT placeholder (object columns) must not take part in dtype
        # resolution: it turned int features into object, which the distribution
        # summary then reported as categorical.
        return concat_non_empty(frames, ignore_index=True) if frames else pd.DataFrame()

    def _feature_source_frame(self, new_features: list[str], incumbent_features: list[str]) -> pd.DataFrame:
        return pd.DataFrame(
            [{"feature": col, "feature_source": "new"} for col in new_features]
            + [{"feature": col, "feature_source": "incumbent"} for col in incumbent_features]
        )

    def _note_absent_group_columns(self, name: str, group_cols: list[str], columns: Any) -> None:
        """Skip a group spec with an absent column: silently for the time and population dimensions (documented), with a
        warning for the groupings that the user wrote (``group_specs`` and the extra ``ivks_group_dims``)."""
        if name.startswith(("time:", "population:", "time_population:")):
            return
        missing = [col for col in group_cols if col not in set(columns)]
        warnings.warn(
            f"FeatureValidationPipeline: the group {name!r} needs the absent column(s) {missing} and is skipped.",
            UserWarning,
            stacklevel=3,
        )

    def _group_specs(self, include_global: bool = True) -> dict[str, list[str]]:
        cfg = self.config
        if cfg.group_specs is not None:
            return {
                str(spec["name"]): list(spec["columns"])
                for spec in normalize_group_specs(cfg.group_specs)
            }
        specs: dict[str, list[str]] = {}
        if include_global:
            specs["global"] = []
        for col in cfg.time_dims:
            specs[f"time:{col}"] = [col]
        for col in cfg.population_dims:
            specs[f"population:{col}"] = [col]
        for time_col in cfg.time_dims:
            for pop_col in cfg.population_dims:
                specs[f"time_population:{time_col}x{pop_col}"] = [time_col, pop_col]
        return specs

    def _run_distribution(self, data: pd.DataFrame, features: list[str]) -> dict[str, pd.DataFrame]:
        from Modeling_Tool.Feature.Distribution_Tool import proc_means_by_grp as _proc_means_by_grp

        q = self.config.distribution_params.get("q")
        spec_missing_value = self.config.distribution_params.get("spec_missing_value")
        feature_block_size = self.config.distribution_params.get("feature_block_size", 128)
        # bool flags go to the categorical table: describe() drops bool columns
        # mixed with numeric ones, so they used to vanish from the report.
        numeric_features = [
            col for col in features
            if pd.api.types.is_numeric_dtype(data[col]) and not pd.api.types.is_bool_dtype(data[col])
        ]
        categorical_features = [col for col in features if col not in numeric_features]
        tables: dict[str, pd.DataFrame] = {}
        for name, group_cols in self._group_specs(include_global=True).items():
            valid_group_cols = [col for col in group_cols if col in data.columns]
            if len(valid_group_cols) != len(group_cols):
                # a group column that is absent used to collapse the table into the global one under another name
                self._note_absent_group_columns(name, group_cols, data.columns)
                continue
            if numeric_features:
                table = _proc_means_by_grp(
                    data,
                    numeric_features,
                    groupby=valid_group_cols,
                    spec_missing_value=spec_missing_value,
                    q=q,
                    feature_block_size=feature_block_size,
                )
                table.insert(0, "group_spec", name)
                tables[f"numeric_{name}"] = table
            if categorical_features:
                tables[f"categorical_{name}"] = self._categorical_distribution(data, categorical_features, valid_group_cols, name)
        return tables

    def _categorical_distribution(
        self,
        data: pd.DataFrame,
        features: list[str],
        group_cols: list[str],
        group_spec: str,
    ) -> pd.DataFrame:
        rows = []
        grouped = [((), data)] if not group_cols else data.groupby(group_cols, dropna=False)
        for group_value, sub in grouped:
            if not isinstance(group_value, tuple):
                group_value = (group_value,)
            group_info = dict(zip(group_cols, group_value))
            for feature in features:
                vc = sub[feature].value_counts(dropna=False)
                top = vc.head(5)
                rows.append(
                    {
                        "group_spec": group_spec,
                        **group_info,
                        "attribute": feature,
                        "N_ALL": len(sub),
                        "N": int(sub[feature].notna().sum()),
                        "MISSING_RATE": float(sub[feature].isna().mean()),
                        "N_UNIQUE": int(sub[feature].nunique(dropna=True)),
                        "TOP_VALUES": "; ".join(f"{idx}:{cnt}" for idx, cnt in top.items()),
                    }
                )
        return pd.DataFrame(rows)

    def _fit_woe(
        self,
        splits: dict[str, pd.DataFrame],
        new_features: list[str],
        incumbent_features: list[str],
        target_cols: list[str],
        fit_features_override: list[str] | None = None,
    ) -> dict[str, Any]:
        from Modeling_Tool.WOE.WOE_Adapter import as_woe_engine

        cfg = self.config
        fit_features = list(
            new_features if fit_features_override is None else fit_features_override
        )
        if cfg.corr_include_incumbent and cfg.corr_use_woe_bins:
            fit_features = list(dict.fromkeys(fit_features + incumbent_features))

        by_target = {}
        woe_tables = []
        refine_rows = []
        for target in target_cols:
            train = splits["ins"][splits["ins"][target].notna()].copy()
            train, fit_filter_row = apply_woe_fit_query(train, cfg.woe_fit_query, target=target)
            if fit_filter_row is not None:
                refine_rows.append(fit_filter_row)
            if len(train) < max(10, cfg.min_group_size):
                refine_rows.append({"target": target, "step": "fit", "status": "skipped_min_group_size", "n": len(train)})
                continue
            try:
                engine = (
                    self._fit_monotone_binner(train, fit_features, target, refine_rows)
                    if cfg.woe_engine.lower() == "monotone"
                    else self._fit_woe_master(train, fit_features, target)
                )
                adapter = as_woe_engine(engine)
                table = adapter.get_woe_table(fit_features)
                table.insert(0, "target", target)
                woe_tables.append(table)
                woe_splits = {}
                categorical_transform_stats_by_split = {}
                unseen_category_stats_by_split = {}
                unseen_special_stats_by_split = {}
                for name, df in splits.items():
                    woe_splits[name] = adapter.transform(df, varlist=fit_features)
                    categorical_transform_stats_by_split[name] = copy.deepcopy(
                        getattr(engine, "_categorical_transform_stats", {})
                    )
                    unseen_category_stats_by_split[name] = copy.deepcopy(
                        getattr(engine, "_unseen_category_stats", {})
                    )
                    unseen_special_stats_by_split[name] = copy.deepcopy(
                        getattr(engine, "_unseen_special_stats", {})
                    )
                self._plot_woe(engine, adapter, train, fit_features, target)
                by_target[target] = {
                    "engine": engine,
                    "adapter": adapter,
                    "woe_splits": woe_splits,
                    "features": fit_features,
                    "categorical_transform_stats_by_split": (
                        categorical_transform_stats_by_split
                    ),
                    "unseen_category_stats_by_split": unseen_category_stats_by_split,
                    "unseen_special_stats_by_split": unseen_special_stats_by_split,
                }
            except Exception as exc:
                refine_rows.append({"target": target, "step": "fit", "status": "error", "error": repr(exc)})
        return {
            "by_target": by_target,
            "woe_table": pd.concat(woe_tables, ignore_index=True) if woe_tables else pd.DataFrame(),
            "refine_summary": pd.DataFrame(refine_rows),
            "categorical_transform_stats_by_target": {
                target: copy.deepcopy(item["categorical_transform_stats_by_split"])
                for target, item in by_target.items()
            },
            "unseen_category_stats_by_target": {
                target: copy.deepcopy(item["unseen_category_stats_by_split"])
                for target, item in by_target.items()
            },
            "unseen_special_stats_by_target": {
                target: copy.deepcopy(item["unseen_special_stats_by_split"])
                for target, item in by_target.items()
            },
        }

    def _fit_monotone_binner(
        self,
        train: pd.DataFrame,
        features: list[str],
        target: str,
        refine_rows: list[dict[str, Any]],
    ) -> Any:
        from Modeling_Tool import MonotoneWOEBinner

        cfg = self.config
        categorical = [col for col in as_list(cfg.categorical_features) if col in features]
        numeric = [col for col in features if col not in set(categorical)]
        params = with_default_special_values(cfg.monotone_woe_params, train, numeric)
        init_params = {k: v for k, v in params.items() if k in self._MONOTONE_INIT_KEYS}
        fit_params = {k: v for k, v in params.items() if k in self._MONOTONE_FIT_KEYS}
        binner = MonotoneWOEBinner(
            feature_cols=numeric,
            target_col=target,
            cate_feats=categorical,
            **init_params,
        )
        with quiet_default_sentinel(cfg.monotone_woe_params):
            binner.fit(train, **fit_params)
        refine_rows.append({"target": target, "step": "fit_monotone", "status": "ok", "features": ",".join(features)})

        if cfg.monotone_refine_cate_enabled:
            params = dict(cfg.monotone_refine_cate_params or {})
            params.setdefault("features", categorical)
            binner.refine_cate(**params)
            refine_rows.append({"target": target, "step": "refine_cate", "status": "ok", "params": repr(params)})
        if cfg.monotone_refine_dtree_enabled:
            params = dict(cfg.monotone_refine_dtree_params or {})
            params.setdefault("features", numeric)
            binner.refine_dtree(train, **params)
            refine_rows.append({"target": target, "step": "refine_dtree", "status": "ok", "params": repr(params)})
        if cfg.monotone_refine_chi2_enabled:
            params = dict(cfg.monotone_refine_chi2_params or {})
            params.setdefault("features", numeric)
            binner.refine_chi2(train, **params)
            refine_rows.append({"target": target, "step": "refine_chi2", "status": "ok", "params": repr(params)})
        return binner

    def _fit_woe_master(self, train: pd.DataFrame, features: list[str], target: str) -> Any:
        from Modeling_Tool import WOE_Master

        cfg = self.config
        graph_dir = str(Path(cfg.output_dir) / "figs" / "woe" / target)
        master = WOE_Master(
            train_data=train,
            varlist=features,
            dep=target,
            graph_save_dir=graph_dir,
            woe_suffix=cfg.woe_params.get("woe_suffix", "_woe"),
            missing_ref_value=cfg.woe_params.get("missing_ref_value", -999999),
        )
        fit_params = {k: v for k, v in cfg.woe_params.items() if k not in {"woe_suffix", "missing_ref_value"}}
        master.fit(**fit_params)
        return master

    def _plot_woe(self, engine: Any, adapter: Any, train: pd.DataFrame, features: list[str], target: str) -> None:
        cfg = self.config
        if not (cfg.write_outputs and cfg.plot_outputs):
            return
        base_dir = Path(cfg.output_dir) / "figs" / "woe" / target
        make_dirs(base_dir)
        try:
            if adapter.get_engine_name() == "monotone" and hasattr(engine, "plot_woe_graph"):
                engine.plot_woe_graph(str(base_dir))
                for group in cfg.woe_plot_groups:
                    if group in train.columns:
                        engine.plot_woe_graph(str(base_dir / f"by_{group}"), group_name=group, _df_for_group=train)
            elif hasattr(engine, "plot_bivar_graph"):
                transformed = engine.transform(train)
                # Explicit group=None: WOE_Master.plot_bivar_graph made `group`
                # optional in 0.6.2; passing it explicitly documents intent and
                # keeps the call correct even on older WOE_Master builds where
                # `group` was positional (would raise TypeError there — which
                # is what the 0.6.2 logger.warning below now surfaces instead
                # of the pre-0.6.2 silent swallow).
                engine.plot_bivar_graph(transformed, group=None, dirname=str(base_dir))
                for group in cfg.woe_plot_groups:
                    if group in transformed.columns:
                        engine.plot_bivar_graph(transformed, group=group, dirname=str(base_dir / f"by_{group}"))
        except Exception as exc:
            # Never silently swallow: pre-0.6.2 a bare `except: return` here
            # hid the equal_freq blind-spot for the entire 0.5.x/0.6.0 line.
            engine_name = adapter.get_engine_name() if adapter is not None else "<unknown>"
            _logger.warning(
                "WOE plot generation failed and was skipped. "
                "engine=%s target=%s output_dir=%s error=%r",
                engine_name,
                target,
                str(base_dir),
                exc,
            )
            return

    def _build_selection_config(self) -> Any:
        from Modeling_Tool.Feature.Feature_Screen import FeatureScreenConfig, screen_config_from_mapping

        cfg = self.config
        params = dict(cfg.selection_params or {})
        corr_params = dict(cfg.corr_params or {})
        psi_params = dict(cfg.psi_params or {})
        missing_rate_threshold = params.get("missing_rate_threshold", cfg.missing_rate_threshold)
        mapping = {
            "psi_enabled": params.get("psi_enabled", cfg.psi_enabled),
            "psi_threshold": params.get("psi_threshold", 0.2),
            "psi_compare_splits": params.get("psi_compare_splits", ["oos"]),
            "psi_buckets": params.get("psi_buckets", psi_params.get("buckets", 10)),
            "psi_use_woe_bins": params.get("psi_use_woe_bins", cfg.psi_use_woe_bins),
            "iv_enabled": params.get("iv_enabled", cfg.ivks_enabled),
            "iv_threshold": params.get("iv_threshold", 0.02),
            # 2.0 since 0.9.3 (an explicit None switches the limit off)
            "iv_upper_threshold": params.get("iv_upper_threshold", FeatureScreenConfig().iv_upper_threshold),
            "iv_gap_ratio_warn": params.get("iv_gap_ratio_warn", FeatureScreenConfig().iv_gap_ratio_warn),
            "iv_gap_abs_warn": params.get("iv_gap_abs_warn", FeatureScreenConfig().iv_gap_abs_warn),
            "iv_nbins": params.get("iv_nbins", psi_params.get("buckets", 10)),
            "iv_min_bin_prop": params.get("iv_min_bin_prop", psi_params.get("min_bin_prop", 0.05)),
            "iv_equal_freq": params.get("iv_equal_freq", True),
            "iv_use_woe_bins": params.get("iv_use_woe_bins", cfg.ivks_use_woe_bins),
            "corr_enabled": params.get("corr_enabled", cfg.corr_enabled),
            "corr_threshold": params.get(
                "corr_threshold",
                params.get("corr_cutpoint", corr_params.get("corr_cutpoint", 0.75)),
            ),
            "corr_max_iterations": params.get(
                "corr_max_iterations",
                corr_params.get("max_iterations", 10),
            ),
            "corr_use_woe_bins": params.get("corr_use_woe_bins", cfg.corr_use_woe_bins),
            "corr_method": params.get("corr_method", corr_params.get("method", "pearson")),
            "corr_base_metric": params.get("corr_base_metric", corr_params.get("base_metric", "iv")),
            "missing_rate_threshold": missing_rate_threshold,
            "missing_rate_ref": params.get(
                "missing_rate_ref",
                cfg.woe_params.get("missing_ref_value", -999999),
            ),
            # Without this, a selection run that has to fit its own screening
            # binner (no prefit WOE engine) would bin declared categorical
            # features as numeric and crash on string levels.
            "categorical_features": params.get("categorical_features", cfg.categorical_features),
            # Post-corr selection gates (G03/G04/G05/G06); all default off.
            "monthly_iv_min": params.get("monthly_iv_min"),
            "monthly_iv_cv_max": params.get("monthly_iv_cv_max"),
            "direction_consistency_min": params.get("direction_consistency_min"),
            "min_group_n": params.get("min_group_n"),
            "insufficient_group_policy": params.get("insufficient_group_policy", "keep_warn"),
            "target_rules": params.get("target_rules"),
            "min_pass_count": params.get("min_pass_count"),
            "per_target_iv_range": params.get("per_target_iv_range"),
            "direction_reference_target": params.get("direction_reference_target"),
            "max_selected_features": params.get("max_selected_features"),
            "min_selected_features": params.get("min_selected_features"),
            "ranking_metric": params.get("ranking_metric", "iv"),
            "tie_breaker": params.get("tie_breaker", "name"),
            "vif_enabled": params.get("vif_enabled", False),
            "vif_threshold": params.get("vif_threshold", 10.0),
            "vif_min_features": params.get("vif_min_features", 2),
            "vif_tie_break_metric": params.get("vif_tie_break_metric", "iv"),
            "vif_use_woe_bins": params.get("vif_use_woe_bins", False),
            "on_empty_stage": params.get("on_empty_stage", "keep_all_warn"),
            "fallback_top_k": params.get("fallback_top_k", FeatureScreenConfig().fallback_top_k),
        }
        return screen_config_from_mapping(
            mapping,
            woe_engine=cfg.woe_engine,
            woe_fit_query=cfg.woe_fit_query,
            woe_params=cfg.woe_params,
            monotone_woe_params=cfg.monotone_woe_params,
            screen_iv_convention=self._screen_iv_convention(),
        )

    def _evidence_bins_frame(
        self,
        data: pd.DataFrame,
        features: list[str],
        target: str,
        woe_artifacts: dict[str, Any],
    ) -> pd.DataFrame | None:
        engine = (woe_artifacts or {}).get("by_target", {}).get(target, {}).get("engine")
        if engine is None:
            return None
        from Modeling_Tool.WOE.WOE_Adapter import as_woe_engine

        adapter = as_woe_engine(engine)
        if adapter is None:
            return None
        usable = [f for f in features if f in data.columns]
        if not usable:
            return None
        return adapter.assign_bins_frame(data, usable)

    @staticmethod
    def _evidence_iv(
        bins: np.ndarray,
        target_values: np.ndarray,
        sample_weight: np.ndarray | None = None,
        screen_iv_convention: str = "guarded",
        smoothed_woes: list[float] | tuple = (),
    ) -> float:
        """Same WOE/IV formula as _ivks_from_assigned_bins_grouped, minus the
        iv_cut/min_group_size report filters (gates need unfiltered evidence).

        ``sample_weight`` (positional-aligned with ``bins``) switches counts
        to weighted mass so gate evidence matches the weighted modeling
        basis; None reproduces the unweighted arithmetic exactly. Weights are
        expected on frequency scale (e.g. FT weights): the max(total, 1.0)
        floors are kept verbatim, so a zero-mass class or slice degrades to
        iv == 0.0 (never NaN) and sub-unit total mass is damped — both err
        toward dropping under a lower IV floor.

        ``screen_iv_convention='guarded'`` leaves the class-pure bins (no bad
        or no good mass in this slice) out of the sum, as the screening does
        (a special-value bin the engine smoothed, whose WOE is in
        ``smoothed_woes``, counts with that WOE); ``'raw'`` sums every bin
        (0.9.1)."""
        from Modeling_Tool._utils.screen_iv import guard_iv_arrays

        bin_codes, labels = pd.factorize(bins, sort=False)
        if sample_weight is None:
            counts = np.bincount(bin_codes).astype(float)
            bad = np.bincount(bin_codes, weights=target_values)
        else:
            w = np.asarray(sample_weight, dtype=float)
            counts = np.bincount(bin_codes, weights=w)
            bad = np.bincount(bin_codes, weights=w * np.asarray(target_values, dtype=float))
        good = counts - bad
        bad_pct = bad / max(float(bad.sum()), 1.0)
        good_pct = good / max(float(good.sum()), 1.0)
        woe = np.log((bad_pct + 1e-6) / (good_pct + 1e-6))
        contributions = (bad_pct - good_pct) * woe
        iv_raw = float(np.sum(contributions))
        if screen_iv_convention != "guarded":
            return iv_raw
        columns, _ = guard_iv_arrays(
            iv_raw, labels, counts, bad, good, bad_pct, good_pct, contributions, "guarded", smoothed_woes,
        )
        return columns["iv"]

    @staticmethod
    def _smoothed_woes(woe_artifacts: dict[str, Any], target: str, var: str) -> list[float]:
        """WOE values of the special-value bins of ``var`` that the target's engine smoothed (see ``_evidence_iv``)."""
        from Modeling_Tool._utils.screen_iv import smoothed_special_woes

        engine = (woe_artifacts or {}).get("by_target", {}).get(target, {}).get("engine")
        return smoothed_special_woes(engine, var) if engine is not None else []

    def _build_selection_evidence(
        self,
        splits: dict[str, pd.DataFrame],
        target_cols: list[str],
        woe_artifacts: dict[str, Any],
        screen_cfg: Any,
    ) -> Any | None:
        """Lazy G03/G04 evidence closures over INS, priced only on the
        post-corr survivor set the gates hand them.

        Evidence IVs and directions respect ``cfg.weight_col`` so gate
        decisions share the weighted modeling basis; the ``n`` column stays a
        RAW observation count (min_group_n gauges statistical sufficiency,
        which weights do not add to)."""
        cfg = self.config
        group_gates_on = any(
            getattr(screen_cfg, name, None) is not None
            for name in ("monthly_iv_min", "monthly_iv_cv_max", "direction_consistency_min")
        )
        target_gates_on = getattr(screen_cfg, "target_rules", None) is not None
        if not group_gates_on and not target_gates_on:
            return None
        from Modeling_Tool.Feature.Screen_Gates import (
            SelectionEvidence,
            point_biserial_direction,
        )

        ins = splits["ins"]
        primary = target_cols[0]
        missing_ref = getattr(screen_cfg, "missing_rate_ref", None)
        w_ins = None
        if cfg.weight_col:
            from Modeling_Tool.Core.sample_weight_utils import resolve_sample_weight

            w_ins = resolve_sample_weight(
                data=ins, weight_col=cfg.weight_col, expected_len=len(ins)
            )
        group_dims = list(cfg.selection_group_dims or [])
        if group_gates_on:
            if not group_dims:
                raise ValueError(
                    "Group-stability gates (monthly_iv_min/monthly_iv_cv_max/"
                    "direction_consistency_min) require selection_group_dims, "
                    "e.g. ['apply_month']."
                )
            missing_dims = [c for c in group_dims if c not in ins.columns]
            if missing_dims:
                raise KeyError(
                    f"selection_group_dims {missing_dims} not found in the INS split."
                )

        def per_group_iv_fn(features: list[str]) -> pd.DataFrame:
            obs_mask = ins[primary].notna()
            data = ins[obs_mask]
            w_data = w_ins[obs_mask.to_numpy()] if w_ins is not None else None
            bins_frame = self._evidence_bins_frame(data, features, primary, woe_artifacts)
            if bins_frame is None:
                raise ValueError(
                    f"Group-stability gates need the WOE engine for target "
                    f"{primary!r}; none is available (woe fit disabled or failed)."
                )
            y = data[primary].astype(int).to_numpy()
            group_key = group_dims[0] if len(group_dims) == 1 else group_dims
            rows: list[dict] = []
            for group_value, positions in data.groupby(group_key, dropna=False, sort=True).indices.items():
                label = (
                    "|".join(str(v) for v in group_value)
                    if isinstance(group_value, tuple) else str(group_value)
                )
                y_g = y[positions]
                w_g = w_data[positions] if w_data is not None else None
                sub = data.iloc[positions]
                for var in features:
                    if var not in bins_frame.columns:
                        continue
                    bins = bins_frame[var].to_numpy(dtype=object)[positions]
                    bins = np.where(pd.isna(bins), "__MISSING__", bins)
                    rows.append({
                        "var": var,
                        "group": label,
                        "n": int(len(positions)),
                        "iv": self._evidence_iv(
                            bins, y_g, sample_weight=w_g, screen_iv_convention=self._screen_iv_convention(),
                            smoothed_woes=self._smoothed_woes(woe_artifacts, primary, var),
                        ),
                        "direction": point_biserial_direction(
                            sub[var], sub[primary], sample_weight=w_g, missing_ref=missing_ref
                        ),
                    })
            return pd.DataFrame(rows, columns=["var", "group", "n", "iv", "direction"])

        def per_target_fn(features: list[str]) -> pd.DataFrame:
            rows: list[dict] = []
            for target in target_cols:
                obs_mask = ins[target].notna()
                data = ins[obs_mask]
                if not len(data):
                    rows.extend(
                        {"var": var, "target": target, "iv": np.nan,
                         "direction": 0, "status": "no_observed_rows"}
                        for var in features
                    )
                    continue
                w_t = w_ins[obs_mask.to_numpy()] if w_ins is not None else None
                bins_frame = self._evidence_bins_frame(data, features, target, woe_artifacts)
                y = data[target].astype(int).to_numpy()
                for var in features:
                    if bins_frame is None or var not in bins_frame.columns:
                        rows.append({
                            "var": var, "target": target, "iv": np.nan,
                            "direction": point_biserial_direction(
                                data[var], data[target], sample_weight=w_t, missing_ref=missing_ref
                            )
                            if var in data.columns else 0,
                            "status": "engine_missing",
                        })
                        continue
                    bins = bins_frame[var].to_numpy(dtype=object)
                    bins = np.where(pd.isna(bins), "__MISSING__", bins)
                    rows.append({
                        "var": var,
                        "target": target,
                        "iv": self._evidence_iv(
                            bins, y, sample_weight=w_t, screen_iv_convention=self._screen_iv_convention(),
                            smoothed_woes=self._smoothed_woes(woe_artifacts, target, var),
                        ),
                        "direction": point_biserial_direction(
                            data[var], data[target], sample_weight=w_t, missing_ref=missing_ref
                        ),
                        "status": "ok",
                    })
            return pd.DataFrame(rows, columns=["var", "target", "iv", "direction", "status"])

        return SelectionEvidence(
            group_dims=group_dims,
            scope="ins",
            per_group_iv_fn=per_group_iv_fn if group_gates_on else None,
            per_target_fn=per_target_fn if target_gates_on else None,
            min_group_n_default=int(cfg.min_group_size),
        )

    def _run_selection(
        self,
        splits: dict[str, pd.DataFrame],
        new_features: list[str],
        incumbent_features: list[str],
        target_cols: list[str],
        woe_artifacts: dict[str, Any],
        config_snapshot: dict[str, Any],
    ) -> tuple[list[str], dict[str, Any], Any | None]:
        from Modeling_Tool.Feature.Feature_Screen import feature_screen
        from .screening_artifact import FeatureScreeningArtifact, screen_result_to_summary

        cfg = self.config
        target = target_cols[0]
        screen_cfg = self._build_selection_config()
        engine = None
        if woe_artifacts:
            engine = woe_artifacts.get("by_target", {}).get(target, {}).get("engine")

        screen_features = list(new_features)
        if cfg.weight_col and cfg.weight_col not in splits["ins"].columns:
            raise KeyError(f"Missing weight_col {cfg.weight_col!r} for weighted selection.")

        selection_evidence = self._build_selection_evidence(
            splits, target_cols, woe_artifacts, screen_cfg,
        )
        result = feature_screen(
            splits,
            screen_features,
            target,
            weight_col=cfg.weight_col,
            config=screen_cfg,
            prefit_woe_engine=engine,
            selection_evidence=selection_evidence,
        )
        selection_config_snapshot = self._selection_config_snapshot(config_snapshot, screen_cfg, incumbent_features)
        selection_summary = {
            "initial_features": list(new_features),
            "final_features": list(result.selected_features),
            "target_col": target,
            "config_snapshot": selection_config_snapshot,
        }
        selection_summary.update(screen_result_to_summary(result, new_features))
        selection_summary["config_snapshot"] = selection_config_snapshot
        gate_dropped = (woe_artifacts or {}).get("missing_gate_dropped")
        if isinstance(gate_dropped, pd.DataFrame) and len(gate_dropped):
            # features removed by the missing-rate gate before the WOE fit never reach the screening: record them here
            selection_summary["missing_gate_dropped"] = gate_dropped["var"].tolist()
        artifact = FeatureScreeningArtifact.from_screen_result(
            result,
            initial_features=list(new_features),
            target_col=target,
            weight_col=cfg.weight_col,
            woe_artifacts=woe_artifacts,
            source="fvp",
            config_snapshot=selection_config_snapshot,
        )
        if "missing_gate_dropped" in selection_summary:
            artifact.selection_summary["missing_gate_dropped"] = list(selection_summary["missing_gate_dropped"])
        return list(result.selected_features), selection_summary, artifact

    def _selection_config_snapshot(
        self, config_snapshot: dict[str, Any], screen_cfg: Any, incumbent_features: list[str]
    ) -> dict[str, Any]:
        cfg = self.config
        selection_config_snapshot = dict(config_snapshot)
        selection_config_snapshot.update({
            "missing_rate_threshold": screen_cfg.missing_rate_threshold,
            "psi_use_woe_bins": screen_cfg.psi_use_woe_bins,
            "iv_use_woe_bins": screen_cfg.iv_use_woe_bins,
            "corr_use_woe_bins": screen_cfg.corr_use_woe_bins,
            "n_incumbent_features": len(incumbent_features),
            "iv_upper_threshold": screen_cfg.iv_upper_threshold,
            "iv_gap_ratio_warn": screen_cfg.iv_gap_ratio_warn,
            "iv_gap_abs_warn": screen_cfg.iv_gap_abs_warn,
            "screen_iv_convention": screen_cfg.screen_iv_convention,
            "monthly_iv_min": screen_cfg.monthly_iv_min,
            "monthly_iv_cv_max": screen_cfg.monthly_iv_cv_max,
            "direction_consistency_min": screen_cfg.direction_consistency_min,
            "min_group_n": screen_cfg.min_group_n,
            "insufficient_group_policy": screen_cfg.insufficient_group_policy,
            "target_rules": screen_cfg.target_rules,
            "min_pass_count": screen_cfg.min_pass_count,
            "per_target_iv_range": screen_cfg.per_target_iv_range,
            "direction_reference_target": screen_cfg.direction_reference_target,
            "max_selected_features": screen_cfg.max_selected_features,
            "min_selected_features": screen_cfg.min_selected_features,
            "vif_enabled": screen_cfg.vif_enabled,
            "vif_threshold": screen_cfg.vif_threshold,
            "vif_use_woe_bins": screen_cfg.vif_use_woe_bins,
            "selection_group_dims": list(cfg.selection_group_dims or []),
            "evidence_weight_col": cfg.weight_col,
            # the thresholds the screening really used (selection_params holds only the keys that the user overrode)
            "psi_threshold": screen_cfg.psi_threshold,
            "psi_compare_splits": list(screen_cfg.psi_compare_splits or []),
            "iv_threshold": screen_cfg.iv_threshold,
            "corr_threshold": screen_cfg.corr_threshold,
            "corr_method": screen_cfg.corr_method,
            "corr_base_metric": screen_cfg.corr_base_metric,
            "on_empty_stage": screen_cfg.on_empty_stage,
            "fallback_top_k": screen_cfg.fallback_top_k,
            "missing_rate_ref": screen_cfg.missing_rate_ref,
        })
        return selection_config_snapshot

    def _run_psi(
        self,
        combined: pd.DataFrame,
        splits: dict[str, pd.DataFrame],
        features: list[str],
        target_cols: list[str],
        woe_artifacts: dict[str, Any],
    ) -> tuple[pd.DataFrame, dict[str, Any]]:
        from Modeling_Tool import PSICalculator

        cfg = self.config
        reference = cfg.psi_reference_data.copy() if cfg.psi_reference_dataset == "external" else splits[cfg.psi_reference_dataset].copy()
        if cfg.psi_reference_dataset == "external":
            self._add_time_columns(reference)
        if cfg.psi_reference_dataset == "external":
            absent = [col for col in features if col not in reference.columns]
            if absent:
                raise ValueError(
                    f"psi_reference_data lacks the feature column(s) {absent[:8]}; the PSI needs every analysed feature in the "
                    "reference sample."
                )
        if reference.empty:
            # an empty reference gave every feature a large PSI (all mass against none) without any message
            warnings.warn(
                f"FeatureValidationPipeline: the PSI reference sample {cfg.psi_reference_dataset!r} has no rows, so the PSI "
                "is not computed.",
                UserWarning,
                stacklevel=3,
            )
            return pd.DataFrame(), {}
        group_cols = self._resolve_group_dim_columns(cfg.psi_group_dims)
        if "sample" in cfg.psi_group_dims:
            group_cols = ["_smf_split"] + group_cols
        group_cols = list(dict.fromkeys([col for col in group_cols if col in combined.columns]))
        # the PSI by sample reports the stand-in OOT as its own group; every other grouping pools the real rows once
        by_sample = (
            self._combine_splits(splits, include_stand_in=True)
            if "_smf_split" in group_cols and self._has_stand_in_oot(splits)
            else combined
        )

        rows = []
        details: dict[str, Any] = {}
        engines = self._psi_engines(target_cols, woe_artifacts)
        for target_key, engine in engines.items():
            use_woe_bins = cfg.psi_use_woe_bins and engine is not None
            calc = PSICalculator(binning_engine=engine, **cfg.psi_params) if use_woe_bins else PSICalculator(**cfg.psi_params)
            psi_features = list(features)
            if not use_woe_bins:
                # the numeric PSI cannot bin text: one categorical feature used to turn the PSI of every feature into a
                # single error row
                psi_features = [col for col in features if pd.api.types.is_numeric_dtype(combined[col])]
                skipped = [col for col in features if col not in set(psi_features)]
                if skipped:
                    warnings.warn(
                        f"FeatureValidationPipeline: the PSI without WOE bins (psi_use_woe_bins=False or no WOE engine) "
                        f"skips the non-numeric feature(s) {skipped[:8]}; set psi_use_woe_bins=True to include them.",
                        UserWarning,
                        stacklevel=3,
                    )
            prepared: dict[int, Any] = {}
            for group_col in group_cols or [None]:
                frame = by_sample if group_col == "_smf_split" else combined
                try:
                    if use_woe_bins and id(frame) not in prepared:
                        prepared[id(frame)] = calc._prepare_woe_bins(reference, frame, psi_features)
                    prepared_bins = prepared.get(id(frame))
                    if prepared_bins is not None:
                        result = calc._calculate_prebinned(
                            prepared_bins[0],
                            prepared_bins[1],
                            frame,
                            psi_features,
                            group_col,
                            True,
                            calc.psi_missing_bucket_policy,
                        )
                    else:
                        result = calc.calculate(
                            reference,
                            frame,
                            psi_features,
                            group_by=None,
                            group_name=group_col,
                            return_details=True,
                        )
                    if isinstance(result, dict) and "psi" in result:
                        if "details" in result:
                            details[f"{target_key}:{group_col}"] = result["details"]
                        result = result["psi"]
                    if isinstance(result, tuple):
                        result, detail = result
                        details[f"{target_key}:{group_col}"] = detail
                    result = pd.DataFrame(result)
                    if group_col is None and "psi" not in result.columns:
                        result = pd.DataFrame({"psi": result.iloc[:, 0]})
                    if group_col is None:
                        result["group_value"] = "global"
                    elif group_col in result.columns:
                        result["group_value"] = result[group_col]
                    else:
                        result["group_value"] = np.nan
                    result["target"] = target_key
                    result["group_col"] = group_col or "global"
                    rows.append(result)
                except Exception as exc:
                    rows.append(pd.DataFrame({"target": [target_key], "group_col": [group_col or "global"], "error": [repr(exc)]}))
        if not rows:
            return pd.DataFrame(), details
        summary = pd.concat(rows, ignore_index=True)
        if "sample" not in summary.columns and "sample" not in combined.columns:
            # the internal split column is called "sample" in the output, like the psi_group_dims entry that asks for it
            summary = summary.rename(columns={"_smf_split": "sample"})
            summary["group_col"] = summary["group_col"].replace({"_smf_split": "sample"})
            details = {key.replace(":_smf_split", ":sample"): value for key, value in details.items()}
        return summary, details

    def _psi_engines(self, target_cols: list[str], woe_artifacts: dict[str, Any]) -> dict[str, Any]:
        by_target = woe_artifacts.get("by_target", {}) if woe_artifacts else {}
        if by_target:
            return {target: item.get("engine") for target, item in by_target.items()}
        return {"no_target": None}

    def _resolve_group_dim_columns(self, dims: list[str]) -> list[str]:
        cfg = self.config
        cols = []
        if "time" in dims:
            cols.extend(cfg.time_dims)
        if "population" in dims:
            cols.extend(cfg.population_dims)
        cols.extend([dim for dim in dims if dim not in {"global", "sample", "time", "population"}])
        return cols

    def _run_ivks(
        self,
        combined: pd.DataFrame,
        features: list[str],
        target_cols: list[str],
        woe_artifacts: dict[str, Any],
    ) -> pd.DataFrame:
        rows = []
        # the class-pure special-value bins of the global rows on WOE bins (``ivks_sv_pure_bins`` of the result)
        self._ivks_sv_pure_rows: list[dict] = []
        for target in target_cols:
            data = combined[combined[target].notna()].copy()
            if len(data) < self.config.min_group_size:
                continue
            binner = None
            if self.config.ivks_use_woe_bins:
                binner = woe_artifacts.get("by_target", {}).get(target, {}).get("engine")
            bins_frame = None
            if binner is not None:
                from Modeling_Tool.WOE.WOE_Adapter import as_woe_engine

                adapter = as_woe_engine(binner)
                if adapter is not None:
                    feature_block_size = (self.config.ivks_params or {}).get(
                        "feature_block_size", 64
                    )
                    bins_frame = adapter.assign_bins_frame(
                        data,
                        features,
                        feature_block_size=feature_block_size,
                    )
            group_specs = self._ivks_group_specs()
            for group_name, group_cols in group_specs.items():
                valid_group_cols = [col for col in group_cols if col in data.columns]
                if len(valid_group_cols) != len(group_cols):
                    self._note_absent_group_columns(group_name, group_cols, data.columns)
                    continue
                if not valid_group_cols:
                    if bins_frame is not None:
                        global_sv_pure: list[dict] = []
                        rows.append(
                            self._ivks_from_assigned_bins(
                                data, bins_frame, features, target, "global", {},
                                adapter=adapter, sv_pure_rows=global_sv_pure,
                            )
                        )
                        self._ivks_sv_pure_rows.extend({"target": target, **row} for row in global_sv_pure)
                    else:
                        rows.append(self._single_ivks(data, features, target, "global", {}, woe_artifacts))
                elif bins_frame is not None:
                    rows.append(
                        self._ivks_from_assigned_bins_grouped(
                            data,
                            bins_frame,
                            features,
                            target,
                            group_name,
                            valid_group_cols,
                            adapter=adapter,
                        )
                    )
                else:
                    grouped = data.groupby(valid_group_cols, dropna=False, sort=True)
                    for group_value, positions in grouped.indices.items():
                        sub = data.iloc[positions]
                        if len(sub) < self.config.min_group_size:
                            continue
                        if not isinstance(group_value, tuple):
                            group_value = (group_value,)
                        group_info = dict(zip(valid_group_cols, group_value))
                        if bins_frame is not None:
                            rows.append(
                                self._ivks_from_assigned_bins(
                                    sub,
                                    bins_frame.iloc[positions],
                                    features,
                                    target,
                                    group_name,
                                    group_info,
                                )
                            )
                        else:
                            rows.append(self._single_ivks(sub, features, target, group_name, group_info, woe_artifacts))
        frames = [df for df in rows if isinstance(df, pd.DataFrame) and not df.empty]
        return pd.concat(frames, ignore_index=True) if frames else pd.DataFrame()

    def _ivks_group_specs(self) -> dict[str, list[str]]:
        dims = set(self.config.ivks_group_dims)
        specs = {}
        if "global" in dims:
            specs["global"] = []
        if "time" in dims:
            specs.update({f"time:{col}": [col] for col in self.config.time_dims})
        if "population" in dims:
            specs.update({f"population:{col}": [col] for col in self.config.population_dims})
        for dim in dims - {"global", "time", "population"}:
            specs[str(dim)] = [str(dim)]
        return specs

    def _single_ivks(
        self,
        data: pd.DataFrame,
        features: list[str],
        target: str,
        group_spec: str,
        group_info: dict[str, Any],
        woe_artifacts: dict[str, Any],
    ) -> pd.DataFrame:
        from Modeling_Tool import VarExtractionInsights

        cfg = self.config
        binner = None
        if cfg.ivks_use_woe_bins:
            binner = woe_artifacts.get("by_target", {}).get(target, {}).get("engine")
        if binner is not None:
            report = self._ivks_from_binner(data, features, target, binner)
            if report.empty:
                return report
            report.insert(0, "target", target)
            report.insert(1, "group_spec", group_spec)
            for key, value in group_info.items():
                report[key] = value
            return report
        params = dict(cfg.ivks_params or {})
        params.pop("feature_block_size", None)
        iv_cut = float(params.pop("iv_cut", 0.0))
        params["missing_rate_ref"] = self._ivks_missing_ref()
        insights = VarExtractionInsights(
            data=data,
            dep=target,
            plot_path=None,
            woe_binner=binner,
            woe_engine="monotone" if binner is not None else "master",
            **params,
        )
        report = insights.get_var_analysis_report(data, features, dep=target, iv_cut=iv_cut)
        if report is None or len(report) == 0:
            return pd.DataFrame()
        report = report.copy()
        if "iv_raw" not in report.columns and "iv" in report.columns:
            # the default binning has its own class-pure handling: screen_iv_convention does not apply to it
            at = report.columns.get_loc("iv") + 1
            report.insert(at, "iv_raw", report["iv"])
            report.insert(at + 1, "iv_sv_pure", np.nan)
            report.insert(at + 2, "n_sv_pure", np.nan)
        report.insert(0, "target", target)
        report.insert(1, "group_spec", group_spec)
        for key, value in group_info.items():
            report[key] = value
        return report

    def _ivks_missing_ref(self) -> Any:
        """Sentinel that the IV/KS report counts as missing on both paths: ``ivks_params['missing_rate_ref']``, else
        ``woe_params['missing_ref_value']`` (-999999)."""
        params = self.config.ivks_params or {}
        if "missing_rate_ref" in params:
            return params["missing_rate_ref"]
        return (self.config.woe_params or {}).get("missing_ref_value", -999999)

    def _ivks_value_stats(self, series: pd.Series) -> dict[str, Any]:
        """``n_all``, ``n``, ``missing_rate``, ``min``, ``mean`` and ``max`` of a feature with the sentinel counted as
        missing, the rule of the report without WOE bins (``proc_means_for_screening``)."""
        is_numeric = pd.api.types.is_numeric_dtype(series) and not pd.api.types.is_bool_dtype(series)
        ref = self._ivks_missing_ref()
        values = series.mask(series == ref) if is_numeric and ref is not None else series
        n_all = len(values)
        n = int(values.notna().sum())
        return {
            "n_all": n_all,
            "n": n,
            "missing_rate": float(1.0 - n / n_all) if n_all else np.nan,
            "min": float(values.min()) if is_numeric else np.nan,
            "mean": float(values.mean()) if is_numeric else np.nan,
            "max": float(values.max()) if is_numeric else np.nan,
        }

    def _ivks_from_binner(self, data: pd.DataFrame, features: list[str], target: str, binner: Any) -> pd.DataFrame:
        from Modeling_Tool.WOE.WOE_Adapter import as_woe_engine

        adapter = as_woe_engine(binner)
        if adapter is None:
            return pd.DataFrame()
        feature_block_size = (self.config.ivks_params or {}).get("feature_block_size", 64)
        assign_bins_frame = getattr(adapter, "assign_bins_frame", None)
        if callable(assign_bins_frame):
            bins_frame = assign_bins_frame(
                data,
                features,
                feature_block_size=feature_block_size,
            )
        else:
            # Custom adapters written against the original protocol may only
            # expose assign_bins(). Keep them compatible without slowing the
            # built-in block transform path.
            bins_frame = pd.DataFrame(
                {var: adapter.assign_bins(data, var) for var in features},
                index=data.index,
            )
        report = self._ivks_from_assigned_bins(
            data, bins_frame, features, target, "global", {}
        )
        return report.drop(columns=["target", "group_spec"], errors="ignore")

    def _ivks_from_assigned_bins(
        self,
        data: pd.DataFrame,
        bins_frame: pd.DataFrame,
        features: list[str],
        target: str,
        group_spec: str,
        group_info: dict[str, Any],
        adapter: Any = None,
        sv_pure_rows: list[dict] | None = None,
    ) -> pd.DataFrame:
        from Modeling_Tool._utils.screen_iv import (
            class_pure_special_bins,
            guard_screen_iv,
            smoothed_special_woes,
            special_bin_woes,
        )

        convention = self._screen_iv_convention()
        target_series = data[target].astype(int)
        total_bad = max(float((target_series == 1).sum()), 1.0)
        total_good = max(float((target_series == 0).sum()), 1.0)
        overall_bad = max(float(target_series.mean()), 1e-6)
        rows = []
        for var in features:
            if var not in data.columns or data[var].nunique(dropna=False) <= 1:
                continue
            try:
                bins = bins_frame[var]
                # Cast bins to object dtype and replace NaN with an explicit sentinel to
                # avoid mixed Interval + NaN sort issues under older pandas/numpy combos
                # (pandas <2.0 raises TypeError when groupby(sort=True) sees such a mix).
                bin_arr = np.asarray(bins, dtype=object)
                bin_arr = np.where(pd.isna(bin_arr), "__MISSING__", bin_arr)
                tmp = pd.DataFrame({"bin": bin_arr, target: target_series.to_numpy()})
                grouped = (
                    tmp.groupby("bin", dropna=False, sort=False)[target]
                    .agg(["count", "sum"])
                    .reset_index()
                )
                grouped = grouped.rename(columns={"count": "n", "sum": "n_bad"})
                grouped["n_good"] = grouped["n"] - grouped["n_bad"]
                grouped["bad_pct"] = grouped["n_bad"] / total_bad
                grouped["good_pct"] = grouped["n_good"] / total_good
                grouped["woe"] = np.log((grouped["bad_pct"] + 1e-6) / (grouped["good_pct"] + 1e-6))
                grouped["iv_component"] = (grouped["bad_pct"] - grouped["good_pct"]) * grouped["woe"]
                grouped["bad_rate"] = grouped["n_bad"] / grouped["n"].replace(0, np.nan)
                ordered = grouped.sort_values("bad_rate", ascending=False).reset_index(drop=True)
                ks = float((ordered["bad_pct"].cumsum() - ordered["good_pct"].cumsum()).abs().max())
                lift = float((ordered["bad_rate"] / overall_bad).replace([np.inf, -np.inf], np.nan).max())
                per_bin = pd.DataFrame({
                    "label": grouped["bin"], "n": grouped["n"], "bad": grouped["n_bad"], "good": grouped["n_good"],
                    "bad_share": grouped["bad_pct"], "good_share": grouped["good_pct"], "woe": grouped["woe"],
                    "iv": grouped["iv_component"],
                })
                iv_columns, pure = guard_screen_iv(
                    float(grouped["iv_component"].sum()), per_bin, convention,
                    smoothed_special_woes(adapter, var) if adapter is not None else [],
                )
                if sv_pure_rows is not None and pure.any():
                    sv_pure_rows.extend(class_pure_special_bins(
                        var, per_bin.assign(pure=pure), bin_arr, data[var].to_numpy(),
                        special_bin_woes(adapter, var) if adapter is not None else {},
                    ))
                rows.append(
                    {
                        "var": var,
                        **self._ivks_value_stats(data[var]),
                        "ks_in_gains": ks,
                        "lift_in_gains": lift,
                        **iv_columns,
                        "n_bump": int(grouped.shape[0]),
                        "n_bins": int(grouped.shape[0]),
                    }
                )
            except Exception:
                continue
        if not rows:
            return pd.DataFrame()
        report = pd.DataFrame(rows).round(4)
        iv_cut = float((self.config.ivks_params or {}).get("iv_cut", 0.0))
        report = report[report["iv"] >= iv_cut].reset_index(drop=True)
        report.insert(0, "target", target)
        report.insert(1, "group_spec", group_spec)
        for key, value in group_info.items():
            report[key] = value
        return report

    def _ivks_from_assigned_bins_grouped(
        self,
        data: pd.DataFrame,
        bins_frame: pd.DataFrame,
        features: list[str],
        target: str,
        group_spec: str,
        group_cols: list[str],
        adapter: Any = None,
    ) -> pd.DataFrame:
        """Aggregate group x bin x target counts once for each feature."""
        if not group_cols:
            return self._ivks_from_assigned_bins(
                data, bins_frame, features, target, group_spec, {}
            )

        group_key = group_cols[0] if len(group_cols) == 1 else group_cols
        grouped = data.groupby(group_key, dropna=False, sort=True)
        group_codes = grouped.ngroup().to_numpy(dtype=np.int64)
        group_values = list(grouped.size().index)
        group_positions = [
            np.flatnonzero(group_codes == group_idx)
            for group_idx in range(len(group_values))
        ]
        eligible_groups = [
            group_idx
            for group_idx, positions in enumerate(group_positions)
            if len(positions) >= self.config.min_group_size
        ]
        if not eligible_groups:
            return pd.DataFrame()

        from Modeling_Tool._utils.screen_iv import guard_iv_arrays, smoothed_special_woes

        convention = self._screen_iv_convention()
        target_values = data[target].astype(int).to_numpy()
        feature_counts: dict[str, tuple[np.ndarray, np.ndarray, np.ndarray, list[float]]] = {}
        n_groups = len(group_values)
        for var in features:
            if var not in data.columns or data[var].nunique(dropna=False) <= 1:
                continue
            bin_values = bins_frame[var].to_numpy(dtype=object)
            bin_values = np.where(pd.isna(bin_values), "__MISSING__", bin_values)
            bin_codes, categories = pd.factorize(bin_values, sort=False)
            n_bins = len(categories)
            flat_codes = group_codes * n_bins + bin_codes
            counts = np.bincount(
                flat_codes,
                minlength=n_groups * n_bins,
            ).reshape(n_groups, n_bins)
            bad_counts = np.bincount(
                flat_codes,
                weights=target_values,
                minlength=n_groups * n_bins,
            ).reshape(n_groups, n_bins)
            smoothed = smoothed_special_woes(adapter, var) if adapter is not None else []
            feature_counts[var] = (counts, bad_counts, np.asarray(categories, dtype=object), smoothed)

        rows = []
        for group_idx in eligible_groups:
            positions = group_positions[group_idx]
            raw_group_value = group_values[group_idx]
            value_tuple = (
                raw_group_value
                if isinstance(raw_group_value, tuple)
                else (raw_group_value,)
            )
            group_info = dict(zip(group_cols, value_tuple))
            group_target = target_values[positions]
            overall_bad = max(float(group_target.mean()), 1e-6)

            for var, (counts_matrix, bad_matrix, categories, smoothed) in feature_counts.items():
                counts = counts_matrix[group_idx].astype(float, copy=False)
                observed = counts > 0
                counts = counts[observed]
                bad_counts = bad_matrix[group_idx][observed]
                good_counts = counts - bad_counts
                total_bad = max(float(bad_counts.sum()), 1.0)
                total_good = max(float(good_counts.sum()), 1.0)
                bad_pct = bad_counts / total_bad
                good_pct = good_counts / total_good
                woe = np.log((bad_pct + 1e-6) / (good_pct + 1e-6))
                contributions = (bad_pct - good_pct) * woe
                # class purity on this group's counts
                iv_columns, _ = guard_iv_arrays(
                    float(np.sum(contributions)), categories[observed], counts, bad_counts, good_counts, bad_pct,
                    good_pct, contributions, convention, smoothed,
                )
                bad_rate = np.divide(
                    bad_counts,
                    counts,
                    out=np.full_like(bad_counts, np.nan, dtype=float),
                    where=counts > 0,
                )
                order = np.argsort(-np.nan_to_num(bad_rate, nan=-np.inf))
                ks = float(
                    np.max(
                        np.abs(
                            np.cumsum(bad_pct[order])
                            - np.cumsum(good_pct[order])
                        )
                    )
                )
                lift_values = bad_rate / overall_bad
                lift = float(np.nanmax(lift_values)) if len(lift_values) else np.nan

                row = {
                    "var": var,
                    **self._ivks_value_stats(data[var].iloc[positions]),
                    "ks_in_gains": ks,
                    "lift_in_gains": lift,
                    **iv_columns,
                    "n_bump": int(observed.sum()),
                    "n_bins": int(observed.sum()),
                }
                row.update(group_info)
                rows.append(row)

        if not rows:
            return pd.DataFrame()
        report = pd.DataFrame(rows).round(4)
        iv_cut = float((self.config.ivks_params or {}).get("iv_cut", 0.0))
        report = report[report["iv"] >= iv_cut].reset_index(drop=True)
        report.insert(0, "target", target)
        report.insert(1, "group_spec", group_spec)
        return report

    def _run_correlation(
        self,
        combined: pd.DataFrame,
        new_features: list[str],
        incumbent_features: list[str],
        target_cols: list[str],
        woe_artifacts: dict[str, Any],
    ) -> tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame]:
        cfg = self.config
        features = list(new_features)
        if cfg.corr_include_incumbent:
            features = list(dict.fromkeys(features + incumbent_features))
        numeric_features = [col for col in features if col in combined.columns and pd.api.types.is_numeric_dtype(combined[col])]
        method = str(cfg.corr_params.get("method", "pearson"))
        threshold = float(cfg.corr_params.get("corr_cutpoint", 0.75))
        corr_matrix = combined[numeric_features].corr(method=method) if len(numeric_features) >= 2 else pd.DataFrame()
        high_corr_pairs = self._high_corr_pairs(corr_matrix, threshold, new_features, incumbent_features)
        correlated_detail = self._correlated_detail(combined, numeric_features, target_cols, woe_artifacts)
        return corr_matrix, high_corr_pairs, correlated_detail

    def _high_corr_pairs(
        self,
        corr_matrix: pd.DataFrame,
        threshold: float,
        new_features: list[str],
        incumbent_features: list[str],
    ) -> pd.DataFrame:
        # Vectorized in v0.4.0: previous O(n^2) Python loop with .loc[] lookups was
        # a real bottleneck at 3000+ features (~4.5M scalar lookups). Uses np.triu_indices
        # to extract the upper triangle once, filters by threshold, then classifies pair_type
        # via boolean masks. Output is numerically identical to the loop version.
        empty_cols = ["var1", "var2", "corr", "abs_corr", "pair_type"]
        if corr_matrix.empty:
            return pd.DataFrame(columns=empty_cols)
        cols = list(corr_matrix.columns)
        n = len(cols)
        if n < 2:
            return pd.DataFrame(columns=empty_cols)
        values = corr_matrix.to_numpy()
        iu, ju = np.triu_indices(n, k=1)
        pair_corr = values[iu, ju]
        with np.errstate(invalid="ignore"):
            mask = np.isfinite(pair_corr) & (np.abs(pair_corr) > threshold)
        if not mask.any():
            return pd.DataFrame(columns=empty_cols)
        iu = iu[mask]
        ju = ju[mask]
        pair_corr = pair_corr[mask]
        col_arr = np.asarray(cols, dtype=object)
        var1 = col_arr[iu]
        var2 = col_arr[ju]
        new_set = set(new_features)
        incumbent_set = set(incumbent_features)
        v1_new = np.fromiter((v in new_set for v in var1), dtype=bool, count=len(var1))
        v2_new = np.fromiter((v in new_set for v in var2), dtype=bool, count=len(var2))
        v1_inc = np.fromiter((v in incumbent_set for v in var1), dtype=bool, count=len(var1))
        v2_inc = np.fromiter((v in incumbent_set for v in var2), dtype=bool, count=len(var2))
        pair_type = np.full(len(var1), "incumbent_incumbent", dtype=object)
        pair_type[(v1_new & v2_inc) | (v2_new & v1_inc)] = "new_incumbent"
        pair_type[v1_new & v2_new] = "new_new"
        return pd.DataFrame(
            {
                "var1": var1,
                "var2": var2,
                "corr": pair_corr,
                "abs_corr": np.abs(pair_corr),
                "pair_type": pair_type,
            }
        )

    def _correlated_detail(
        self,
        combined: pd.DataFrame,
        features: list[str],
        target_cols: list[str],
        woe_artifacts: dict[str, Any],
    ) -> pd.DataFrame:
        if not target_cols or len(features) < 2:
            return pd.DataFrame()
        from Modeling_Tool import CorrelationFilter

        cfg = self.config
        rows = []
        for target in target_cols:
            data = combined[combined[target].notna()].copy()
            if len(data) < cfg.min_group_size or data[target].nunique() < 2:
                continue
            params = dict(cfg.corr_params or {})
            max_iterations = int(params.pop("max_iterations", 10))
            corr_method = params.get("method", "pearson")
            binner = woe_artifacts.get("by_target", {}).get(target, {}).get("engine") if cfg.corr_use_woe_bins else None
            params["screen_iv_convention"] = self._screen_iv_convention()
            filt = CorrelationFilter(data=data, dep=target, woe_binner=binner, **params)
            keep = filt.remove_highly_correlated(features, max_iterations=max_iterations)
            removed = [feature for feature in features if feature not in keep]
            for anchor, payload in getattr(filt, "correlated_dict", {}).items():
                corr = payload.get("corr", pd.DataFrame()).copy()
                gains = payload.get("gains", pd.DataFrame()).copy()
                metric_cols = [col for col in ["var", "iv", "ks_in_gains", "lift_in_gains"] if col in gains.columns]
                metric_map = gains[metric_cols].set_index("var").to_dict("index") if "var" in gains.columns else {}
                for _, row in corr.iterrows():
                    for var in [row.get("VAR1"), row.get("VAR2")]:
                        metrics = metric_map.get(var, {})
                        rows.append(
                            {
                                "target": target,
                                "anchor_var": anchor,
                                "var": var,
                                "corr_var1": row.get("VAR1"),
                                "corr_var2": row.get("VAR2"),
                                "corr": row.get("CORR"),
                                "recommended_action": "keep" if var in keep else "remove",
                                "iv": metrics.get("iv"),
                                "ks_in_gains": metrics.get("ks_in_gains"),
                                "lift_in_gains": metrics.get("lift_in_gains"),
                                **self._corr_detail_fields(
                                    row.get("VAR1"),
                                    row.get("VAR2"),
                                    var,
                                    corr_method,
                                    "within_batch",
                                ),
                            }
                        )
            if not rows and removed:
                rows.append(
                    {
                        "target": target,
                        "recommended_action": "remove",
                        "var": ",".join(removed),
                        "metric_var": ",".join(removed),
                        "corr_method": corr_method,
                        "detail_scope": "within_batch",
                    }
                )
        return pd.DataFrame(rows)

    @staticmethod
    def _corr_detail_fields(
        corr_var1: Any,
        corr_var2: Any,
        metric_var: Any,
        corr_method: str,
        detail_scope: str,
    ) -> dict[str, Any]:
        corr_var1_str = str(corr_var1) if pd.notna(corr_var1) else ""
        corr_var2_str = str(corr_var2) if pd.notna(corr_var2) else ""
        metric_var_str = str(metric_var) if pd.notna(metric_var) else ""
        if metric_var_str == corr_var1_str:
            metric_var_position = "corr_var1"
        elif metric_var_str == corr_var2_str:
            metric_var_position = "corr_var2"
        else:
            metric_var_position = np.nan
        return {
            "corr_pair_id": f"{corr_var1_str}||{corr_var2_str}" if corr_var1_str or corr_var2_str else np.nan,
            "metric_var": metric_var,
            "metric_var_position": metric_var_position,
            "corr_method": corr_method,
            "detail_scope": detail_scope,
        }

    def _build_validation_summary(
        self,
        data: pd.DataFrame,
        new_features: list[str],
        incumbent_features: list[str],
        target_cols: list[str],
        feature_sources: pd.DataFrame,
        distribution_summary: dict[str, pd.DataFrame],
        psi_summary: pd.DataFrame,
        ivks_summary: pd.DataFrame,
        high_corr_pairs: pd.DataFrame,
        woe_artifacts: dict[str, Any],
        selected_features: list[str] | None = None,
    ) -> pd.DataFrame:
        rows = [
            {"metric": "n_rows", "value": len(data)},
            {"metric": "n_new_features", "value": len(new_features)},
            {"metric": "n_incumbent_features", "value": len(incumbent_features)},
            {"metric": "n_targets", "value": len(target_cols)},
            {"metric": "woe_engine", "value": self.config.woe_engine},
            {"metric": "distribution_tables", "value": len(distribution_summary)},
            {"metric": "psi_rows", "value": len(psi_summary)},
            {"metric": "ivks_rows", "value": len(ivks_summary)},
            {"metric": "high_corr_pairs", "value": len(high_corr_pairs)},
            {"metric": "woe_targets", "value": len(woe_artifacts.get("by_target", {})) if woe_artifacts else 0},
            {"metric": "feature_sources", "value": feature_sources["feature_source"].value_counts().to_dict()},
        ]
        if self.config.selection_enabled:
            rows.append({"metric": "selection_enabled", "value": True})
            rows.append({"metric": "n_selected_features", "value": len(selected_features or [])})
        return pd.DataFrame(rows)

    def _collect_tables(
        self,
        feature_sources: pd.DataFrame,
        distribution_summary: dict[str, pd.DataFrame],
        woe_artifacts: dict[str, Any],
        psi_summary: pd.DataFrame,
        ivks_summary: pd.DataFrame,
        corr_matrix: pd.DataFrame,
        high_corr_pairs: pd.DataFrame,
        correlated_detail: pd.DataFrame,
        validation_summary: pd.DataFrame,
        selected_features: list[str] | None = None,
        selection_summary: dict[str, Any] | None = None,
        ivks_sv_pure_bins: pd.DataFrame | None = None,
    ) -> dict[str, pd.DataFrame]:
        tables = {
            "validation_summary": validation_summary,
            "feature_sources": feature_sources,
            "woe_table": woe_artifacts.get("woe_table", pd.DataFrame()) if woe_artifacts else pd.DataFrame(),
            "woe_refine_summary": woe_artifacts.get("refine_summary", pd.DataFrame()) if woe_artifacts else pd.DataFrame(),
            "psi_summary": psi_summary,
            "ivks_summary": ivks_summary,
            "corr_matrix": corr_matrix.reset_index(names="feature") if not corr_matrix.empty else corr_matrix,
            "high_corr_pairs": high_corr_pairs,
            "correlated_detail": correlated_detail,
        }
        if selected_features:
            tables["selected_features"] = pd.DataFrame({"feature": selected_features})
        if selection_summary:
            for key in ("missing_rate", "missing_rate_dropped", "screen_summary", "batch_screen_summary"):
                value = selection_summary.get(key)
                if isinstance(value, pd.DataFrame) and not value.empty:
                    tables[f"selection_{key}"] = value
            if "final_features" in selection_summary:
                tables["selection_final_features"] = pd.DataFrame(
                    {"feature": list(selection_summary["final_features"])},
                )
            sv_pure_selection = (selection_summary.get("stage_tables") or {}).get("iv_sv_pure_bins")
            if isinstance(sv_pure_selection, pd.DataFrame) and not sv_pure_selection.empty:
                tables["selection_iv_sv_pure_bins"] = sv_pure_selection
        if isinstance(ivks_sv_pure_bins, pd.DataFrame) and not ivks_sv_pure_bins.empty:
            tables["ivks_sv_pure_bins"] = ivks_sv_pure_bins
        for name, df in distribution_summary.items():
            tables[f"distribution_{name}"] = df
        return tables

    def _write_outputs(self, tables: dict[str, pd.DataFrame]) -> tuple[dict[str, str], str | None]:
        cfg = self.config
        output_paths: dict[str, str] = {}
        report_path = None
        if not (cfg.write_outputs or cfg.write_excel):
            return output_paths, report_path
        output_dir = Path(cfg.output_dir)
        make_dirs(output_dir)
        if cfg.write_outputs:
            for name, df in tables.items():
                path = output_dir / f"{name}.csv"
                safe_to_csv(df, path, index=False)
                output_paths[name] = str(path.resolve())
        if cfg.write_excel:
            excel_path = output_dir / "Feature_Validation_Report.xlsx"
            self._write_excel_report(excel_path, tables)
            report_path = str(excel_path.resolve())
            output_paths["excel_report"] = report_path
        return output_paths, report_path

    def _write_excel_report(self, excel_path: Path, tables: dict[str, pd.DataFrame]) -> None:
        from ExcelMaster.ExcelMaster import ExcelMaster

        em = ExcelMaster(str(excel_path), verbose=False)
        used_names: set[str] = set()
        for name, df in tables.items():
            if not isinstance(df, pd.DataFrame) or df.empty:
                continue
            sheet_name = self._excel_sheet_name(name, used_names)
            ws = em.add_worksheet(sheet_name, zoom_perc=90)
            em.write_dataframe(ws, df=df, title=name, index=False)
        if not used_names:
            ws = em.add_worksheet("Summary", zoom_perc=90)
            em.write_dataframe(ws, df=pd.DataFrame({"message": ["No non-empty tables generated."]}), title="Summary", index=False)
        em.close_workbook()

    @staticmethod
    def _excel_sheet_name(name: str, used_names: set[str]) -> str:
        invalid = set("[]:*?/\\")
        base = "".join(ch if ch not in invalid else "_" for ch in name)[:31] or "Sheet"
        candidate = base
        idx = 1
        while candidate in used_names:
            suffix = f"_{idx}"
            candidate = f"{base[:31 - len(suffix)]}{suffix}"
            idx += 1
        used_names.add(candidate)
        return candidate

run

run(data: DataFrame | str | Path) -> FeatureValidationPipelineResult

Validate the new features in data and write the configured reports.

Parameters:

Name	Type	Description	Default
data	DataFrame or str or Path	The feature table, or the path of a CSV file (see input_type). It must contain id_col, apply_time_col, the new and incumbent feature columns, weight_col and split_col when configured. A DataFrame is not modified.	required

Returns:

Type	Description
FeatureValidationPipelineResult	Splits, distribution, WOE, PSI, IV/KS, correlation and selection results, and the paths of the files written.

Raises:

Type	Description
KeyError	If a required column is missing from the data.
ValueError	If a configuration value is invalid (input_type, woe_engine, psi_reference_dataset, batch settings, woe_fit_query), if there are no new features, if split_col lacks ins or oos rows, or if every batch fails.
TypeError	If input_type='dataframe' and data is not a DataFrame, or oot_col is not numeric.
FileNotFoundError	If a CSV path does not exist.

Notes

In CSV batch mode a failing batch is recorded in batch_metadata (status='error') and the run continues with the remaining batches.

Source code in Modeling_Tool/Pipeline/feature_validation.py

def run(self, data: pd.DataFrame | str | Path) -> FeatureValidationPipelineResult:
    """Validate the new features in ``data`` and write the configured reports.

    Parameters
    ----------
    data : pandas.DataFrame or str or pathlib.Path
        The feature table, or the path of a CSV file (see ``input_type``). It must contain ``id_col``,
        ``apply_time_col``, the new and incumbent feature columns, ``weight_col`` and ``split_col`` when configured.
        A DataFrame is not modified.

    Returns
    -------
    FeatureValidationPipelineResult
        Splits, distribution, WOE, PSI, IV/KS, correlation and selection results, and the paths of the files written.

    Raises
    ------
    KeyError
        If a required column is missing from the data.
    ValueError
        If a configuration value is invalid (``input_type``, ``woe_engine``, ``psi_reference_dataset``, batch
        settings, ``woe_fit_query``), if there are no new features, if ``split_col`` lacks ``ins`` or ``oos`` rows, or
        if every batch fails.
    TypeError
        If ``input_type='dataframe'`` and ``data`` is not a DataFrame, or ``oot_col`` is not numeric.
    FileNotFoundError
        If a CSV path does not exist.

    Notes
    -----
    In CSV batch mode a failing batch is recorded in ``batch_metadata`` (``status='error'``) and the run continues
    with the remaining batches.
    """
    cfg = self.config
    manifest = None
    if not getattr(self, "_nested_run", False) and (cfg.write_outputs or cfg.write_excel):
        manifest = OutputManifest.begin(cfg.output_dir, "feature_validation", clean=cfg.clean_output_dir)
    try:
        result = self._run(data)
    except BaseException:
        if manifest is not None:
            try:
                manifest.finish(clean=False)  # list what the failed run wrote; remove nothing
            except Exception:
                pass
        raise
    if manifest is not None:
        removed = manifest.finish()
        if removed:
            _logger.info(
                "FeatureValidationPipeline: clean_output_dir removed %d stale file(s): %s", len(removed), removed
            )
    return result

Reject Inference Pipeline: reject_inference

reject_inference

RejectInferencePipelineConfig dataclass

Configuration of :class:RejectInferencePipeline.

Parameters:

Name	Type	Description	Default
output_dir	str	Root output directory: datasets (RI datasets as CSV), report (CSV tables, performance reports, Excel report, perf_figs plots) and, by default, models.	"output/reject_inference"
approved_col	str	Approval flag column: 1 marks approved rows and 0 rejected rows. Rows with any other value (including missing) belong to neither group.	"approved"
target_col	str	Performance label (1 = bad). The labels of rejected rows are inferred, so any existing values are overwritten in the RI datasets. Approved rows whose target is missing (not yet performed) stay in the RI datasets but are left out of the pre-score, of the reference statistics, of model training and of the validation and OOT samples, and so are rejected rows that get no inferred label (hard_cutoff with a missing score); ri_summary and ri_model_perf count them and a UserWarning names them.	"badflag"
score_col	str	Pre-score column holding the probability of bad (see ri_score_direction). It is created, or overwritten with a UserWarning, when the pre-score is trained.	"prescore_prob"
feature_cols	list of str or None	Feature columns of the pre-score and RI models. None uses every numeric column except target_col, approved_col, score_col, split_col and a column named true_badflag.	None
split_col	str or None	Column with ins / oos / oot labels (case and surrounding spaces ignored; other values raise ValueError). Only INS and OOS rows enter the pipeline and need both, and OOT rows are kept aside for evaluation; rows with a missing label are dropped. oot_data takes precedence over the OOT rows. Approved OOS rows serve as the validation sample and are excluded from training.	None
random_state	int	Seed for the pre-score split, the OOT and validation sampling, the reference sampling, the random labels of the inference methods and the models (unless a model param sets its own seed).	42
write_outputs	bool	Whether to write CSV files and plots: datasets/ri_<method>.csv (see write_ri_datasets) and, in report, ri_comparison_summary.csv, ri_model_perf.csv, oot_summary.csv, model_paths.csv, perf_<method>.csv and perf_figs/perf_<method>.png.	True
write_excel	bool	Whether to write report/RI_Pipeline_Report.xlsx (its path is report_path).	True
train_prescore	bool	True trains a pre-score model on approved rows with an observed target and scores all rows into score_col. False reuses the supplied score_col; if that column is absent the pre-score is trained anyway. The rows held out as random OOT (see oot_frac) are left out of the pre-score's training data. A trained pre-score is the probability of bad, so it requires ri_score_direction='high_bad'.	True
prescore_model_type	str	Pre-score model: "lgb", "xgb", "cat" or "lr" (aliases lightgbm, xgboost, catboost, logistic, logistic_regression; case-insensitive). Anything else raises ValueError.	"lgb"
prescore_params	dict	Model parameters that override the built-in defaults of prescore_model_type.	{}
prescore_test_size	float	Share of the approved rows held out as the (stratified) validation set when the pre-score is trained.	0.3
ri_methods	list of str	Reject-inference methods to run; the aliases simple, hard, fuzzy and parcel are accepted. Unknown names raise ValueError. Results are keyed by the full method name.	['simple_augment', 'hard_cutoff', 'fuzzy_augment', 'parceling']
ri_method_params	dict of str to dict	Per-method parameters keyed by the full method name: simple_augment bad_rate (default: mean target of the reference sample), hard_cutoff cutoff (default: 25th, or 75th for high_good, percentile of the bad reference scores), fuzzy_augment weight_factor (1.0) and parceling n_parcels (10). Other keys, and entries filed under an alias, are ignored.	{}
ri_score_direction	('high_bad', 'high_good')	Meaning of score_col. "high_bad" means a higher score is a higher bad probability, "high_good" the opposite. It is passed to the inference methods and sets the default hard cutoff and the sign of prescore_AUC. Any other value raises ValueError, and so does "high_good" whenever the pipeline trains the pre-score (train_prescore=True or score_col absent), because that pre-score is the probability of bad: use "high_good" only with your own score and train_prescore=False.	"high_bad"
train_ri_models	bool	Whether to train a model on every RI dataset (and the benchmark) and compare their train, validation and OOT performance in ri_model_perf.	True
ri_model_type	str	Model type of the RI models; same options as prescore_model_type. The _weight column of fuzzy_augment is passed as sample weight.	"lgb"
ri_model_params	dict	Model parameters that override the built-in defaults of ri_model_type.	{}
lr_nan_handling	('fillna_median', 'fillna_mean', 'fillna_0', 'raise')	NaN/Inf handling for logistic-regression models only (pre-score and RI models). The fill modes impute with values learned on the training frame, re-applied at scoring time, with a UserWarning when values were filled. "raise" raises ValueError on non-finite features. Gradient-boosting models ignore it.	"fillna_median"
include_no_ri_benchmark	bool	Whether to also train an approved-only model named no_ri_benchmark that takes part in the ri_model_perf ranking; it is not added to ri_datasets.	True
ri_validation_frac	float	Share of the approved training pool with an observed target sampled (stratified by target) as validation data unless approved OOS rows with an observed target are available. Must lie in (0, 1), and oot_frac + ri_validation_frac must be below 1.	0.2
save_models	bool	Whether to pickle the models with save_model into model_output_dir: prescore_model.pkl (only if the pre-score was trained) and ri_model_<method>.pkl for every RI model and the benchmark.	False
model_output_dir	str or None	Folder for the model files. None uses <output_dir>/models.	None
model_include_metadata	bool	Whether the saved models carry the SMF metadata envelope (passed as include_metadata to save_model).	True
write_ri_datasets	bool	Whether to write each RI dataset as datasets/ri_<method>.csv; used only when write_outputs is True. Turn it off for very wide or large data.	True
ri_dataset_output_cols	list of str or None	Columns to keep in the RI dataset CSVs (missing names trigger a UserWarning). None writes all columns. The in-memory ri_datasets are not affected.	None
ri_dataset_warn_mb	float	A UserWarning is issued when a dataset is estimated at this many MB or more before the CSV is written.	1024.0
oot_data	DataFrame or None	External out-of-time sample used for evaluation instead of a random hold-out. It needs target_col and the feature columns; rows with a missing target are dropped with a UserWarning and ValueError is raised if none remains.	None
oot_frac	float	Share of the approved rows with an observed target (at least one row) randomly held out as OOT when neither oot_data nor OOT rows from split_col exist. Must lie in [0, 1). The OOT is drawn before the pre-score is trained and is left out of its training data. With ri_approved_scope="output_subset" it is drawn from all approved rows and only the drawn rows inside the output subset are used, so the subset's share is about oot_frac; ValueError is raised when none falls inside.	0.2
perf_pct_bins	int	Number of percentile bins of PerformanceEvaluator for the model reports.	10
min_bin_prop	float	Minimum share of samples per bin in PerformanceEvaluator.	0.03
ri_approved_data	DataFrame or None	External approved sample used only as the reference that fits the inference rules; the approved rows of the main data are still the approved part of the output. It needs target_col and the feature columns (and score_col unless the pre-score is trained). When the pre-score is trained in the run, the reference is scored with it (an existing score_col is replaced, with a UserWarning), so that the rules fitted on the reference and the rejected rows use the same score; otherwise its own score_col is used. Only approved_col == 1 rows are kept when that column exists. It cannot be combined with ri_approved_query, ri_approved_func or ri_approved_scope="output_subset".	None
ri_approved_query	str or None	pandas query expression selecting the reference rows among the approved rows of the main data.	None
ri_approved_func	callable or None	Function f(df) returning a boolean mask that selects the reference rows among the approved rows of the main data; applied after ri_approved_query when both are given.	None
ri_approved_frac	float or None	Fraction in (0, 1] of the reference sample drawn at random. It cannot be combined with ri_approved_n.	None
ri_approved_n	int or None	Number of rows drawn at random from the reference sample (must be positive and not exceed its size). It cannot be combined with ri_approved_frac.	None
ri_approved_scope	('reference_only', 'output_subset')	"reference_only" uses the reference only to fit the inference rules and keeps all approved rows in the output. "output_subset" also restricts the approved rows of the output (and approved_data) to the reference; it is not allowed with ri_approved_data.	"reference_only"

Source code in Modeling_Tool/Pipeline/reject_inference.py

@dataclass
class RejectInferencePipelineConfig:
    """Configuration of :class:`RejectInferencePipeline`.

    Parameters
    ----------
    output_dir : str, default "output/reject_inference"
        Root output directory: ``datasets`` (RI datasets as CSV), ``report`` (CSV tables, performance reports, Excel
        report, ``perf_figs`` plots) and, by default, ``models``.
    approved_col : str, default "approved"
        Approval flag column: 1 marks approved rows and 0 rejected rows. Rows with any other value (including missing)
        belong to neither group.
    target_col : str, default "badflag"
        Performance label (1 = bad). The labels of rejected rows are inferred, so any existing values are overwritten in
        the RI datasets. Approved rows whose target is missing (not yet performed) stay in the RI datasets but are left
        out of the pre-score, of the reference statistics, of model training and of the validation and OOT samples, and
        so are rejected rows that get no inferred label (``hard_cutoff`` with a missing score); ``ri_summary`` and
        ``ri_model_perf`` count them and a ``UserWarning`` names them.
    score_col : str, default "prescore_prob"
        Pre-score column holding the probability of bad (see ``ri_score_direction``). It is created, or overwritten with
        a ``UserWarning``, when the pre-score is trained.
    feature_cols : list of str or None, default None
        Feature columns of the pre-score and RI models. ``None`` uses every numeric column except ``target_col``,
        ``approved_col``, ``score_col``, ``split_col`` and a column named ``true_badflag``.
    split_col : str or None, default None
        Column with ``ins`` / ``oos`` / ``oot`` labels (case and surrounding spaces ignored; other values raise
        ``ValueError``). Only INS and OOS rows enter the pipeline and need both, and OOT rows are kept aside for
        evaluation; rows with a missing label are dropped. ``oot_data`` takes precedence over the OOT rows. Approved OOS
        rows serve as the validation sample and are excluded from training.
    random_state : int, default 42
        Seed for the pre-score split, the OOT and validation sampling, the reference sampling, the random labels of the
        inference methods and the models (unless a model param sets its own seed).
    write_outputs : bool, default True
        Whether to write CSV files and plots: ``datasets/ri_<method>.csv`` (see ``write_ri_datasets``) and, in ``report``,
        ``ri_comparison_summary.csv``, ``ri_model_perf.csv``, ``oot_summary.csv``, ``model_paths.csv``,
        ``perf_<method>.csv`` and ``perf_figs/perf_<method>.png``.
    write_excel : bool, default True
        Whether to write ``report/RI_Pipeline_Report.xlsx`` (its path is ``report_path``).
    train_prescore : bool, default True
        True trains a pre-score model on approved rows with an observed target and scores all rows into ``score_col``.
        False reuses the supplied ``score_col``; if that column is absent the pre-score is trained anyway. The rows held
        out as random OOT (see ``oot_frac``) are left out of the pre-score's training data. A trained pre-score is the
        probability of bad, so it requires ``ri_score_direction='high_bad'``.
    prescore_model_type : str, default "lgb"
        Pre-score model: ``"lgb"``, ``"xgb"``, ``"cat"`` or ``"lr"`` (aliases ``lightgbm``, ``xgboost``, ``catboost``,
        ``logistic``, ``logistic_regression``; case-insensitive). Anything else raises ``ValueError``.
    prescore_params : dict, default {}
        Model parameters that override the built-in defaults of ``prescore_model_type``.
    prescore_test_size : float, default 0.3
        Share of the approved rows held out as the (stratified) validation set when the pre-score is trained.
    ri_methods : list of str, default ['simple_augment', 'hard_cutoff', 'fuzzy_augment', 'parceling']
        Reject-inference methods to run; the aliases ``simple``, ``hard``, ``fuzzy`` and ``parcel`` are accepted. Unknown
        names raise ``ValueError``. Results are keyed by the full method name.
    ri_method_params : dict of str to dict, default {}
        Per-method parameters keyed by the full method name: ``simple_augment`` ``bad_rate`` (default: mean target of the
        reference sample), ``hard_cutoff`` ``cutoff`` (default: 25th, or 75th for ``high_good``, percentile of the bad
        reference scores), ``fuzzy_augment`` ``weight_factor`` (1.0) and ``parceling`` ``n_parcels`` (10). Other keys, and
        entries filed under an alias, are ignored.
    ri_score_direction : {"high_bad", "high_good"}, default "high_bad"
        Meaning of ``score_col``. ``"high_bad"`` means a higher score is a higher bad probability, ``"high_good"`` the
        opposite. It is passed to the inference methods and sets the default hard cutoff and the sign of
        ``prescore_AUC``. Any other value raises ``ValueError``, and so does ``"high_good"`` whenever the pipeline trains
        the pre-score (``train_prescore=True`` or ``score_col`` absent), because that pre-score is the probability of
        bad: use ``"high_good"`` only with your own score and ``train_prescore=False``.
    train_ri_models : bool, default True
        Whether to train a model on every RI dataset (and the benchmark) and compare their train, validation and OOT
        performance in ``ri_model_perf``.
    ri_model_type : str, default "lgb"
        Model type of the RI models; same options as ``prescore_model_type``. The ``_weight`` column of
        ``fuzzy_augment`` is passed as sample weight.
    ri_model_params : dict, default {}
        Model parameters that override the built-in defaults of ``ri_model_type``.
    lr_nan_handling : {"fillna_median", "fillna_mean", "fillna_0", "raise"}, default "fillna_median"
        NaN/Inf handling for logistic-regression models only (pre-score and RI models). The fill modes impute with values
        learned on the training frame, re-applied at scoring time, with a ``UserWarning`` when values were filled.
        ``"raise"`` raises ``ValueError`` on non-finite features. Gradient-boosting models ignore it.
    include_no_ri_benchmark : bool, default True
        Whether to also train an approved-only model named ``no_ri_benchmark`` that takes part in the ``ri_model_perf``
        ranking; it is not added to ``ri_datasets``.
    ri_validation_frac : float, default 0.2
        Share of the approved training pool with an observed target sampled (stratified by target) as validation data
        unless approved OOS rows with an observed target are available. Must lie in (0, 1), and
        ``oot_frac + ri_validation_frac`` must be below 1.
    save_models : bool, default False
        Whether to pickle the models with ``save_model`` into ``model_output_dir``: ``prescore_model.pkl`` (only if the
        pre-score was trained) and ``ri_model_<method>.pkl`` for every RI model and the benchmark.
    model_output_dir : str or None, default None
        Folder for the model files. ``None`` uses ``<output_dir>/models``.
    model_include_metadata : bool, default True
        Whether the saved models carry the SMF metadata envelope (passed as ``include_metadata`` to ``save_model``).
    write_ri_datasets : bool, default True
        Whether to write each RI dataset as ``datasets/ri_<method>.csv``; used only when ``write_outputs`` is True.
        Turn it off for very wide or large data.
    ri_dataset_output_cols : list of str or None, default None
        Columns to keep in the RI dataset CSVs (missing names trigger a ``UserWarning``). ``None`` writes all columns. The
        in-memory ``ri_datasets`` are not affected.
    ri_dataset_warn_mb : float, default 1024.0
        A ``UserWarning`` is issued when a dataset is estimated at this many MB or more before the CSV is written.
    oot_data : pandas.DataFrame or None, default None
        External out-of-time sample used for evaluation instead of a random hold-out. It needs ``target_col`` and the
        feature columns; rows with a missing target are dropped with a ``UserWarning`` and ``ValueError`` is raised if
        none remains.
    oot_frac : float, default 0.2
        Share of the approved rows with an observed target (at least one row) randomly held out as OOT when neither
        ``oot_data`` nor OOT rows from ``split_col`` exist. Must lie in [0, 1). The OOT is drawn before the pre-score is
        trained and is left out of its training data. With ``ri_approved_scope="output_subset"`` it is drawn from all
        approved rows and only the drawn rows inside the output subset are used, so the subset's share is about
        ``oot_frac``; ``ValueError`` is raised when none falls inside.
    perf_pct_bins : int, default 10
        Number of percentile bins of ``PerformanceEvaluator`` for the model reports.
    min_bin_prop : float, default 0.03
        Minimum share of samples per bin in ``PerformanceEvaluator``.
    ri_approved_data : pandas.DataFrame or None, default None
        External approved sample used only as the reference that fits the inference rules; the approved rows of the main
        data are still the approved part of the output. It needs ``target_col`` and the feature columns (and ``score_col``
        unless the pre-score is trained). When the pre-score is trained in the run, the reference is scored with it (an
        existing ``score_col`` is replaced, with a ``UserWarning``), so that the rules fitted on the reference and the
        rejected rows use the same score; otherwise its own ``score_col`` is used. Only ``approved_col == 1`` rows are
        kept when that column exists. It cannot be combined with ``ri_approved_query``,
        ``ri_approved_func`` or ``ri_approved_scope="output_subset"``.
    ri_approved_query : str or None, default None
        pandas ``query`` expression selecting the reference rows among the approved rows of the main data.
    ri_approved_func : callable or None, default None
        Function ``f(df)`` returning a boolean mask that selects the reference rows among the approved rows of the main
        data; applied after ``ri_approved_query`` when both are given.
    ri_approved_frac : float or None, default None
        Fraction in (0, 1] of the reference sample drawn at random. It cannot be combined with ``ri_approved_n``.
    ri_approved_n : int or None, default None
        Number of rows drawn at random from the reference sample (must be positive and not exceed its size). It cannot be
        combined with ``ri_approved_frac``.
    ri_approved_scope : {"reference_only", "output_subset"}, default "reference_only"
        ``"reference_only"`` uses the reference only to fit the inference rules and keeps all approved rows in the output.
        ``"output_subset"`` also restricts the approved rows of the output (and ``approved_data``) to the reference; it
        is not allowed with ``ri_approved_data``.
    """

    output_dir: str = "output/reject_inference"
    approved_col: str = "approved"
    target_col: str = "badflag"
    score_col: str = "prescore_prob"
    feature_cols: list[str] | None = None
    split_col: str | None = None
    random_state: int = 42
    write_outputs: bool = True
    write_excel: bool = True

    train_prescore: bool = True
    prescore_model_type: str = "lgb"
    prescore_params: dict[str, Any] = field(default_factory=dict)
    prescore_test_size: float = 0.3

    ri_methods: list[str] = field(
        default_factory=lambda: ["simple_augment", "hard_cutoff", "fuzzy_augment", "parceling"]
    )
    ri_method_params: dict[str, dict[str, Any]] = field(default_factory=dict)
    ri_score_direction: Literal["high_bad", "high_good"] = "high_bad"

    train_ri_models: bool = True
    ri_model_type: str = "lgb"
    ri_model_params: dict[str, Any] = field(default_factory=dict)
    # NaN/Inf handling for lr models (both prescore_model_type="lr" and
    # ri_model_type="lr"). GBM backends accept NaN natively; sklearn
    # LogisticRegression does not, so lr features are imputed with
    # train-derived fill values that are re-applied at scoring time.
    # "raise" forbids imputation and fails fast on non-finite features.
    lr_nan_handling: Literal["fillna_median", "fillna_mean", "fillna_0", "raise"] = "fillna_median"
    include_no_ri_benchmark: bool = True
    ri_validation_frac: float = 0.2
    save_models: bool = False
    model_output_dir: str | None = None
    model_include_metadata: bool = True
    write_ri_datasets: bool = True
    ri_dataset_output_cols: list[str] | None = None
    ri_dataset_warn_mb: float = 1024.0
    oot_data: pd.DataFrame | None = None
    oot_frac: float = 0.2
    perf_pct_bins: int = 10
    min_bin_prop: float = 0.03

    ri_approved_data: pd.DataFrame | None = None
    ri_approved_query: str | None = None
    ri_approved_func: Callable[[pd.DataFrame], pd.Series] | None = None
    ri_approved_frac: float | None = None
    ri_approved_n: int | None = None
    ri_approved_scope: Literal["reference_only", "output_subset"] = "reference_only"

RejectInferencePipelineResult dataclass

Result returned by :meth:RejectInferencePipeline.run.

The frames may contain the helper columns _smf_ri_row_id (and _smf_ri_split with split_col).

Parameters:

Name	Type	Description	Default
approved_data	DataFrame	Approved rows used as the approved part of the RI datasets, with the pre-score: all approved rows, or only the reference rows with ri_approved_scope="output_subset".	required
rejected_data	DataFrame	Rejected rows (approved_col == 0) with the pre-score and without inferred labels.	required
ri_datasets	dict of str to pandas.DataFrame	{method: dataset} keyed by full method name: approved_data followed by the rejected rows with inferred labels, plus an ri_method column and, for fuzzy_augment, a _weight column (each rejected row appears twice, as bad and as good).	required
ri_summary	DataFrame	One row per method: ri_method, N_total, N_approved, N_rejected, N_approved_unlabelled and N_rejected_unlabelled (rows whose target is missing), bad_rate_appr, bad_rate_rej, bad_rate_total (over the labelled rows, weighted when _weight exists), has_weight_col, prescore_AUC (on the approved rows with an observed target, oriented so that high means high risk), prescore_AUC_raw and prescore_score_direction.	required
ri_model_perf	DataFrame or None	One row per trained model (each RI method and no_ri_benchmark) with train_N (rows trained on), train_unlabelled_n (rows of the training pool left out because their target is missing), oot_N, weighted_train and the train, validation and OOT AUC, KS and Gini, sorted by oot_AUC descending. None when train_ri_models is False.	None
best_method	str or None	Method with the highest OOT AUC (the first row of ri_model_perf); it can be no_ri_benchmark. None when no models were trained.	None
prescore_model	object or None	The fitted pre-score model. None when the pre-score was not trained.	None
ri_models	dict	{method: fitted model} for every RI method and the benchmark; empty when train_ri_models is False.	{}
report_path	str or None	Path of the Excel report; None when write_excel is False.	None
approved_full_data	DataFrame or None	All approved rows of the input (with pre-score), before any ri_approved_* restriction.	None
ri_approved_reference_data	DataFrame or None	The approved reference sample that was used to fit the inference rules.	None
ri_approved_summary	DataFrame or None	Two columns metric and value: the sizes of the full approved, reference and output approved samples, the number of rejected rows, the reference source and scope, the reference share and the ri_approved_* settings.	None
model_paths	dict of str to str	{key: file path} of the saved models (prescore and the method names); empty unless save_models is True.	{}
oot_summary	DataFrame or None	Two columns metric and value: oot_source, oot_raw_n, oot_observed_n, oot_dropped_missing_target_n and oot_missing_target_rate. None when train_ri_models is False.	None

Source code in Modeling_Tool/Pipeline/reject_inference.py

@dataclass
class RejectInferencePipelineResult:
    """Result returned by :meth:`RejectInferencePipeline.run`.

    The frames may contain the helper columns ``_smf_ri_row_id`` (and ``_smf_ri_split`` with ``split_col``).

    Parameters
    ----------
    approved_data : pandas.DataFrame
        Approved rows used as the approved part of the RI datasets, with the pre-score: all approved rows, or only the
        reference rows with ``ri_approved_scope="output_subset"``.
    rejected_data : pandas.DataFrame
        Rejected rows (``approved_col == 0``) with the pre-score and without inferred labels.
    ri_datasets : dict of str to pandas.DataFrame
        ``{method: dataset}`` keyed by full method name: ``approved_data`` followed by the rejected rows with inferred
        labels, plus an ``ri_method`` column and, for ``fuzzy_augment``, a ``_weight`` column (each rejected row appears
        twice, as bad and as good).
    ri_summary : pandas.DataFrame
        One row per method: ``ri_method``, ``N_total``, ``N_approved``, ``N_rejected``, ``N_approved_unlabelled`` and
        ``N_rejected_unlabelled`` (rows whose target is missing), ``bad_rate_appr``, ``bad_rate_rej``,
        ``bad_rate_total`` (over the labelled rows, weighted when ``_weight`` exists), ``has_weight_col``,
        ``prescore_AUC`` (on the approved rows with an observed target, oriented so that high means high risk),
        ``prescore_AUC_raw`` and ``prescore_score_direction``.
    ri_model_perf : pandas.DataFrame or None, default None
        One row per trained model (each RI method and ``no_ri_benchmark``) with ``train_N`` (rows trained on),
        ``train_unlabelled_n`` (rows of the training pool left out because their target is missing), ``oot_N``,
        ``weighted_train`` and the train, validation and OOT ``AUC``, ``KS`` and ``Gini``, sorted by ``oot_AUC``
        descending. None when ``train_ri_models`` is False.
    best_method : str or None, default None
        Method with the highest OOT AUC (the first row of ``ri_model_perf``); it can be ``no_ri_benchmark``. None when no
        models were trained.
    prescore_model : object or None, default None
        The fitted pre-score model. None when the pre-score was not trained.
    ri_models : dict, default {}
        ``{method: fitted model}`` for every RI method and the benchmark; empty when ``train_ri_models`` is False.
    report_path : str or None, default None
        Path of the Excel report; None when ``write_excel`` is False.
    approved_full_data : pandas.DataFrame or None, default None
        All approved rows of the input (with pre-score), before any ``ri_approved_*`` restriction.
    ri_approved_reference_data : pandas.DataFrame or None, default None
        The approved reference sample that was used to fit the inference rules.
    ri_approved_summary : pandas.DataFrame or None, default None
        Two columns ``metric`` and ``value``: the sizes of the full approved, reference and output approved samples, the
        number of rejected rows, the reference source and scope, the reference share and the ``ri_approved_*`` settings.
    model_paths : dict of str to str, default {}
        ``{key: file path}`` of the saved models (``prescore`` and the method names); empty unless ``save_models`` is
        True.
    oot_summary : pandas.DataFrame or None, default None
        Two columns ``metric`` and ``value``: ``oot_source``, ``oot_raw_n``, ``oot_observed_n``,
        ``oot_dropped_missing_target_n`` and ``oot_missing_target_rate``. None when ``train_ri_models`` is False.
    """

    approved_data: pd.DataFrame
    rejected_data: pd.DataFrame
    ri_datasets: dict[str, pd.DataFrame]
    ri_summary: pd.DataFrame
    ri_model_perf: pd.DataFrame | None = None
    best_method: str | None = None
    prescore_model: Any | None = None
    ri_models: dict[str, Any] = field(default_factory=dict)
    report_path: str | None = None
    approved_full_data: pd.DataFrame | None = None
    ri_approved_reference_data: pd.DataFrame | None = None
    ri_approved_summary: pd.DataFrame | None = None
    model_paths: dict[str, str] = field(default_factory=dict)
    oot_summary: pd.DataFrame | None = None

RejectInferencePipeline

Reusable reject-inference workflow for approved/rejected application data.

The pipeline (optionally) trains a pre-score model on the approved rows, infers labels for the rejected rows with each configured method, optionally trains a model per augmented dataset and ranks the methods by OOT AUC.

Parameters:

Name	Type	Description	Default
config	RejectInferencePipelineConfig or None	Pipeline configuration. None (or any falsy value) uses RejectInferencePipelineConfig() with its defaults.	None

Attributes:

Name	Type	Description
config	RejectInferencePipelineConfig	The configuration in use.
predict_positive_nan_stats	dict	{method: {dataset: count}} number of non-finite predictions per evaluation dataset (train, validation, oot) of each model trained by run; empty before the first run.

Source code in Modeling_Tool/Pipeline/reject_inference.py

class RejectInferencePipeline:
    """Reusable reject-inference workflow for approved/rejected application data.

    The pipeline (optionally) trains a pre-score model on the approved rows, infers labels for the rejected rows with
    each configured method, optionally trains a model per augmented dataset and ranks the methods by OOT AUC.

    Parameters
    ----------
    config : RejectInferencePipelineConfig or None, default None
        Pipeline configuration. ``None`` (or any falsy value) uses ``RejectInferencePipelineConfig()`` with its defaults.

    Attributes
    ----------
    config : RejectInferencePipelineConfig
        The configuration in use.
    predict_positive_nan_stats : dict
        ``{method: {dataset: count}}`` number of non-finite predictions per evaluation dataset (``train``,
        ``validation``, ``oot``) of each model trained by ``run``; empty before the first run.
    """

    _METHOD_ALIASES = {
        "simple": "simple_augment",
        "simple_augment": "simple_augment",
        "hard": "hard_cutoff",
        "hard_cutoff": "hard_cutoff",
        "fuzzy": "fuzzy_augment",
        "fuzzy_augment": "fuzzy_augment",
        "parcel": "parceling",
        "parceling": "parceling",
    }

    _MODEL_TYPE_ALIASES = {
        "lgb": "lgb",
        "lightgbm": "lgb",
        "xgb": "xgb",
        "xgboost": "xgb",
        "cat": "cat",
        "catboost": "cat",
        "lr": "lr",
        "logistic": "lr",
        "logistic_regression": "lr",
    }

    _DEFAULT_PRESCORE_PARAMS_BY_TYPE = {
        "lgb": {
            "n_estimators": 200,
            "learning_rate": 0.05,
            "num_leaves": 31,
            "subsample": 0.8,
            "colsample_bytree": 0.8,
            "n_jobs": -1,
            "verbose": -1,
            "early_stopping_rounds": 30,
            "eval_metric": "auc",
        },
        "xgb": {
            "n_estimators": 200,
            "learning_rate": 0.05,
            "max_depth": 5,
            "subsample": 0.8,
            "colsample_bytree": 0.8,
            "n_jobs": -1,
            "verbosity": 0,
            "early_stopping_rounds": 30,
            "eval_metric": "auc",
        },
        "cat": {
            "n_estimators": 200,
            "learning_rate": 0.05,
            "max_depth": 6,
            "verbose": False,
            "early_stopping_rounds": 30,
            "eval_metric": "auc",
            "allow_writing_files": False,
        },
        "lr": {"C": 1.0, "solver": "lbfgs", "max_iter": 1000},
    }

    _DEFAULT_RI_MODEL_PARAMS_BY_TYPE = {
        "lgb": {
            "n_estimators": 200,
            "learning_rate": 0.05,
            "num_leaves": 31,
            "subsample": 0.8,
            "colsample_bytree": 0.8,
            "min_child_samples": 30,
            "n_jobs": -1,
            "verbose": -1,
            "early_stopping_rounds": 30,
            "eval_metric": "auc",
        },
        "xgb": {
            "n_estimators": 200,
            "learning_rate": 0.05,
            "max_depth": 5,
            "subsample": 0.8,
            "colsample_bytree": 0.8,
            "min_child_weight": 1,
            "n_jobs": -1,
            "verbosity": 0,
            "early_stopping_rounds": 30,
            "eval_metric": "auc",
        },
        "cat": {
            "n_estimators": 200,
            "learning_rate": 0.05,
            "max_depth": 6,
            "verbose": False,
            "early_stopping_rounds": 30,
            "eval_metric": "auc",
            "allow_writing_files": False,
        },
        "lr": {"C": 1.0, "solver": "lbfgs", "max_iter": 1000},
    }

    def __init__(self, config: RejectInferencePipelineConfig | None = None):
        self.config = config or RejectInferencePipelineConfig()
        self.predict_positive_nan_stats: dict[str, dict[str, int]] = {}

    @classmethod
    def _normalize_model_type(cls, model_type: str, *, config_name: str) -> str:
        normalized = cls._MODEL_TYPE_ALIASES.get(str(model_type).strip().lower())
        if normalized is None:
            raise ValueError(
                f"{config_name} must be one of lgb/xgb/cat/lr; got {model_type!r}"
            )
        return normalized

    def _resolve_model_params(
        self,
        model_type: str,
        overrides: dict[str, Any],
        *,
        role: Literal["prescore", "ri"],
    ) -> tuple[str, dict[str, Any]]:
        config_name = "prescore_model_type" if role == "prescore" else "ri_model_type"
        normalized = self._normalize_model_type(model_type, config_name=config_name)
        defaults_by_type = (
            self._DEFAULT_PRESCORE_PARAMS_BY_TYPE
            if role == "prescore"
            else self._DEFAULT_RI_MODEL_PARAMS_BY_TYPE
        )
        params = merge_dict(defaults_by_type[normalized], overrides)
        if normalized == "cat":
            if "random_seed" not in params:
                params.setdefault("random_state", self.config.random_state)
        else:
            params.setdefault("random_state", self.config.random_state)
        return normalized, params

    def _fit_pipeline_model(
        self,
        *,
        model_type: str,
        params: dict[str, Any],
        train: pd.DataFrame,
        val: pd.DataFrame,
        feature_cols: list[str],
        weight_col: str | None = None,
    ) -> Any:
        from Modeling_Tool import GradientBoostingModel, LRMaster

        cfg = self.config
        # The cast below would turn a missing target into 0 (good), so callers
        # must leave unlabelled rows out before fitting.
        for frame_name, frame in (("training", train), ("validation", val)):
            n_missing = int(pd.to_numeric(frame[cfg.target_col], errors="coerce").isna().sum())
            if n_missing:
                raise ValueError(
                    f"{frame_name} frame has {n_missing} row(s) with a missing {cfg.target_col!r}; "
                    f"unlabelled rows must be left out before fitting"
                )
        train_fit = train.copy()
        val_fit = val.copy()
        train_fit[cfg.target_col] = (pd.to_numeric(train_fit[cfg.target_col]) > 0.5).astype(int)
        val_fit[cfg.target_col] = (pd.to_numeric(val_fit[cfg.target_col]) > 0.5).astype(int)

        if model_type == "lr":
            lr_params = dict(params)
            standardize = bool(lr_params.pop("standardize", False))
            train_fit, val_fit, fill_values = self._prepare_lr_frames(
                train_fit, val_fit, feature_cols,
            )
            model = LRMaster(params=lr_params or None, standardize=standardize)
            model.fit(
                data=train_fit,
                varlist=feature_cols,
                tgt_name=cfg.target_col,
                val_data=val_fit,
                val_varlist=feature_cols,
                val_tgt_name=cfg.target_col,
                weight_col=weight_col,
            )
            if fill_values is not None:
                return _LRNanFillWrapper(model, fill_values, cfg.lr_nan_handling)
            return model

        model = GradientBoostingModel(model_type, params)
        fit_kwargs = {}
        if weight_col is not None:
            fit_kwargs["sample_weight"] = train_fit[weight_col].to_numpy(dtype=float)
        model.fit(
            x=train_fit[feature_cols],
            y=train_fit[cfg.target_col],
            valx=val_fit[feature_cols],
            valy=val_fit[cfg.target_col],
            **fit_kwargs,
        )
        return model

    def _prepare_lr_frames(
        self,
        train_fit: pd.DataFrame,
        val_fit: pd.DataFrame,
        feature_cols: list[str],
    ) -> tuple[pd.DataFrame, pd.DataFrame, dict[str, float] | None]:
        """Apply cfg.lr_nan_handling to the lr feature frames.

        GBM backends consume NaN natively; sklearn LogisticRegression raises
        ``Input X contains NaN`` on raw pipeline features. Fill values are
        computed on the training frame only and returned so the fitted model
        re-applies the identical imputation at scoring time via
        _LRNanFillWrapper. Callers pass throwaway copies, so frames are
        modified in place.
        """
        cfg = self.config
        allowed = {"fillna_median", "fillna_mean", "fillna_0", "raise"}
        mode = str(cfg.lr_nan_handling)
        if mode not in allowed:
            raise ValueError(
                f"lr_nan_handling must be one of {sorted(allowed)}; got {mode!r}"
            )
        train_block = train_fit[feature_cols].replace([np.inf, -np.inf], np.nan)
        val_block = val_fit[feature_cols].replace([np.inf, -np.inf], np.nan)
        n_bad_train = int(train_block.isna().sum().sum())
        n_bad_val = int(val_block.isna().sum().sum())
        if mode == "raise":
            if n_bad_train or n_bad_val:
                bad_cols = sorted(
                    set(train_block.columns[train_block.isna().any()])
                    | set(val_block.columns[val_block.isna().any()])
                )
                raise ValueError(
                    f"lr model features contain {n_bad_train + n_bad_val} "
                    f"non-finite value(s) across {len(bad_cols)} feature(s) "
                    f"(e.g. {bad_cols[:5]}) and lr_nan_handling='raise'. "
                    f"Clean the features first, or set lr_nan_handling to "
                    f"'fillna_median'/'fillna_mean'/'fillna_0'."
                )
            return train_fit, val_fit, None
        if mode == "fillna_0":
            fill_series = pd.Series(0.0, index=feature_cols)
        elif mode == "fillna_mean":
            fill_series = train_block.mean(numeric_only=True).reindex(feature_cols).fillna(0.0)
        else:
            fill_series = train_block.median(numeric_only=True).reindex(feature_cols).fillna(0.0)
        fill_values = {str(col): float(fill_series[col]) for col in feature_cols}
        if n_bad_train or n_bad_val:
            n_cols = int((train_block.isna().any() | val_block.isna().any()).sum())
            warnings.warn(
                f"lr model: {n_bad_train + n_bad_val} non-finite feature "
                f"value(s) across {n_cols}/{len(feature_cols)} feature(s) "
                f"filled with train-derived {mode} values (sklearn LR cannot "
                f"handle NaN). Fill values are stored on the model and "
                f"re-applied at scoring time; set lr_nan_handling='raise' to "
                f"forbid imputation.",
                UserWarning,
                stacklevel=2,
            )
        train_fit[feature_cols] = train_block.fillna(fill_values)
        val_fit[feature_cols] = val_block.fillna(fill_values)
        return train_fit, val_fit, fill_values

    def run(self, data: pd.DataFrame) -> RejectInferencePipelineResult:
        """Run the reject-inference workflow on the approved and rejected applications in ``data``.

        Parameters
        ----------
        data : pandas.DataFrame
            Application data with all approved (``approved_col == 1``) and rejected (``approved_col == 0``) rows. It must
            contain ``approved_col``, the feature columns, ``split_col`` when configured, ``target_col`` whenever the
            pre-score is trained (``train_prescore`` is True or ``score_col`` is absent) and ``score_col`` when
            ``train_prescore`` is False. The input frame is not modified.

        Returns
        -------
        RejectInferencePipelineResult
            Approved and rejected data, the RI datasets and summary, the model comparison, the best method and the report
            and model paths.

        Raises
        ------
        KeyError
            If a required column is missing from ``data``, ``oot_data`` or ``ri_approved_data``.
        ValueError
            If the configuration is inconsistent (unknown model type, method or direction,
            ``ri_score_direction="high_good"`` with a pre-score trained by the pipeline, invalid fractions, conflicting
            ``ri_approved_*`` settings, invalid ``split_col`` labels), if no approved row has an observed target for the
            pre-score or (with ``train_ri_models``) for the models, or if a training, validation, OOT or reference sample
            is empty.

        Notes
        -----
        Warnings are issued when ``train_prescore=True`` overwrites an existing ``score_col`` (of ``data`` or of
        ``ri_approved_data``), when rows with a missing target are dropped from an external OOT or left out of model
        training, and when non-finite values are imputed or predicted. With ``save_models=True`` the models are pickled
        even if ``write_outputs`` is False.
        """
        cfg = self.config
        feature_cols = self._resolve_feature_cols(data)
        self._validate_input(data, feature_cols)
        self._validate_ri_approved_config()
        self._validate_split_col_schema(data)
        self._validate_external_ri_approved_schema(data, feature_cols)

        datasets_dir = Path(cfg.output_dir) / "datasets"
        report_dir = Path(cfg.output_dir) / "report"
        figs_dir = report_dir / "perf_figs"
        model_dir = Path(cfg.model_output_dir) if cfg.model_output_dir else Path(cfg.output_dir) / "models"
        if cfg.write_outputs or cfg.write_excel:
            make_dirs(datasets_dir, report_dir, figs_dir)
        if cfg.save_models:
            make_dirs(model_dir)

        work = data.copy()
        work["_smf_ri_row_id"] = np.arange(len(work))
        split_oot_data = None
        if cfg.split_col:
            work, split_oot_data = self._prepare_split_col_data(work)
        # Drawn before the pre-score so that the OOT rows stay out of its training data
        random_oot_ids = self._draw_random_oot_ids(work, split_oot_data)
        prescore_model = None
        model_paths: dict[str, str] = {}
        if cfg.train_prescore or cfg.score_col not in work.columns:
            if cfg.train_prescore and cfg.score_col in work.columns:
                warnings.warn(
                    f"train_prescore=True will overwrite existing column "
                    f"{cfg.score_col!r} in input data. Set train_prescore=False "
                    f"to reuse the supplied scores, or rename cfg.score_col to "
                    f"a fresh column name to avoid this overwrite.",
                    UserWarning,
                    stacklevel=2,
                )
            work, prescore_model = self._fit_prescore(work, feature_cols, exclude_ids=random_oot_ids)
            if cfg.save_models:
                model_paths["prescore"] = self._save_pipeline_model(
                    model=prescore_model,
                    path=model_dir / "prescore_model.pkl",
                    feature_cols=feature_cols,
                    model_role="prescore",
                    ri_method=None,
                    model_type=self._normalize_model_type(
                        cfg.prescore_model_type,
                        config_name="prescore_model_type",
                    ),
                )

        approved_full = work[work[cfg.approved_col] == 1].copy().reset_index(drop=True)
        rejected = work[work[cfg.approved_col] == 0].copy().reset_index(drop=True)
        ri_approved_ref, approved_output, ri_approved_summary = self._prepare_ri_approved_reference(
            work=work,
            approved_full=approved_full,
            feature_cols=feature_cols,
            prescore_model=prescore_model,
        )
        hard_cutoff = self._default_hard_cutoff(ri_approved_ref)

        ri_datasets: dict[str, pd.DataFrame] = {}
        inferrers: dict[str, Any] = {}
        for requested in as_list(cfg.ri_methods):
            method = self._METHOD_ALIASES.get(str(requested).lower())
            if method is None:
                raise ValueError(f"Unsupported reject inference method: {requested!r}")
            inferrer = self._build_inferrer(method, hard_cutoff)
            df_inferred = inferrer.infer(
                df_approved=ri_approved_ref,
                df_rejected=rejected,
                score_col=cfg.score_col,
            )
            df_ri = self._compose_ri_dataset(
                df_inferred=df_inferred,
                approved_output=approved_output,
                method=method,
            )
            df_ri["ri_method"] = method
            ri_datasets[method] = df_ri
            inferrers[method] = inferrer
            if cfg.write_outputs and cfg.write_ri_datasets:
                self._write_ri_dataset(df_ri, datasets_dir / f"ri_{method}.csv", method)

        ri_summary = self._summarize_ri(ri_datasets)
        if cfg.write_outputs:
            safe_to_csv(ri_summary, report_dir / "ri_comparison_summary.csv", index=False)

        ri_model_perf = None
        best_method = None
        ri_models: dict[str, Any] = {}
        oot_summary = None
        if cfg.train_ri_models:
            ri_model_perf, ri_models, ri_model_paths, oot_summary = self._train_and_evaluate_models(
                ri_datasets=ri_datasets,
                approved=approved_output,
                feature_cols=feature_cols,
                report_dir=report_dir,
                model_dir=model_dir,
                split_oot_data=split_oot_data,
                random_oot_ids=random_oot_ids,
            )
            model_paths.update(ri_model_paths)
            if ri_model_perf is not None and len(ri_model_perf):
                best_method = str(ri_model_perf.iloc[0]["ri_method"])
                if cfg.write_outputs:
                    safe_to_csv(ri_model_perf, report_dir / "ri_model_perf.csv", index=False)
            if cfg.write_outputs and oot_summary is not None:
                safe_to_csv(oot_summary, report_dir / "oot_summary.csv", index=False)
        if cfg.write_outputs and model_paths:
            safe_to_csv(self._model_paths_to_frame(model_paths), report_dir / "model_paths.csv", index=False)

        report_path = None
        if cfg.write_excel:
            report_path = str(report_dir / "RI_Pipeline_Report.xlsx")
            sheets = {
                "RI_Dataset_Stats": ri_summary,
                "Model_Performance": ri_model_perf,
                "RI_Approved_Sample": ri_approved_summary,
                "OOT_Sample": oot_summary,
                "Model_Paths": self._model_paths_to_frame(model_paths) if model_paths else None,
            }
            write_basic_excel(report_path, sheets, title="SMF Reject Inference Pipeline Report")

        return RejectInferencePipelineResult(
            approved_data=approved_output,
            rejected_data=rejected,
            ri_datasets=ri_datasets,
            ri_summary=ri_summary,
            ri_model_perf=ri_model_perf,
            best_method=best_method,
            prescore_model=prescore_model,
            ri_models=ri_models,
            report_path=report_path,
            approved_full_data=approved_full,
            ri_approved_reference_data=ri_approved_ref,
            ri_approved_summary=ri_approved_summary,
            model_paths=model_paths,
            oot_summary=oot_summary,
        )

    def _resolve_feature_cols(self, data: pd.DataFrame) -> list[str]:
        cfg = self.config
        if cfg.feature_cols:
            return list(cfg.feature_cols)
        excluded = {cfg.target_col, cfg.approved_col, cfg.score_col, "true_badflag"}
        if cfg.split_col:
            excluded.add(cfg.split_col)
        numeric_cols = data.select_dtypes(include=[np.number]).columns
        return [col for col in numeric_cols if col not in excluded]

    def _validate_input(self, data: pd.DataFrame, feature_cols: list[str]) -> None:
        cfg = self.config
        missing = [cfg.approved_col] + [c for c in feature_cols if c not in data.columns]
        missing = [c for c in missing if c not in data.columns]
        if missing:
            raise KeyError(f"Missing required columns: {missing}")
        # Prescore is trained either explicitly (train_prescore=True) or
        # implicitly when score_col is absent (see _run, L154). Either path
        # requires target_col — validate both up-front instead of failing
        # deep inside _fit_prescore with a confusing KeyError.
        will_train_prescore = cfg.train_prescore or cfg.score_col not in data.columns
        if will_train_prescore and cfg.target_col not in data.columns:
            raise KeyError(
                f"Missing target column {cfg.target_col!r} for pre-score training "
                f"(train_prescore={cfg.train_prescore}, score_col {cfg.score_col!r} "
                f"{'present' if cfg.score_col in data.columns else 'absent'})"
            )
        # The pre-score the pipeline trains is the probability of bad, so reading it as
        # a high-good score would invert hard_cutoff and fuzzy_augment
        if will_train_prescore and cfg.ri_score_direction == "high_good":
            raise ValueError(
                f"ri_score_direction='high_good' cannot be used with a pre-score trained by the pipeline "
                f"(train_prescore={cfg.train_prescore}, score_col {cfg.score_col!r} "
                f"{'present' if cfg.score_col in data.columns else 'absent'}): that pre-score is the "
                f"probability of bad. Use ri_score_direction='high_bad', or supply your own high-good score in "
                f"score_col with train_prescore=False."
            )

    def _validate_ri_approved_config(self) -> None:
        cfg = self.config
        self._normalize_model_type(cfg.prescore_model_type, config_name="prescore_model_type")
        self._normalize_model_type(cfg.ri_model_type, config_name="ri_model_type")
        if cfg.ri_score_direction not in {"high_bad", "high_good"}:
            raise ValueError("ri_score_direction must be 'high_bad' or 'high_good'")
        if not 0 < float(cfg.ri_validation_frac) < 1:
            raise ValueError("ri_validation_frac must be in (0, 1)")
        if not 0 <= float(cfg.oot_frac) < 1:
            raise ValueError("oot_frac must be in [0, 1)")
        # OOT + validation combined must leave room for a real training pool.
        # Otherwise _sample_validation_ids' fallback (pool = approved when the
        # exclude set covers everything) silently overlaps validation with OOT,
        # leaking OOT rows into the validation metric.
        combined = float(cfg.oot_frac) + float(cfg.ri_validation_frac)
        if combined >= 1.0:
            raise ValueError(
                f"oot_frac ({cfg.oot_frac}) + ri_validation_frac ({cfg.ri_validation_frac}) "
                f"= {combined:.3f} must be < 1.0 to leave a non-empty training pool disjoint "
                f"from OOT and validation"
            )
        if cfg.ri_approved_scope not in {"reference_only", "output_subset"}:
            raise ValueError("ri_approved_scope must be 'reference_only' or 'output_subset'")
        if cfg.ri_approved_frac is not None and cfg.ri_approved_n is not None:
            raise ValueError("ri_approved_frac and ri_approved_n cannot be used together")
        if cfg.ri_approved_frac is not None and not 0 < float(cfg.ri_approved_frac) <= 1:
            raise ValueError("ri_approved_frac must be in (0, 1]")
        if cfg.ri_approved_n is not None and int(cfg.ri_approved_n) <= 0:
            raise ValueError("ri_approved_n must be a positive integer")
        if cfg.ri_approved_data is not None and (cfg.ri_approved_query or cfg.ri_approved_func is not None):
            raise ValueError("ri_approved_data cannot be combined with ri_approved_query or ri_approved_func")
        if cfg.ri_approved_data is not None and cfg.ri_approved_scope == "output_subset":
            raise ValueError("ri_approved_scope='output_subset' cannot be used with external ri_approved_data")

    def _validate_split_col_schema(self, data: pd.DataFrame) -> None:
        cfg = self.config
        if not cfg.split_col:
            return
        if cfg.split_col not in data.columns:
            raise KeyError(f"Missing split_col {cfg.split_col!r}")
        raw_split = data[cfg.split_col]
        valid = {"ins", "oos", "oot"}
        invalid = sorted(set(raw_split.dropna().astype(str).str.strip().str.lower()) - valid)
        if invalid:
            raise ValueError(f"split_col {cfg.split_col!r} only supports ins/oos/oot values, got {invalid}")

    def _validate_external_ri_approved_schema(
        self,
        data: pd.DataFrame,
        feature_cols: list[str],
    ) -> None:
        cfg = self.config
        if cfg.ri_approved_data is None:
            return
        ri_ref = cfg.ri_approved_data
        required = [cfg.target_col] + feature_cols
        will_train_prescore = cfg.train_prescore or cfg.score_col not in data.columns
        if cfg.score_col not in ri_ref.columns and not will_train_prescore:
            required.append(cfg.score_col)
        missing = [col for col in dict.fromkeys(required) if col not in ri_ref.columns]
        if missing:
            raise KeyError(f"External ri_approved_data missing required columns: {missing}")

    def _prepare_split_col_data(self, work: pd.DataFrame) -> tuple[pd.DataFrame, pd.DataFrame]:
        cfg = self.config
        if cfg.split_col not in work.columns:
            raise KeyError(f"Missing split_col {cfg.split_col!r}")
        raw_split = work[cfg.split_col]
        lower = raw_split.astype(str).str.strip().str.lower()
        valid = {"ins", "oos", "oot"}
        invalid = sorted(set(raw_split.dropna().astype(str).str.strip().str.lower()) - valid)
        if invalid:
            raise ValueError(f"split_col {cfg.split_col!r} only supports ins/oos/oot values, got {invalid}")
        work = work.copy()
        work["_smf_ri_split"] = lower
        train = work[lower.isin(["ins", "oos"])].copy()
        oot = work[lower == "oot"].copy()
        if not (lower == "ins").any() or not (lower == "oos").any():
            raise ValueError(f"split_col {cfg.split_col!r} must contain non-empty ins and oos samples")
        return train.reset_index(drop=True), oot.reset_index(drop=True)

    def _prepare_ri_approved_reference(
        self,
        work: pd.DataFrame,
        approved_full: pd.DataFrame,
        feature_cols: list[str],
        prescore_model: Any | None,
    ) -> tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame]:
        cfg = self.config
        if cfg.ri_approved_data is not None:
            source = "external_data"
            ri_ref = cfg.ri_approved_data.copy()
            if cfg.approved_col in ri_ref.columns:
                ri_ref = ri_ref[ri_ref[cfg.approved_col] == 1].copy()
            else:
                ri_ref[cfg.approved_col] = 1
            self._validate_ri_approved_frame(ri_ref, feature_cols, prescore_model)
            if prescore_model is not None:
                # The rules fitted on the reference are applied to the rejects' new pre-score,
                # so the reference must be scored by the same model
                if cfg.score_col in ri_ref.columns:
                    warnings.warn(
                        f"External ri_approved_data has its own {cfg.score_col!r} column; it is replaced by the "
                        f"pre-score trained in this run so that the reference and the rejected rows share one "
                        f"score. Set train_prescore=False to use the supplied scores everywhere.",
                        UserWarning,
                        stacklevel=3,
                    )
                ri_ref[cfg.score_col] = predict_positive(prescore_model, ri_ref, feature_cols)
            ri_ref = self._sample_ri_approved_reference(ri_ref.reset_index(drop=True))
            approved_output = approved_full.copy()
        else:
            source = "main_approved"
            ri_ref = approved_full.copy()
            if cfg.ri_approved_query:
                ri_ref = ri_ref.query(cfg.ri_approved_query).copy()
                source = "main_query"
            if cfg.ri_approved_func is not None:
                mask = cfg.ri_approved_func(ri_ref)
                mask = pd.Series(mask, index=ri_ref.index).astype(bool)
                ri_ref = ri_ref[mask].copy()
                source = "main_func" if source == "main_approved" else f"{source}+func"
            ri_ref = self._sample_ri_approved_reference(ri_ref.reset_index(drop=True))
            approved_output = ri_ref.copy() if cfg.ri_approved_scope == "output_subset" else approved_full.copy()

        self._validate_ri_approved_not_empty(ri_ref)
        summary = self._summarize_ri_approved_reference(
            approved_full=approved_full,
            ri_ref=ri_ref,
            approved_output=approved_output,
            rejected_n=int((work[cfg.approved_col] == 0).sum()),
            source=source,
        )
        return ri_ref.reset_index(drop=True), approved_output.reset_index(drop=True), summary

    def _validate_ri_approved_frame(
        self,
        ri_ref: pd.DataFrame,
        feature_cols: list[str],
        prescore_model: Any | None,
    ) -> None:
        cfg = self.config
        required = [cfg.target_col] + feature_cols
        if cfg.score_col not in ri_ref.columns and prescore_model is None:
            required.append(cfg.score_col)
        missing = [col for col in dict.fromkeys(required) if col not in ri_ref.columns]
        if missing:
            raise KeyError(f"External ri_approved_data missing required columns: {missing}")

    def _validate_ri_approved_not_empty(self, ri_ref: pd.DataFrame) -> None:
        cfg = self.config
        if ri_ref.empty:
            raise ValueError("RI approved reference sample is empty")
        if cfg.target_col not in ri_ref.columns or ri_ref[cfg.target_col].notna().sum() == 0:
            raise ValueError("RI approved reference target column must not be empty")

    def _sample_ri_approved_reference(self, ri_ref: pd.DataFrame) -> pd.DataFrame:
        cfg = self.config
        if cfg.ri_approved_n is not None:
            n = int(cfg.ri_approved_n)
            if n > len(ri_ref):
                raise ValueError("ri_approved_n cannot exceed RI approved reference sample size")
            return ri_ref.sample(n=n, random_state=cfg.random_state).copy()
        if cfg.ri_approved_frac is not None:
            return ri_ref.sample(frac=float(cfg.ri_approved_frac), random_state=cfg.random_state).copy()
        return ri_ref.copy()

    def _summarize_ri_approved_reference(
        self,
        approved_full: pd.DataFrame,
        ri_ref: pd.DataFrame,
        approved_output: pd.DataFrame,
        rejected_n: int,
        source: str,
    ) -> pd.DataFrame:
        cfg = self.config
        approved_full_n = len(approved_full)
        rows = [
            ("approved_full_n", approved_full_n),
            ("ri_approved_ref_n", len(ri_ref)),
            ("approved_output_n", len(approved_output)),
            ("rejected_n", rejected_n),
            ("ri_approved_source", source),
            ("ri_approved_scope", cfg.ri_approved_scope),
            ("ref_share_of_full_approved", len(ri_ref) / approved_full_n if approved_full_n else np.nan),
            ("ri_approved_query", cfg.ri_approved_query),
            ("ri_approved_frac", cfg.ri_approved_frac),
            ("ri_approved_n", cfg.ri_approved_n),
        ]
        return pd.DataFrame(rows, columns=["metric", "value"])

    def _draw_random_oot_ids(
        self,
        work: pd.DataFrame,
        split_oot_data: pd.DataFrame | None,
    ) -> set[Any] | None:
        """Row ids of the random OOT; None when the OOT comes from ``oot_data`` / ``split_col`` or no model is trained."""
        cfg = self.config
        if not cfg.train_ri_models or cfg.oot_data is not None:
            return None
        if split_oot_data is not None and len(split_oot_data):
            return None
        if cfg.target_col not in work.columns:
            return None
        labelled = work[(work[cfg.approved_col] == 1) & work[cfg.target_col].notna()]
        if labelled.empty:
            return None
        rng = np.random.default_rng(cfg.random_state)
        n_oot = max(1, int(len(labelled) * cfg.oot_frac))
        return set(rng.choice(labelled["_smf_ri_row_id"].to_numpy(), size=n_oot, replace=False))

    def _fit_prescore(
        self,
        data: pd.DataFrame,
        feature_cols: list[str],
        exclude_ids: set[Any] | None = None,
    ) -> tuple[pd.DataFrame, Any]:
        from Modeling_Tool import SampleSplitter

        cfg = self.config
        trainable = (data[cfg.approved_col] == 1) & data[cfg.target_col].notna()
        if exclude_ids:
            # The random OOT rows evaluate the RI models, so the pre-score must not learn their labels
            trainable &= ~data["_smf_ri_row_id"].isin(exclude_ids)
        approved = data[trainable].copy()
        if len(approved) == 0:
            n_approved = int((data[cfg.approved_col] == 1).sum())
            n_target_obs = int(data[cfg.target_col].notna().sum())
            raise ValueError(
                f"Cannot train pre-score model: no rows satisfy "
                f"({cfg.approved_col!r}==1 & {cfg.target_col!r} not null). "
                f"Diagnostics: approved={n_approved}, target-observed={n_target_obs}, "
                f"total={len(data)}. Check approved_col/target_col semantics or "
                f"supply a pre-computed score_col and set train_prescore=False."
            )
        splitter = SampleSplitter(
            test_size=cfg.prescore_test_size,
            random_state=cfg.random_state,
            stratify=True,
        )
        train, val = splitter.split_df(approved, target=cfg.target_col)
        model_type, params = self._resolve_model_params(
            cfg.prescore_model_type,
            cfg.prescore_params,
            role="prescore",
        )
        model = self._fit_pipeline_model(
            model_type=model_type,
            params=params,
            train=train,
            val=val,
            feature_cols=feature_cols,
        )
        scored = data.copy()
        scored[cfg.score_col] = predict_positive(model, scored, feature_cols)
        return scored, model

    def _default_hard_cutoff(self, approved: pd.DataFrame) -> float:
        cfg = self.config
        bad_scores = approved.loc[approved[cfg.target_col] == 1, cfg.score_col]
        if len(bad_scores.dropna()) == 0:
            scores = approved[cfg.score_col]
            # With all-NaN scores the median is NaN anyway (an error is raised below), so do
            # not let numpy emit an empty-slice warning
            fallback = float(scores.median()) if scores.notna().any() else float("nan")
            if not np.isfinite(fallback):
                raise ValueError(
                    "Cannot derive default hard cutoff: no valid bad-sample scores and "
                    "overall score median is NaN. Check prescore model output, or set "
                    "ri_method_params['hard_cutoff']['cutoff'] explicitly."
                )
            warnings.warn(
                f"No bad samples with valid {cfg.score_col!r}; using overall score "
                f"median {fallback:.6f} as default hard cutoff.",
                RuntimeWarning,
                stacklevel=2,
            )
            return fallback
        percentile = 25 if cfg.ri_score_direction == "high_bad" else 75
        return float(np.percentile(bad_scores.dropna(), percentile))

    def _build_inferrer(self, method: str, default_hard_cutoff: float) -> Any:
        from Modeling_Tool.Sample.Reject_Infer import (
            FuzzyAugmentInferrer,
            HardCutoffInferrer,
            ParcelingInferrer,
            SimpleAugmentInferrer,
        )

        cfg = self.config
        params = dict(cfg.ri_method_params.get(method, {}))
        common = {
            "target_col": cfg.target_col,
            "score_col": cfg.score_col,
            "score_direction": cfg.ri_score_direction,
            "random_state": cfg.random_state,
        }
        if method == "simple_augment":
            return SimpleAugmentInferrer(**common, bad_rate=params.get("bad_rate"))
        if method == "hard_cutoff":
            return HardCutoffInferrer(**common, cutoff=float(params.get("cutoff", default_hard_cutoff)))
        if method == "fuzzy_augment":
            return FuzzyAugmentInferrer(**common, weight_factor=float(params.get("weight_factor", 1.0)))
        if method == "parceling":
            return ParcelingInferrer(**common, n_parcels=int(params.get("n_parcels", 10)))
        raise ValueError(f"Unsupported reject inference method: {method!r}")

    def _compose_ri_dataset(
        self,
        df_inferred: pd.DataFrame,
        approved_output: pd.DataFrame,
        method: str,
    ) -> pd.DataFrame:
        cfg = self.config
        rejected_inferred = df_inferred[df_inferred[cfg.approved_col] == 0].copy()
        approved_part = approved_output.copy()
        if method == "fuzzy_augment":
            approved_part["_weight"] = 1.0
            if "_weight" not in rejected_inferred.columns:
                rejected_inferred["_weight"] = 1.0
        elif "_weight" in approved_part.columns and "_weight" not in rejected_inferred.columns:
            approved_part = approved_part.drop(columns=["_weight"])
        return pd.concat([approved_part, rejected_inferred], ignore_index=True, sort=False)

    def _write_ri_dataset(self, df: pd.DataFrame, path: Path, method: str) -> None:
        cfg = self.config
        out = df
        if cfg.ri_dataset_output_cols is not None:
            keep = [col for col in cfg.ri_dataset_output_cols if col in out.columns]
            missing = [col for col in cfg.ri_dataset_output_cols if col not in out.columns]
            if missing:
                warnings.warn(
                    f"ri_dataset_output_cols for {method} contains missing columns: {missing}",
                    UserWarning,
                )
            out = out.loc[:, keep].copy()
        size_mb = float(out.memory_usage(index=True, deep=True).sum()) / (1024.0 * 1024.0)
        if size_mb >= float(cfg.ri_dataset_warn_mb):
            warnings.warn(
                f"RI dataset {method!r} is estimated at {size_mb:.1f} MB before CSV serialization. "
                "Consider write_ri_datasets=False or ri_dataset_output_cols for wide full-scale runs.",
                UserWarning,
            )
        safe_to_csv(out, path, index=False)

    def _summarize_ri(self, datasets: dict[str, pd.DataFrame]) -> pd.DataFrame:
        cfg = self.config

        def _target_mean(data: pd.DataFrame) -> float:
            if data.empty:
                return np.nan
            y = pd.to_numeric(data[cfg.target_col], errors="coerce")
            if "_weight" not in data.columns:
                return float(y.mean())

            weight = pd.to_numeric(data["_weight"], errors="coerce").replace([np.inf, -np.inf], np.nan)
            valid = y.notna() & weight.notna() & (weight >= 0)
            if not bool(valid.any()):
                return np.nan
            weight_sum = float(weight.loc[valid].sum())
            if weight_sum <= 0:
                return np.nan
            return float(np.average(y.loc[valid], weights=weight.loc[valid]))

        rows = []
        for method, df in datasets.items():
            appr = df[cfg.approved_col] == 1
            rej = df[cfg.approved_col] == 0
            unlabelled = pd.to_numeric(df[cfg.target_col], errors="coerce").isna()
            try:
                # AUC of the pre-score on the approved rows with an observed target
                approved_target = df.loc[appr & ~unlabelled, cfg.target_col]
                approved_score = df.loc[appr & ~unlabelled, cfg.score_col]
                raw_auc = roc_auc_score(approved_target, approved_score)
                risk_score = approved_score if cfg.ri_score_direction == "high_bad" else -approved_score
                direction_adjusted_auc = roc_auc_score(approved_target, risk_score)
            except Exception:
                raw_auc = np.nan
                direction_adjusted_auc = np.nan
            rows.append(
                {
                    "ri_method": method,
                    "N_total": len(df),
                    "N_approved": int(appr.sum()),
                    "N_rejected": int(rej.sum()),
                    "N_approved_unlabelled": int((appr & unlabelled).sum()),
                    "N_rejected_unlabelled": int((rej & unlabelled).sum()),
                    "bad_rate_appr": _target_mean(df.loc[appr]),
                    "bad_rate_rej": _target_mean(df.loc[rej]),
                    "bad_rate_total": _target_mean(df),
                    "has_weight_col": "_weight" in df.columns,
                    "prescore_AUC": direction_adjusted_auc,
                    "prescore_AUC_raw": raw_auc,
                    "prescore_score_direction": cfg.ri_score_direction,
                }
            )
        return pd.DataFrame(rows)

    def _train_and_evaluate_models(
        self,
        ri_datasets: dict[str, pd.DataFrame],
        approved: pd.DataFrame,
        feature_cols: list[str],
        report_dir: Path,
        model_dir: Path,
        split_oot_data: pd.DataFrame | None = None,
        random_oot_ids: set[Any] | None = None,
    ) -> tuple[pd.DataFrame, dict[str, Any], dict[str, str], pd.DataFrame]:
        from Modeling_Tool import PerformanceEvaluator

        cfg = self.config
        rng = np.random.default_rng(cfg.random_state)
        exclude_train_ids: set[Any] = set()
        exclude_train_split = None
        # Only approved rows with an observed target can be trained on or evaluated
        labelled_approved = approved[approved[cfg.target_col].notna()]
        if labelled_approved.empty:
            raise ValueError(
                f"No approved row has an observed {cfg.target_col!r}, so the RI models cannot be "
                f"validated or evaluated; set train_ri_models=False to only build the RI datasets"
            )
        if cfg.oot_data is not None:
            oot, oot_summary = self._prepare_external_oot_data(feature_cols)
            val_ids = self._sample_validation_ids(approved, rng, exclude_ids=set())
            exclude_train_ids.update(val_ids)
            val = approved[approved["_smf_ri_row_id"].isin(val_ids)].copy()
        elif split_oot_data is not None and len(split_oot_data):
            oot, oot_summary = self._prepare_observed_oot_data(
                split_oot_data,
                feature_cols=feature_cols,
                source="split_col",
            )
            if "_smf_ri_split" in approved.columns and (labelled_approved["_smf_ri_split"] == "oos").any():
                val = labelled_approved[labelled_approved["_smf_ri_split"] == "oos"].copy()
                exclude_train_split = "oos"
            else:
                val_ids = self._sample_validation_ids(approved, rng, exclude_ids=set())
                exclude_train_ids.update(val_ids)
                val = approved[approved["_smf_ri_row_id"].isin(val_ids)].copy()
        else:
            if random_oot_ids is None:
                n_oot = max(1, int(len(labelled_approved) * cfg.oot_frac))
                oot_ids = set(rng.choice(labelled_approved["_smf_ri_row_id"].to_numpy(), size=n_oot, replace=False))
            else:
                # Drawn from all labelled approved rows before the pre-score; with
                # ri_approved_scope='output_subset' only those in the output subset remain
                in_output = labelled_approved["_smf_ri_row_id"].isin(random_oot_ids)
                oot_ids = set(labelled_approved.loc[in_output, "_smf_ri_row_id"])
                if not oot_ids:
                    raise ValueError(
                        "None of the rows held out as random OOT is among the approved output rows "
                        "(ri_approved_scope='output_subset' with a small reference sample); raise oot_frac, "
                        "or pass oot_data or split_col"
                    )
            oot = approved[approved["_smf_ri_row_id"].isin(oot_ids)].copy()
            exclude_train_ids.update(oot_ids)
            val_ids = self._sample_validation_ids(approved, rng, exclude_ids=oot_ids)
            exclude_train_ids.update(val_ids)
            val = approved[approved["_smf_ri_row_id"].isin(val_ids)].copy()
            oot_summary = self._summarize_oot_data(
                source="approved_random_split",
                raw_n=len(oot),
                observed_n=len(oot),
                dropped_n=0,
            )

        rows = []
        models: dict[str, Any] = {}
        model_paths: dict[str, str] = {}
        training_datasets: dict[str, pd.DataFrame] = {}
        if cfg.include_no_ri_benchmark:
            benchmark = approved.copy()
            if "_weight" in benchmark.columns:
                benchmark = benchmark.drop(columns=["_weight"])
            benchmark["ri_method"] = "no_ri_benchmark"
            training_datasets["no_ri_benchmark"] = benchmark
        training_datasets.update(ri_datasets)

        unlabelled_by_method: dict[str, tuple[int, int]] = {}
        for method, df_ri in training_datasets.items():
            train = df_ri.copy()
            if "_smf_ri_row_id" in train.columns and exclude_train_ids:
                train = train[~train["_smf_ri_row_id"].isin(exclude_train_ids)]
            if exclude_train_split and "_smf_ri_split" in train.columns:
                train = train[train["_smf_ri_split"] != exclude_train_split]
            # Approved rows without an observed target and rejected rows without an
            # inferred label (hard_cutoff with a missing score) carry no label to learn
            unlabelled = pd.to_numeric(train[cfg.target_col], errors="coerce").isna()
            n_unlabelled = int(unlabelled.sum())
            if n_unlabelled:
                n_unlabelled_appr = int((unlabelled & (train[cfg.approved_col] == 1)).sum())
                unlabelled_by_method[str(method)] = (n_unlabelled_appr, n_unlabelled - n_unlabelled_appr)
                train = train[~unlabelled]
            if len(train) == 0:
                raise ValueError(f"No training rows remain for RI method {method!r} after OOT/validation exclusion")
            if len(val) == 0:
                raise ValueError("RI validation sample is empty after splitting")

            model_type, params = self._resolve_model_params(
                cfg.ri_model_type,
                cfg.ri_model_params,
                role="ri",
            )
            weight_col = "_weight" if "_weight" in train.columns else None
            model = self._fit_pipeline_model(
                model_type=model_type,
                params=params,
                train=train,
                val=val,
                feature_cols=feature_cols,
                weight_col=weight_col,
            )
            models[method] = model

            train_eval = train[train[cfg.approved_col] == 1].copy()
            eval_sets = {"train": train_eval, "validation": val.copy(), "oot": oot.copy()}
            nan_stats: dict[str, int] = {}
            for ds in eval_sets.values():
                ds["pred_prob"] = predict_positive(model, ds, feature_cols, warn_nan=False)
            for ds_name, ds in eval_sets.items():
                nan_stats[ds_name] = int((~np.isfinite(ds["pred_prob"].to_numpy(dtype=float))).sum())
            self.predict_positive_nan_stats[str(method)] = nan_stats
            total_bad = sum(nan_stats.values())
            if total_bad:
                total_rows = sum(len(ds) for ds in eval_sets.values())
                detail = ", ".join(f"{k}={v}" for k, v in nan_stats.items() if v)
                warnings.warn(
                    f"{method}: NaN/Inf predictions detected across RI evaluation datasets: "
                    f"{detail}; total {total_bad}/{total_rows}.",
                    RuntimeWarning,
                    stacklevel=2,
                )

            evaluator = PerformanceEvaluator(
                tgt_name=cfg.target_col,
                scr_name="pred_prob",
                pct_bins=cfg.perf_pct_bins,
                min_bin_prop=cfg.min_bin_prop,
                equal_freq=True,
            )
            for name, ds in eval_sets.items():
                add_dataset_with_optional_weight(evaluator, name, ds, "_weight" if "_weight" in ds.columns else None)
            row = {
                "ri_method": method,
                "train_N": len(train),
                "train_unlabelled_n": n_unlabelled,
                "oot_N": len(oot),
                "weighted_train": "_weight" in train.columns,
            }
            for ds_name in ["train", "validation", "oot"]:
                metrics = self._binary_eval_metrics(eval_sets[ds_name])
                for metric_name, metric_value in metrics.items():
                    row[f"{ds_name}_{metric_name}"] = metric_value
            try:
                perf = evaluator.evaluate(
                    to_show=False,
                    display=False,
                    fig_save_path=str(report_dir / "perf_figs" / f"perf_{method}.png") if cfg.write_outputs else None,
                    rpt_save_path=str(report_dir / f"perf_{method}.csv") if cfg.write_outputs else None,
                )
            except Exception as exc:
                perf = pd.DataFrame()
                row["perf_eval_error"] = repr(exc)
                warnings.warn(
                    f"PerformanceEvaluator failed for RI method {method!r}; "
                    "ri_model_perf keeps internally computed AUC/KS/Gini. "
                    f"Original error: {exc!r}",
                    UserWarning,
                )
            if isinstance(perf, pd.DataFrame):
                for ds_name in ["train", "validation", "oot"]:
                    subset = perf[perf["dataset"] == ds_name] if "dataset" in perf.columns else pd.DataFrame()
                    if len(subset):
                        for output_col, candidates in {
                            "AUC": ["AUC", "auc"],
                            "KS": ["KS", "ks"],
                            "Gini": ["Gini", "gini"],
                        }.items():
                            for col in candidates:
                                if col in subset.columns:
                                    row[f"{ds_name}_{output_col}"] = float(subset.iloc[0][col])
                                    break
            rows.append(row)
            if cfg.save_models:
                model_paths[method] = self._save_pipeline_model(
                    model=model,
                    path=model_dir / f"ri_model_{method}.pkl",
                    feature_cols=feature_cols,
                    model_role="ri_model",
                    ri_method=method,
                    model_type=model_type,
                    metrics=row,
                )

        if unlabelled_by_method:
            detail = "; ".join(
                f"{method}: {n_appr} approved without an observed target, {n_rej} rejected without an inferred label"
                for method, (n_appr, n_rej) in unlabelled_by_method.items()
            )
            warnings.warn(
                f"Rows with a missing {cfg.target_col!r} were left out of RI model training ({detail}). "
                f"They stay in ri_datasets; ri_model_perf['train_unlabelled_n'] counts them per model.",
                UserWarning,
                stacklevel=2,
            )

        perf_df = pd.DataFrame(rows)
        sort_col = "oot_AUC" if "oot_AUC" in perf_df.columns else None
        if sort_col:
            perf_df = perf_df.sort_values(sort_col, ascending=False).reset_index(drop=True)
        return perf_df, models, model_paths, oot_summary

    def _binary_eval_metrics(self, data: pd.DataFrame) -> dict[str, float]:
        cfg = self.config
        y = pd.to_numeric(data[cfg.target_col], errors="coerce")
        score = pd.to_numeric(data["pred_prob"], errors="coerce")
        valid = y.notna() & score.notna()
        if valid.sum() == 0 or y[valid].nunique() < 2:
            return {"AUC": np.nan, "KS": np.nan, "Gini": np.nan}
        weight = None
        if "_weight" in data.columns:
            weight = pd.to_numeric(data.loc[valid, "_weight"], errors="coerce").fillna(0).to_numpy(dtype=float)
        y_arr = y[valid].astype(int).to_numpy()
        score_arr = score[valid].to_numpy(dtype=float)
        auc = float(roc_auc_score(y_arr, score_arr, sample_weight=weight))
        fpr, tpr, _ = roc_curve(y_arr, score_arr, sample_weight=weight)
        ks = float(np.max(tpr - fpr)) if len(tpr) else np.nan
        return {"AUC": auc, "KS": ks, "Gini": float(2 * auc - 1)}

    def _sample_validation_ids(
        self,
        approved: pd.DataFrame,
        rng: np.random.Generator,
        exclude_ids: set[Any],
    ) -> set[Any]:
        cfg = self.config
        if "_smf_ri_row_id" not in approved.columns:
            return set()
        # Validation rows need an observed target
        approved = approved[approved[cfg.target_col].notna()]
        pool = approved[~approved["_smf_ri_row_id"].isin(exclude_ids)].copy()
        if len(pool) == 0:
            pool = approved.copy()
        n_val = max(1, int(round(len(pool) * float(cfg.ri_validation_frac))))
        if n_val >= len(pool) and len(pool) > 1:
            n_val = max(1, len(pool) // 5)
        if pool[cfg.target_col].nunique(dropna=True) > 1 and n_val >= 2:
            sampled_parts = []
            for _, grp in pool.groupby(cfg.target_col):
                grp_n = max(1, int(round(len(grp) * float(cfg.ri_validation_frac))))
                grp_n = min(grp_n, len(grp))
                sampled_parts.append(grp.sample(n=grp_n, random_state=cfg.random_state))
            sampled = pd.concat(sampled_parts, ignore_index=False)
            if len(sampled) > n_val:
                sampled = sampled.sample(n=n_val, random_state=cfg.random_state)
        else:
            sampled = pool.sample(n=n_val, random_state=cfg.random_state)
        return set(sampled["_smf_ri_row_id"].tolist())

    def _prepare_external_oot_data(self, feature_cols: list[str]) -> tuple[pd.DataFrame, pd.DataFrame]:
        return self._prepare_observed_oot_data(
            self.config.oot_data,
            feature_cols=feature_cols,
            source="external_oot_data",
        )

    def _prepare_observed_oot_data(
        self,
        oot_data: pd.DataFrame,
        feature_cols: list[str],
        source: str,
    ) -> tuple[pd.DataFrame, pd.DataFrame]:
        cfg = self.config
        oot = oot_data.copy()
        label = "External oot_data" if source == "external_oot_data" else source
        required = [cfg.target_col] + feature_cols
        missing = [col for col in dict.fromkeys(required) if col not in oot.columns]
        if missing:
            raise KeyError(f"{label} missing required columns: {missing}")
        raw_n = len(oot)
        observed_mask = oot[cfg.target_col].notna()
        observed_n = int(observed_mask.sum())
        dropped_n = raw_n - observed_n
        if dropped_n:
            warnings.warn(
                f"{label} contains missing target rows; "
                f"dropped {dropped_n} of {raw_n} rows and kept {observed_n} observed rows.",
                UserWarning,
                stacklevel=2,
            )
            oot = oot[observed_mask].copy()
        if observed_n == 0:
            raise ValueError("OOT data has no observed target rows after filtering missing target values")
        summary = self._summarize_oot_data(
            source=source,
            raw_n=raw_n,
            observed_n=observed_n,
            dropped_n=dropped_n,
        )
        return oot.reset_index(drop=True), summary

    def _summarize_oot_data(
        self,
        source: str,
        raw_n: int,
        observed_n: int,
        dropped_n: int,
    ) -> pd.DataFrame:
        missing_rate = dropped_n / raw_n if raw_n else np.nan
        rows = [
            ("oot_source", source),
            ("oot_raw_n", raw_n),
            ("oot_observed_n", observed_n),
            ("oot_dropped_missing_target_n", dropped_n),
            ("oot_missing_target_rate", missing_rate),
        ]
        return pd.DataFrame(rows, columns=["metric", "value"])

    def _save_pipeline_model(
        self,
        model: Any,
        path: Path,
        feature_cols: list[str],
        model_role: str,
        ri_method: str | None,
        model_type: str,
        metrics: dict[str, Any] | None = None,
    ) -> str:
        from Modeling_Tool import save_model

        cfg = self.config
        metadata = {
            "pipeline": "RejectInferencePipeline",
            "model_role": model_role,
            "ri_method": ri_method,
            "target_col": cfg.target_col,
            "score_col": cfg.score_col,
            "model_type": model_type,
            "random_state": cfg.random_state,
        }
        fill_values = getattr(model, "fill_values_", None)
        if fill_values is not None:
            metadata["lr_nan_handling"] = getattr(model, "nan_handling", None)
            metadata["lr_fill_values"] = dict(fill_values)
        save_model(
            model,
            path,
            metadata=metadata,
            feature_cols=feature_cols,
            metrics=metrics,
            model_name=f"reject_inference_{model_role}",
            model_version=str(ri_method or model_role),
            include_metadata=cfg.model_include_metadata,
        )
        return str(path)

    def _model_paths_to_frame(self, model_paths: dict[str, str]) -> pd.DataFrame:
        return pd.DataFrame(
            [{"model_key": key, "model_path": path} for key, path in model_paths.items()]
        )

run

run(data: DataFrame) -> RejectInferencePipelineResult

Run the reject-inference workflow on the approved and rejected applications in data.

Parameters:

Name	Type	Description	Default
data	DataFrame	Application data with all approved (approved_col == 1) and rejected (approved_col == 0) rows. It must contain approved_col, the feature columns, split_col when configured, target_col whenever the pre-score is trained (train_prescore is True or score_col is absent) and score_col when train_prescore is False. The input frame is not modified.	required

Returns:

Type	Description
RejectInferencePipelineResult	Approved and rejected data, the RI datasets and summary, the model comparison, the best method and the report and model paths.

Raises:

Type	Description
KeyError	If a required column is missing from data, oot_data or ri_approved_data.
ValueError	If the configuration is inconsistent (unknown model type, method or direction, ri_score_direction="high_good" with a pre-score trained by the pipeline, invalid fractions, conflicting ri_approved_* settings, invalid split_col labels), if no approved row has an observed target for the pre-score or (with train_ri_models) for the models, or if a training, validation, OOT or reference sample is empty.

Notes

Warnings are issued when train_prescore=True overwrites an existing score_col (of data or of ri_approved_data), when rows with a missing target are dropped from an external OOT or left out of model training, and when non-finite values are imputed or predicted. With save_models=True the models are pickled even if write_outputs is False.

Source code in Modeling_Tool/Pipeline/reject_inference.py

def run(self, data: pd.DataFrame) -> RejectInferencePipelineResult:
    """Run the reject-inference workflow on the approved and rejected applications in ``data``.

    Parameters
    ----------
    data : pandas.DataFrame
        Application data with all approved (``approved_col == 1``) and rejected (``approved_col == 0``) rows. It must
        contain ``approved_col``, the feature columns, ``split_col`` when configured, ``target_col`` whenever the
        pre-score is trained (``train_prescore`` is True or ``score_col`` is absent) and ``score_col`` when
        ``train_prescore`` is False. The input frame is not modified.

    Returns
    -------
    RejectInferencePipelineResult
        Approved and rejected data, the RI datasets and summary, the model comparison, the best method and the report
        and model paths.

    Raises
    ------
    KeyError
        If a required column is missing from ``data``, ``oot_data`` or ``ri_approved_data``.
    ValueError
        If the configuration is inconsistent (unknown model type, method or direction,
        ``ri_score_direction="high_good"`` with a pre-score trained by the pipeline, invalid fractions, conflicting
        ``ri_approved_*`` settings, invalid ``split_col`` labels), if no approved row has an observed target for the
        pre-score or (with ``train_ri_models``) for the models, or if a training, validation, OOT or reference sample
        is empty.

    Notes
    -----
    Warnings are issued when ``train_prescore=True`` overwrites an existing ``score_col`` (of ``data`` or of
    ``ri_approved_data``), when rows with a missing target are dropped from an external OOT or left out of model
    training, and when non-finite values are imputed or predicted. With ``save_models=True`` the models are pickled
    even if ``write_outputs`` is False.
    """
    cfg = self.config
    feature_cols = self._resolve_feature_cols(data)
    self._validate_input(data, feature_cols)
    self._validate_ri_approved_config()
    self._validate_split_col_schema(data)
    self._validate_external_ri_approved_schema(data, feature_cols)

    datasets_dir = Path(cfg.output_dir) / "datasets"
    report_dir = Path(cfg.output_dir) / "report"
    figs_dir = report_dir / "perf_figs"
    model_dir = Path(cfg.model_output_dir) if cfg.model_output_dir else Path(cfg.output_dir) / "models"
    if cfg.write_outputs or cfg.write_excel:
        make_dirs(datasets_dir, report_dir, figs_dir)
    if cfg.save_models:
        make_dirs(model_dir)

    work = data.copy()
    work["_smf_ri_row_id"] = np.arange(len(work))
    split_oot_data = None
    if cfg.split_col:
        work, split_oot_data = self._prepare_split_col_data(work)
    # Drawn before the pre-score so that the OOT rows stay out of its training data
    random_oot_ids = self._draw_random_oot_ids(work, split_oot_data)
    prescore_model = None
    model_paths: dict[str, str] = {}
    if cfg.train_prescore or cfg.score_col not in work.columns:
        if cfg.train_prescore and cfg.score_col in work.columns:
            warnings.warn(
                f"train_prescore=True will overwrite existing column "
                f"{cfg.score_col!r} in input data. Set train_prescore=False "
                f"to reuse the supplied scores, or rename cfg.score_col to "
                f"a fresh column name to avoid this overwrite.",
                UserWarning,
                stacklevel=2,
            )
        work, prescore_model = self._fit_prescore(work, feature_cols, exclude_ids=random_oot_ids)
        if cfg.save_models:
            model_paths["prescore"] = self._save_pipeline_model(
                model=prescore_model,
                path=model_dir / "prescore_model.pkl",
                feature_cols=feature_cols,
                model_role="prescore",
                ri_method=None,
                model_type=self._normalize_model_type(
                    cfg.prescore_model_type,
                    config_name="prescore_model_type",
                ),
            )

    approved_full = work[work[cfg.approved_col] == 1].copy().reset_index(drop=True)
    rejected = work[work[cfg.approved_col] == 0].copy().reset_index(drop=True)
    ri_approved_ref, approved_output, ri_approved_summary = self._prepare_ri_approved_reference(
        work=work,
        approved_full=approved_full,
        feature_cols=feature_cols,
        prescore_model=prescore_model,
    )
    hard_cutoff = self._default_hard_cutoff(ri_approved_ref)

    ri_datasets: dict[str, pd.DataFrame] = {}
    inferrers: dict[str, Any] = {}
    for requested in as_list(cfg.ri_methods):
        method = self._METHOD_ALIASES.get(str(requested).lower())
        if method is None:
            raise ValueError(f"Unsupported reject inference method: {requested!r}")
        inferrer = self._build_inferrer(method, hard_cutoff)
        df_inferred = inferrer.infer(
            df_approved=ri_approved_ref,
            df_rejected=rejected,
            score_col=cfg.score_col,
        )
        df_ri = self._compose_ri_dataset(
            df_inferred=df_inferred,
            approved_output=approved_output,
            method=method,
        )
        df_ri["ri_method"] = method
        ri_datasets[method] = df_ri
        inferrers[method] = inferrer
        if cfg.write_outputs and cfg.write_ri_datasets:
            self._write_ri_dataset(df_ri, datasets_dir / f"ri_{method}.csv", method)

    ri_summary = self._summarize_ri(ri_datasets)
    if cfg.write_outputs:
        safe_to_csv(ri_summary, report_dir / "ri_comparison_summary.csv", index=False)

    ri_model_perf = None
    best_method = None
    ri_models: dict[str, Any] = {}
    oot_summary = None
    if cfg.train_ri_models:
        ri_model_perf, ri_models, ri_model_paths, oot_summary = self._train_and_evaluate_models(
            ri_datasets=ri_datasets,
            approved=approved_output,
            feature_cols=feature_cols,
            report_dir=report_dir,
            model_dir=model_dir,
            split_oot_data=split_oot_data,
            random_oot_ids=random_oot_ids,
        )
        model_paths.update(ri_model_paths)
        if ri_model_perf is not None and len(ri_model_perf):
            best_method = str(ri_model_perf.iloc[0]["ri_method"])
            if cfg.write_outputs:
                safe_to_csv(ri_model_perf, report_dir / "ri_model_perf.csv", index=False)
        if cfg.write_outputs and oot_summary is not None:
            safe_to_csv(oot_summary, report_dir / "oot_summary.csv", index=False)
    if cfg.write_outputs and model_paths:
        safe_to_csv(self._model_paths_to_frame(model_paths), report_dir / "model_paths.csv", index=False)

    report_path = None
    if cfg.write_excel:
        report_path = str(report_dir / "RI_Pipeline_Report.xlsx")
        sheets = {
            "RI_Dataset_Stats": ri_summary,
            "Model_Performance": ri_model_perf,
            "RI_Approved_Sample": ri_approved_summary,
            "OOT_Sample": oot_summary,
            "Model_Paths": self._model_paths_to_frame(model_paths) if model_paths else None,
        }
        write_basic_excel(report_path, sheets, title="SMF Reject Inference Pipeline Report")

    return RejectInferencePipelineResult(
        approved_data=approved_output,
        rejected_data=rejected,
        ri_datasets=ri_datasets,
        ri_summary=ri_summary,
        ri_model_perf=ri_model_perf,
        best_method=best_method,
        prescore_model=prescore_model,
        ri_models=ri_models,
        report_path=report_path,
        approved_full_data=approved_full,
        ri_approved_reference_data=ri_approved_ref,
        ri_approved_summary=ri_approved_summary,
        model_paths=model_paths,
        oot_summary=oot_summary,
    )

Score Comparison Pipeline: score_comparison

score_comparison

ScoreComparisonPipelineConfig dataclass

Configuration of :class:ScoreComparisonPipeline.

Parameters:

Name	Type	Description	Default
output_dir	str	Root output directory. CSV files and the Excel report go to <output_dir>/report.	"output/score_comparison"
target_col	str	Binary target column (1 = bad). It must exist in the input data.	"badflag"
score_cols	list of str or None	All score columns to compare. Without base_score the first entry is the base score and the rest are the comparison scores. When None, both base_score and comp_scores must be given, otherwise ValueError.	None
base_score	str or None	Baseline score column. None uses score_cols[0].	None
comp_scores	list of str or None	Comparison score columns. None uses the entries of score_cols other than base_score.	None
weight_col	str or None	Sample-weight column (must exist in the data). It weights the performance tables (global_perf and group_perf) and gains; the cross-risk tables (cross_results and pairwise_cross) are unweighted.	None
split_col	str or None	Column with evaluation-set labels (any non-empty names such as ins / oos / oot). Values are stripped and lower-cased and must include at least one non-empty value. It only adds one entry to group_perf (when group_specs is None); it does not split the data for the other tables.	None
random_state	int	Not used by this pipeline; kept so that all high-level Pipeline configs share the same interface.	42
write_outputs	bool	Whether to write CSV files to <output_dir>/report: step1_global_perf.csv, step2_by_<group>.csv, step3_gains_with_metrics.csv, step4_<score>__<cross_var>__<metric>.csv and step4_pairwise.csv. In the file names, characters that a file name cannot hold (/ \ : * ? " < > | and control characters) become _, and a name that would repeat another one (ignoring case) gets a _2, _3 ... suffix, so every table is written to its own file in report. The keys of group_perf and cross_results are not changed.	True
write_excel	bool	Whether to write <output_dir>/report/Score_Comparison_Report.xlsx (its path is returned in report_path).	True
nbins	int	Number of bins for the performance tables, the gains tables and both cross-risk analyses.	10
min_bin_prop	float	Minimum share of samples per bin.	0.02
equal_freq	bool	Whether to use equal-frequency binning.	True
min_data_size	int	Minimum number of rows a group value needs to be evaluated (smaller groups are skipped; a group with exactly this many rows is evaluated, for single columns and crossed groups alike). It is also the default for group_min_size.	50
precision	int	Decimal precision of bin boundaries and of the score ranges (not applied to the cross_vars tables).	5
include_missing	bool	Whether missing values get their own bin in the gains and cross-risk tables. The performance tables always exclude missing scores. With drop_missing_group_values=False it also turns the missing group tokens into a [Missing] group.	False
fillna	any	Value used to fill missing scores before binning the gains tables and the cross_vars cross-risk tables.	-999999
positive_score_only	bool	In the performance tables a score counts as valid only where it is greater than 0, so zero, negative and missing scores are dropped (missing and infinite scores are never valid). It does not change the gains tables, and the pairwise cross always requires both scores to be greater than 0.	True
perf_common_rows	bool	True evaluates every score of global_perf and group_perf, the base score included, on the same rows: the rows with an observed target where all the scores are valid, so that the scores are compared on one population. A score without any valid row is left out with a UserWarning. False evaluates each score on its own valid rows (the metrics then describe different populations). The column N_OWN gives each score's own number of valid rows.	True
group_missing_values	list	Text values (compared after stripping whitespace) treated as missing in the grouping columns (split_col, time_dims, population_dims and the columns of group_specs). Only text columns are inspected, and all their values are stripped of surrounding whitespace.	['', ' ', 'NA', 'NULL', 'nan']
drop_missing_group_values	bool	True sets those values to missing so that they do not form a group. False keeps them as ordinary group labels, or as [Missing] when include_missing is True.	True
time_dims	list of str	Time columns; each one is evaluated as a group in group_perf. Columns absent from the data are skipped silently.	['apply_month']
population_dims	list of str	Population columns; each one is evaluated as a group in group_perf. Columns absent from the data are skipped silently.	['channel']
segment_dims	list of str or None	Alias of population_dims: when not None its content replaces population_dims at construction.	None
include_time_population_cross	bool	Whether to also evaluate every population column crossed with every time column (group name <population>_x_<time>).	True
group_min_size	int or None	Minimum rows per group value in the group evaluation (an entry of group_specs may set its own min_size). None uses min_data_size.	None
group_specs	dict or list or None	Custom grouping that replaces the automatic groups (split_col, time_dims, population_dims and their cross). Either {name: [columns]} or a list whose items are column lists or dicts with columns (or cols) and optional name and min_size. Specs with a column missing from the data are skipped silently; invalid specs raise ValueError.	None
gains_add_func	callable or None	Function f(df) -> pandas.Series applied to every gains bin to add extra metric columns, with and without weight_col (the function sees the weight column among the columns of the bin). None adds the mean of every custom_metric_cols column present in the data as <col>_mean, weighted by weight_col when it is set.	None
custom_metric_cols	list of str	Business columns averaged in the default gains metrics and used in the default cross_metrics (mean) and pairwise_cross_agg_dict (count and mean). Columns absent from the data are skipped silently.	['credit_limit', 'age', 'apr']
gains_display_metric_list	list of str	Gains columns to display. It is passed to the evaluation tool, which applies it only when no add_func is given; this pipeline always supplies one, so the gains result currently keeps all gains columns.	['MIN', 'MAX', 'N', 'PROP', 'AVG_SCORE', 'AVG_BAD', 'CUM_BAD_PCT', 'KS_PER_BIN', 'LIFT', 'RANK_ORDER_BUMP']
cross_vars	list of str	Second-dimension variables for cross_results: every score in the score list is crossed with each variable for each cross_metrics entry. Variables absent from the data are skipped with a warning. The default is no cross analysis (in 0.4.0 it stopped defaulting to ["rating"]).	[]
cross_metrics	dict	{metric_name: (column, aggregation)} evaluated in each cross table; the aggregation is a pandas aggregation name or a callable. An empty dict uses the bad rate (mean of target_col) and the mean of each custom_metric_cols column present in the data. Invalid items raise TypeError, ValueError or KeyError.	{}
cross_binning_numeric	list of bool or bool	[bin the score, bin the cross variable] for numeric columns in cross_results; a single bool applies to both. A list or tuple of another length raises ValueError. It does not affect pairwise_cross.	[True, False]
pairwise_cross_enabled	bool	Whether to compute pairwise_cross, the cross of the base score with each comparison score.	True
pairwise_cross_agg_dict	dict or None	{column: aggregation or [aggregations]} evaluated in the pairwise cross (a row count and share of flow_id is always added). None uses the count and bad rate of target_col plus the count and mean of each custom_metric_cols column present. Columns that do not exist raise KeyError and a wrong format raises TypeError.	None

Source code in Modeling_Tool/Pipeline/score_comparison.py

@dataclass
class ScoreComparisonPipelineConfig:
    """Configuration of :class:`ScoreComparisonPipeline`.

    Parameters
    ----------
    output_dir : str, default "output/score_comparison"
        Root output directory. CSV files and the Excel report go to ``<output_dir>/report``.
    target_col : str, default "badflag"
        Binary target column (1 = bad). It must exist in the input data.
    score_cols : list of str or None, default None
        All score columns to compare. Without ``base_score`` the first entry is the base score and the rest are the
        comparison scores. When None, both ``base_score`` and ``comp_scores`` must be given, otherwise ``ValueError``.
    base_score : str or None, default None
        Baseline score column. ``None`` uses ``score_cols[0]``.
    comp_scores : list of str or None, default None
        Comparison score columns. ``None`` uses the entries of ``score_cols`` other than ``base_score``.
    weight_col : str or None, default None
        Sample-weight column (must exist in the data). It weights the performance tables (``global_perf`` and
        ``group_perf``) and ``gains``; the cross-risk tables (``cross_results`` and ``pairwise_cross``) are unweighted.
    split_col : str or None, default None
        Column with evaluation-set labels (any non-empty names such as ``ins`` / ``oos`` / ``oot``). Values are
        stripped and lower-cased and must include at least one non-empty value. It only adds one entry to
        ``group_perf`` (when ``group_specs`` is None); it does not split the data for the other tables.
    random_state : int, default 42
        Not used by this pipeline; kept so that all high-level Pipeline configs share the same interface.
    write_outputs : bool, default True
        Whether to write CSV files to ``<output_dir>/report``: ``step1_global_perf.csv``, ``step2_by_<group>.csv``,
        ``step3_gains_with_metrics.csv``, ``step4_<score>__<cross_var>__<metric>.csv`` and ``step4_pairwise.csv``. In the
        file names, characters that a file name cannot hold (``/ \\ : * ? " < > |`` and control characters) become
        ``_``, and a name that would repeat another one (ignoring case) gets a ``_2``, ``_3`` ... suffix, so every
        table is written to its own file in ``report``. The keys of ``group_perf`` and ``cross_results`` are not
        changed.
    write_excel : bool, default True
        Whether to write ``<output_dir>/report/Score_Comparison_Report.xlsx`` (its path is returned in
        ``report_path``).
    nbins : int, default 10
        Number of bins for the performance tables, the gains tables and both cross-risk analyses.
    min_bin_prop : float, default 0.02
        Minimum share of samples per bin.
    equal_freq : bool, default True
        Whether to use equal-frequency binning.
    min_data_size : int, default 50
        Minimum number of rows a group value needs to be evaluated (smaller groups are skipped; a group with exactly
        this many rows is evaluated, for single columns and crossed groups alike). It is also the default for
        ``group_min_size``.
    precision : int, default 5
        Decimal precision of bin boundaries and of the score ranges (not applied to the ``cross_vars`` tables).
    include_missing : bool, default False
        Whether missing values get their own bin in the gains and cross-risk tables. The performance tables always
        exclude missing scores. With ``drop_missing_group_values=False`` it also turns the missing group tokens into a
        ``[Missing]`` group.
    fillna : any, default -999999
        Value used to fill missing scores before binning the gains tables and the ``cross_vars`` cross-risk tables.
    positive_score_only : bool, default True
        In the performance tables a score counts as valid only where it is greater than 0, so zero, negative and missing
        scores are dropped (missing and infinite scores are never valid). It does not change the gains tables, and the
        pairwise cross always requires both scores to be greater than 0.
    perf_common_rows : bool, default True
        True evaluates every score of ``global_perf`` and ``group_perf``, the base score included, on the same rows: the
        rows with an observed target where all the scores are valid, so that the scores are compared on one population.
        A score without any valid row is left out with a ``UserWarning``. False evaluates each score on its own valid
        rows (the metrics then describe different populations). The column ``N_OWN`` gives each score's own number of
        valid rows.
    group_missing_values : list, default ['', ' ', 'NA', 'NULL', 'nan']
        Text values (compared after stripping whitespace) treated as missing in the grouping columns (``split_col``,
        ``time_dims``, ``population_dims`` and the columns of ``group_specs``). Only text columns are inspected, and all
        their values are stripped of surrounding whitespace.
    drop_missing_group_values : bool, default True
        True sets those values to missing so that they do not form a group. False keeps them as ordinary group labels,
        or as ``[Missing]`` when ``include_missing`` is True.
    time_dims : list of str, default ['apply_month']
        Time columns; each one is evaluated as a group in ``group_perf``. Columns absent from the data are skipped
        silently.
    population_dims : list of str, default ['channel']
        Population columns; each one is evaluated as a group in ``group_perf``. Columns absent from the data are
        skipped silently.
    segment_dims : list of str or None, default None
        Alias of ``population_dims``: when not None its content replaces ``population_dims`` at construction.
    include_time_population_cross : bool, default True
        Whether to also evaluate every population column crossed with every time column (group name
        ``<population>_x_<time>``).
    group_min_size : int or None, default None
        Minimum rows per group value in the group evaluation (an entry of ``group_specs`` may set its own
        ``min_size``). ``None`` uses ``min_data_size``.
    group_specs : dict or list or None, default None
        Custom grouping that replaces the automatic groups (``split_col``, ``time_dims``, ``population_dims`` and their
        cross). Either ``{name: [columns]}`` or a list whose items are column lists or dicts with ``columns`` (or
        ``cols``) and optional ``name`` and ``min_size``. Specs with a column missing from the data are skipped
        silently; invalid specs raise ``ValueError``.
    gains_add_func : callable or None, default None
        Function ``f(df) -> pandas.Series`` applied to every gains bin to add extra metric columns, with and without
        ``weight_col`` (the function sees the weight column among the columns of the bin). ``None`` adds the mean of
        every ``custom_metric_cols`` column present in the data as ``<col>_mean``, weighted by ``weight_col`` when it is
        set.
    custom_metric_cols : list of str, default ['credit_limit', 'age', 'apr']
        Business columns averaged in the default gains metrics and used in the default ``cross_metrics`` (mean) and
        ``pairwise_cross_agg_dict`` (count and mean). Columns absent from the data are skipped silently.
    gains_display_metric_list : list of str, default ['MIN', 'MAX', 'N', 'PROP', 'AVG_SCORE', 'AVG_BAD', 'CUM_BAD_PCT', 'KS_PER_BIN', 'LIFT', 'RANK_ORDER_BUMP']
        Gains columns to display. It is passed to the evaluation tool, which applies it only when no ``add_func`` is
        given; this pipeline always supplies one, so the ``gains`` result currently keeps all gains columns.
    cross_vars : list of str, default []
        Second-dimension variables for ``cross_results``: every score in the score list is crossed with each variable
        for each ``cross_metrics`` entry. Variables absent from the data are skipped with a warning. The default is no
        cross analysis (in 0.4.0 it stopped defaulting to ``["rating"]``).
    cross_metrics : dict, default {}
        ``{metric_name: (column, aggregation)}`` evaluated in each cross table; the aggregation is a pandas aggregation
        name or a callable. An empty dict uses the bad rate (mean of ``target_col``) and the mean of each
        ``custom_metric_cols`` column present in the data. Invalid items raise ``TypeError``, ``ValueError`` or
        ``KeyError``.
    cross_binning_numeric : list of bool or bool, default [True, False]
        ``[bin the score, bin the cross variable]`` for numeric columns in ``cross_results``; a single ``bool`` applies to
        both. A list or tuple of another length raises ``ValueError``. It does not affect ``pairwise_cross``.
    pairwise_cross_enabled : bool, default True
        Whether to compute ``pairwise_cross``, the cross of the base score with each comparison score.
    pairwise_cross_agg_dict : dict or None, default None
        ``{column: aggregation or [aggregations]}`` evaluated in the pairwise cross (a row count and share of
        ``flow_id`` is always added). ``None`` uses the count and bad rate of ``target_col`` plus the count and mean of
        each ``custom_metric_cols`` column present. Columns that do not exist raise ``KeyError`` and a wrong format
        raises ``TypeError``.
    """

    output_dir: str = "output/score_comparison"
    target_col: str = "badflag"
    score_cols: list[str] | None = None
    base_score: str | None = None
    comp_scores: list[str] | None = None
    weight_col: str | None = None
    split_col: str | None = None
    random_state: int = 42
    write_outputs: bool = True
    write_excel: bool = True

    nbins: int = 10
    min_bin_prop: float = 0.02
    equal_freq: bool = True
    min_data_size: int = 50
    precision: int = 5
    include_missing: bool = False
    fillna: Any = -999999
    positive_score_only: bool = True
    perf_common_rows: bool = True
    group_missing_values: list[Any] = field(default_factory=lambda: ["", " ", "NA", "NULL", "nan"])
    drop_missing_group_values: bool = True

    time_dims: list[str] = field(default_factory=lambda: ["apply_month"])
    population_dims: list[str] = field(default_factory=lambda: ["channel"])
    segment_dims: list[str] | None = None
    include_time_population_cross: bool = True
    group_min_size: int | None = None
    group_specs: dict[str, list[str]] | list[Any] | None = None
    gains_add_func: Callable[[pd.DataFrame], pd.Series] | None = None
    custom_metric_cols: list[str] = field(default_factory=lambda: ["credit_limit", "age", "apr"])
    gains_display_metric_list: list[str] = field(
        default_factory=lambda: [
            "MIN",
            "MAX",
            "N",
            "PROP",
            "AVG_SCORE",
            "AVG_BAD",
            "CUM_BAD_PCT",
            "KS_PER_BIN",
            "LIFT",
            "RANK_ORDER_BUMP",
        ]
    )

    # v0.4.0 behavior change: default is now [] (no cross-var breakdown).
    # Previous default ["rating"] silently required a 'rating' column and either
    # crashed or produced misleading breakdowns when it was absent. Callers who
    # want a rating breakdown must now set cross_vars=["rating"] explicitly.
    cross_vars: list[str] = field(default_factory=list)
    cross_metrics: dict[str, tuple[str, Any]] = field(default_factory=dict)
    cross_binning_numeric: list[bool] | bool = field(default_factory=lambda: [True, False])
    pairwise_cross_enabled: bool = True
    pairwise_cross_agg_dict: dict[str, Any] | None = None

    def __post_init__(self) -> None:
        if self.segment_dims is not None:
            self.population_dims = list(self.segment_dims)

ScoreComparisonPipelineResult dataclass

Result returned by :meth:ScoreComparisonPipeline.run.

Parameters:

Name	Type	Description	Default
global_perf	DataFrame	Performance (N, KS, AUC, top/bottom decile lift and so on) of the base and comparison scores on the whole sample, on common rows unless perf_common_rows is False. One row per score (score_name) with N_OWN, the score's own number of valid rows; sample_scope is "global".	required
group_perf	dict of str to pandas.DataFrame	Same performance metrics per group value, keyed by group name: split_col, each time and population column, <population>_x_<time> crosses, or the group_specs names. Empty when no group is evaluated.	required
gains	DataFrame	Gains table of every score on the whole sample (one block of bins per score_name, each ending with its Grand Summary row), with the metrics of gains_add_func or the custom_metric_cols means. With weight_col it is the weighted table, with the same extra columns and summary row.	required
cross_results	dict of str to pandas.DataFrame	Cross-risk tables keyed <score>__<cross_var>__<metric_name>. Empty when cross_vars is empty.	required
pairwise_cross	DataFrame or None	Long table with columns base_scr_range, eval_metric, score_name, comp_scr_range and value for the base score crossed with each comparison score. None when pairwise_cross_enabled is False.	None
report_path	str or None	Path of the Excel report; None when write_excel is False.	None

Source code in Modeling_Tool/Pipeline/score_comparison.py

@dataclass
class ScoreComparisonPipelineResult:
    """Result returned by :meth:`ScoreComparisonPipeline.run`.

    Parameters
    ----------
    global_perf : pandas.DataFrame
        Performance (``N``, ``KS``, ``AUC``, top/bottom decile lift and so on) of the base and comparison scores on the
        whole sample, on common rows unless ``perf_common_rows`` is False. One row per score (``score_name``) with
        ``N_OWN``, the score's own number of valid rows; ``sample_scope`` is ``"global"``.
    group_perf : dict of str to pandas.DataFrame
        Same performance metrics per group value, keyed by group name: ``split_col``, each time and population
        column, ``<population>_x_<time>`` crosses, or the ``group_specs`` names. Empty when no group is evaluated.
    gains : pandas.DataFrame
        Gains table of every score on the whole sample (one block of bins per ``score_name``, each ending with its
        ``Grand Summary`` row), with the metrics of ``gains_add_func`` or the ``custom_metric_cols`` means. With
        ``weight_col`` it is the weighted table, with the same extra columns and summary row.
    cross_results : dict of str to pandas.DataFrame
        Cross-risk tables keyed ``<score>__<cross_var>__<metric_name>``. Empty when ``cross_vars`` is empty.
    pairwise_cross : pandas.DataFrame or None, default None
        Long table with columns ``base_scr_range``, ``eval_metric``, ``score_name``, ``comp_scr_range`` and ``value``
        for the base score crossed with each comparison score. None when ``pairwise_cross_enabled`` is False.
    report_path : str or None, default None
        Path of the Excel report; None when ``write_excel`` is False.
    """

    global_perf: pd.DataFrame
    group_perf: dict[str, pd.DataFrame]
    gains: pd.DataFrame
    cross_results: dict[str, pd.DataFrame]
    pairwise_cross: pd.DataFrame | None = None
    report_path: str | None = None

ScoreComparisonPipeline

Reusable multi-score comparison workflow built on Model_Evaluation_Tool.

Parameters:

Name	Type	Description	Default
config	ScoreComparisonPipelineConfig or None	Pipeline configuration. None (or any falsy value) uses ScoreComparisonPipelineConfig() with its defaults, which still requires base_score and comp_scores or score_cols to be set before run.	None

Attributes:

Name	Type	Description
config	ScoreComparisonPipelineConfig	The configuration in use.

Source code in Modeling_Tool/Pipeline/score_comparison.py

class ScoreComparisonPipeline:
    """Reusable multi-score comparison workflow built on Model_Evaluation_Tool.

    Parameters
    ----------
    config : ScoreComparisonPipelineConfig or None, default None
        Pipeline configuration. ``None`` (or any falsy value) uses ``ScoreComparisonPipelineConfig()`` with its
        defaults, which still requires ``base_score`` and ``comp_scores`` or ``score_cols`` to be set before ``run``.

    Attributes
    ----------
    config : ScoreComparisonPipelineConfig
        The configuration in use.
    """

    def __init__(self, config: ScoreComparisonPipelineConfig | None = None):
        self.config = config or ScoreComparisonPipelineConfig()

    def run(self, data: pd.DataFrame) -> ScoreComparisonPipelineResult:
        """Compare the configured scores globally, per group, by gains and by cross risk.

        Parameters
        ----------
        data : pandas.DataFrame
            Scored sample. It must contain ``target_col``, the score columns and ``weight_col`` / ``split_col`` when
            configured; time, population, ``group_specs`` and ``cross_vars`` columns are optional. A missing
            ``flow_id`` column is created as a running number. The input frame is not modified.

        Returns
        -------
        ScoreComparisonPipelineResult
            Global and group performance, gains, cross-risk tables and the Excel report path.

        Raises
        ------
        ValueError
            If neither ``score_cols`` nor ``base_score`` with ``comp_scores`` is configured, if ``split_col`` has no
            non-empty value, or if a ``cross_metrics`` / ``group_specs`` entry is malformed.
        KeyError
            If a required column (target, scores, ``weight_col``, ``split_col``) or a column referenced by
            ``cross_metrics`` / ``pairwise_cross_agg_dict`` is missing.
        TypeError
            If ``cross_metrics`` or ``pairwise_cross_agg_dict`` has an invalid structure.
        """
        from Modeling_Tool import EvaluationPipeline, Model_Evaluation_Tool, cross_risk

        cfg = self.config
        work = data.copy()
        if "flow_id" not in work.columns:
            work["flow_id"] = range(len(work))

        score_cols = self._resolve_scores(work)
        base_score = cfg.base_score or score_cols[0]
        comp_scores = list(cfg.comp_scores or [s for s in score_cols if s != base_score])
        self._normalize_group_values(work)
        self._validate_input(work, score_cols, base_score, comp_scores)

        report_dir = Path(cfg.output_dir) / "report"
        if cfg.write_outputs or cfg.write_excel:
            make_dirs(cfg.output_dir, Path(cfg.output_dir) / "figs", report_dir)

        cross_agg_dict = cfg.pairwise_cross_agg_dict or self._default_cross_agg_dict(work)
        cross_agg_dict = self._validate_pairwise_cross_agg_dict(work, cross_agg_dict)
        cross_metrics = cfg.cross_metrics or self._default_cross_metrics(work)
        cross_metrics = self._validate_cross_metrics(work, cross_metrics)
        met = Model_Evaluation_Tool(
            data=work,
            dep=cfg.target_col,
            comp_scrlist=comp_scores,
            base_score=base_score,
            nbins=cfg.nbins,
            min_bin_prop=cfg.min_bin_prop,
            equal_freq=cfg.equal_freq,
            min_data_size=cfg.min_data_size,
            precision=cfg.precision,
            include_missing=cfg.include_missing,
            fillna=cfg.fillna,
            weight_col=cfg.weight_col,
            positive_score_only=cfg.positive_score_only,
            cross_agg_dict=cross_agg_dict,
            gains_display_metric_list=cfg.gains_display_metric_list,
        )

        global_perf = self._normalize_global_perf(
            met.model_perf_compare(
                pct_bins=cfg.nbins,
                min_data_size=cfg.min_data_size,
                sample_name="global",
                sync_data_size=cfg.perf_common_rows,
            )
        )
        gains = met.get_gains_summary(
            grp_name=None,
            disp=False,
            withSummary=True,
            add_func=cfg.gains_add_func or self._custom_metrics_func,
        )
        group_perf = self._run_group_perf(met, EvaluationPipeline, work)

        cross_results = {}
        active_cross_vars = []
        for cross_var in cfg.cross_vars:
            if cross_var in work.columns:
                active_cross_vars.append(cross_var)
            else:
                _logger.warning(
                    "ScoreComparisonPipeline: cross_var %r not found in input columns; skipping.",
                    cross_var,
                )
        for score in score_cols:
            for cross_var in active_cross_vars:
                for metric_name, (agg_col, agg_func) in cross_metrics.items():
                    key = f"{score}__{cross_var}__{metric_name}"
                    cross_results[key] = cross_risk(
                        data=work,
                        score_list=[score, cross_var],
                        dep=cfg.target_col,
                        nbins=[cfg.nbins, cfg.nbins],
                        agg_col=agg_col,
                        agg_func=agg_func,
                        equal_freq=cfg.equal_freq,
                        binning_numeric=cfg.cross_binning_numeric,
                        min_bin_prop=cfg.min_bin_prop,
                        include_missing=cfg.include_missing,
                        fillna=cfg.fillna,
                    )

        pairwise_cross = None
        if cfg.pairwise_cross_enabled:
            pairwise_cross = met.get_cross_risk_summary(
                cross_agg_dict=cross_agg_dict,
                nbins=cfg.nbins,
                equal_freq=cfg.equal_freq,
                disp=False,
            )

        if cfg.write_outputs:
            # Group names, scores, cross variables and metric names come from the user: a "/" in one of them would
            # write into a subfolder, so they are made safe (and unique) as file names
            used_names = {"step1_global_perf", "step3_gains_with_metrics", "step4_pairwise"}
            safe_to_csv(global_perf, report_dir / "step1_global_perf.csv", index=False)
            for name, df in group_perf.items():
                stem = self._safe_file_stem(f"step2_by_{name}", used_names)
                safe_to_csv(df, report_dir / f"{stem}.csv", index=False)
            safe_to_csv(gains, report_dir / "step3_gains_with_metrics.csv", index=False)
            for key, df in cross_results.items():
                stem = self._safe_file_stem(f"step4_{key}", used_names)
                safe_to_csv(df, report_dir / f"{stem}.csv", index=True)
            safe_to_csv(pairwise_cross, report_dir / "step4_pairwise.csv", index=False)

        report_path = None
        if cfg.write_excel:
            report_path = str(report_dir / "Score_Comparison_Report.xlsx")
            sheets = {
                "Global_AUC_KS": global_perf,
                "Global_Gains": gains,
                "Cross_Pairwise": pairwise_cross,
            }
            for name, df in group_perf.items():
                sheets[f"Dim_{name}"] = df
            first_cross = next(iter(cross_results.values()), None)
            sheets["Cross_Risk_Sample"] = self._cross_risk_for_excel(first_cross)
            write_basic_excel(report_path, sheets, title="SMF Model Score Comparison Report")

        return ScoreComparisonPipelineResult(
            global_perf=global_perf,
            group_perf=group_perf,
            gains=gains,
            cross_results=cross_results,
            pairwise_cross=pairwise_cross,
            report_path=report_path,
        )

    def _normalize_global_perf(self, global_perf: pd.DataFrame) -> pd.DataFrame:
        if not isinstance(global_perf, pd.DataFrame):
            return global_perf
        result = global_perf.copy()
        # The 'index' label is set at the source via model_perf_compare(sample_name=...),
        # so no oot->global remap is needed here anymore.
        if "sample_scope" not in result.columns:
            result.insert(0, "sample_scope", "global")
        return result

    @staticmethod
    def _unique_labels(labels: list[str]) -> list[str]:
        counts: dict[str, int] = {}
        output: list[str] = []
        for raw_label in labels:
            label = str(raw_label) or "value"
            count = counts.get(label, 0)
            counts[label] = count + 1
            output.append(label if count == 0 else f"{label}_{count + 1}")
        return output

    @classmethod
    def _cross_risk_for_excel(cls, frame: pd.DataFrame | None) -> pd.DataFrame | None:
        """Return an Excel-safe copy without duplicate MultiIndex labels."""
        if frame is None:
            return None
        if not isinstance(frame, pd.DataFrame):
            raise TypeError(f"Cross risk result must be a DataFrame, got {type(frame)!r}")

        index_labels = []
        for level, name in enumerate(frame.index.names):
            suffix = str(name) if name not in (None, "") else "level"
            index_labels.append(f"index_{level}_{suffix}")
        index_labels = cls._unique_labels(index_labels)
        index_frame = pd.DataFrame(
            {
                label: frame.index.get_level_values(level).to_numpy()
                for level, label in enumerate(index_labels)
            }
        )

        values = frame.reset_index(drop=True).copy()
        value_labels = []
        for col in values.columns:
            parts = col if isinstance(col, tuple) else (col,)
            label = "__".join(str(part) for part in parts if part not in (None, ""))
            value_labels.append(label or "value")
        values.columns = cls._unique_labels(value_labels)
        return pd.concat([index_frame.reset_index(drop=True), values], axis=1)

    def _normalize_group_values(self, data: pd.DataFrame) -> None:
        cfg = self.config
        cols: list[str] = []
        cols.extend(str(col) for col in as_list(cfg.time_dims))
        cols.extend(str(col) for col in as_list(cfg.population_dims))
        if cfg.split_col:
            cols.append(cfg.split_col)
        if cfg.group_specs is not None:
            for spec in normalize_group_specs(cfg.group_specs):
                cols.extend(str(col) for col in spec["columns"])
        missing_tokens = {str(x).strip() for x in as_list(cfg.group_missing_values)}
        for col in dict.fromkeys(cols):
            if col not in data.columns:
                continue
            series = data[col]
            if not (pd.api.types.is_object_dtype(series) or pd.api.types.is_string_dtype(series)):
                continue
            stripped = series.astype("string").str.strip()
            missing_mask = stripped.isin(missing_tokens)
            data[col] = stripped
            if missing_mask.any():
                if cfg.drop_missing_group_values:
                    data.loc[missing_mask, col] = pd.NA
                elif cfg.include_missing:
                    data.loc[missing_mask, col] = "[Missing]"
        if cfg.split_col and cfg.split_col in data.columns:
            data[cfg.split_col] = normalize_split_values(data[cfg.split_col])

    def _resolve_scores(self, data: pd.DataFrame) -> list[str]:
        cfg = self.config
        if cfg.score_cols:
            return list(cfg.score_cols)
        if cfg.base_score and cfg.comp_scores:
            return [cfg.base_score] + list(cfg.comp_scores)
        raise ValueError("Provide score_cols or base_score + comp_scores")

    def _validate_input(
        self,
        data: pd.DataFrame,
        score_cols: list[str],
        base_score: str,
        comp_scores: list[str],
    ) -> None:
        cfg = self.config
        # cross_vars are validated softly (warn+skip in run()); do not require them here.
        required = [cfg.target_col, base_score] + comp_scores + score_cols
        if cfg.weight_col:
            required.append(cfg.weight_col)
        if cfg.split_col:
            required.append(cfg.split_col)
        missing = [col for col in dict.fromkeys(required) if col not in data.columns]
        if missing:
            raise KeyError(f"Missing required columns: {missing}")
        if cfg.split_col:
            values = normalize_split_values(data[cfg.split_col]).dropna()
            if values.empty:
                raise ValueError(f"split_col {cfg.split_col!r} must contain at least one non-empty value")

    @staticmethod
    def _safe_file_stem(stem: str, used: set[str]) -> str:
        clean = re.sub(r'[\x00-\x1f/\\:*?"<>|]', "_", str(stem)).rstrip(" .") or "_"
        candidate, i = clean, 2
        while candidate.lower() in used:
            candidate = f"{clean}_{i}"
            i += 1
        used.add(candidate.lower())
        return candidate

    def _custom_metrics_func(self, sub_df: pd.DataFrame) -> pd.Series:
        cfg = self.config
        weights = None
        if cfg.weight_col and cfg.weight_col in sub_df.columns:
            # The weighted gains table: the means are weighted like the rest of the table
            weights = sub_df[cfg.weight_col].astype(float)
        metrics = {}
        for col in cfg.custom_metric_cols:
            if col not in sub_df.columns:
                continue
            if weights is None:
                metrics[f"{col}_mean"] = round(sub_df[col].mean(), 4)
                continue
            values = sub_df[col]
            observed = values.notna().to_numpy()
            total = float(weights[observed].sum())
            metrics[f"{col}_mean"] = (
                round(float(np.sum(values[observed].astype(float) * weights[observed]) / total), 4)
                if total > 0
                else np.nan
            )
        return pd.Series(metrics)

    def _default_cross_metrics(self, data: pd.DataFrame | None = None) -> dict[str, tuple[str, Any]]:
        cfg = self.config
        metrics: dict[str, tuple[str, Any]] = {"bad_rate": (cfg.target_col, "mean")}
        for col in cfg.custom_metric_cols:
            if data is None or col in data.columns:
                metrics[col] = (col, "mean")
        return metrics

    def _validate_cross_metrics(
        self,
        data: pd.DataFrame,
        metrics: dict[str, Any],
    ) -> dict[str, tuple[str, Any]]:
        if not isinstance(metrics, dict):
            raise TypeError("cross_metrics must be a mapping of metric_name to (column, aggregation)")
        normalized: dict[str, tuple[str, Any]] = {}
        for name, spec in metrics.items():
            if not isinstance(spec, (list, tuple)) or len(spec) != 2:
                raise ValueError(
                    f"cross_metrics[{name!r}] must be a two-item (column, aggregation) pair; "
                    f"got {spec!r}"
                )
            agg_col, agg_func = spec
            if agg_col not in data.columns:
                raise KeyError(f"cross_metrics[{name!r}] references missing column {agg_col!r}")
            if not (isinstance(agg_func, str) or callable(agg_func)):
                raise TypeError(
                    f"cross_metrics[{name!r}] aggregation must be a string or callable; "
                    f"got {type(agg_func).__name__}"
                )
            normalized[str(name)] = (str(agg_col), agg_func)
        return normalized

    def _validate_pairwise_cross_agg_dict(
        self,
        data: pd.DataFrame,
        agg_dict: dict[str, Any],
    ) -> dict[str, Any]:
        if not isinstance(agg_dict, dict):
            raise TypeError("pairwise_cross_agg_dict must be a mapping of column to aggregation(s)")
        for col, funcs in agg_dict.items():
            if col not in data.columns:
                raise KeyError(f"pairwise_cross_agg_dict references missing column {col!r}")
            func_list = list(funcs) if isinstance(funcs, (list, tuple)) else [funcs]
            if not func_list or any(not (isinstance(func, str) or callable(func)) for func in func_list):
                raise TypeError(
                    f"pairwise_cross_agg_dict[{col!r}] must be an aggregation or a non-empty "
                    "list of string/callable aggregations"
                )
        return dict(agg_dict)

    def _default_cross_agg_dict(self, data: pd.DataFrame | None = None) -> dict[str, Any]:
        cfg = self.config
        agg: dict[str, Any] = {
            cfg.target_col: ["count", lambda x: round(x.sum() / x.count(), 4)],
        }
        for col in cfg.custom_metric_cols:
            if data is None or col in data.columns:
                agg[col] = ["count", lambda x: round(x.mean(), 4)]
        return agg

    def _run_group_perf(self, met: Any, evaluation_pipeline_cls: Any, data: pd.DataFrame) -> dict[str, pd.DataFrame]:
        cfg = self.config
        results: dict[str, pd.DataFrame] = {}
        groups_with_dropped_scores: list[str] = []

        def perf_compare(**kwargs: Any) -> pd.DataFrame:
            # A score missing from one group, or groups whose scores do not overlap, would repeat the same
            # model_perf_compare warning for every group value: gather them into one warning instead
            with warnings.catch_warnings(record=True) as caught:
                warnings.simplefilter("always")
                output = met.model_perf_compare(**kwargs)
            for record in caught:
                if str(record.message).startswith("model_perf_compare:"):
                    dropped_in.add(current_spec[0])
                else:
                    warnings.warn_explicit(record.message, record.category, record.filename, record.lineno)
            return output

        dropped_in: set[str] = set()
        current_spec = [""]
        for spec in self._resolve_group_specs(data):
            name = str(spec.get("name") or "_".join(spec.get("columns", [])))
            columns = list(spec.get("columns", []))
            min_size = int(spec.get("min_size", cfg.min_data_size))
            if not columns or any(col not in data.columns for col in columns):
                continue
            current_spec[0] = name
            if len(columns) == 1:
                # multi_group_wrapper keeps the groups with MORE than min_subset_size rows, while min_size is the
                # minimum a group needs (as in the crossed groups below)
                results[name] = met.multi_group_wrapper(
                    group_name=columns[0],
                    group_var_name=columns[0],
                    group_eval_func=perf_compare,
                    min_subset_size=min_size - 1,
                    pct_bins=cfg.nbins,
                    sample_name="global",
                    sync_data_size=cfg.perf_common_rows,
                )
            else:
                pipeline = evaluation_pipeline_cls(met)
                for col in columns:
                    pipeline = pipeline.group_by(col, min_size=min_size, group_var_name=col)
                output = pipeline.apply(
                    perf_compare,
                    pct_bins=cfg.nbins,
                    sample_name="global",
                    sync_data_size=cfg.perf_common_rows,
                )
                if isinstance(output, pd.DataFrame):
                    results[name] = output
            if name in dropped_in:
                groups_with_dropped_scores.append(name)
        if groups_with_dropped_scores:
            warnings.warn(
                f"ScoreComparisonPipeline: in some values of {groups_with_dropped_scores} a score has no valid row, or "
                f"the scores have no row in common; that score (or that group value) is missing from those group_perf "
                f"tables.",
                UserWarning,
                stacklevel=3,
            )
        return results

    def _resolve_group_specs(self, data: pd.DataFrame) -> list[dict[str, Any]]:
        cfg = self.config
        if cfg.group_specs is not None:
            min_size = cfg.group_min_size if cfg.group_min_size is not None else cfg.min_data_size
            return normalize_group_specs(cfg.group_specs, default_min_size=min_size)

        min_size = cfg.group_min_size if cfg.group_min_size is not None else cfg.min_data_size
        specs: list[dict[str, Any]] = []
        seen: set[tuple[str, ...]] = set()

        def add(columns: list[str], name: str | None = None) -> None:
            existing = [col for col in columns if col in data.columns]
            if len(existing) != len(columns):
                return
            key = tuple(existing)
            if key in seen:
                return
            seen.add(key)
            specs.append({"name": name or "_x_".join(existing), "columns": existing, "min_size": min_size})

        time_dims = [str(col) for col in as_list(cfg.time_dims)]
        population_dims = [str(col) for col in as_list(cfg.population_dims)]

        if cfg.split_col:
            add([cfg.split_col], name=cfg.split_col)
        for time_col in time_dims:
            add([time_col], name=time_col)
        for pop_col in population_dims:
            add([pop_col], name=pop_col)
        if cfg.include_time_population_cross:
            for pop_col in population_dims:
                for time_col in time_dims:
                    add([pop_col, time_col], name=f"{pop_col}_x_{time_col}")

        return specs

run

run(data: DataFrame) -> ScoreComparisonPipelineResult

Compare the configured scores globally, per group, by gains and by cross risk.

Parameters:

Name	Type	Description	Default
data	DataFrame	Scored sample. It must contain target_col, the score columns and weight_col / split_col when configured; time, population, group_specs and cross_vars columns are optional. A missing flow_id column is created as a running number. The input frame is not modified.	required

Returns:

Type	Description
ScoreComparisonPipelineResult	Global and group performance, gains, cross-risk tables and the Excel report path.

Raises:

Type	Description
ValueError	If neither score_cols nor base_score with comp_scores is configured, if split_col has no non-empty value, or if a cross_metrics / group_specs entry is malformed.
KeyError	If a required column (target, scores, weight_col, split_col) or a column referenced by cross_metrics / pairwise_cross_agg_dict is missing.
TypeError	If cross_metrics or pairwise_cross_agg_dict has an invalid structure.

Source code in Modeling_Tool/Pipeline/score_comparison.py

def run(self, data: pd.DataFrame) -> ScoreComparisonPipelineResult:
    """Compare the configured scores globally, per group, by gains and by cross risk.

    Parameters
    ----------
    data : pandas.DataFrame
        Scored sample. It must contain ``target_col``, the score columns and ``weight_col`` / ``split_col`` when
        configured; time, population, ``group_specs`` and ``cross_vars`` columns are optional. A missing
        ``flow_id`` column is created as a running number. The input frame is not modified.

    Returns
    -------
    ScoreComparisonPipelineResult
        Global and group performance, gains, cross-risk tables and the Excel report path.

    Raises
    ------
    ValueError
        If neither ``score_cols`` nor ``base_score`` with ``comp_scores`` is configured, if ``split_col`` has no
        non-empty value, or if a ``cross_metrics`` / ``group_specs`` entry is malformed.
    KeyError
        If a required column (target, scores, ``weight_col``, ``split_col``) or a column referenced by
        ``cross_metrics`` / ``pairwise_cross_agg_dict`` is missing.
    TypeError
        If ``cross_metrics`` or ``pairwise_cross_agg_dict`` has an invalid structure.
    """
    from Modeling_Tool import EvaluationPipeline, Model_Evaluation_Tool, cross_risk

    cfg = self.config
    work = data.copy()
    if "flow_id" not in work.columns:
        work["flow_id"] = range(len(work))

    score_cols = self._resolve_scores(work)
    base_score = cfg.base_score or score_cols[0]
    comp_scores = list(cfg.comp_scores or [s for s in score_cols if s != base_score])
    self._normalize_group_values(work)
    self._validate_input(work, score_cols, base_score, comp_scores)

    report_dir = Path(cfg.output_dir) / "report"
    if cfg.write_outputs or cfg.write_excel:
        make_dirs(cfg.output_dir, Path(cfg.output_dir) / "figs", report_dir)

    cross_agg_dict = cfg.pairwise_cross_agg_dict or self._default_cross_agg_dict(work)
    cross_agg_dict = self._validate_pairwise_cross_agg_dict(work, cross_agg_dict)
    cross_metrics = cfg.cross_metrics or self._default_cross_metrics(work)
    cross_metrics = self._validate_cross_metrics(work, cross_metrics)
    met = Model_Evaluation_Tool(
        data=work,
        dep=cfg.target_col,
        comp_scrlist=comp_scores,
        base_score=base_score,
        nbins=cfg.nbins,
        min_bin_prop=cfg.min_bin_prop,
        equal_freq=cfg.equal_freq,
        min_data_size=cfg.min_data_size,
        precision=cfg.precision,
        include_missing=cfg.include_missing,
        fillna=cfg.fillna,
        weight_col=cfg.weight_col,
        positive_score_only=cfg.positive_score_only,
        cross_agg_dict=cross_agg_dict,
        gains_display_metric_list=cfg.gains_display_metric_list,
    )

    global_perf = self._normalize_global_perf(
        met.model_perf_compare(
            pct_bins=cfg.nbins,
            min_data_size=cfg.min_data_size,
            sample_name="global",
            sync_data_size=cfg.perf_common_rows,
        )
    )
    gains = met.get_gains_summary(
        grp_name=None,
        disp=False,
        withSummary=True,
        add_func=cfg.gains_add_func or self._custom_metrics_func,
    )
    group_perf = self._run_group_perf(met, EvaluationPipeline, work)

    cross_results = {}
    active_cross_vars = []
    for cross_var in cfg.cross_vars:
        if cross_var in work.columns:
            active_cross_vars.append(cross_var)
        else:
            _logger.warning(
                "ScoreComparisonPipeline: cross_var %r not found in input columns; skipping.",
                cross_var,
            )
    for score in score_cols:
        for cross_var in active_cross_vars:
            for metric_name, (agg_col, agg_func) in cross_metrics.items():
                key = f"{score}__{cross_var}__{metric_name}"
                cross_results[key] = cross_risk(
                    data=work,
                    score_list=[score, cross_var],
                    dep=cfg.target_col,
                    nbins=[cfg.nbins, cfg.nbins],
                    agg_col=agg_col,
                    agg_func=agg_func,
                    equal_freq=cfg.equal_freq,
                    binning_numeric=cfg.cross_binning_numeric,
                    min_bin_prop=cfg.min_bin_prop,
                    include_missing=cfg.include_missing,
                    fillna=cfg.fillna,
                )

    pairwise_cross = None
    if cfg.pairwise_cross_enabled:
        pairwise_cross = met.get_cross_risk_summary(
            cross_agg_dict=cross_agg_dict,
            nbins=cfg.nbins,
            equal_freq=cfg.equal_freq,
            disp=False,
        )

    if cfg.write_outputs:
        # Group names, scores, cross variables and metric names come from the user: a "/" in one of them would
        # write into a subfolder, so they are made safe (and unique) as file names
        used_names = {"step1_global_perf", "step3_gains_with_metrics", "step4_pairwise"}
        safe_to_csv(global_perf, report_dir / "step1_global_perf.csv", index=False)
        for name, df in group_perf.items():
            stem = self._safe_file_stem(f"step2_by_{name}", used_names)
            safe_to_csv(df, report_dir / f"{stem}.csv", index=False)
        safe_to_csv(gains, report_dir / "step3_gains_with_metrics.csv", index=False)
        for key, df in cross_results.items():
            stem = self._safe_file_stem(f"step4_{key}", used_names)
            safe_to_csv(df, report_dir / f"{stem}.csv", index=True)
        safe_to_csv(pairwise_cross, report_dir / "step4_pairwise.csv", index=False)

    report_path = None
    if cfg.write_excel:
        report_path = str(report_dir / "Score_Comparison_Report.xlsx")
        sheets = {
            "Global_AUC_KS": global_perf,
            "Global_Gains": gains,
            "Cross_Pairwise": pairwise_cross,
        }
        for name, df in group_perf.items():
            sheets[f"Dim_{name}"] = df
        first_cross = next(iter(cross_results.values()), None)
        sheets["Cross_Risk_Sample"] = self._cross_risk_for_excel(first_cross)
        write_basic_excel(report_path, sheets, title="SMF Model Score Comparison Report")

    return ScoreComparisonPipelineResult(
        global_perf=global_perf,
        group_perf=group_perf,
        gains=gains,
        cross_results=cross_results,
        pairwise_cross=pairwise_cross,
        report_path=report_path,
    )

Score Consistency UAT Pipeline: score_consistency_uat

score_consistency_uat

ScoreConsistencyUATPipelineConfig dataclass

Configuration of :class:ScoreConsistencyUATPipeline.

Two data modes exist. In DataFrame mode (offline_data / online_data given here or to run) the frames are used directly. In SQL mode the two SQL files are executed with sqlrunner.

Parameters:

Name	Type	Description	Default
output_dir	str	Root output directory. CSV files and the default Excel report go to <output_dir>/report.	"output/score_consistency_uat"
random_state	int	Not used by this pipeline; kept so that all high-level Pipeline configs share the same interface.	42
write_outputs	bool	Whether to write the CSV tables coverage_summary, main_score_summary, submodel_summary, feature_diff_summary, time_summary, per_flow_report and summary to <output_dir>/report.	True
write_excel	bool	Whether to export the Excel report with UATConsistencyChecker.export_excel (path in report_path).	True
sql_dir	str	Directory that contains the SQL files (SQL mode). It is resolved to an absolute path.	"sql"
offline_sql	str	File name, inside sql_dir, of the SQL that pulls the offline (backtest) data.	"pull_offline.sql"
online_sql	str	File name, inside sql_dir, of the SQL that pulls the online data.	"pull_online.sql"
sqlrunner	object or None	SQL runner whose run_sql(sql, n_process=...) returns a DataFrame (SQL mode). None creates Modeling_Tool.Core.ODPSRunner(), which reads its credentials from environment variables (ALIBABA_CLOUD_ACCESS_KEY_ID, ALIBABA_CLOUD_ACCESS_KEY_SECRET, ODPS_PROJECT and ODPS_ENDPOINT, all required). In DataFrame mode the runner is never called.	None
env_path	str or None	Path of a .env file loaded with python-dotenv at the start of run without overriding variables that are already set (~ and environment variables in the path are expanded). ImportError if python-dotenv is not installed.	None
n_process	int or str or None	Number of processes passed to sqlrunner.run_sql when the SQL files are pulled. None or "auto" means max(1, cpu_count - 1); any other value must be an integer >= 1, otherwise ValueError (checked in both data modes).	"auto"
offline_data	DataFrame or None	Offline data for DataFrame mode; the offline_data argument of run takes precedence. It must contain a flow_id column.	None
online_data	DataFrame or None	Online data for DataFrame mode; the online_data argument of run takes precedence. It must contain a flow_id column.	None
main_model_score_col	str	Name of the main model score column, identical offline and online. After the outer merge on flow_id the online column is <name>_online. If either column is missing, the main score check only reports the two column names (the missing one as None).	"credit_risk_v31_cdc_submodel_score"
tol_score	float	Absolute tolerance for the main score and the dedicated submodel check. A row mismatches when |online - offline| > tol_score or when exactly one side is null.	1e-06
tol_feat	float	Absolute tolerance for numeric feature pairs (col offline versus col_online). The same one-side-null rule applies.	0.01
time_featlist	list of str	Time fields (same name offline and online) compared as datetimes with tol_time_seconds. They are excluded from the numeric feature comparison. Empty means no time check.	[]
tol_time_seconds	float	Tolerance in seconds on |online - offline| for the time_featlist fields. A value that cannot be parsed on one side counts as a mismatch.	60.0
comparison_block_size	int	Number of feature columns compared per block when per_flow_report is built; a smaller value lowers peak memory for wide tables. Must be positive, otherwise ValueError.	128
excel_output_path	str or None	Excel report path (~ and environment variables are expanded, relative paths become absolute). None uses <output_dir>/report/Score_Consistency_UAT_Report.xlsx. Parent directories are created when write_excel is True.	None
excel_font	str	Font name applied to the whole Excel report.	"Arial"
info_list	list of str	Identifier columns (for example user_id or launch_time) written after flow_id in the detail tables and per_flow_report. They are excluded from the numeric feature comparison. Columns that do not exist in the data are ignored (with a warning in SQL mode).	[]
include_submodel_scores	bool	True: submodel scores are compared like any other feature and the dedicated submodel check is skipped. False: run the dedicated check for submodel_pairs and fill submodel_summary.	True
submodel_pairs	dict of str to str	{offline_col: online_col} submodel score column pairs. The dedicated check uses them only when include_submodel_scores is False, but per_flow_report gets an <offline_col>_diff column for every pair whose two columns exist, whatever the flag.	{}
numeric_coercion_mode	str	DataFrame mode only: how object-dtype columns of the merged frame become numeric. "safe" converts a column only if at least numeric_coercion_min_ratio of its non-null values parse as numbers (a warning is logged when some, but not enough, values parse); "aggressive" always converts and turns unparsable values into NaN (warning); "off" converts nothing. Any other value raises ValueError. In SQL mode the checker converts every object column that has at least one numeric value, and this field is ignored.	"safe"
numeric_coercion_min_ratio	float	DataFrame mode with numeric_coercion_mode="safe" only: minimum share of parsable non-null values required to convert an object column.	0.99

Source code in Modeling_Tool/Pipeline/score_consistency_uat.py

@dataclass
class ScoreConsistencyUATPipelineConfig:
    """Configuration of :class:`ScoreConsistencyUATPipeline`.

    Two data modes exist. In DataFrame mode (``offline_data`` / ``online_data`` given here or to ``run``) the frames are
    used directly. In SQL mode the two SQL files are executed with ``sqlrunner``.

    Parameters
    ----------
    output_dir : str, default "output/score_consistency_uat"
        Root output directory. CSV files and the default Excel report go to ``<output_dir>/report``.
    random_state : int, default 42
        Not used by this pipeline; kept so that all high-level Pipeline configs share the same interface.
    write_outputs : bool, default True
        Whether to write the CSV tables ``coverage_summary``, ``main_score_summary``, ``submodel_summary``,
        ``feature_diff_summary``, ``time_summary``, ``per_flow_report`` and ``summary`` to ``<output_dir>/report``.
    write_excel : bool, default True
        Whether to export the Excel report with ``UATConsistencyChecker.export_excel`` (path in ``report_path``).
    sql_dir : str, default "sql"
        Directory that contains the SQL files (SQL mode). It is resolved to an absolute path.
    offline_sql : str, default "pull_offline.sql"
        File name, inside ``sql_dir``, of the SQL that pulls the offline (backtest) data.
    online_sql : str, default "pull_online.sql"
        File name, inside ``sql_dir``, of the SQL that pulls the online data.
    sqlrunner : object or None, default None
        SQL runner whose ``run_sql(sql, n_process=...)`` returns a DataFrame (SQL mode). ``None`` creates
        ``Modeling_Tool.Core.ODPSRunner()``, which reads its credentials from environment variables
        (``ALIBABA_CLOUD_ACCESS_KEY_ID``, ``ALIBABA_CLOUD_ACCESS_KEY_SECRET``, ``ODPS_PROJECT`` and ``ODPS_ENDPOINT``, all
        required). In DataFrame mode the runner is never called.
    env_path : str or None, default None
        Path of a ``.env`` file loaded with python-dotenv at the start of ``run`` without overriding variables that are
        already set (``~`` and environment variables in the path are expanded). ``ImportError`` if python-dotenv is not
        installed.
    n_process : int or str or None, default "auto"
        Number of processes passed to ``sqlrunner.run_sql`` when the SQL files are pulled. ``None`` or ``"auto"`` means
        ``max(1, cpu_count - 1)``; any other value must be an integer >= 1, otherwise ``ValueError`` (checked in both
        data modes).
    offline_data : pandas.DataFrame or None, default None
        Offline data for DataFrame mode; the ``offline_data`` argument of ``run`` takes precedence. It must contain a
        ``flow_id`` column.
    online_data : pandas.DataFrame or None, default None
        Online data for DataFrame mode; the ``online_data`` argument of ``run`` takes precedence. It must contain a
        ``flow_id`` column.
    main_model_score_col : str, default "credit_risk_v31_cdc_submodel_score"
        Name of the main model score column, identical offline and online. After the outer merge on ``flow_id`` the
        online column is ``<name>_online``. If either column is missing, the main score check only reports the two
        column names (the missing one as None).
    tol_score : float, default 1e-06
        Absolute tolerance for the main score and the dedicated submodel check. A row mismatches when
        ``|online - offline| > tol_score`` or when exactly one side is null.
    tol_feat : float, default 0.01
        Absolute tolerance for numeric feature pairs (``col`` offline versus ``col_online``). The same one-side-null
        rule applies.
    time_featlist : list of str, default []
        Time fields (same name offline and online) compared as datetimes with ``tol_time_seconds``. They are excluded
        from the numeric feature comparison. Empty means no time check.
    tol_time_seconds : float, default 60.0
        Tolerance in seconds on ``|online - offline|`` for the ``time_featlist`` fields. A value that cannot be parsed
        on one side counts as a mismatch.
    comparison_block_size : int, default 128
        Number of feature columns compared per block when ``per_flow_report`` is built; a smaller value lowers peak
        memory for wide tables. Must be positive, otherwise ``ValueError``.
    excel_output_path : str or None, default None
        Excel report path (``~`` and environment variables are expanded, relative paths become absolute). ``None`` uses
        ``<output_dir>/report/Score_Consistency_UAT_Report.xlsx``. Parent directories are created when ``write_excel``
        is True.
    excel_font : str, default "Arial"
        Font name applied to the whole Excel report.
    info_list : list of str, default []
        Identifier columns (for example user_id or launch_time) written after ``flow_id`` in the detail tables and
        ``per_flow_report``. They are excluded from the numeric feature comparison. Columns that do not exist in the
        data are ignored (with a warning in SQL mode).
    include_submodel_scores : bool, default True
        True: submodel scores are compared like any other feature and the dedicated submodel check is skipped.
        False: run the dedicated check for ``submodel_pairs`` and fill ``submodel_summary``.
    submodel_pairs : dict of str to str, default {}
        ``{offline_col: online_col}`` submodel score column pairs. The dedicated check uses them only when
        ``include_submodel_scores`` is False, but ``per_flow_report`` gets an ``<offline_col>_diff`` column for every
        pair whose two columns exist, whatever the flag.
    numeric_coercion_mode : str, default "safe"
        DataFrame mode only: how object-dtype columns of the merged frame become numeric. ``"safe"`` converts a column
        only if at least ``numeric_coercion_min_ratio`` of its non-null values parse as numbers (a warning is logged when
        some, but not enough, values parse); ``"aggressive"`` always converts and turns unparsable values into NaN
        (warning); ``"off"`` converts nothing. Any other value raises ``ValueError``. In SQL mode the checker converts
        every object column that has at least one numeric value, and this field is ignored.
    numeric_coercion_min_ratio : float, default 0.99
        DataFrame mode with ``numeric_coercion_mode="safe"`` only: minimum share of parsable non-null values required
        to convert an object column.
    """

    output_dir: str = "output/score_consistency_uat"
    random_state: int = 42
    write_outputs: bool = True
    write_excel: bool = True

    sql_dir: str = "sql"
    offline_sql: str = "pull_offline.sql"
    online_sql: str = "pull_online.sql"
    sqlrunner: Any | None = None
    env_path: str | None = None
    n_process: int | str | None = "auto"
    offline_data: pd.DataFrame | None = None
    online_data: pd.DataFrame | None = None

    main_model_score_col: str = "credit_risk_v31_cdc_submodel_score"
    tol_score: float = 1e-6
    tol_feat: float = 1e-2
    time_featlist: list[str] = field(default_factory=list)
    tol_time_seconds: float = 60.0
    comparison_block_size: int = 128

    excel_output_path: str | None = None
    excel_font: str = "Arial"
    info_list: list[str] = field(default_factory=list)

    include_submodel_scores: bool = True
    submodel_pairs: dict[str, str] = field(default_factory=dict)

    # Object-column numeric coercion controls (v0.4.0)
    # In v0.3.x, object columns were silently coerced to numeric whenever ANY value
    # could be parsed, discarding legitimate string labels (e.g. "A1", "NA") without
    # warning. From 0.4.0, coercion is opt-in via safety modes below.
    numeric_coercion_mode: str = "safe"  # "safe" | "aggressive" | "off"
    numeric_coercion_min_ratio: float = 0.99  # only used in "safe" mode

ScoreConsistencyUATPipelineResult dataclass

Result returned by :meth:ScoreConsistencyUATPipeline.run.

Parameters:

Name	Type	Description	Default
offline_data	DataFrame or None	Offline data held by the checker (a copy of the input in DataFrame mode).	required
online_data	DataFrame or None	Online data held by the checker (a copy of the input in DataFrame mode).	required
compare_data	DataFrame or None	Outer merge of the two frames on flow_id. Columns present on both sides get the suffix _online on the online side, and the _merge column marks left_only / right_only / both.	required
both_data	DataFrame or None	Rows of compare_data whose flow_id exists both offline and online; all consistency checks use it.	required
coverage_summary	dict	flow_id coverage counts: n_offline, n_online, n_common, n_only_offline, n_only_online, dup_offline and dup_online (duplicated flow_id rows).	required
main_score_summary	dict	Main score comparison: offline_score_col, online_score_col, n_compared, n_null, n_one_side_null, mean_diff, max_abs_diff, n_mismatch and consistent. Only the two column names are present when one of the score columns is missing.	required
submodel_summary	list of dict	One dict per submodel_pairs entry with submodel, n_compared, n_one_side_null, n_mismatch, n_mismatch_gt_1e6 and max_abs_diff. Empty when include_submodel_scores is True.	required
feature_diff_summary	DataFrame	One row per col / col_online feature pair, sorted by n_mismatch descending: feature, n_compared, n_one_side_null, n_mismatch, pct_mismatch, mean_diff, max_abs_diff.	required
time_summary	DataFrame	One row per time_featlist field (time_field, offline_col, online_col, n_compared, n_one_side_null, n_mismatch, pct_mismatch, mean_diff_sec, max_abs_diff_sec); empty when time_featlist is empty.	required
per_flow_report	DataFrame	One row per common flow_id: flow_id, the info_list columns, main_score_diff, main_score_ok, <submodel>_diff columns, n_feature_mismatch and mismatch_features (comma-separated feature names).	required
summary	DataFrame	Overall conclusion with columns Check Item, Detail and Status; the last row is OVERALL.	required
report_path	str or None	Path of the Excel report; None when write_excel is False.	None
checker	UATConsistencyChecker or None	The underlying checker, for access to intermediate attributes such as main_score_mismatch_df.	None

Source code in Modeling_Tool/Pipeline/score_consistency_uat.py

@dataclass
class ScoreConsistencyUATPipelineResult:
    """Result returned by :meth:`ScoreConsistencyUATPipeline.run`.

    Parameters
    ----------
    offline_data : pandas.DataFrame or None
        Offline data held by the checker (a copy of the input in DataFrame mode).
    online_data : pandas.DataFrame or None
        Online data held by the checker (a copy of the input in DataFrame mode).
    compare_data : pandas.DataFrame or None
        Outer merge of the two frames on ``flow_id``. Columns present on both sides get the suffix ``_online`` on the
        online side, and the ``_merge`` column marks ``left_only`` / ``right_only`` / ``both``.
    both_data : pandas.DataFrame or None
        Rows of ``compare_data`` whose ``flow_id`` exists both offline and online; all consistency checks use it.
    coverage_summary : dict
        ``flow_id`` coverage counts: ``n_offline``, ``n_online``, ``n_common``, ``n_only_offline``, ``n_only_online``,
        ``dup_offline`` and ``dup_online`` (duplicated ``flow_id`` rows).
    main_score_summary : dict
        Main score comparison: ``offline_score_col``, ``online_score_col``, ``n_compared``, ``n_null``,
        ``n_one_side_null``, ``mean_diff``, ``max_abs_diff``, ``n_mismatch`` and ``consistent``. Only the two column
        names are present when one of the score columns is missing.
    submodel_summary : list of dict
        One dict per ``submodel_pairs`` entry with ``submodel``, ``n_compared``, ``n_one_side_null``, ``n_mismatch``,
        ``n_mismatch_gt_1e6`` and ``max_abs_diff``. Empty when ``include_submodel_scores`` is True.
    feature_diff_summary : pandas.DataFrame
        One row per ``col`` / ``col_online`` feature pair, sorted by ``n_mismatch`` descending: ``feature``,
        ``n_compared``, ``n_one_side_null``, ``n_mismatch``, ``pct_mismatch``, ``mean_diff``, ``max_abs_diff``.
    time_summary : pandas.DataFrame
        One row per ``time_featlist`` field (``time_field``, ``offline_col``, ``online_col``, ``n_compared``,
        ``n_one_side_null``, ``n_mismatch``, ``pct_mismatch``, ``mean_diff_sec``, ``max_abs_diff_sec``); empty when
        ``time_featlist`` is empty.
    per_flow_report : pandas.DataFrame
        One row per common ``flow_id``: ``flow_id``, the ``info_list`` columns, ``main_score_diff``, ``main_score_ok``,
        ``<submodel>_diff`` columns, ``n_feature_mismatch`` and ``mismatch_features`` (comma-separated feature names).
    summary : pandas.DataFrame
        Overall conclusion with columns ``Check Item``, ``Detail`` and ``Status``; the last row is ``OVERALL``.
    report_path : str or None, default None
        Path of the Excel report; None when ``write_excel`` is False.
    checker : UATConsistencyChecker or None, default None
        The underlying checker, for access to intermediate attributes such as ``main_score_mismatch_df``.
    """

    offline_data: pd.DataFrame | None
    online_data: pd.DataFrame | None
    compare_data: pd.DataFrame | None
    both_data: pd.DataFrame | None
    coverage_summary: dict[str, Any]
    main_score_summary: dict[str, Any]
    submodel_summary: list[dict[str, Any]]
    feature_diff_summary: pd.DataFrame
    time_summary: pd.DataFrame
    per_flow_report: pd.DataFrame
    summary: pd.DataFrame
    report_path: str | None = None
    checker: Any | None = None

ScoreConsistencyUATPipeline

Reusable online/offline score consistency UAT workflow.

Wraps Modeling_Tool.UAT.UATConsistencyChecker: loads the offline and online data (SQL files or DataFrames), merges them on flow_id and checks coverage, main score, submodel scores, all features, time fields and per-flow differences.

Parameters:

Name	Type	Description	Default
config	ScoreConsistencyUATPipelineConfig or None	Pipeline configuration. None (or any falsy value) uses ScoreConsistencyUATPipelineConfig() with its defaults.	None

Attributes:

Name	Type	Description
config	ScoreConsistencyUATPipelineConfig	The configuration in use.

Source code in Modeling_Tool/Pipeline/score_consistency_uat.py

class ScoreConsistencyUATPipeline:
    """Reusable online/offline score consistency UAT workflow.

    Wraps ``Modeling_Tool.UAT.UATConsistencyChecker``: loads the offline and online data (SQL files or DataFrames),
    merges them on ``flow_id`` and checks coverage, main score, submodel scores, all features, time fields and
    per-flow differences.

    Parameters
    ----------
    config : ScoreConsistencyUATPipelineConfig or None, default None
        Pipeline configuration. ``None`` (or any falsy value) uses ``ScoreConsistencyUATPipelineConfig()`` with its
        defaults.

    Attributes
    ----------
    config : ScoreConsistencyUATPipelineConfig
        The configuration in use.
    """

    def __init__(self, config: ScoreConsistencyUATPipelineConfig | None = None):
        self.config = config or ScoreConsistencyUATPipelineConfig()

    def run(
        self,
        offline_data: pd.DataFrame | None = None,
        online_data: pd.DataFrame | None = None,
    ) -> ScoreConsistencyUATPipelineResult:
        """Run the consistency checks and write the configured reports.

        Parameters
        ----------
        offline_data : pandas.DataFrame or None, default None
            Offline data; overrides ``config.offline_data``. It must contain a ``flow_id`` column.
        online_data : pandas.DataFrame or None, default None
            Online data; overrides ``config.online_data``. It must contain a ``flow_id`` column.

        Returns
        -------
        ScoreConsistencyUATPipelineResult
            The checker tables, the merged data, the Excel report path and the checker itself.

        Raises
        ------
        ValueError
            If only one of the two DataFrames is available, or if ``n_process``, ``numeric_coercion_mode`` or
            ``comparison_block_size`` is invalid.
        KeyError
            If a DataFrame has no ``flow_id`` column.
        ImportError
            If ``config.env_path`` is set and python-dotenv is not installed.

        Notes
        -----
        DataFrame mode is used as soon as one of the two frames is given (as argument or in the config); then both are
        required. Otherwise the SQL files in ``sql_dir`` are run with ``config.sqlrunner`` (``ODPSRunner()`` when None).
        """
        from Modeling_Tool.UAT import UATConsistencyChecker

        cfg = self.config
        self._load_env(cfg.env_path)
        report_dir = Path(cfg.output_dir) / "report"
        if cfg.write_outputs or cfg.write_excel:
            make_dirs(cfg.output_dir, report_dir)

        offline = offline_data if offline_data is not None else cfg.offline_data
        online = online_data if online_data is not None else cfg.online_data
        use_dataframes = offline is not None or online is not None
        checker = UATConsistencyChecker(self._build_uat_config(), self._resolve_sqlrunner(use_dataframes))

        if use_dataframes:
            if offline is None or online is None:
                raise ValueError("Provide both offline_data and online_data for DataFrame mode.")
            self._load_dataframes(checker, offline, online)
        else:
            checker.load_data()

        coverage_summary = checker.check_coverage()
        main_score_summary = checker.check_main_score()
        submodel_summary = checker.check_submodel_features()
        feature_diff_summary = checker.check_all_features()
        time_summary = checker.check_time_fields()
        per_flow_report = checker.build_per_flow_report()
        summary = checker.build_summary()

        report_path = None
        if cfg.write_outputs:
            safe_to_csv(pd.DataFrame([coverage_summary]), report_dir / "coverage_summary.csv", index=False)
            safe_to_csv(pd.DataFrame([main_score_summary]), report_dir / "main_score_summary.csv", index=False)
            safe_to_csv(pd.DataFrame(submodel_summary), report_dir / "submodel_summary.csv", index=False)
            safe_to_csv(feature_diff_summary, report_dir / "feature_diff_summary.csv", index=False)
            safe_to_csv(time_summary, report_dir / "time_summary.csv", index=False)
            safe_to_csv(per_flow_report, report_dir / "per_flow_report.csv", index=False)
            safe_to_csv(summary, report_dir / "summary.csv", index=False)

        if cfg.write_excel:
            report_path = checker.export_excel()

        return ScoreConsistencyUATPipelineResult(
            offline_data=checker.df_offline,
            online_data=checker.df_online,
            compare_data=checker.df_compare,
            both_data=checker.df_both,
            coverage_summary=coverage_summary,
            main_score_summary=main_score_summary,
            submodel_summary=submodel_summary,
            feature_diff_summary=feature_diff_summary,
            time_summary=time_summary,
            per_flow_report=per_flow_report,
            summary=summary,
            report_path=report_path,
            checker=checker,
        )

    def _build_uat_config(self):
        from Modeling_Tool.UAT import UATConfig

        cfg = self.config
        return UATConfig(
            main_model_score_col=cfg.main_model_score_col,
            include_submodel_scores=cfg.include_submodel_scores,
            excel_output_path=self._resolve_excel_output_path(),
            sql_dir=str(Path(cfg.sql_dir).expanduser().resolve()),
            offline_sql=cfg.offline_sql,
            online_sql=cfg.online_sql,
            tol_score=float(cfg.tol_score),
            tol_feat=float(cfg.tol_feat),
            n_process=self._resolve_n_process(cfg.n_process),
            submodel_pairs=dict(cfg.submodel_pairs or {}),
            excel_font=cfg.excel_font,
            info_list=list(cfg.info_list or []),
            time_featlist=list(cfg.time_featlist or []),
            tol_time_seconds=float(cfg.tol_time_seconds),
            comparison_block_size=int(cfg.comparison_block_size),
        )

    def _resolve_excel_output_path(self) -> str:
        cfg = self.config
        if cfg.excel_output_path:
            path = Path(os.path.expandvars(cfg.excel_output_path)).expanduser()
        else:
            path = Path(cfg.output_dir) / "report" / "Score_Consistency_UAT_Report.xlsx"
        if not path.is_absolute():
            path = path.resolve()
        if cfg.write_excel:
            path.parent.mkdir(parents=True, exist_ok=True)
        return str(path)

    def _resolve_n_process(self, value: int | str | None) -> int:
        if value in (None, "auto"):
            try:
                import multiprocessing

                return max(1, multiprocessing.cpu_count() - 1)
            except Exception:
                return 1
        if isinstance(value, int) and value >= 1:
            return value
        raise ValueError("n_process must be a positive integer, null, or 'auto'.")

    def _resolve_sqlrunner(self, use_dataframes: bool = False) -> Any:
        cfg = self.config
        if cfg.sqlrunner is not None:
            return cfg.sqlrunner
        if use_dataframes:
            return _NoOpSQLRunner()
        from Modeling_Tool.Core import ODPSRunner

        return ODPSRunner()

    def _load_env(self, env_path: str | None) -> None:
        if not env_path:
            return
        try:
            from dotenv import load_dotenv
        except ImportError as exc:
            raise ImportError("python-dotenv is required when env_path is provided.") from exc
        path = Path(os.path.expandvars(env_path)).expanduser()
        load_dotenv(path, override=False)

    def _load_dataframes(
        self,
        checker: Any,
        offline_data: pd.DataFrame,
        online_data: pd.DataFrame,
    ) -> None:
        for name, frame in {"offline_data": offline_data, "online_data": online_data}.items():
            if "flow_id" not in frame.columns:
                raise KeyError(f"{name} must contain a flow_id column.")

        checker.df_offline = offline_data.copy()
        checker.df_online = online_data.copy()
        online_extra = [col for col in checker.df_online.columns if col != "flow_id"]
        checker.df_compare = checker.df_offline.merge(
            checker.df_online[["flow_id"] + online_extra],
            on="flow_id",
            how="outer",
            suffixes=("", "_online"),
            indicator=True,
        )

        mode = getattr(self.config, "numeric_coercion_mode", "safe")
        min_ratio = float(getattr(self.config, "numeric_coercion_min_ratio", 0.99))
        if mode not in {"safe", "aggressive", "off"}:
            raise ValueError(
                f"numeric_coercion_mode must be one of 'safe','aggressive','off'; got {mode!r}"
            )
        if mode != "off":
            for col in checker.df_compare.columns:
                if col in ("flow_id", "_merge"):
                    continue
                if checker.df_compare[col].dtype != object:
                    continue
                original = checker.df_compare[col]
                as_num = pd.to_numeric(original, errors="coerce")
                original_notna = original.notna()
                original_count = int(original_notna.sum())
                if original_count == 0:
                    continue
                coerced_count = int((as_num.notna() & original_notna).sum())
                parse_ratio = coerced_count / original_count
                if mode == "aggressive":
                    lost = original_count - coerced_count
                    if lost > 0:
                        _logger.warning(
                            "Column %r: aggressive coercion turned %d/%d non-numeric values into NaN.",
                            col,
                            lost,
                            original_count,
                        )
                    checker.df_compare[col] = as_num
                else:  # safe
                    if parse_ratio >= min_ratio:
                        checker.df_compare[col] = as_num
                    elif coerced_count > 0:
                        _logger.warning(
                            "Column %r: only %d/%d (%.1f%%) values parseable as numeric (< min_ratio=%.2f); "
                            "kept as object. Set numeric_coercion_mode='aggressive' to force coercion.",
                            col,
                            coerced_count,
                            original_count,
                            parse_ratio * 100,
                            min_ratio,
                        )

        checker.df_both = checker.df_compare[checker.df_compare["_merge"] == "both"].copy()
        checker._info_cols = [
            col for col in checker.cfg.info_list if col != "flow_id" and col in checker.df_both.columns
        ]

run

run(offline_data: DataFrame | None = None, online_data: DataFrame | None = None) -> ScoreConsistencyUATPipelineResult

Run the consistency checks and write the configured reports.

Parameters:

Name	Type	Description	Default
offline_data	DataFrame or None	Offline data; overrides config.offline_data. It must contain a flow_id column.	None
online_data	DataFrame or None	Online data; overrides config.online_data. It must contain a flow_id column.	None

Returns:

Type	Description
ScoreConsistencyUATPipelineResult	The checker tables, the merged data, the Excel report path and the checker itself.

Raises:

Type	Description
ValueError	If only one of the two DataFrames is available, or if n_process, numeric_coercion_mode or comparison_block_size is invalid.
KeyError	If a DataFrame has no flow_id column.
ImportError	If config.env_path is set and python-dotenv is not installed.

Notes

DataFrame mode is used as soon as one of the two frames is given (as argument or in the config); then both are required. Otherwise the SQL files in sql_dir are run with config.sqlrunner (ODPSRunner() when None).

Source code in Modeling_Tool/Pipeline/score_consistency_uat.py

def run(
    self,
    offline_data: pd.DataFrame | None = None,
    online_data: pd.DataFrame | None = None,
) -> ScoreConsistencyUATPipelineResult:
    """Run the consistency checks and write the configured reports.

    Parameters
    ----------
    offline_data : pandas.DataFrame or None, default None
        Offline data; overrides ``config.offline_data``. It must contain a ``flow_id`` column.
    online_data : pandas.DataFrame or None, default None
        Online data; overrides ``config.online_data``. It must contain a ``flow_id`` column.

    Returns
    -------
    ScoreConsistencyUATPipelineResult
        The checker tables, the merged data, the Excel report path and the checker itself.

    Raises
    ------
    ValueError
        If only one of the two DataFrames is available, or if ``n_process``, ``numeric_coercion_mode`` or
        ``comparison_block_size`` is invalid.
    KeyError
        If a DataFrame has no ``flow_id`` column.
    ImportError
        If ``config.env_path`` is set and python-dotenv is not installed.

    Notes
    -----
    DataFrame mode is used as soon as one of the two frames is given (as argument or in the config); then both are
    required. Otherwise the SQL files in ``sql_dir`` are run with ``config.sqlrunner`` (``ODPSRunner()`` when None).
    """
    from Modeling_Tool.UAT import UATConsistencyChecker

    cfg = self.config
    self._load_env(cfg.env_path)
    report_dir = Path(cfg.output_dir) / "report"
    if cfg.write_outputs or cfg.write_excel:
        make_dirs(cfg.output_dir, report_dir)

    offline = offline_data if offline_data is not None else cfg.offline_data
    online = online_data if online_data is not None else cfg.online_data
    use_dataframes = offline is not None or online is not None
    checker = UATConsistencyChecker(self._build_uat_config(), self._resolve_sqlrunner(use_dataframes))

    if use_dataframes:
        if offline is None or online is None:
            raise ValueError("Provide both offline_data and online_data for DataFrame mode.")
        self._load_dataframes(checker, offline, online)
    else:
        checker.load_data()

    coverage_summary = checker.check_coverage()
    main_score_summary = checker.check_main_score()
    submodel_summary = checker.check_submodel_features()
    feature_diff_summary = checker.check_all_features()
    time_summary = checker.check_time_fields()
    per_flow_report = checker.build_per_flow_report()
    summary = checker.build_summary()

    report_path = None
    if cfg.write_outputs:
        safe_to_csv(pd.DataFrame([coverage_summary]), report_dir / "coverage_summary.csv", index=False)
        safe_to_csv(pd.DataFrame([main_score_summary]), report_dir / "main_score_summary.csv", index=False)
        safe_to_csv(pd.DataFrame(submodel_summary), report_dir / "submodel_summary.csv", index=False)
        safe_to_csv(feature_diff_summary, report_dir / "feature_diff_summary.csv", index=False)
        safe_to_csv(time_summary, report_dir / "time_summary.csv", index=False)
        safe_to_csv(per_flow_report, report_dir / "per_flow_report.csv", index=False)
        safe_to_csv(summary, report_dir / "summary.csv", index=False)

    if cfg.write_excel:
        report_path = checker.export_excel()

    return ScoreConsistencyUATPipelineResult(
        offline_data=checker.df_offline,
        online_data=checker.df_online,
        compare_data=checker.df_compare,
        both_data=checker.df_both,
        coverage_summary=coverage_summary,
        main_score_summary=main_score_summary,
        submodel_summary=submodel_summary,
        feature_diff_summary=feature_diff_summary,
        time_summary=time_summary,
        per_flow_report=per_flow_report,
        summary=summary,
        report_path=report_path,
        checker=checker,
    )

Sample Analysis Pipeline: sample_analysis

sample_analysis

SampleAnalysisPipelineConfig dataclass

Configuration of :class:SampleAnalysisPipeline.

The defaults for the column names (target_cols, time_dims, population_dims, profile_cols, approved_col) match one particular dataset. run raises KeyError when a configured column is missing from the data, so override them for your own data.

Parameters:

Name	Type	Description	Default
target_cols	list of str	Candidate target label columns (1 = bad). Each target is analysed separately and only rows where the label is not null (mature rows) take part.	['y_flag_dpd7_in_mob1', 'y_flag_dpd7_in_mob3', 'y_flag_dpd7_in_mob6', 'y_flag_dpd7_in_mob12']
time_col	str	Application time column; converted with pandas.to_datetime on a copy of the data.	"apply_time"
time_dims	list of str	Existing time-period columns used for the bad-rate and profile summaries (alone and crossed with population_dims).	['apply_week', 'apply_month', 'apply_quarter']
population_dims	list of str	Existing population columns used for the bad-rate and profile summaries (alone and crossed with time_dims).	['channel', 'strategy_version']
profile_cols	list of str	Columns profiled in profile_summary: the missing rate for every column, mean and median for numeric columns, and number of distinct values, top value, its count and its share for the other columns. All must exist in the data.	['age', 'income', 'education', 'credit_limit']
oot_time_dim	str	Column whose sorted distinct values define the out-of-time (OOT) windows: the last N distinct values among a target's mature rows form the OOT sample.	"apply_month"
oot_windows	list of int	Candidate OOT window lengths N in oot_time_dim periods.	[1, 2, 3, 6]
ins_oos_ratios	list of float	Candidate INS shares of the non-OOT pool; the OOS share is 1 - ratio. The pool is split with SampleSplitter(stratify=True), falling back to an unstratified split when stratification fails.	[0.7, 0.75, 0.8]
random_seeds	range or list of int or tuple of int	Random seeds tried for every INS/OOS split. The number of candidates per target is len(oot_windows) * len(ins_oos_ratios) * len(random_seeds).	range(3000, 3020)
min_sample_size	int	Minimum rows required in each of INS, OOS and OOT for a candidate to be eligible for the recommendation. If a target has no eligible candidate, all of its candidates are considered.	500
output_dir	str	Directory for the CSV files, the Excel report and the optional split files.	"output/sample_analysis"
write_outputs	bool	Whether to write the five summary tables as <table_name>.csv files in output_dir.	True
write_excel	bool	Whether to write Sample_Analysis_Report.xlsx (a Charts sheet plus one sheet per table) in output_dir.	True
approved_col	str or None	Approval flag column (1 = approved) used to count n_approved_observed in label_coverage_summary. If it is set but absent from the data a UserWarning is issued and that count is NaN; None turns the count off.	"is_approved"
dry_run	bool	True makes run validate the input and return only the split-count estimate (a one-row split_candidate_summary, the other tables empty, output_paths empty) without any analysis or file output.	False
id_col	str or None	Unique row identifier column, required when materialize_split is True (ValueError if empty, KeyError if absent from the data). It must have no duplicates among the mature rows of any target.	None
materialize_split	bool	True replays the recommended (OOT window, ratio, seed) of every target into row-level INS/OOS/OOT labels (row_level_split) and an audit record (split_artifact). A ValueError is raised if no recommendation exists.	False
oot_cutoff	any	Used only when materialize_split is True: the OOT sample becomes the mature rows with oot_time_dim >= oot_cutoff (artifact oot_basis="cutoff") instead of the recommended trailing window, and the INS/OOS pool is rebuilt from that boundary. The candidate statistics still use the trailing windows.	None
split_col_name	str	Name of the label column in row_level_split; its values are "ins", "oos" and "oot".	"sample_split"
persist_split_map	bool	With materialize_split=True, write row_level_split.csv and split_artifact.json to output_dir (even when write_outputs and write_excel are False) and add their absolute paths to output_paths.	False

Source code in Modeling_Tool/Pipeline/sample_analysis.py

@dataclass
class SampleAnalysisPipelineConfig:
    """Configuration of :class:`SampleAnalysisPipeline`.

    The defaults for the column names (``target_cols``, ``time_dims``, ``population_dims``, ``profile_cols``,
    ``approved_col``) match one particular dataset. ``run`` raises ``KeyError`` when a configured column is missing from
    the data, so override them for your own data.

    Parameters
    ----------
    target_cols : list of str, default ['y_flag_dpd7_in_mob1', 'y_flag_dpd7_in_mob3', 'y_flag_dpd7_in_mob6', 'y_flag_dpd7_in_mob12']
        Candidate target label columns (1 = bad). Each target is analysed separately and only rows where the label is
        not null (mature rows) take part.
    time_col : str, default "apply_time"
        Application time column; converted with ``pandas.to_datetime`` on a copy of the data.
    time_dims : list of str, default ['apply_week', 'apply_month', 'apply_quarter']
        Existing time-period columns used for the bad-rate and profile summaries (alone and crossed with
        ``population_dims``).
    population_dims : list of str, default ['channel', 'strategy_version']
        Existing population columns used for the bad-rate and profile summaries (alone and crossed with
        ``time_dims``).
    profile_cols : list of str, default ['age', 'income', 'education', 'credit_limit']
        Columns profiled in ``profile_summary``: the missing rate for every column, mean and median for numeric columns,
        and number of distinct values, top value, its count and its share for the other columns. All must exist in the
        data.
    oot_time_dim : str, default "apply_month"
        Column whose sorted distinct values define the out-of-time (OOT) windows: the last ``N`` distinct values among a
        target's mature rows form the OOT sample.
    oot_windows : list of int, default [1, 2, 3, 6]
        Candidate OOT window lengths ``N`` in ``oot_time_dim`` periods.
    ins_oos_ratios : list of float, default [0.7, 0.75, 0.8]
        Candidate INS shares of the non-OOT pool; the OOS share is ``1 - ratio``. The pool is split with
        ``SampleSplitter(stratify=True)``, falling back to an unstratified split when stratification fails.
    random_seeds : range or list of int or tuple of int, default range(3000, 3020)
        Random seeds tried for every INS/OOS split. The number of candidates per target is
        ``len(oot_windows) * len(ins_oos_ratios) * len(random_seeds)``.
    min_sample_size : int, default 500
        Minimum rows required in each of INS, OOS and OOT for a candidate to be eligible for the recommendation. If a
        target has no eligible candidate, all of its candidates are considered.
    output_dir : str, default "output/sample_analysis"
        Directory for the CSV files, the Excel report and the optional split files.
    write_outputs : bool, default True
        Whether to write the five summary tables as ``<table_name>.csv`` files in ``output_dir``.
    write_excel : bool, default True
        Whether to write ``Sample_Analysis_Report.xlsx`` (a ``Charts`` sheet plus one sheet per table) in
        ``output_dir``.
    approved_col : str or None, default "is_approved"
        Approval flag column (1 = approved) used to count ``n_approved_observed`` in ``label_coverage_summary``. If it
        is set but absent from the data a ``UserWarning`` is issued and that count is NaN; ``None`` turns the count off.
    dry_run : bool, default False
        True makes ``run`` validate the input and return only the split-count estimate (a one-row
        ``split_candidate_summary``, the other tables empty, ``output_paths`` empty) without any analysis or file
        output.
    id_col : str or None, default None
        Unique row identifier column, required when ``materialize_split`` is True (``ValueError`` if empty,
        ``KeyError`` if absent from the data). It must have no duplicates among the mature rows of any target.
    materialize_split : bool, default False
        True replays the recommended (OOT window, ratio, seed) of every target into row-level INS/OOS/OOT labels
        (``row_level_split``) and an audit record (``split_artifact``). A ``ValueError`` is raised if no recommendation
        exists.
    oot_cutoff : any, default None
        Used only when ``materialize_split`` is True: the OOT sample becomes the mature rows with
        ``oot_time_dim >= oot_cutoff`` (artifact ``oot_basis="cutoff"``) instead of the recommended trailing window, and
        the INS/OOS pool is rebuilt from that boundary. The candidate statistics still use the trailing windows.
    split_col_name : str, default "sample_split"
        Name of the label column in ``row_level_split``; its values are ``"ins"``, ``"oos"`` and ``"oot"``.
    persist_split_map : bool, default False
        With ``materialize_split=True``, write ``row_level_split.csv`` and ``split_artifact.json`` to ``output_dir``
        (even when ``write_outputs`` and ``write_excel`` are False) and add their absolute paths to ``output_paths``.
    """

    target_cols: list[str] = field(
        default_factory=lambda: [
            "y_flag_dpd7_in_mob1",
            "y_flag_dpd7_in_mob3",
            "y_flag_dpd7_in_mob6",
            "y_flag_dpd7_in_mob12",
        ]
    )
    time_col: str = "apply_time"
    time_dims: list[str] = field(default_factory=lambda: ["apply_week", "apply_month", "apply_quarter"])
    population_dims: list[str] = field(default_factory=lambda: ["channel", "strategy_version"])
    profile_cols: list[str] = field(default_factory=lambda: ["age", "income", "education", "credit_limit"])
    oot_time_dim: str = "apply_month"
    oot_windows: list[int] = field(default_factory=lambda: [1, 2, 3, 6])
    ins_oos_ratios: list[float] = field(default_factory=lambda: [0.7, 0.75, 0.8])
    random_seeds: range | list[int] | tuple[int, ...] = field(default_factory=lambda: range(3000, 3020))
    min_sample_size: int = 500
    output_dir: str = "output/sample_analysis"
    write_outputs: bool = True
    write_excel: bool = True
    approved_col: str | None = "is_approved"

    # v0.4.0: split-grid preview controls.
    # Default combinations (4 targets * 4 windows * 3 ratios * 20 seeds = 960 splits)
    # can silently take hours. Set dry_run=True to short-circuit run() and only emit
    # an estimated split count. Alternatively call SampleAnalysisPipeline
    # .estimate_split_count() directly without running.
    dry_run: bool = False

    # G01: row-level materialization of the recommended splits. Off by default
    # (legacy: SAP only emits statistics). With materialize_split=True the
    # recommended (window, ratio, seed) combination per target is REPLAYED via
    # the same SampleSplitter, producing a long frame
    # [id_col, target_col, split_col_name] plus a verifiable split_artifact.
    id_col: str | None = None
    materialize_split: bool = False
    # When set, OOT = rows with oot_time_dim >= oot_cutoff (artifact records
    # oot_basis="cutoff"); otherwise the recommended trailing window is used.
    oot_cutoff: Any = None
    split_col_name: str = "sample_split"
    persist_split_map: bool = False

SampleAnalysisPipelineResult dataclass

Result returned by :meth:SampleAnalysisPipeline.run.

Parameters:

Name	Type	Description	Default
label_coverage_summary	DataFrame	One row per target: target_col, approved_col, n_total, n_observed, observed_rate, bad_rate, apply_time_min, apply_time_max (over observed rows) and n_approved_observed.	required
segment_bad_rate_summary	DataFrame	Mature-row counts and bad rates per target and group: target_col, group_type (global, time, population or time_x_population), group_cols, group_value, n and bad_rate.	required
profile_summary	DataFrame	Same group keys as segment_bad_rate_summary with n, bad_rate and the profile statistics of every profile_cols column (<col>_missing_rate, <col>_mean / <col>_median for numeric columns, <col>_nunique / <col>_top / <col>_top_count / <col>_top_rate for the others).	required
split_candidate_summary	DataFrame	One row per target, OOT window, INS ratio and seed with the OOT periods, sample sizes (n_ins, n_oos, n_oot), bad rates (bad_rate_ins, bad_rate_oos, bad_rate_oot), the pairwise absolute gaps and max_abs_bad_rate_gap. With dry_run=True it holds a single row with the split-count estimate instead.	required
split_recommendation	DataFrame	One row per target: the eligible candidate with the smallest max_abs_bad_rate_gap (ties go to the larger n_oot, then the ratio closest to 0.75), with the extra columns ratio_distance_to_75_25 and recommend_reason. Empty when there are no candidates.	required
output_paths	dict of str to str	Absolute paths of the files written: one entry per table name (CSV), excel_report, and row_level_split and split_artifact when persist_split_map is used. Empty when nothing was written.	required
row_level_split	DataFrame or None	None unless materialize_split=True. Otherwise a long frame with the id_col column, a target_col column holding the target name and the split_col_name column (ins / oos / oot), with one row per mature row of each target.	None
split_artifact	dict or None	None unless materialize_split=True. Audit record with split_col_name, id_col, smf_version and, per target, the seed, INS ratio, OOT basis (window or cutoff) and specification, the row counts and hashes of the sorted ids of each segment (ins, oos, oot, full).	None

Source code in Modeling_Tool/Pipeline/sample_analysis.py

@dataclass
class SampleAnalysisPipelineResult:
    """Result returned by :meth:`SampleAnalysisPipeline.run`.

    Parameters
    ----------
    label_coverage_summary : pandas.DataFrame
        One row per target: ``target_col``, ``approved_col``, ``n_total``, ``n_observed``, ``observed_rate``,
        ``bad_rate``, ``apply_time_min``, ``apply_time_max`` (over observed rows) and ``n_approved_observed``.
    segment_bad_rate_summary : pandas.DataFrame
        Mature-row counts and bad rates per target and group: ``target_col``, ``group_type`` (``global``, ``time``,
        ``population`` or ``time_x_population``), ``group_cols``, ``group_value``, ``n`` and ``bad_rate``.
    profile_summary : pandas.DataFrame
        Same group keys as ``segment_bad_rate_summary`` with ``n``, ``bad_rate`` and the profile statistics of every
        ``profile_cols`` column (``<col>_missing_rate``, ``<col>_mean`` / ``<col>_median`` for numeric columns,
        ``<col>_nunique`` / ``<col>_top`` / ``<col>_top_count`` / ``<col>_top_rate`` for the others).
    split_candidate_summary : pandas.DataFrame
        One row per target, OOT window, INS ratio and seed with the OOT periods, sample sizes (``n_ins``, ``n_oos``,
        ``n_oot``), bad rates (``bad_rate_ins``, ``bad_rate_oos``, ``bad_rate_oot``), the pairwise absolute gaps and
        ``max_abs_bad_rate_gap``. With ``dry_run=True`` it holds a single row with the split-count estimate instead.
    split_recommendation : pandas.DataFrame
        One row per target: the eligible candidate with the smallest ``max_abs_bad_rate_gap`` (ties go to the larger
        ``n_oot``, then the ratio closest to 0.75), with the extra columns ``ratio_distance_to_75_25`` and
        ``recommend_reason``. Empty when there are no candidates.
    output_paths : dict of str to str
        Absolute paths of the files written: one entry per table name (CSV), ``excel_report``, and ``row_level_split``
        and ``split_artifact`` when ``persist_split_map`` is used. Empty when nothing was written.
    row_level_split : pandas.DataFrame or None, default None
        None unless ``materialize_split=True``. Otherwise a long frame with the ``id_col`` column, a ``target_col``
        column holding the target name and the ``split_col_name`` column (``ins`` / ``oos`` / ``oot``), with one row per
        mature row of each target.
    split_artifact : dict or None, default None
        None unless ``materialize_split=True``. Audit record with ``split_col_name``, ``id_col``, ``smf_version`` and,
        per target, the seed, INS ratio, OOT basis (``window`` or ``cutoff``) and specification, the row counts and
        hashes of the sorted ids of each segment (``ins``, ``oos``, ``oot``, ``full``).
    """

    label_coverage_summary: pd.DataFrame
    segment_bad_rate_summary: pd.DataFrame
    profile_summary: pd.DataFrame
    split_candidate_summary: pd.DataFrame
    split_recommendation: pd.DataFrame
    output_paths: dict[str, str]
    # G01: long frame [id_col, target_col, split_col_name] and its audit
    # artifact; None unless materialize_split=True.
    row_level_split: pd.DataFrame | None = None
    split_artifact: dict[str, Any] | None = None

SampleAnalysisPipeline

Analyze label maturity, segment drift, and INS/OOS/OOT split stability.

Parameters:

Name	Type	Description	Default
config	SampleAnalysisPipelineConfig or None	Pipeline configuration. None (or any falsy value) uses SampleAnalysisPipelineConfig() with its defaults.	None

Attributes:

Name	Type	Description
config	SampleAnalysisPipelineConfig	The configuration in use.

Source code in Modeling_Tool/Pipeline/sample_analysis.py

class SampleAnalysisPipeline:
    """Analyze label maturity, segment drift, and INS/OOS/OOT split stability.

    Parameters
    ----------
    config : SampleAnalysisPipelineConfig or None, default None
        Pipeline configuration. ``None`` (or any falsy value) uses ``SampleAnalysisPipelineConfig()`` with its defaults.

    Attributes
    ----------
    config : SampleAnalysisPipelineConfig
        The configuration in use.
    """

    def __init__(self, config: SampleAnalysisPipelineConfig | None = None):
        self.config = config or SampleAnalysisPipelineConfig()

    def estimate_split_count(self, data: pd.DataFrame | None = None) -> dict[str, Any]:
        """Return an upper-bound estimate of split candidates that _split_candidates would produce.

        If `data` is provided, refine the OOT window count against the actual number of
        distinct oot_time_dim values available (some windows may collapse if history is short).

        Parameters
        ----------
        data : pandas.DataFrame or None, default None
            Optional data. When it contains the ``oot_time_dim`` column, the number of distinct non-null periods in it is
            added to the estimate.

        Returns
        -------
        dict
            ``n_targets``, ``n_oot_windows``, ``n_ins_oos_ratios``, ``n_random_seeds`` and ``estimated_max_splits`` (their
            product). With ``data``, also ``available_oot_periods`` (distinct ``oot_time_dim`` values over all rows) and
            ``windows_usable`` (number of ``oot_windows`` smaller than that count).

        Notes
        -----
        Nothing is computed from the labels, so the real number of candidates can be lower: a target without usable
        OOT and pool segments contributes none.
        """
        cfg = self.config
        n_targets = len(cfg.target_cols)
        n_windows = len(cfg.oot_windows)
        n_ratios = len(cfg.ins_oos_ratios)
        n_seeds = len(list(cfg.random_seeds))
        estimated = n_targets * n_windows * n_ratios * n_seeds
        info: dict[str, Any] = {
            "n_targets": n_targets,
            "n_oot_windows": n_windows,
            "n_ins_oos_ratios": n_ratios,
            "n_random_seeds": n_seeds,
            "estimated_max_splits": estimated,
        }
        if data is not None and cfg.oot_time_dim in data.columns:
            n_periods = data[cfg.oot_time_dim].dropna().nunique()
            info["available_oot_periods"] = int(n_periods)
            info["windows_usable"] = sum(1 for w in cfg.oot_windows if int(w) < n_periods)
        return info

    def run(self, data: pd.DataFrame) -> SampleAnalysisPipelineResult:
        """Run the label coverage, segment, profile and split-stability analyses.

        Parameters
        ----------
        data : pandas.DataFrame
            Sample to analyse. It must contain ``time_col``, ``oot_time_dim``, every column of ``target_cols``,
            ``time_dims``, ``population_dims`` and ``profile_cols``, and ``id_col`` when ``materialize_split`` is True.
            The input frame is not modified.

        Returns
        -------
        SampleAnalysisPipelineResult
            The five summary tables, the paths of the files written and, with ``materialize_split=True``, the row-level
            split and its audit artifact.

        Raises
        ------
        KeyError
            If a required column (or ``id_col`` for ``materialize_split``) is missing from ``data``.
        ValueError
            If ``materialize_split`` is True without ``id_col``, if no recommendation could be produced, if ``id_col``
            has duplicates among the mature rows of a target, or if the OOT or the INS/OOS pool segment is empty.
        AssertionError
            If the materialized INS, OOS and OOT segments overlap or do not cover all mature rows of a target.

        Notes
        -----
        With ``dry_run=True`` the input is validated, a warning with the estimated split count is logged and the
        analysis is skipped (see :meth:`estimate_split_count`). Existing output files are overwritten.
        """
        self._validate_input(data)
        if self.config.dry_run:
            info = self.estimate_split_count(data)
            _logger.warning(
                "SampleAnalysisPipeline dry_run=True: skipping execution. Estimated splits: %s",
                info,
            )
            empty = pd.DataFrame()
            return SampleAnalysisPipelineResult(
                label_coverage_summary=empty,
                segment_bad_rate_summary=empty,
                profile_summary=empty,
                split_candidate_summary=pd.DataFrame([info]),
                split_recommendation=empty,
                output_paths={},
            )
        work = data.copy()
        work[self.config.time_col] = pd.to_datetime(work[self.config.time_col])

        label_coverage = self._label_coverage(work)
        segment_bad_rate = self._segment_bad_rate(work)
        profile_summary = self._profile_summary(work)
        split_candidates = self._split_candidates(work)
        split_recommendation = self._recommend_splits(split_candidates)

        row_level_split = None
        split_artifact = None
        if self.config.materialize_split:
            if split_recommendation.empty:
                raise ValueError(
                    "materialize_split=True but no split recommendation was produced "
                    "(no target had usable OOT/pool segments)."
                )
            row_level_split, split_artifact = self._materialize_recommended_splits(
                work, split_recommendation,
            )

        output_paths = self._write_outputs(
            {
                "label_coverage_summary": label_coverage,
                "segment_bad_rate_summary": segment_bad_rate,
                "profile_summary": profile_summary,
                "split_candidate_summary": split_candidates,
                "split_recommendation": split_recommendation,
            }
        )
        if row_level_split is not None and self.config.persist_split_map:
            import json

            output_dir = Path(self.config.output_dir)
            output_dir.mkdir(parents=True, exist_ok=True)
            split_csv = output_dir / "row_level_split.csv"
            row_level_split.to_csv(split_csv, index=False)
            output_paths["row_level_split"] = str(split_csv.resolve())
            artifact_json = output_dir / "split_artifact.json"
            artifact_json.write_text(
                json.dumps(split_artifact, ensure_ascii=False, indent=2),
                encoding="utf-8",
            )
            output_paths["split_artifact"] = str(artifact_json.resolve())

        return SampleAnalysisPipelineResult(
            label_coverage_summary=label_coverage,
            segment_bad_rate_summary=segment_bad_rate,
            profile_summary=profile_summary,
            split_candidate_summary=split_candidates,
            split_recommendation=split_recommendation,
            output_paths=output_paths,
            row_level_split=row_level_split,
            split_artifact=split_artifact,
        )

    def _validate_input(self, data: pd.DataFrame) -> None:
        cfg = self.config
        required = (
            [cfg.time_col, cfg.oot_time_dim]
            + cfg.target_cols
            + cfg.time_dims
            + cfg.population_dims
            + cfg.profile_cols
        )
        missing = [col for col in dict.fromkeys(required) if col not in data.columns]
        if missing:
            raise KeyError(f"Missing required columns: {missing}")
        if cfg.materialize_split:
            if not cfg.id_col:
                raise ValueError("materialize_split=True requires id_col.")
            if cfg.id_col not in data.columns:
                raise KeyError(f"Missing id_col {cfg.id_col!r} for materialize_split.")

    def _label_coverage(self, data: pd.DataFrame) -> pd.DataFrame:
        cfg = self.config
        rows = []
        approved_col = cfg.approved_col
        approved_available = approved_col is not None and approved_col in data.columns
        if approved_col is not None and not approved_available:
            warnings.warn(
                f"approved_col={approved_col!r} was not found in input data; "
                "n_approved_observed will be empty.",
                UserWarning,
                stacklevel=2,
            )
        for target in cfg.target_cols:
            mature = data[data[target].notna()]
            rows.append(
                {
                    "target_col": target,
                    "approved_col": approved_col if approved_available else None,
                    "n_total": len(data),
                    "n_observed": len(mature),
                    "observed_rate": len(mature) / len(data) if len(data) else np.nan,
                    "bad_rate": mature[target].mean(),
                    "apply_time_min": mature[cfg.time_col].min(),
                    "apply_time_max": mature[cfg.time_col].max(),
                    "n_approved_observed": int(mature[approved_col].eq(1).sum()) if approved_available else np.nan,
                }
            )
        return pd.DataFrame(rows)

    def _segment_bad_rate(self, data: pd.DataFrame) -> pd.DataFrame:
        rows = []
        for target in self.config.target_cols:
            mature = data[data[target].notna()].copy()
            rows.extend(self._summarize_bad_rate(mature, target, [], "global"))
            for dim in self.config.time_dims:
                rows.extend(self._summarize_bad_rate(mature, target, [dim], "time"))
            for dim in self.config.population_dims:
                rows.extend(self._summarize_bad_rate(mature, target, [dim], "population"))
            for time_dim in self.config.time_dims:
                for pop_dim in self.config.population_dims:
                    rows.extend(self._summarize_bad_rate(mature, target, [time_dim, pop_dim], "time_x_population"))
        return pd.DataFrame(rows)

    def _profile_summary(self, data: pd.DataFrame) -> pd.DataFrame:
        rows = []
        group_specs = [([], "global")]
        group_specs.extend(([dim], "time") for dim in self.config.time_dims)
        group_specs.extend(([dim], "population") for dim in self.config.population_dims)
        group_specs.extend(
            ([time_dim, pop_dim], "time_x_population")
            for time_dim in self.config.time_dims
            for pop_dim in self.config.population_dims
        )

        for target in self.config.target_cols:
            mature = data[data[target].notna()].copy()
            for group_cols, group_type in group_specs:
                rows.extend(self._summarize_profile(mature, target, list(group_cols), group_type))
        return pd.DataFrame(rows)

    def _split_candidates(self, data: pd.DataFrame) -> pd.DataFrame:
        from Modeling_Tool import SampleSplitter

        cfg = self.config
        rows = []
        for target in cfg.target_cols:
            mature = data[data[target].notna()]
            time_values = sorted(mature[cfg.oot_time_dim].dropna().unique())
            if not time_values:
                continue
            for oot_window in cfg.oot_windows:
                oot_values = set(time_values[-int(oot_window):])
                oot_mask = mature[cfg.oot_time_dim].isin(oot_values)
                oot_index = mature.index[oot_mask]
                pool_index = mature.index[~oot_mask]
                if len(oot_index) == 0 or len(pool_index) == 0:
                    continue
                pool_target = mature.loc[pool_index, target]
                br_oot = mature.loc[oot_index, target].mean()

                for ins_ratio in cfg.ins_oos_ratios:
                    for seed in list(cfg.random_seeds):
                        ins_index, oos_index = self._materialize_one_split(
                            pool_index, pool_target, float(ins_ratio), int(seed),
                        )

                        br_ins = mature.loc[ins_index, target].mean()
                        br_oos = mature.loc[oos_index, target].mean()
                        rows.append(
                            {
                                "target_col": target,
                                "oot_time_dim": cfg.oot_time_dim,
                                "oot_window_periods": int(oot_window),
                                "oot_window_months": int(oot_window),
                                "oot_periods": ",".join(str(x) for x in sorted(oot_values)),
                                "oot_months": ",".join(str(x) for x in sorted(oot_values)),
                                "ins_ratio": float(ins_ratio),
                                "oos_ratio": 1 - float(ins_ratio),
                                "seed": int(seed),
                                "n_ins": len(ins_index),
                                "n_oos": len(oos_index),
                                "n_oot": len(oot_index),
                                "bad_rate_ins": br_ins,
                                "bad_rate_oos": br_oos,
                                "bad_rate_oot": br_oot,
                                "abs_diff_ins_oos": abs(br_ins - br_oos),
                                "abs_diff_oos_oot": abs(br_oos - br_oot),
                                "max_abs_bad_rate_gap": max(
                                    abs(br_ins - br_oos),
                                    abs(br_ins - br_oot),
                                    abs(br_oos - br_oot),
                                ),
                                "oot_sample_pct": len(oot_index) / len(mature) if len(mature) else np.nan,
                            }
                        )
        return pd.DataFrame(rows)

    @staticmethod
    def _materialize_one_split(
        pool_index: pd.Index,
        pool_target: pd.Series,
        ins_ratio: float,
        seed: int,
    ) -> tuple[pd.Index, pd.Index]:
        """The single INS/OOS index generator shared by candidate scoring and
        row-level materialization — replaying a recommended (ratio, seed) is
        guaranteed to reproduce the exact indices the statistics came from."""
        from Modeling_Tool import SampleSplitter

        splitter = SampleSplitter(
            test_size=1 - float(ins_ratio),
            random_state=int(seed),
            stratify=True,
        )
        try:
            return splitter.split_indices(pool_index, pool_target)
        except ValueError:
            splitter = SampleSplitter(
                test_size=1 - float(ins_ratio),
                random_state=int(seed),
                stratify=False,
            )
            return splitter.split_indices(pool_index, pool_target)

    def _materialize_recommended_splits(
        self,
        data: pd.DataFrame,
        recommendation: pd.DataFrame,
    ) -> tuple[pd.DataFrame, dict[str, Any]]:
        """G01: replay each target's recommended split into row-level
        membership, with loud integrity assertions and verifiable ID hashes."""
        from Modeling_Tool.Pipeline._common import hash_id_values

        cfg = self.config
        id_col = cfg.id_col
        split_col = cfg.split_col_name
        frames: list[pd.DataFrame] = []
        per_target: dict[str, Any] = {}
        for _, rec in recommendation.iterrows():
            target = str(rec["target_col"])
            mature = data[data[target].notna()]
            dup_count = int(mature[id_col].duplicated().sum())
            if dup_count:
                raise ValueError(
                    f"materialize_split: id_col {id_col!r} has {dup_count} duplicate "
                    f"value(s) among mature rows of target {target!r}; row-level "
                    f"membership would be ambiguous."
                )
            if cfg.oot_cutoff is not None:
                oot_mask = mature[cfg.oot_time_dim] >= cfg.oot_cutoff
                oot_basis = "cutoff"
                oot_spec: Any = str(cfg.oot_cutoff)
            else:
                time_values = sorted(mature[cfg.oot_time_dim].dropna().unique())
                window = int(rec["oot_window_periods"])
                oot_values = set(time_values[-window:])
                oot_mask = mature[cfg.oot_time_dim].isin(oot_values)
                oot_basis = "window"
                oot_spec = ",".join(str(x) for x in sorted(oot_values))
            oot_index = mature.index[oot_mask]
            pool_index = mature.index[~oot_mask]
            if len(oot_index) == 0 or len(pool_index) == 0:
                raise ValueError(
                    f"materialize_split: target {target!r} produced an empty "
                    f"{'OOT' if len(oot_index) == 0 else 'INS/OOS pool'} segment "
                    f"(oot_basis={oot_basis!r}, spec={oot_spec!r})."
                )
            ins_index, oos_index = self._materialize_one_split(
                pool_index, mature.loc[pool_index, target],
                float(rec["ins_ratio"]), int(rec["seed"]),
            )
            # split_indices may hand back plain ndarrays; the set-algebra
            # assertions below need pandas Index semantics.
            ins_index, oos_index = pd.Index(ins_index), pd.Index(oos_index)

            segments = {"ins": ins_index, "oos": oos_index, "oot": oot_index}
            for (name_a, idx_a), (name_b, idx_b) in (
                (("ins", ins_index), ("oos", oos_index)),
                (("ins", ins_index), ("oot", oot_index)),
                (("oos", oos_index), ("oot", oot_index)),
            ):
                overlap = len(idx_a.intersection(idx_b))
                if overlap:
                    raise AssertionError(
                        f"materialize_split: {overlap} row(s) fall in both "
                        f"{name_a} and {name_b} for target {target!r}."
                    )
            covered = len(ins_index) + len(oos_index) + len(oot_index)
            if covered != len(mature):
                raise AssertionError(
                    f"materialize_split: segments cover {covered} rows but target "
                    f"{target!r} has {len(mature)} mature rows."
                )

            long = pd.DataFrame({
                id_col: np.concatenate([
                    mature.loc[idx, id_col].to_numpy() for idx in segments.values()
                ]),
                "target_col": target,
                split_col: np.concatenate([
                    np.full(len(idx), name, dtype=object)
                    for name, idx in segments.items()
                ]),
            })
            frames.append(long)
            per_target[target] = {
                "seed": int(rec["seed"]),
                "ins_ratio": float(rec["ins_ratio"]),
                "oot_time_dim": cfg.oot_time_dim,
                "oot_basis": oot_basis,
                "oot_spec": oot_spec,
                "counts": {name: int(len(idx)) for name, idx in segments.items()},
                "hashes": {
                    **{
                        name: hash_id_values(mature.loc[idx, id_col])
                        for name, idx in segments.items()
                    },
                    "full": hash_id_values(mature[id_col]),
                },
            }

        from Modeling_Tool import __version__ as smf_version

        row_level = pd.concat(frames, ignore_index=True)
        artifact = {
            "split_col_name": split_col,
            "id_col": id_col,
            "targets": per_target,
            "smf_version": str(smf_version),
        }
        return row_level, artifact

    def _recommend_splits(self, split_candidates: pd.DataFrame) -> pd.DataFrame:
        if split_candidates.empty:
            return split_candidates.copy()

        cfg = self.config
        recs = []
        for _, sub in split_candidates.groupby("target_col", dropna=False):
            valid = sub[
                (sub["n_ins"] >= cfg.min_sample_size)
                & (sub["n_oos"] >= cfg.min_sample_size)
                & (sub["n_oot"] >= cfg.min_sample_size)
            ].copy()
            if valid.empty:
                valid = sub.copy()
            valid["ratio_distance_to_75_25"] = (valid["ins_ratio"] - 0.75).abs()
            valid = valid.sort_values(
                ["max_abs_bad_rate_gap", "n_oot", "ratio_distance_to_75_25"],
                ascending=[True, False, True],
            )
            rec = valid.head(1).copy()
            rec["recommend_reason"] = (
                "Min bad-rate gap after sample-size filter; tie-break by larger OOT and 75/25 proximity"
            )
            recs.append(rec)
        return pd.concat(recs, ignore_index=True)

    def _summarize_bad_rate(
        self,
        data: pd.DataFrame,
        target: str,
        group_cols: list[str],
        group_type: str,
    ) -> list[dict[str, Any]]:
        if not group_cols:
            return [
                {
                    "target_col": target,
                    "group_type": group_type,
                    "group_cols": "global",
                    "group_value": "ALL",
                    "n": len(data),
                    "bad_rate": data[target].mean(),
                }
            ]

        summary = data.groupby(group_cols, dropna=False, sort=True).agg(
            n=(target, "size"),
            bad_rate=(target, "mean"),
        ).reset_index()
        group_value = summary[group_cols[0]].astype(str)
        for col in group_cols[1:]:
            group_value = group_value.str.cat(summary[col].astype(str), sep=" x ")
        summary["target_col"] = target
        summary["group_type"] = group_type
        summary["group_cols"] = " x ".join(group_cols)
        summary["group_value"] = group_value
        return summary[
            ["target_col", "group_type", "group_cols", "group_value", "n", "bad_rate"]
        ].to_dict("records")

    def _summarize_profile(
        self,
        data: pd.DataFrame,
        target: str,
        group_cols: list[str],
        group_type: str,
    ) -> list[dict[str, Any]]:
        profile_cols = [col for col in self.config.profile_cols if col in data.columns]
        work_cols = list(dict.fromkeys(group_cols + [target] + profile_cols))
        work = data[work_cols].copy()
        internal_group_cols = list(group_cols)
        if not internal_group_cols:
            global_col = "_smf_profile_global"
            work[global_col] = "ALL"
            internal_group_cols = [global_col]

        named_aggs: dict[str, tuple[str, str]] = {
            "n": (target, "size"),
            "bad_rate": (target, "mean"),
        }
        categorical_cols = []
        desired_profile_cols = []
        for idx, col in enumerate(profile_cols):
            missing_col = f"_smf_profile_missing_{idx}"
            work[missing_col] = work[col].isna().astype(float)
            named_aggs[f"{col}_missing_rate"] = (missing_col, "mean")
            desired_profile_cols.append(f"{col}_missing_rate")
            if pd.api.types.is_numeric_dtype(work[col]):
                named_aggs[f"{col}_mean"] = (col, "mean")
                named_aggs[f"{col}_median"] = (col, "median")
                desired_profile_cols.extend([f"{col}_mean", f"{col}_median"])
            else:
                categorical_cols.append(col)
                named_aggs[f"{col}_nunique"] = (col, "nunique")
                named_aggs[f"{col}_non_missing"] = (col, "count")
                desired_profile_cols.extend(
                    [f"{col}_nunique", f"{col}_top", f"{col}_top_count", f"{col}_top_rate"]
                )

        summary = (
            work.groupby(internal_group_cols, dropna=False, sort=True)
            .agg(**named_aggs)
            .reset_index()
        )
        for col in categorical_cols:
            if col in internal_group_cols:
                non_missing = summary[f"{col}_non_missing"].to_numpy(dtype=float)
                summary[f"{col}_top"] = summary[col].where(non_missing > 0, None)
                summary[f"{col}_top_count"] = non_missing.astype(int)
            else:
                counts = (
                    work.dropna(subset=[col])
                    .groupby(internal_group_cols + [col], dropna=False, sort=True)
                    .size()
                    .rename(f"{col}_top_count")
                    .reset_index()
                )
                if counts.empty:
                    summary[f"{col}_top"] = None
                    summary[f"{col}_top_count"] = 0
                else:
                    counts["_smf_value_sort"] = counts[col].astype(str)
                    counts = counts.sort_values(
                        internal_group_cols + [f"{col}_top_count", "_smf_value_sort"],
                        ascending=[True] * len(internal_group_cols) + [False, True],
                        kind="mergesort",
                    ).drop_duplicates(internal_group_cols, keep="first")
                    top = counts[
                        internal_group_cols + [col, f"{col}_top_count"]
                    ].rename(columns={col: f"{col}_top"})
                    summary = summary.merge(top, on=internal_group_cols, how="left", sort=False)
                    summary[f"{col}_top_count"] = summary[f"{col}_top_count"].fillna(0).astype(int)
            denominator = summary[f"{col}_non_missing"].to_numpy(dtype=float)
            summary[f"{col}_top_rate"] = np.divide(
                summary[f"{col}_top_count"].to_numpy(dtype=float),
                denominator,
                out=np.full(len(summary), np.nan),
                where=denominator > 0,
            )
            summary = summary.drop(columns=f"{col}_non_missing")

        if group_cols:
            group_value = summary[group_cols[0]].astype(str)
            for col in group_cols[1:]:
                group_value = group_value.str.cat(summary[col].astype(str), sep=" x ")
            group_cols_label = " x ".join(group_cols)
        else:
            group_value = pd.Series("ALL", index=summary.index)
            group_cols_label = "global"
        summary["target_col"] = target
        summary["group_type"] = group_type
        summary["group_cols"] = group_cols_label
        summary["group_value"] = group_value
        output_cols = [
            "target_col", "group_type", "group_cols", "group_value", "n", "bad_rate",
            *desired_profile_cols,
        ]
        return summary.reindex(columns=output_cols).to_dict("records")

    def _write_outputs(self, tables: dict[str, pd.DataFrame]) -> dict[str, str]:
        cfg = self.config
        output_dir = Path(cfg.output_dir)
        output_paths: dict[str, str] = {}
        if not (cfg.write_outputs or cfg.write_excel):
            return output_paths

        output_dir.mkdir(parents=True, exist_ok=True)
        if cfg.write_outputs:
            for name, df in tables.items():
                path = output_dir / f"{name}.csv"
                df.to_csv(path, index=False)
                output_paths[name] = str(path.resolve())

        if cfg.write_excel:
            excel_path = output_dir / "Sample_Analysis_Report.xlsx"
            self._write_excel_report(excel_path, tables)
            output_paths["excel_report"] = str(excel_path.resolve())
        return output_paths

    def _write_excel_report(self, excel_path: Path, tables: dict[str, pd.DataFrame]) -> None:
        from ExcelMaster.ExcelMaster import ExcelMaster

        em = ExcelMaster(str(excel_path), verbose=False)
        used_sheet_names: set[str] = set()

        self._write_chart_sheet(em, tables, used_sheet_names)
        for name, df in tables.items():
            sheet_name = self._excel_sheet_name(name, used_sheet_names)
            ws = em.add_worksheet(sheet_name, zoom_perc=90)
            em.write_dataframe(ws, df=df, title=name, index=False)
        em.close_workbook()

    def _write_chart_sheet(
        self,
        em: Any,
        tables: dict[str, pd.DataFrame],
        used_sheet_names: set[str],
    ) -> None:
        ws = em.add_worksheet(self._excel_sheet_name("Charts", used_sheet_names), zoom_perc=90)
        em.merge_col(ws, loc=(0, 0), ncols=8, text="Sample Analysis Charts", cformat="BLUE_H4")

        coverage_chart = self._label_coverage_chart_data(tables["label_coverage_summary"])
        rec_chart = self._recommendation_chart_data(tables["split_recommendation"])
        oot_chart = self._oot_window_chart_data(tables["split_candidate_summary"])
        seed_chart = self._seed_stability_chart_data(tables["split_candidate_summary"])
        segment_chart = self._segment_chart_data(tables["segment_bad_rate_summary"])

        if not coverage_chart.empty:
            em.write_duo_chart(
                worksheet=ws,
                df=coverage_chart,
                x="label",
                y1_list=["n_observed"],
                y2_list=["observed_rate", "bad_rate"],
                c1_type="column",
                c2_type="line",
                y1_axis_range=(0, None),
                y2_axis_range=(0, None),
                y2_num_format="0.0%",
                loc=(2, 0),
                title="Label maturity coverage and observed bad rate",
                chart_size=(18, 13),
                xy_axes_name=("Target label", "Observed N", "Rate"),
                major_gridlines=False,
            )

        if not rec_chart.empty:
            em.write_chart(
                worksheet=ws,
                df=rec_chart,
                x="label",
                y_list=["bad_rate_ins", "bad_rate_oos", "bad_rate_oot"],
                chart_type="line",
                y_axis_range=(0, None),
                y_num_format="0.0%",
                loc=(2, 15),
                title="Recommended split bad-rate comparison",
                chart_size=(18, 13),
                xy_axes_name=("Target label", "Bad rate"),
                major_gridlines=True,
            )

        if not oot_chart.empty:
            em.write_chart(
                worksheet=ws,
                df=oot_chart,
                x="oot_window_periods",
                y_list=[col for col in oot_chart.columns if col != "oot_window_periods"],
                chart_type="line",
                y_axis_range=(0, None),
                y_num_format="0.0%",
                loc=(23, 0),
                title="Average max bad-rate gap by OOT window",
                chart_size=(18, 13),
                xy_axes_name=("OOT window periods", "Avg max bad-rate gap"),
                major_gridlines=True,
            )

        if not seed_chart.empty:
            em.write_chart(
                worksheet=ws,
                df=seed_chart,
                x="seed",
                y_list=[col for col in seed_chart.columns if col != "seed"],
                chart_type="line",
                y_axis_range=(0, None),
                y_num_format="0.0%",
                loc=(23, 15),
                title="Seed stability: average max bad-rate gap",
                chart_size=(18, 13),
                xy_axes_name=("Random seed", "Avg max bad-rate gap"),
                major_gridlines=True,
            )

        if not segment_chart.empty:
            em.write_chart(
                worksheet=ws,
                df=segment_chart,
                x="segment",
                y_list=[col for col in segment_chart.columns if col != "segment"],
                chart_type="column",
                y_axis_range=(0, None),
                y_num_format="0.0%",
                loc=(44, 0),
                title="Population bad-rate snapshot",
                chart_size=(18, 13),
                xy_axes_name=("Segment", "Bad rate"),
                major_gridlines=True,
            )

        em.write_dataframe(
            ws,
            df=tables["split_recommendation"],
            loc=(44, 15),
            title="Recommended split details",
            index=False,
        )

    def _excel_sheet_name(self, name: str, used_sheet_names: set[str]) -> str:
        clean = str(name).replace("/", "_").replace("\\", "_").replace(":", "-")
        clean = clean.replace("[", "(").replace("]", ")").replace("*", "_").replace("?", "_")
        clean = clean.strip("' ") or "Sheet"
        base = clean[:31]
        candidate = base
        i = 2
        while candidate.lower() in used_sheet_names:
            suffix = f"_{i}"
            candidate = f"{base[:31 - len(suffix)]}{suffix}"
            i += 1
        used_sheet_names.add(candidate.lower())
        return candidate

    def _label_coverage_chart_data(self, df: pd.DataFrame) -> pd.DataFrame:
        cols = ["target_col", "n_observed", "observed_rate", "bad_rate"]
        if df.empty or not set(cols).issubset(df.columns):
            return pd.DataFrame(columns=["label", "n_observed", "observed_rate", "bad_rate"])
        out = df[cols].copy()
        out["label"] = out["target_col"].map(self._short_target_name)
        return out[["label", "n_observed", "observed_rate", "bad_rate"]]

    def _recommendation_chart_data(self, df: pd.DataFrame) -> pd.DataFrame:
        cols = ["target_col", "bad_rate_ins", "bad_rate_oos", "bad_rate_oot"]
        if df.empty or not set(cols).issubset(df.columns):
            return pd.DataFrame(columns=["label", "bad_rate_ins", "bad_rate_oos", "bad_rate_oot"])
        out = df[cols].copy()
        out["label"] = out["target_col"].map(self._short_target_name)
        return out[["label", "bad_rate_ins", "bad_rate_oos", "bad_rate_oot"]]

    def _oot_window_chart_data(self, df: pd.DataFrame) -> pd.DataFrame:
        if df.empty:
            return pd.DataFrame(columns=["oot_window_periods"])
        grouped = (
            df.groupby(["oot_window_periods", "target_col"], dropna=False)["max_abs_bad_rate_gap"]
            .mean()
            .reset_index()
        )
        grouped["target_col"] = grouped["target_col"].map(self._short_target_name)
        out = grouped.pivot(index="oot_window_periods", columns="target_col", values="max_abs_bad_rate_gap")
        return out.reset_index().sort_values("oot_window_periods")

    def _seed_stability_chart_data(self, df: pd.DataFrame) -> pd.DataFrame:
        if df.empty:
            return pd.DataFrame(columns=["seed"])
        grouped = (
            df.groupby(["seed", "target_col"], dropna=False)["max_abs_bad_rate_gap"]
            .mean()
            .reset_index()
        )
        grouped["target_col"] = grouped["target_col"].map(self._short_target_name)
        out = grouped.pivot(index="seed", columns="target_col", values="max_abs_bad_rate_gap")
        return out.reset_index().sort_values("seed")

    def _segment_chart_data(self, df: pd.DataFrame) -> pd.DataFrame:
        if df.empty:
            return pd.DataFrame(columns=["segment"])
        sub = df[
            (df["group_type"] == "population")
            & (df["group_cols"].isin(self.config.population_dims))
        ].copy()
        if sub.empty:
            return pd.DataFrame(columns=["segment"])

        top_segments = (
            sub.groupby("group_value", dropna=False)["n"].sum().sort_values(ascending=False).head(10).index
        )
        sub = sub[sub["group_value"].isin(top_segments)]
        sub["target_col"] = sub["target_col"].map(self._short_target_name)
        out = sub.pivot_table(
            index="group_value",
            columns="target_col",
            values="bad_rate",
            aggfunc="mean",
        )
        return out.reset_index().rename(columns={"group_value": "segment"})

    def _short_target_name(self, target_col: str) -> str:
        return target_col.replace("y_flag_dpd7_in_", "").upper()

estimate_split_count

estimate_split_count(data: DataFrame | None = None) -> dict[str, Any]

Return an upper-bound estimate of split candidates that _split_candidates would produce.

If data is provided, refine the OOT window count against the actual number of distinct oot_time_dim values available (some windows may collapse if history is short).

Parameters:

Name	Type	Description	Default
data	DataFrame or None	Optional data. When it contains the oot_time_dim column, the number of distinct non-null periods in it is added to the estimate.	None

Returns:

Type	Description
dict	n_targets, n_oot_windows, n_ins_oos_ratios, n_random_seeds and estimated_max_splits (their product). With data, also available_oot_periods (distinct oot_time_dim values over all rows) and windows_usable (number of oot_windows smaller than that count).

Notes

Nothing is computed from the labels, so the real number of candidates can be lower: a target without usable OOT and pool segments contributes none.

Source code in Modeling_Tool/Pipeline/sample_analysis.py

def estimate_split_count(self, data: pd.DataFrame | None = None) -> dict[str, Any]:
    """Return an upper-bound estimate of split candidates that _split_candidates would produce.

    If `data` is provided, refine the OOT window count against the actual number of
    distinct oot_time_dim values available (some windows may collapse if history is short).

    Parameters
    ----------
    data : pandas.DataFrame or None, default None
        Optional data. When it contains the ``oot_time_dim`` column, the number of distinct non-null periods in it is
        added to the estimate.

    Returns
    -------
    dict
        ``n_targets``, ``n_oot_windows``, ``n_ins_oos_ratios``, ``n_random_seeds`` and ``estimated_max_splits`` (their
        product). With ``data``, also ``available_oot_periods`` (distinct ``oot_time_dim`` values over all rows) and
        ``windows_usable`` (number of ``oot_windows`` smaller than that count).

    Notes
    -----
    Nothing is computed from the labels, so the real number of candidates can be lower: a target without usable
    OOT and pool segments contributes none.
    """
    cfg = self.config
    n_targets = len(cfg.target_cols)
    n_windows = len(cfg.oot_windows)
    n_ratios = len(cfg.ins_oos_ratios)
    n_seeds = len(list(cfg.random_seeds))
    estimated = n_targets * n_windows * n_ratios * n_seeds
    info: dict[str, Any] = {
        "n_targets": n_targets,
        "n_oot_windows": n_windows,
        "n_ins_oos_ratios": n_ratios,
        "n_random_seeds": n_seeds,
        "estimated_max_splits": estimated,
    }
    if data is not None and cfg.oot_time_dim in data.columns:
        n_periods = data[cfg.oot_time_dim].dropna().nunique()
        info["available_oot_periods"] = int(n_periods)
        info["windows_usable"] = sum(1 for w in cfg.oot_windows if int(w) < n_periods)
    return info

run

run(data: DataFrame) -> SampleAnalysisPipelineResult

Run the label coverage, segment, profile and split-stability analyses.

Parameters:

Name	Type	Description	Default
data	DataFrame	Sample to analyse. It must contain time_col, oot_time_dim, every column of target_cols, time_dims, population_dims and profile_cols, and id_col when materialize_split is True. The input frame is not modified.	required

Returns:

Type	Description
SampleAnalysisPipelineResult	The five summary tables, the paths of the files written and, with materialize_split=True, the row-level split and its audit artifact.

Raises:

Type	Description
KeyError	If a required column (or id_col for materialize_split) is missing from data.
ValueError	If materialize_split is True without id_col, if no recommendation could be produced, if id_col has duplicates among the mature rows of a target, or if the OOT or the INS/OOS pool segment is empty.
AssertionError	If the materialized INS, OOS and OOT segments overlap or do not cover all mature rows of a target.

Notes

With dry_run=True the input is validated, a warning with the estimated split count is logged and the analysis is skipped (see :meth:estimate_split_count). Existing output files are overwritten.

Source code in Modeling_Tool/Pipeline/sample_analysis.py

def run(self, data: pd.DataFrame) -> SampleAnalysisPipelineResult:
    """Run the label coverage, segment, profile and split-stability analyses.

    Parameters
    ----------
    data : pandas.DataFrame
        Sample to analyse. It must contain ``time_col``, ``oot_time_dim``, every column of ``target_cols``,
        ``time_dims``, ``population_dims`` and ``profile_cols``, and ``id_col`` when ``materialize_split`` is True.
        The input frame is not modified.

    Returns
    -------
    SampleAnalysisPipelineResult
        The five summary tables, the paths of the files written and, with ``materialize_split=True``, the row-level
        split and its audit artifact.

    Raises
    ------
    KeyError
        If a required column (or ``id_col`` for ``materialize_split``) is missing from ``data``.
    ValueError
        If ``materialize_split`` is True without ``id_col``, if no recommendation could be produced, if ``id_col``
        has duplicates among the mature rows of a target, or if the OOT or the INS/OOS pool segment is empty.
    AssertionError
        If the materialized INS, OOS and OOT segments overlap or do not cover all mature rows of a target.

    Notes
    -----
    With ``dry_run=True`` the input is validated, a warning with the estimated split count is logged and the
    analysis is skipped (see :meth:`estimate_split_count`). Existing output files are overwritten.
    """
    self._validate_input(data)
    if self.config.dry_run:
        info = self.estimate_split_count(data)
        _logger.warning(
            "SampleAnalysisPipeline dry_run=True: skipping execution. Estimated splits: %s",
            info,
        )
        empty = pd.DataFrame()
        return SampleAnalysisPipelineResult(
            label_coverage_summary=empty,
            segment_bad_rate_summary=empty,
            profile_summary=empty,
            split_candidate_summary=pd.DataFrame([info]),
            split_recommendation=empty,
            output_paths={},
        )
    work = data.copy()
    work[self.config.time_col] = pd.to_datetime(work[self.config.time_col])

    label_coverage = self._label_coverage(work)
    segment_bad_rate = self._segment_bad_rate(work)
    profile_summary = self._profile_summary(work)
    split_candidates = self._split_candidates(work)
    split_recommendation = self._recommend_splits(split_candidates)

    row_level_split = None
    split_artifact = None
    if self.config.materialize_split:
        if split_recommendation.empty:
            raise ValueError(
                "materialize_split=True but no split recommendation was produced "
                "(no target had usable OOT/pool segments)."
            )
        row_level_split, split_artifact = self._materialize_recommended_splits(
            work, split_recommendation,
        )

    output_paths = self._write_outputs(
        {
            "label_coverage_summary": label_coverage,
            "segment_bad_rate_summary": segment_bad_rate,
            "profile_summary": profile_summary,
            "split_candidate_summary": split_candidates,
            "split_recommendation": split_recommendation,
        }
    )
    if row_level_split is not None and self.config.persist_split_map:
        import json

        output_dir = Path(self.config.output_dir)
        output_dir.mkdir(parents=True, exist_ok=True)
        split_csv = output_dir / "row_level_split.csv"
        row_level_split.to_csv(split_csv, index=False)
        output_paths["row_level_split"] = str(split_csv.resolve())
        artifact_json = output_dir / "split_artifact.json"
        artifact_json.write_text(
            json.dumps(split_artifact, ensure_ascii=False, indent=2),
            encoding="utf-8",
        )
        output_paths["split_artifact"] = str(artifact_json.resolve())

    return SampleAnalysisPipelineResult(
        label_coverage_summary=label_coverage,
        segment_bad_rate_summary=segment_bad_rate,
        profile_summary=profile_summary,
        split_candidate_summary=split_candidates,
        split_recommendation=split_recommendation,
        output_paths=output_paths,
        row_level_split=row_level_split,
        split_artifact=split_artifact,
    )

Mock Sample Pipeline: mock_sample

mock_sample

MockSamplePipelineConfig dataclass

Configuration of :class:MockSamplePipeline.

Parameters:

Name	Type	Description	Default
n_samples	int	Number of full application rows generated before any filtering. Must be positive. With applied_sample=0 the output keeps only the approved rows, so it has about n_samples * approve_rate rows.	80000
applied_sample	int	1 returns all applications; 0 returns only approved applications (is_approved is 1 on every row). Any other value raises ValueError.	1
approve_rate	float	Share of applications flagged is_approved=1. Must lie strictly between 0 and 1. The number of approved rows is round(n_samples * approve_rate) (kept between 1 and n_samples - 1) and the lowest noisy-risk applications are approved.	0.25
num_online_scores	int	Number of simulated online bad-probability columns online_model_pb_1 ... online_model_pb_n (probabilities rounded to 6 decimals). Must be non-negative; 0 adds none.	5
y_flag_candidates	list of int	Performance windows in days. One label column y_flag_dpd7_in_{days}d is created per distinct value (sorted ascending). The list must be non-empty and contain positive values. A label is filled only for approved applications that have matured by observation_timestamp and is NaN otherwise.	[15, 30, 45]
num_features	int	Number of random feature columns feat_{business_type}_{idx}. Must be non-negative; 0 adds none and feature_metadata is then empty.	20
min_num_feature_business_type	int	Lower bound on the number of distinct business types covered by the features. It must be between 1 and min(num_features, 10), or exactly 0 when num_features is 0, otherwise ValueError is raised. Any valid value gives the same result because the generator always covers min(num_features, 10) types.	5
random_state	int	Seed of the NumPy random generator; the same configuration gives the same data (for a fixed observation_timestamp).	42
observation_timestamp	str or Timestamp or None	Observation date used to decide which labels have matured. Application times are drawn uniformly between application_months months before this date and the date itself. None uses today's date at midnight, so set it explicitly for reproducible output.	None
application_months	int	Length in months of the application window that ends at the observation date. Must be positive.	18
write_csv	bool	Whether to write the generated data to output_path as CSV (without the index).	False
output_path	str	CSV file path; parent directories are created. Used only when write_csv is True.	"output/mock_sample/mock_sample.csv"

Source code in Modeling_Tool/Pipeline/mock_sample.py

@dataclass
class MockSamplePipelineConfig:
    """Configuration of :class:`MockSamplePipeline`.

    Parameters
    ----------
    n_samples : int, default 80000
        Number of full application rows generated before any filtering. Must be positive. With ``applied_sample=0`` the
        output keeps only the approved rows, so it has about ``n_samples * approve_rate`` rows.
    applied_sample : int, default 1
        ``1`` returns all applications; ``0`` returns only approved applications (``is_approved`` is 1 on every
        row). Any other value raises ``ValueError``.
    approve_rate : float, default 0.25
        Share of applications flagged ``is_approved=1``. Must lie strictly between 0 and 1. The number of approved rows
        is ``round(n_samples * approve_rate)`` (kept between 1 and ``n_samples - 1``) and the lowest noisy-risk
        applications are approved.
    num_online_scores : int, default 5
        Number of simulated online bad-probability columns ``online_model_pb_1 ... online_model_pb_n`` (probabilities
        rounded to 6 decimals). Must be non-negative; ``0`` adds none.
    y_flag_candidates : list of int, default [15, 30, 45]
        Performance windows in days. One label column ``y_flag_dpd7_in_{days}d`` is created per distinct value
        (sorted ascending). The list must be non-empty and contain positive values. A label is filled only for approved
        applications that have matured by ``observation_timestamp`` and is NaN otherwise.
    num_features : int, default 20
        Number of random feature columns ``feat_{business_type}_{idx}``. Must be non-negative; ``0`` adds none and
        ``feature_metadata`` is then empty.
    min_num_feature_business_type : int, default 5
        Lower bound on the number of distinct business types covered by the features. It must be between 1 and
        ``min(num_features, 10)``, or exactly 0 when ``num_features`` is 0, otherwise ``ValueError`` is raised. Any
        valid value gives the same result because the generator always covers ``min(num_features, 10)`` types.
    random_state : int, default 42
        Seed of the NumPy random generator; the same configuration gives the same data (for a fixed
        ``observation_timestamp``).
    observation_timestamp : str or pandas.Timestamp or None, default None
        Observation date used to decide which labels have matured. Application times are drawn uniformly between
        ``application_months`` months before this date and the date itself. ``None`` uses today's date at midnight, so
        set it explicitly for reproducible output.
    application_months : int, default 18
        Length in months of the application window that ends at the observation date. Must be positive.
    write_csv : bool, default False
        Whether to write the generated data to ``output_path`` as CSV (without the index).
    output_path : str, default "output/mock_sample/mock_sample.csv"
        CSV file path; parent directories are created. Used only when ``write_csv`` is True.
    """

    n_samples: int = 80000
    applied_sample: int = 1
    approve_rate: float = 0.25
    num_online_scores: int = 5
    y_flag_candidates: list[int] = field(default_factory=lambda: [15, 30, 45])
    num_features: int = 20
    min_num_feature_business_type: int = 5
    random_state: int = 42
    observation_timestamp: str | pd.Timestamp | None = None
    application_months: int = 18
    write_csv: bool = False
    output_path: str = "output/mock_sample/mock_sample.csv"

MockSamplePipelineResult dataclass

Result returned by :meth:MockSamplePipeline.run.

Parameters:

Name	Type	Description	Default
data	DataFrame	Generated sample. Columns: flow_id, apply_timestamp, apply_week, apply_month, apply_quarter, is_approved, the feat_* features, the online_model_pb_* scores and the y_flag_dpd7_in_{days}d labels (1 = bad, 0 = good, NaN = not matured or not approved). Only approved rows remain when applied_sample=0.	required
summary	DataFrame	Two columns metric and value: row count, applied_sample, approval rate, number of online scores and features, and per label the observed count, observed rate and bad rate.	required
feature_metadata	DataFrame	One row per generated feature with columns feature, business_type and distribution (normal, count, binary or ratio); empty when num_features=0.	required
output_path	str or None	Absolute path of the CSV file that was written, or None when write_csv is False.	required

Source code in Modeling_Tool/Pipeline/mock_sample.py

@dataclass
class MockSamplePipelineResult:
    """Result returned by :meth:`MockSamplePipeline.run`.

    Parameters
    ----------
    data : pandas.DataFrame
        Generated sample. Columns: ``flow_id``, ``apply_timestamp``, ``apply_week``, ``apply_month``,
        ``apply_quarter``, ``is_approved``, the ``feat_*`` features, the ``online_model_pb_*`` scores and the
        ``y_flag_dpd7_in_{days}d`` labels (1 = bad, 0 = good, NaN = not matured or not approved). Only approved rows
        remain when ``applied_sample=0``.
    summary : pandas.DataFrame
        Two columns ``metric`` and ``value``: row count, ``applied_sample``, approval rate, number of online scores and
        features, and per label the observed count, observed rate and bad rate.
    feature_metadata : pandas.DataFrame
        One row per generated feature with columns ``feature``, ``business_type`` and ``distribution`` (``normal``,
        ``count``, ``binary`` or ``ratio``); empty when ``num_features=0``.
    output_path : str or None
        Absolute path of the CSV file that was written, or None when ``write_csv`` is False.
    """

    data: pd.DataFrame
    summary: pd.DataFrame
    feature_metadata: pd.DataFrame
    output_path: str | None

MockSamplePipeline

Generate synthetic application or approved samples for SMF demos.

The pipeline only produces data (and optionally a CSV file); it runs no modelling analysis and writes no Excel report.

Parameters:

Name	Type	Description	Default
config	MockSamplePipelineConfig or None	Pipeline configuration. None (or any falsy value) uses MockSamplePipelineConfig() with its defaults.	None

Attributes:

Name	Type	Description
config	MockSamplePipelineConfig	The configuration in use.

Source code in Modeling_Tool/Pipeline/mock_sample.py

class MockSamplePipeline:
    """Generate synthetic application or approved samples for SMF demos.

    The pipeline only produces data (and optionally a CSV file); it runs no modelling analysis and writes no Excel report.

    Parameters
    ----------
    config : MockSamplePipelineConfig or None, default None
        Pipeline configuration. ``None`` (or any falsy value) uses ``MockSamplePipelineConfig()`` with its defaults.

    Attributes
    ----------
    config : MockSamplePipelineConfig
        The configuration in use.
    """

    def __init__(self, config: MockSamplePipelineConfig | None = None):
        self.config = config or MockSamplePipelineConfig()

    def run(self) -> MockSamplePipelineResult:
        """Generate the sample described by the configuration.

        Returns
        -------
        MockSamplePipelineResult
            Generated ``data``, ``summary``, ``feature_metadata`` and the CSV ``output_path`` (None unless
            ``write_csv`` is True).

        Raises
        ------
        ValueError
            If the configuration is invalid (see :class:`MockSamplePipelineConfig`) or if the numbers of observed labels
            do not strictly decrease from the shortest to the longest ``y_flag_candidates`` window (for example with very
            few approved rows).

        Notes
        -----
        Labels exist only for approved applications; ``is_approved`` is 1 for them. The ``apply_week`` column uses the ISO
        week format ``%G-W%V``.
        """
        cfg = self.config
        self._validate_config()
        rng = np.random.default_rng(cfg.random_state)
        observation_ts = self._observation_timestamp()

        data = self._base_applications(rng, observation_ts)
        latent_risk = self._latent_risk(rng, data)
        data["is_approved"] = self._approval_flag(rng, latent_risk)
        feature_metadata = self._add_features(rng, data, latent_risk)
        self._add_online_scores(rng, data, latent_risk)
        self._add_y_flags(rng, data, latent_risk, observation_ts)

        if cfg.applied_sample == 0:
            data = data[data["is_approved"].eq(1)].reset_index(drop=True)

        summary = self._summary(data)
        output_path = self._write_csv(data)
        return MockSamplePipelineResult(
            data=data,
            summary=summary,
            feature_metadata=feature_metadata,
            output_path=output_path,
        )

    def _validate_config(self) -> None:
        cfg = self.config
        if cfg.n_samples <= 0:
            raise ValueError("n_samples must be positive")
        if cfg.applied_sample not in {0, 1}:
            raise ValueError("applied_sample must be 1 for full applications or 0 for approved samples")
        if not 0 < cfg.approve_rate < 1:
            raise ValueError("approve_rate must be between 0 and 1")
        if cfg.num_online_scores < 0:
            raise ValueError("num_online_scores must be non-negative")
        if cfg.num_features < 0:
            raise ValueError("num_features must be non-negative")
        if cfg.application_months <= 0:
            raise ValueError("application_months must be positive")
        if not cfg.y_flag_candidates:
            raise ValueError("y_flag_candidates must not be empty")
        if any(int(days) <= 0 for days in cfg.y_flag_candidates):
            raise ValueError("y_flag_candidates must contain positive day values")
        if cfg.num_features == 0:
            if cfg.min_num_feature_business_type != 0:
                raise ValueError("min_num_feature_business_type must be 0 when num_features is 0")
        elif not 1 <= cfg.min_num_feature_business_type <= min(cfg.num_features, len(_BUSINESS_TYPES)):
            raise ValueError(
                "min_num_feature_business_type must be between 1 and min(num_features, 10)"
            )

    def _observation_timestamp(self) -> pd.Timestamp:
        if self.config.observation_timestamp is None:
            return pd.Timestamp.today().normalize()
        return pd.Timestamp(self.config.observation_timestamp)

    def _base_applications(self, rng: np.random.Generator, observation_ts: pd.Timestamp) -> pd.DataFrame:
        cfg = self.config
        start_ts = observation_ts - pd.DateOffset(months=cfg.application_months)
        total_seconds = max(int((observation_ts - start_ts).total_seconds()), 1)
        offsets = rng.integers(0, total_seconds + 1, size=cfg.n_samples)
        apply_ts = start_ts + pd.to_timedelta(offsets, unit="s")
        data = pd.DataFrame(
            {
                "flow_id": [f"FLOW_{i:08d}" for i in range(cfg.n_samples)],
                "apply_timestamp": apply_ts,
            }
        )
        data["apply_week"] = data["apply_timestamp"].dt.strftime("%G-W%V")
        data["apply_month"] = data["apply_timestamp"].dt.strftime("%Y-%m")
        data["apply_quarter"] = data["apply_timestamp"].dt.to_period("Q").astype(str)
        return data

    def _latent_risk(self, rng: np.random.Generator, data: pd.DataFrame) -> np.ndarray:
        hour = data["apply_timestamp"].dt.hour.to_numpy()
        month = data["apply_timestamp"].dt.month.to_numpy()
        return (
            0.10 * np.sin(hour / 24 * 2 * np.pi)
            + 0.08 * np.cos(month / 12 * 2 * np.pi)
            + rng.normal(0, 1, size=len(data))
        )

    def _approval_flag(self, rng: np.random.Generator, latent_risk: np.ndarray) -> np.ndarray:
        n_approved = int(round(len(latent_risk) * self.config.approve_rate))
        n_approved = min(max(n_approved, 1), len(latent_risk) - 1)
        approval_score = latent_risk + rng.normal(0, 0.25, size=len(latent_risk))
        approved_idx = np.argsort(approval_score)[:n_approved]
        approved = np.zeros(len(latent_risk), dtype=int)
        approved[approved_idx] = 1
        return approved

    def _add_features(
        self,
        rng: np.random.Generator,
        data: pd.DataFrame,
        latent_risk: np.ndarray,
    ) -> pd.DataFrame:
        cfg = self.config
        rows = []
        if cfg.num_features == 0:
            return pd.DataFrame(columns=["feature", "business_type", "distribution"])

        n_types = min(max(cfg.min_num_feature_business_type, min(cfg.num_features, len(_BUSINESS_TYPES))), len(_BUSINESS_TYPES))
        chosen_types = list(rng.choice(_BUSINESS_TYPES, size=n_types, replace=False))
        assigned_types = chosen_types.copy()
        if cfg.num_features > n_types:
            assigned_types.extend(rng.choice(chosen_types, size=cfg.num_features - n_types, replace=True).tolist())
        rng.shuffle(assigned_types)

        type_counter: dict[str, int] = {}
        for business_type in assigned_types:
            type_counter[business_type] = type_counter.get(business_type, 0) + 1
            idx = type_counter[business_type]
            col = f"feat_{business_type}_{idx:02d}"
            distribution = self._feature_distribution(business_type)
            if distribution == "normal":
                data[col] = np.round(latent_risk * rng.uniform(0.1, 0.4) + rng.normal(0, 1, len(data)), 6)
            elif distribution == "count":
                rate = np.clip(np.exp(0.35 * latent_risk + rng.normal(0, 0.15, len(data))), 0.05, 20)
                data[col] = rng.poisson(rate)
            elif distribution == "binary":
                prob = 1 / (1 + np.exp(-(0.4 * latent_risk + rng.normal(0, 0.5, len(data)))))
                data[col] = (rng.random(len(data)) < prob).astype(int)
            else:
                raw = 1 / (1 + np.exp(-(0.5 * latent_risk + rng.normal(0, 0.7, len(data)))))
                data[col] = np.round(raw, 6)
            rows.append({"feature": col, "business_type": business_type, "distribution": distribution})
        return pd.DataFrame(rows)

    def _feature_distribution(self, business_type: str) -> str:
        if business_type in {"multi_loan", "query_count", "overdue_status"}:
            return "count"
        if business_type in {"basic_info", "address_data"}:
            return "binary"
        if business_type in {"telecom_data", "consumption_data"}:
            return "ratio"
        return "normal"

    def _add_online_scores(
        self,
        rng: np.random.Generator,
        data: pd.DataFrame,
        latent_risk: np.ndarray,
    ) -> None:
        for i in range(1, self.config.num_online_scores + 1):
            shift = rng.normal(0, 0.08)
            noise = rng.normal(0, 0.45, len(data))
            score = 1 / (1 + np.exp(-(latent_risk + shift + noise)))
            data[f"online_model_pb_{i}"] = np.round(score, 6)

    def _add_y_flags(
        self,
        rng: np.random.Generator,
        data: pd.DataFrame,
        latent_risk: np.ndarray,
        observation_ts: pd.Timestamp,
    ) -> None:
        approved = data["is_approved"].to_numpy() == 1
        base_bad = 1 / (1 + np.exp(-(latent_risk - 1.35)))
        for days in sorted({int(x) for x in self.config.y_flag_candidates}):
            col = f"y_flag_dpd7_in_{days}d"
            matured = approved & ((data["apply_timestamp"] + pd.to_timedelta(days, unit="D")) <= observation_ts).to_numpy()
            y = np.full(len(data), np.nan)
            multiplier = 0.65 + min(days, 180) / 180
            prob = np.clip(base_bad * multiplier, 0.003, 0.85)
            y[matured] = (rng.random(matured.sum()) < prob[matured]).astype(float)
            data[col] = y

        self._validate_y_maturity(data)

    def _validate_y_maturity(self, data: pd.DataFrame) -> None:
        cols = [f"y_flag_dpd7_in_{days}d" for days in sorted({int(x) for x in self.config.y_flag_candidates})]
        observed_counts = [int(data[col].notna().sum()) for col in cols]
        if observed_counts != sorted(observed_counts, reverse=True) or len(set(observed_counts)) != len(observed_counts):
            raise ValueError(
                "Observed y-label counts should strictly decrease for longer horizons: "
                f"{dict(zip(cols, observed_counts))}"
            )

    def _summary(self, data: pd.DataFrame) -> pd.DataFrame:
        rows = [
            {"metric": "n_rows", "value": len(data)},
            {"metric": "applied_sample", "value": self.config.applied_sample},
            {"metric": "approval_rate", "value": float(data["is_approved"].mean())},
            {"metric": "num_online_scores", "value": self.config.num_online_scores},
            {"metric": "num_features", "value": self.config.num_features},
        ]
        for days in sorted({int(x) for x in self.config.y_flag_candidates}):
            col = f"y_flag_dpd7_in_{days}d"
            observed = data[col].notna()
            rows.extend(
                [
                    {"metric": f"{col}_observed_n", "value": int(observed.sum())},
                    {"metric": f"{col}_observed_rate", "value": float(observed.mean())},
                    {"metric": f"{col}_bad_rate", "value": float(data[col].mean())},
                ]
            )
        return pd.DataFrame(rows)

    def _write_csv(self, data: pd.DataFrame) -> str | None:
        if not self.config.write_csv:
            return None
        path = Path(self.config.output_path)
        path.parent.mkdir(parents=True, exist_ok=True)
        data.to_csv(path, index=False)
        return str(path.resolve())

run

run() -> MockSamplePipelineResult

Generate the sample described by the configuration.

Returns:

Type	Description
MockSamplePipelineResult	Generated data, summary, feature_metadata and the CSV output_path (None unless write_csv is True).

Raises:

Type	Description
ValueError	If the configuration is invalid (see :class:MockSamplePipelineConfig) or if the numbers of observed labels do not strictly decrease from the shortest to the longest y_flag_candidates window (for example with very few approved rows).

Notes

Labels exist only for approved applications; is_approved is 1 for them. The apply_week column uses the ISO week format %G-W%V.

Source code in Modeling_Tool/Pipeline/mock_sample.py

def run(self) -> MockSamplePipelineResult:
    """Generate the sample described by the configuration.

    Returns
    -------
    MockSamplePipelineResult
        Generated ``data``, ``summary``, ``feature_metadata`` and the CSV ``output_path`` (None unless
        ``write_csv`` is True).

    Raises
    ------
    ValueError
        If the configuration is invalid (see :class:`MockSamplePipelineConfig`) or if the numbers of observed labels
        do not strictly decrease from the shortest to the longest ``y_flag_candidates`` window (for example with very
        few approved rows).

    Notes
    -----
    Labels exist only for approved applications; ``is_approved`` is 1 for them. The ``apply_week`` column uses the ISO
    week format ``%G-W%V``.
    """
    cfg = self.config
    self._validate_config()
    rng = np.random.default_rng(cfg.random_state)
    observation_ts = self._observation_timestamp()

    data = self._base_applications(rng, observation_ts)
    latent_risk = self._latent_risk(rng, data)
    data["is_approved"] = self._approval_flag(rng, latent_risk)
    feature_metadata = self._add_features(rng, data, latent_risk)
    self._add_online_scores(rng, data, latent_risk)
    self._add_y_flags(rng, data, latent_risk, observation_ts)

    if cfg.applied_sample == 0:
        data = data[data["is_approved"].eq(1)].reset_index(drop=True)

    summary = self._summary(data)
    output_path = self._write_csv(data)
    return MockSamplePipelineResult(
        data=data,
        summary=summary,
        feature_metadata=feature_metadata,
        output_path=output_path,
    )

Modeling from Validation Results: orchestrator

orchestrator

High-level orchestration helpers across SMF pipelines.

run_modeling_from_validation

run_modeling_from_validation(data: DataFrame, *, fvp_config: FeatureValidationPipelineConfig | None = None, cm_config: CreditModelPipelineConfig | None = None, selection_enabled: bool = True, reuse_screening_woe: bool = True) -> tuple[FeatureValidationPipelineResult, CreditModelPipelineResult]

Run feature validation/selection, then credit modeling on the same dataset.

The feature-validation result is turned into a FeatureScreeningArtifact, which is handed to the credit-model pipeline: the artifact's target_col and weight_col replace those of cm_config (a UserWarning names a column of cm_config that is replaced by another one), and its selected features replace cm_config.feature_cols when there are any.

Parameters:

Name	Type	Description	Default
data	DataFrame	The dataset both pipelines run on.	required
fvp_config	FeatureValidationPipelineConfig or None	Settings of the feature-validation pipeline; None uses the defaults.	None
cm_config	CreditModelPipelineConfig or None	Settings of the credit-model pipeline; None uses the defaults.	None
selection_enabled	bool	When True and fvp_config.selection_enabled is False, the feature-validation run uses a copy of the config with selection switched on. False leaves the setting of fvp_config as it is.	True
reuse_screening_woe	bool	Passed to the credit-model config as reuse_screening_woe: reuse the WOE engine fitted during screening.	True

Returns:

Type	Description
tuple	(FeatureValidationPipelineResult, CreditModelPipelineResult).

Source code in Modeling_Tool/Pipeline/orchestrator.py

def run_modeling_from_validation(
    data: pd.DataFrame,
    *,
    fvp_config: FeatureValidationPipelineConfig | None = None,
    cm_config: CreditModelPipelineConfig | None = None,
    selection_enabled: bool = True,
    reuse_screening_woe: bool = True,
) -> tuple[FeatureValidationPipelineResult, CreditModelPipelineResult]:
    """Run feature validation/selection, then credit modeling on the same dataset.

    The feature-validation result is turned into a ``FeatureScreeningArtifact``, which is handed to the credit-model
    pipeline: the artifact's ``target_col`` and ``weight_col`` replace those of ``cm_config`` (a ``UserWarning`` names a
    column of ``cm_config`` that is replaced by another one), and its selected features replace ``cm_config.feature_cols``
    when there are any.

    Parameters
    ----------
    data : pandas.DataFrame
        The dataset both pipelines run on.
    fvp_config : FeatureValidationPipelineConfig or None, default None
        Settings of the feature-validation pipeline; None uses the defaults.
    cm_config : CreditModelPipelineConfig or None, default None
        Settings of the credit-model pipeline; None uses the defaults.
    selection_enabled : bool, default True
        When True and ``fvp_config.selection_enabled`` is False, the feature-validation run uses a copy of the config with
        selection switched on. False leaves the setting of ``fvp_config`` as it is.
    reuse_screening_woe : bool, default True
        Passed to the credit-model config as ``reuse_screening_woe``: reuse the WOE engine fitted during screening.

    Returns
    -------
    tuple
        ``(FeatureValidationPipelineResult, CreditModelPipelineResult)``.
    """
    fvp_cfg = fvp_config or FeatureValidationPipelineConfig()
    if selection_enabled and not fvp_cfg.selection_enabled:
        fvp_cfg = replace(fvp_cfg, selection_enabled=True)

    fvp_result = FeatureValidationPipeline(fvp_cfg).run(data)
    artifact = FeatureScreeningArtifact.from_fvp_result(fvp_result)

    cm_cfg = cm_config or CreditModelPipelineConfig()
    if cm_config is not None:
        defaults = {item.name: item.default for item in fields(CreditModelPipelineConfig)}
        for name, kept in (("target_col", artifact.target_col), ("weight_col", artifact.weight_col)):
            given = getattr(cm_config, name)
            if given != defaults.get(name) and given != kept:
                # the validation's column replaces the one in cm_config; a weight column that silently disappears makes
                # the model unweighted and a different target trains another model
                warnings.warn(
                    f"run_modeling_from_validation: cm_config.{name}={given!r} is replaced by {kept!r}, the {name} of the "
                    "feature-validation run. Set the same column in both configs to silence this warning.",
                    UserWarning,
                    stacklevel=2,
                )
    cm_overrides: dict[str, Any] = {
        "screening_artifact": artifact,
        "reuse_screening_woe": reuse_screening_woe,
        "target_col": artifact.target_col,
        "weight_col": artifact.weight_col,
    }
    if artifact.selected_features:
        cm_overrides["feature_cols"] = list(artifact.selected_features)
    cm_cfg = replace(cm_cfg, **cm_overrides)

    cm_result = CreditModelPipeline(cm_cfg).run(data)
    return fvp_result, cm_result

Screening Artifact: screening_artifact

screening_artifact

Handoff contract between feature validation and credit modeling pipelines.

FeatureScreeningArtifact dataclass

Handoff contract between feature validation and credit modeling.

Parameters:

Name	Type	Description	Default
selected_features	list of str	Features that survived the screening.	required
selection_summary	dict	The screening summary in the layout of screen_result_to_summary.	required
woe_artifacts	dict or None	Reusable WOE engines and tables, or None when the screen did not keep any.	required
source	('fvp', 'cm', 'standalone')	Which pipeline produced the artifact.	"fvp"
target_col	str	Target the screening was run for.	required
weight_col	str or None	Sample-weight column used by the screening, or None.	required
config_snapshot	dict	Copy of the configuration that produced the screening.	empty dict
created_at	str or None	UTC creation time in ISO format.	None

Source code in Modeling_Tool/Pipeline/screening_artifact.py

@dataclass
class FeatureScreeningArtifact:
    """Handoff contract between feature validation and credit modeling.

    Parameters
    ----------
    selected_features : list of str
        Features that survived the screening.
    selection_summary : dict
        The screening summary in the layout of ``screen_result_to_summary``.
    woe_artifacts : dict or None
        Reusable WOE engines and tables, or None when the screen did not keep any.
    source : {"fvp", "cm", "standalone"}
        Which pipeline produced the artifact.
    target_col : str
        Target the screening was run for.
    weight_col : str or None
        Sample-weight column used by the screening, or None.
    config_snapshot : dict, default empty dict
        Copy of the configuration that produced the screening.
    created_at : str or None, default None
        UTC creation time in ISO format.
    """

    selected_features: list[str]
    selection_summary: dict[str, Any]
    woe_artifacts: dict[str, Any] | None
    source: Literal["fvp", "cm", "standalone"]
    target_col: str
    weight_col: str | None
    config_snapshot: dict[str, Any] = field(default_factory=dict)
    created_at: str | None = None

    @classmethod
    def from_screen_result(
        cls,
        result: WeightedScreenResult,
        *,
        initial_features: list[str],
        target_col: str,
        weight_col: str | None,
        woe_artifacts: dict[str, Any] | None,
        source: Literal["fvp", "cm", "standalone"],
        config_snapshot: dict[str, Any] | None = None,
    ) -> FeatureScreeningArtifact:
        """Build an artifact from the result of a weighted feature screen.

        Parameters
        ----------
        result : WeightedScreenResult
            Result of the screen.
        initial_features : list of str
            The features that entered the screen.
        target_col : str
            Target the screen was run for.
        weight_col : str or None
            Sample-weight column used by the screen.
        woe_artifacts : dict or None
            WOE artifacts to store; when None or empty, the engine that the screen itself fitted is wrapped (if there is one).
        source : {"fvp", "cm", "standalone"}
            Which pipeline produced the result.
        config_snapshot : dict or None, default None
            Copy of the configuration to store.

        Returns
        -------
        FeatureScreeningArtifact
        """
        if not woe_artifacts:
            # G00: reuse the engine the screen itself fitted, when available (an empty dict, which a run with
            # woe_enabled=False passes, carries nothing to reuse either).
            woe_artifacts = woe_artifacts_from_screen_result(result, target_col)
        return cls(
            selected_features=list(result.selected_features),
            selection_summary=screen_result_to_summary(result, initial_features),
            woe_artifacts=woe_artifacts,
            source=source,
            target_col=target_col,
            weight_col=weight_col,
            config_snapshot=dict(config_snapshot or {}),
            created_at=datetime.now(timezone.utc).isoformat(),
        )

    @classmethod
    def from_fvp_result(
        cls,
        result: Any,
        *,
        target_col: str | None = None,
        weight_col: str | None = None,
    ) -> FeatureScreeningArtifact:
        """Build an artifact from a feature-validation pipeline result.

        When the result already carries a ``screening_artifact`` it is returned unchanged. Otherwise the artifact is built
        from the result's ``selection_summary`` and ``config_snapshot``. If the pipeline ran with selection switched off
        and selected nothing, the configured ``new_feature_cols`` become the selected features.

        Parameters
        ----------
        result : FeatureValidationPipelineResult
            Result of a feature-validation run.
        target_col : str or None, default None
            Target column; when None it is read from the summary, then from the config snapshot (``target_col``, then the
            first of ``target_cols``).
        weight_col : str or None, default None
            Weight column. A ``weight_col`` entry in the config snapshot takes precedence over this argument.

        Returns
        -------
        FeatureScreeningArtifact

        Raises
        ------
        ValueError
            If no target column can be determined.
        """
        if getattr(result, "screening_artifact", None) is not None:
            return result.screening_artifact
        summary = dict(getattr(result, "selection_summary", {}) or {})
        summary_snapshot = dict(summary.get("config_snapshot", {}) or {})
        result_snapshot = dict(getattr(result, "config_snapshot", {}) or {})
        config_snapshot = {**summary_snapshot, **result_snapshot}
        resolved_target = target_col
        if not resolved_target:
            resolved_target = summary.get("target_col")
        if not resolved_target:
            resolved_target = config_snapshot.get("target_col")
        if not resolved_target:
            target_cols = list(config_snapshot.get("target_cols") or [])
            resolved_target = target_cols[0] if target_cols else None
        if not resolved_target:
            raise ValueError("target_col is required when building artifact from FVP result.")
        selected_features = list(getattr(result, "selected_features", []) or [])
        if not selected_features and config_snapshot.get("selection_enabled") is False:
            selected_features = list(config_snapshot.get("new_feature_cols") or [])
        if not summary and config_snapshot.get("selection_enabled") is False:
            summary = {
                "initial_features": list(selected_features),
                "final_features": list(selected_features),
                "target_col": str(resolved_target),
                "selection_skipped": True,
                "config_snapshot": config_snapshot,
            }
        resolved_weight_col = (
            config_snapshot["weight_col"]
            if "weight_col" in config_snapshot
            else weight_col
        )
        return cls(
            selected_features=selected_features,
            selection_summary=summary,
            woe_artifacts=getattr(result, "woe_artifacts", None),
            source="fvp",
            target_col=str(resolved_target),
            weight_col=resolved_weight_col,
            config_snapshot=config_snapshot,
            created_at=datetime.now(timezone.utc).isoformat(),
        )

    def validate_for_cm(self, *, target_col: str, weight_col: str | None = None) -> None:
        """Check that the artifact was built for the same target and weight columns as a credit-model run.

        Parameters
        ----------
        target_col : str
            Target column of the credit-model configuration.
        weight_col : str or None, default None
            Weight column of the credit-model configuration.

        Raises
        ------
        ValueError
            If the artifact's ``target_col`` or ``weight_col`` differs.
        """
        if self.target_col != target_col:
            raise ValueError(
                f"screening artifact target_col {self.target_col!r} does not match CM target_col {target_col!r}"
            )
        if self.weight_col != weight_col:
            raise ValueError(
                f"screening artifact weight_col {self.weight_col!r} does not match CM weight_col {weight_col!r}"
            )

    def to_dict(self) -> dict[str, Any]:
        """Return the artifact as a dictionary; top-level DataFrame values become lists of records."""
        payload = asdict(self)
        for key, value in list(payload.items()):
            if isinstance(value, pd.DataFrame):
                payload[key] = value.to_dict(orient="records")
        return payload

from_screen_result classmethod

from_screen_result(result: WeightedScreenResult, *, initial_features: list[str], target_col: str, weight_col: str | None, woe_artifacts: dict[str, Any] | None, source: Literal['fvp', 'cm', 'standalone'], config_snapshot: dict[str, Any] | None = None) -> FeatureScreeningArtifact

Build an artifact from the result of a weighted feature screen.

Parameters:

Name	Type	Description	Default
result	WeightedScreenResult	Result of the screen.	required
initial_features	list of str	The features that entered the screen.	required
target_col	str	Target the screen was run for.	required
weight_col	str or None	Sample-weight column used by the screen.	required
woe_artifacts	dict or None	WOE artifacts to store; when None or empty, the engine that the screen itself fitted is wrapped (if there is one).	required
source	('fvp', 'cm', 'standalone')	Which pipeline produced the result.	"fvp"
config_snapshot	dict or None	Copy of the configuration to store.	None

Returns:

Type	Description
FeatureScreeningArtifact

Source code in Modeling_Tool/Pipeline/screening_artifact.py

@classmethod
def from_screen_result(
    cls,
    result: WeightedScreenResult,
    *,
    initial_features: list[str],
    target_col: str,
    weight_col: str | None,
    woe_artifacts: dict[str, Any] | None,
    source: Literal["fvp", "cm", "standalone"],
    config_snapshot: dict[str, Any] | None = None,
) -> FeatureScreeningArtifact:
    """Build an artifact from the result of a weighted feature screen.

    Parameters
    ----------
    result : WeightedScreenResult
        Result of the screen.
    initial_features : list of str
        The features that entered the screen.
    target_col : str
        Target the screen was run for.
    weight_col : str or None
        Sample-weight column used by the screen.
    woe_artifacts : dict or None
        WOE artifacts to store; when None or empty, the engine that the screen itself fitted is wrapped (if there is one).
    source : {"fvp", "cm", "standalone"}
        Which pipeline produced the result.
    config_snapshot : dict or None, default None
        Copy of the configuration to store.

    Returns
    -------
    FeatureScreeningArtifact
    """
    if not woe_artifacts:
        # G00: reuse the engine the screen itself fitted, when available (an empty dict, which a run with
        # woe_enabled=False passes, carries nothing to reuse either).
        woe_artifacts = woe_artifacts_from_screen_result(result, target_col)
    return cls(
        selected_features=list(result.selected_features),
        selection_summary=screen_result_to_summary(result, initial_features),
        woe_artifacts=woe_artifacts,
        source=source,
        target_col=target_col,
        weight_col=weight_col,
        config_snapshot=dict(config_snapshot or {}),
        created_at=datetime.now(timezone.utc).isoformat(),
    )

from_fvp_result classmethod

from_fvp_result(result: Any, *, target_col: str | None = None, weight_col: str | None = None) -> FeatureScreeningArtifact

Build an artifact from a feature-validation pipeline result.

When the result already carries a screening_artifact it is returned unchanged. Otherwise the artifact is built from the result's selection_summary and config_snapshot. If the pipeline ran with selection switched off and selected nothing, the configured new_feature_cols become the selected features.

Parameters:

Name	Type	Description	Default
result	FeatureValidationPipelineResult	Result of a feature-validation run.	required
target_col	str or None	Target column; when None it is read from the summary, then from the config snapshot (target_col, then the first of target_cols).	None
weight_col	str or None	Weight column. A weight_col entry in the config snapshot takes precedence over this argument.	None

Returns:

Type	Description
FeatureScreeningArtifact

Raises:

Type	Description
ValueError	If no target column can be determined.

Source code in Modeling_Tool/Pipeline/screening_artifact.py

@classmethod
def from_fvp_result(
    cls,
    result: Any,
    *,
    target_col: str | None = None,
    weight_col: str | None = None,
) -> FeatureScreeningArtifact:
    """Build an artifact from a feature-validation pipeline result.

    When the result already carries a ``screening_artifact`` it is returned unchanged. Otherwise the artifact is built
    from the result's ``selection_summary`` and ``config_snapshot``. If the pipeline ran with selection switched off
    and selected nothing, the configured ``new_feature_cols`` become the selected features.

    Parameters
    ----------
    result : FeatureValidationPipelineResult
        Result of a feature-validation run.
    target_col : str or None, default None
        Target column; when None it is read from the summary, then from the config snapshot (``target_col``, then the
        first of ``target_cols``).
    weight_col : str or None, default None
        Weight column. A ``weight_col`` entry in the config snapshot takes precedence over this argument.

    Returns
    -------
    FeatureScreeningArtifact

    Raises
    ------
    ValueError
        If no target column can be determined.
    """
    if getattr(result, "screening_artifact", None) is not None:
        return result.screening_artifact
    summary = dict(getattr(result, "selection_summary", {}) or {})
    summary_snapshot = dict(summary.get("config_snapshot", {}) or {})
    result_snapshot = dict(getattr(result, "config_snapshot", {}) or {})
    config_snapshot = {**summary_snapshot, **result_snapshot}
    resolved_target = target_col
    if not resolved_target:
        resolved_target = summary.get("target_col")
    if not resolved_target:
        resolved_target = config_snapshot.get("target_col")
    if not resolved_target:
        target_cols = list(config_snapshot.get("target_cols") or [])
        resolved_target = target_cols[0] if target_cols else None
    if not resolved_target:
        raise ValueError("target_col is required when building artifact from FVP result.")
    selected_features = list(getattr(result, "selected_features", []) or [])
    if not selected_features and config_snapshot.get("selection_enabled") is False:
        selected_features = list(config_snapshot.get("new_feature_cols") or [])
    if not summary and config_snapshot.get("selection_enabled") is False:
        summary = {
            "initial_features": list(selected_features),
            "final_features": list(selected_features),
            "target_col": str(resolved_target),
            "selection_skipped": True,
            "config_snapshot": config_snapshot,
        }
    resolved_weight_col = (
        config_snapshot["weight_col"]
        if "weight_col" in config_snapshot
        else weight_col
    )
    return cls(
        selected_features=selected_features,
        selection_summary=summary,
        woe_artifacts=getattr(result, "woe_artifacts", None),
        source="fvp",
        target_col=str(resolved_target),
        weight_col=resolved_weight_col,
        config_snapshot=config_snapshot,
        created_at=datetime.now(timezone.utc).isoformat(),
    )

validate_for_cm

validate_for_cm(*, target_col: str, weight_col: str | None = None) -> None

Check that the artifact was built for the same target and weight columns as a credit-model run.

Parameters:

Name	Type	Description	Default
target_col	str	Target column of the credit-model configuration.	required
weight_col	str or None	Weight column of the credit-model configuration.	None

Raises:

Type	Description
ValueError	If the artifact's target_col or weight_col differs.

Source code in Modeling_Tool/Pipeline/screening_artifact.py

def validate_for_cm(self, *, target_col: str, weight_col: str | None = None) -> None:
    """Check that the artifact was built for the same target and weight columns as a credit-model run.

    Parameters
    ----------
    target_col : str
        Target column of the credit-model configuration.
    weight_col : str or None, default None
        Weight column of the credit-model configuration.

    Raises
    ------
    ValueError
        If the artifact's ``target_col`` or ``weight_col`` differs.
    """
    if self.target_col != target_col:
        raise ValueError(
            f"screening artifact target_col {self.target_col!r} does not match CM target_col {target_col!r}"
        )
    if self.weight_col != weight_col:
        raise ValueError(
            f"screening artifact weight_col {self.weight_col!r} does not match CM weight_col {weight_col!r}"
        )

to_dict

to_dict() -> dict[str, Any]

Return the artifact as a dictionary; top-level DataFrame values become lists of records.

Source code in Modeling_Tool/Pipeline/screening_artifact.py

def to_dict(self) -> dict[str, Any]:
    """Return the artifact as a dictionary; top-level DataFrame values become lists of records."""
    payload = asdict(self)
    for key, value in list(payload.items()):
        if isinstance(value, pd.DataFrame):
            payload[key] = value.to_dict(orient="records")
    return payload

iv_table_for_summary

iv_table_for_summary(iv_table: DataFrame) -> DataFrame

The iv table of a feature_selection_summary: iv_table with iv_weighted named iv.

Since 0.9.2 iv_table carries iv itself (equal to iv_weighted), so iv_weighted is dropped instead of renamed and the summary keeps one iv column, followed by iv_raw, iv_sv_pure and n_sv_pure.

Source code in Modeling_Tool/Pipeline/screening_artifact.py

def iv_table_for_summary(iv_table: pd.DataFrame) -> pd.DataFrame:
    """The ``iv`` table of a ``feature_selection_summary``: ``iv_table`` with ``iv_weighted`` named ``iv``.

    Since 0.9.2 ``iv_table`` carries ``iv`` itself (equal to ``iv_weighted``), so ``iv_weighted`` is dropped instead of
    renamed and the summary keeps one ``iv`` column, followed by ``iv_raw``, ``iv_sv_pure`` and ``n_sv_pure``.
    """
    iv = iv_table.copy()
    if "iv_weighted" in iv.columns:
        if "iv" in iv.columns:
            iv = iv.drop(columns=["iv_weighted"])
        else:
            iv = iv.rename(columns={"iv_weighted": "iv"})
    leading = [c for c in ("var", "iv") if c in iv.columns]
    return iv[leading + [c for c in iv.columns if c not in leading]]

screen_result_to_summary

screen_result_to_summary(result: WeightedScreenResult, initial_features: list[str]) -> dict[str, Any]

Convert a screening result into CM-compatible feature_selection_summary.

Parameters:

Name	Type	Description	Default
result	WeightedScreenResult	Result of a weighted feature screen.	required
initial_features	list of str	The features that entered the screen.	required

Returns:

Type

Description

dict

Always initial_features, corr_features, screen_summary and final_features (the last two lists are the selected features). Present only when not empty: psi (with a psi column taken from psi_ins_oos or psi_max), iv (the iv_table with iv_weighted as iv, see iv_table_for_summary), corr_dropped, missing_rate, missing_rate_dropped, dropped_detail and stage_tables (which holds iv_sv_pure_bins when the IV stage found class-pure special-value bins).

Source code in Modeling_Tool/Pipeline/screening_artifact.py

def screen_result_to_summary(
    result: WeightedScreenResult,
    initial_features: list[str],
) -> dict[str, Any]:
    """Convert a screening result into CM-compatible ``feature_selection_summary``.

    Parameters
    ----------
    result : WeightedScreenResult
        Result of a weighted feature screen.
    initial_features : list of str
        The features that entered the screen.

    Returns
    -------
    dict
        Always ``initial_features``, ``corr_features``, ``screen_summary`` and ``final_features`` (the last two lists are
        the selected features). Present only when not empty: ``psi`` (with a ``psi`` column taken from ``psi_ins_oos`` or
        ``psi_max``), ``iv`` (the ``iv_table`` with ``iv_weighted`` as ``iv``, see ``iv_table_for_summary``),
        ``corr_dropped``, ``missing_rate``, ``missing_rate_dropped``, ``dropped_detail`` and ``stage_tables`` (which
        holds ``iv_sv_pure_bins`` when the IV stage found class-pure special-value bins).
    """
    summary: dict[str, Any] = {"initial_features": list(initial_features)}
    if not result.psi_table.empty:
        psi = result.psi_table.copy()
        if "psi_ins_oos" in psi.columns:
            psi["psi"] = psi["psi_ins_oos"]
        elif "psi_max" in psi.columns:
            psi["psi"] = psi["psi_max"]
        summary["psi"] = psi
    if not result.iv_table.empty:
        summary["iv"] = iv_table_for_summary(result.iv_table)
    if not result.corr_dropped.empty:
        summary["corr_dropped"] = result.corr_dropped
    if not result.missing_rate_table.empty:
        summary["missing_rate"] = result.missing_rate_table
    if not result.missing_rate_dropped.empty:
        summary["missing_rate_dropped"] = result.missing_rate_dropped
    summary["corr_features"] = list(result.selected_features)
    summary["screen_summary"] = result.summary
    summary["final_features"] = list(result.selected_features)
    dropped_detail = getattr(result, "dropped_detail", None)
    if dropped_detail is not None and len(dropped_detail):
        summary["dropped_detail"] = dropped_detail
    stage_tables = getattr(result, "stage_tables", None)
    if stage_tables:
        summary["stage_tables"] = dict(stage_tables)
    return summary

woe_artifacts_from_screen_result

woe_artifacts_from_screen_result(result: WeightedScreenResult, target_col: str) -> dict[str, Any] | None

Wrap a screen-fitted WOE engine into the CM reuse contract (G00).

The returned dict matches what CreditModelPipeline._reuse_screening_woe consumes: by_target[target] = {engine, adapter, features} plus a top-level woe_table and the engine_meta of the screen.

Parameters:

Name	Type	Description	Default
result	WeightedScreenResult	Result of a weighted feature screen; its woe_engine and woe_engine_meta attributes are used.	required
target_col	str	Target the engine was fitted for; it becomes the key of by_target.	required

Returns:

Type	Description
dict or None	The reuse payload, or None when the screen did not attach a reusable engine (for example WOE_Master screens attach table-only metadata).

Source code in Modeling_Tool/Pipeline/screening_artifact.py

def woe_artifacts_from_screen_result(
    result: WeightedScreenResult,
    target_col: str,
) -> dict[str, Any] | None:
    """Wrap a screen-fitted WOE engine into the CM reuse contract (G00).

    The returned dict matches what CreditModelPipeline._reuse_screening_woe
    consumes: ``by_target[target] = {engine, adapter, features}`` plus a
    top-level ``woe_table`` and the ``engine_meta`` of the screen.

    Parameters
    ----------
    result : WeightedScreenResult
        Result of a weighted feature screen; its ``woe_engine`` and ``woe_engine_meta`` attributes are used.
    target_col : str
        Target the engine was fitted for; it becomes the key of ``by_target``.

    Returns
    -------
    dict or None
        The reuse payload, or None when the screen did not attach a reusable engine (for example WOE_Master screens
        attach table-only metadata).
    """
    engine = getattr(result, "woe_engine", None)
    meta = dict(getattr(result, "woe_engine_meta", {}) or {})
    if engine is None:
        return None
    from Modeling_Tool.WOE.WOE_Adapter import as_woe_engine

    adapter = as_woe_engine(engine)
    features = list(meta.get("fit_features") or result.selected_features)
    try:
        woe_table = adapter.get_woe_table(features)
    except Exception:
        woe_table = pd.DataFrame()
    return {
        "by_target": {
            target_col: {
                "engine": engine,
                "adapter": adapter,
                "features": features,
            }
        },
        "woe_table": woe_table,
        "engine_meta": meta,
    }

Config Schema and Registry: field_meta

Helpers for building a configuration GUI or loading configs from YAML. See Pipeline GUI Schema.

field_meta

FieldMeta dataclass

Human-readable metadata for a Pipeline Config field.

The metadata is intentionally dependency-free so that GUI applications can introspect SMF configs without importing Streamlit or any frontend package.

Parameters:

Name	Type	Description	Default
label	str	Short display name of the field.	required
description	str	Help text for the field. For fields without curated text it repeats label.	required
widget	('text', 'number', 'select', 'multiselect', 'toggle', 'slider', 'textarea', 'json', 'hidden')	Suggested form control. "hidden" marks code-only objects (DataFrames, callables, connections).	"text"
options	list or None	Allowed values for the "select" and "multiselect" widgets.	None
min_val	float or None	Lower bound hint for numeric controls such as sliders.	None
max_val	float or None	Upper bound hint for numeric controls such as sliders.	None
step	float or None	Step hint for numeric controls such as sliders.	None
required	bool	Whether a form should ask for this field. It comes from a fixed list per Pipeline and does not mean that the dataclass field has no default.	False
group	str	Name of the suggested form section.	"Basic settings"
depends_on	dict or None	Display condition {field_name: value}: show this field only while the other field has that value.	None
since_version	str or None	Version that introduced the field, when recorded.	None
is_dict_subkey	bool	True when the metadata describes a key inside a dictionary-valued config field (see nested_fields).	False
parent_field	str or None	Name of the dictionary-valued field that owns this sub-key.	None
yaml_serializable	bool	False for objects that cannot be written to YAML/JSON (DataFrames, callables, connections, artifacts).	True
gui_editable	bool	False for fields that a GUI should not render (the same code-only objects).	True
advanced	bool	Hint to collapse the field under an "advanced" section.	False
expert_only	bool	Hint that the field is meant for expert users only.	False
placeholder	str or None	Example text for an empty input control.	None
nested_fields	list of FieldMeta	Metadata of the known keys of a dictionary-valued field (for example test_size in split_config).	empty list

Source code in Modeling_Tool/Pipeline/field_meta.py

@dataclass
class FieldMeta:
    """Human-readable metadata for a Pipeline Config field.

    The metadata is intentionally dependency-free so that GUI applications can
    introspect SMF configs without importing Streamlit or any frontend package.

    Parameters
    ----------
    label : str
        Short display name of the field.
    description : str
        Help text for the field. For fields without curated text it repeats ``label``.
    widget : {"text", "number", "select", "multiselect", "toggle", "slider", "textarea", "json", "hidden"}, default "text"
        Suggested form control. ``"hidden"`` marks code-only objects (DataFrames, callables, connections).
    options : list or None, default None
        Allowed values for the ``"select"`` and ``"multiselect"`` widgets.
    min_val : float or None, default None
        Lower bound hint for numeric controls such as sliders.
    max_val : float or None, default None
        Upper bound hint for numeric controls such as sliders.
    step : float or None, default None
        Step hint for numeric controls such as sliders.
    required : bool, default False
        Whether a form should ask for this field. It comes from a fixed list per Pipeline and does not mean that the
        dataclass field has no default.
    group : str, default "Basic settings"
        Name of the suggested form section.
    depends_on : dict or None, default None
        Display condition ``{field_name: value}``: show this field only while the other field has that value.
    since_version : str or None, default None
        Version that introduced the field, when recorded.
    is_dict_subkey : bool, default False
        True when the metadata describes a key inside a dictionary-valued config field (see ``nested_fields``).
    parent_field : str or None, default None
        Name of the dictionary-valued field that owns this sub-key.
    yaml_serializable : bool, default True
        False for objects that cannot be written to YAML/JSON (DataFrames, callables, connections, artifacts).
    gui_editable : bool, default True
        False for fields that a GUI should not render (the same code-only objects).
    advanced : bool, default False
        Hint to collapse the field under an "advanced" section.
    expert_only : bool, default False
        Hint that the field is meant for expert users only.
    placeholder : str or None, default None
        Example text for an empty input control.
    nested_fields : list of FieldMeta, default empty list
        Metadata of the known keys of a dictionary-valued field (for example ``test_size`` in ``split_config``).
    """

    label: str
    description: str
    widget: WidgetType = "text"
    options: list[Any] | None = None
    min_val: float | None = None
    max_val: float | None = None
    step: float | None = None
    required: bool = False
    group: str = "Basic settings"
    depends_on: dict[str, Any] | None = None
    since_version: str | None = None
    is_dict_subkey: bool = False
    parent_field: str | None = None
    yaml_serializable: bool = True
    gui_editable: bool = True
    advanced: bool = False
    expert_only: bool = False
    placeholder: str | None = None
    nested_fields: list["FieldMeta"] = dataclass_field(default_factory=list)

    def to_dict(self) -> dict[str, Any]:
        """Return the metadata as a plain dictionary (nested fields become dictionaries too)."""
        return asdict(self)

to_dict

to_dict() -> dict[str, Any]

Return the metadata as a plain dictionary (nested fields become dictionaries too).

Source code in Modeling_Tool/Pipeline/field_meta.py

def to_dict(self) -> dict[str, Any]:
    """Return the metadata as a plain dictionary (nested fields become dictionaries too)."""
    return asdict(self)

PipelineRegistryEntry dataclass

One high-level Pipeline in the registry: card text, classes and how to call it.

Parameters:

Name	Type	Description	Default
key	str	Registry key, for example "credit_model".	required
display_name	str	Title for a Pipeline card.	required
description	str	What the Pipeline does.	required
use_case	str	When to use it.	required
audience	list of str	Reader roles the Pipeline is meant for.	required
pipeline_class	type	The Pipeline class.	required
config_class	type	The Config dataclass the Pipeline takes.	required
result_class	type	The result dataclass that run returns.	required
module_path	str	Dotted path of the module that defines the classes.	required
import_path	str	Package that exports the classes; generated code imports from here.	"Modeling_Tool.Pipeline"
run_requires_data	bool	False when run() takes no DataFrame.	True
run_method	str	The call to show on a card.	"run(data=your_dataframe)"
result_attrs	list of str	Headline attributes of the result object.	empty list

Source code in Modeling_Tool/Pipeline/field_meta.py

@dataclass
class PipelineRegistryEntry:
    """One high-level Pipeline in the registry: card text, classes and how to call it.

    Parameters
    ----------
    key : str
        Registry key, for example ``"credit_model"``.
    display_name : str
        Title for a Pipeline card.
    description : str
        What the Pipeline does.
    use_case : str
        When to use it.
    audience : list of str
        Reader roles the Pipeline is meant for.
    pipeline_class : type
        The Pipeline class.
    config_class : type
        The Config dataclass the Pipeline takes.
    result_class : type
        The result dataclass that ``run`` returns.
    module_path : str
        Dotted path of the module that defines the classes.
    import_path : str, default "Modeling_Tool.Pipeline"
        Package that exports the classes; generated code imports from here.
    run_requires_data : bool, default True
        False when ``run()`` takes no DataFrame.
    run_method : str, default "run(data=your_dataframe)"
        The call to show on a card.
    result_attrs : list of str, default empty list
        Headline attributes of the result object.
    """

    key: str
    display_name: str
    description: str
    use_case: str
    audience: list[str]
    pipeline_class: type
    config_class: type
    result_class: type
    module_path: str
    import_path: str = "Modeling_Tool.Pipeline"
    run_requires_data: bool = True
    run_method: str = "run(data=your_dataframe)"
    result_attrs: list[str] = dataclass_field(default_factory=list)

    def to_dict(self, include_classes: bool = False) -> dict[str, Any]:
        """Return the entry as a plain dictionary.

        Parameters
        ----------
        include_classes : bool, default False
            When True, also include the ``pipeline_class``, ``config_class`` and ``result_class`` objects. Otherwise
            only their class names are present and the dictionary is JSON/YAML-friendly.

        Returns
        -------
        dict
            The entry fields; the classes appear as ``pipeline_class_name``, ``config_class_name`` and
            ``result_class_name``.
        """
        payload = {
            "key": self.key,
            "display_name": self.display_name,
            "description": self.description,
            "use_case": self.use_case,
            "audience": list(self.audience),
            "pipeline_class_name": self.pipeline_class.__name__,
            "config_class_name": self.config_class.__name__,
            "result_class_name": self.result_class.__name__,
            "module_path": self.module_path,
            "import_path": self.import_path,
            "run_requires_data": self.run_requires_data,
            "run_method": self.run_method,
            "result_attrs": list(self.result_attrs),
        }
        if include_classes:
            payload.update(
                {
                    "pipeline_class": self.pipeline_class,
                    "config_class": self.config_class,
                    "result_class": self.result_class,
                }
            )
        return payload

to_dict

to_dict(include_classes: bool = False) -> dict[str, Any]

Return the entry as a plain dictionary.

Parameters:

Name	Type	Description	Default
include_classes	bool	When True, also include the pipeline_class, config_class and result_class objects. Otherwise only their class names are present and the dictionary is JSON/YAML-friendly.	False

Returns:

Type	Description
dict	The entry fields; the classes appear as pipeline_class_name, config_class_name and result_class_name.

Source code in Modeling_Tool/Pipeline/field_meta.py

def to_dict(self, include_classes: bool = False) -> dict[str, Any]:
    """Return the entry as a plain dictionary.

    Parameters
    ----------
    include_classes : bool, default False
        When True, also include the ``pipeline_class``, ``config_class`` and ``result_class`` objects. Otherwise
        only their class names are present and the dictionary is JSON/YAML-friendly.

    Returns
    -------
    dict
        The entry fields; the classes appear as ``pipeline_class_name``, ``config_class_name`` and
        ``result_class_name``.
    """
    payload = {
        "key": self.key,
        "display_name": self.display_name,
        "description": self.description,
        "use_case": self.use_case,
        "audience": list(self.audience),
        "pipeline_class_name": self.pipeline_class.__name__,
        "config_class_name": self.config_class.__name__,
        "result_class_name": self.result_class.__name__,
        "module_path": self.module_path,
        "import_path": self.import_path,
        "run_requires_data": self.run_requires_data,
        "run_method": self.run_method,
        "result_attrs": list(self.result_attrs),
    }
    if include_classes:
        payload.update(
            {
                "pipeline_class": self.pipeline_class,
                "config_class": self.config_class,
                "result_class": self.result_class,
            }
        )
    return payload

get_pipeline_registry

get_pipeline_registry(include_classes: bool = True) -> dict[str, Any]

Return the public high-level Pipeline registry.

Parameters:

Name	Type	Description	Default
include_classes	bool	When True, each entry includes actual class objects. Set False for a JSON/YAML-friendly registry payload.	True

Returns:

Type	Description
dict	Registry key to entry dictionary (see PipelineRegistryEntry.to_dict).

Source code in Modeling_Tool/Pipeline/field_meta.py

def get_pipeline_registry(include_classes: bool = True) -> dict[str, Any]:
    """Return the public high-level Pipeline registry.

    Parameters
    ----------
    include_classes : bool, default True
        When True, each entry includes actual class objects. Set False for a
        JSON/YAML-friendly registry payload.

    Returns
    -------
    dict
        Registry key to entry dictionary (see ``PipelineRegistryEntry.to_dict``).
    """

    return {
        key: entry.to_dict(include_classes=include_classes)
        for key, entry in PIPELINE_REGISTRY.items()
    }

get_pipeline_registry_schema

get_pipeline_registry_schema() -> dict[str, Any]

Return a JSON-serializable registry summary without class objects.

Returns:

Type	Description
dict	Registry key to entry dictionary, with class names instead of class objects.

Source code in Modeling_Tool/Pipeline/field_meta.py

def get_pipeline_registry_schema() -> dict[str, Any]:
    """Return a JSON-serializable registry summary without class objects.

    Returns
    -------
    dict
        Registry key to entry dictionary, with class names instead of class objects.
    """

    return get_pipeline_registry(include_classes=False)

get_config_field_meta

get_config_field_meta(config_class_or_key: str | type | Any) -> dict[str, FieldMeta]

Return FieldMeta objects keyed by config field name.

Parameters:

Name	Type	Description	Default
config_class_or_key	(str, type or object)	A registry key (for example "credit_model"), the name of a Config, Pipeline or result class, one of those classes, or an instance of one of them.	required

Returns:

Type	Description
dict	Field name to FieldMeta. The result is a deep copy, so editing it does not change the registry.

Raises:

Type	Description
KeyError	If the reference does not match a registered Pipeline.

Source code in Modeling_Tool/Pipeline/field_meta.py

def get_config_field_meta(config_class_or_key: str | type | Any) -> dict[str, FieldMeta]:
    """Return FieldMeta objects keyed by config field name.

    Parameters
    ----------
    config_class_or_key : str, type or object
        A registry key (for example ``"credit_model"``), the name of a Config, Pipeline or result class, one of those
        classes, or an instance of one of them.

    Returns
    -------
    dict
        Field name to ``FieldMeta``. The result is a deep copy, so editing it does not change the registry.

    Raises
    ------
    KeyError
        If the reference does not match a registered Pipeline.
    """

    entry = _resolve_entry(config_class_or_key)
    meta = getattr(entry.config_class, "__smf_field_meta__", None)
    if meta is None:
        meta = _build_class_field_meta(entry.config_class)
    return copy.deepcopy(meta)

extract_config_schema

extract_config_schema(config_class_or_key: str | type | Any) -> dict[str, Any]

Extract a GUI-friendly schema for one Pipeline Config class.

Parameters:

Name	Type	Description	Default
config_class_or_key	(str, type or object)	A registry key (for example "credit_model"), the name of a Config, Pipeline or result class, one of those classes, or an instance of one of them.	required

Returns:

Type	Description
dict	The registry card of the Pipeline (PipelineRegistryEntry.to_dict(include_classes=False)) plus "fields", a list with one dictionary per config field: the FieldMeta keys and name, type, default and has_default.

Raises:

Type	Description
KeyError	If the reference does not match a registered Pipeline.

Source code in Modeling_Tool/Pipeline/field_meta.py

def extract_config_schema(config_class_or_key: str | type | Any) -> dict[str, Any]:
    """Extract a GUI-friendly schema for one Pipeline Config class.

    Parameters
    ----------
    config_class_or_key : str, type or object
        A registry key (for example ``"credit_model"``), the name of a Config, Pipeline or result class, one of those
        classes, or an instance of one of them.

    Returns
    -------
    dict
        The registry card of the Pipeline (``PipelineRegistryEntry.to_dict(include_classes=False)``) plus ``"fields"``,
        a list with one dictionary per config field: the ``FieldMeta`` keys and ``name``, ``type``, ``default`` and
        ``has_default``.

    Raises
    ------
    KeyError
        If the reference does not match a registered Pipeline.
    """

    entry = _resolve_entry(config_class_or_key)
    config_class = entry.config_class
    defaults = _config_defaults(config_class)
    type_hints = _type_hints(config_class)
    meta = get_config_field_meta(config_class)
    field_payload = [
        _field_schema(config_class, f, type_hints.get(f.name, f.type), defaults.get(f.name), meta[f.name])
        for f in fields(config_class)
    ]
    return {
        **entry.to_dict(include_classes=False),
        "fields": field_payload,
    }

extract_pipeline_schema

extract_pipeline_schema(pipeline_key: str | type | Any | None = None) -> dict[str, Any]

Extract one schema or all pipeline schemas.

Passing None returns {"pipelines": {...}} for all registered pipelines.

Parameters:

Name	Type	Description	Default
pipeline_key	(str, type, object or None)	A registry key, a Config, Pipeline or result class (or its name), or an instance of one of them. None selects every registered Pipeline.	None

Returns:

Type	Description
dict	The result of extract_config_schema for one Pipeline, or {"pipelines": {key: schema}} when pipeline_key is None.

Source code in Modeling_Tool/Pipeline/field_meta.py

def extract_pipeline_schema(pipeline_key: str | type | Any | None = None) -> dict[str, Any]:
    """Extract one schema or all pipeline schemas.

    Passing None returns ``{"pipelines": {...}}`` for all registered pipelines.

    Parameters
    ----------
    pipeline_key : str, type, object or None, default None
        A registry key, a Config, Pipeline or result class (or its name), or an instance of one of them. None selects
        every registered Pipeline.

    Returns
    -------
    dict
        The result of ``extract_config_schema`` for one Pipeline, or ``{"pipelines": {key: schema}}`` when
        ``pipeline_key`` is None.
    """

    if pipeline_key is None:
        return {"pipelines": {key: extract_config_schema(key) for key in PIPELINE_REGISTRY}}
    return extract_config_schema(pipeline_key)

extract_schema

extract_schema(config_class_or_key: str | type | Any) -> list[dict[str, Any]]

Compatibility helper returning just the list of field schemas.

Parameters:

Name	Type	Description	Default
config_class_or_key	(str, type or object)	A registry key (for example "credit_model"), the name of a Config, Pipeline or result class, one of those classes, or an instance of one of them.	required

Returns:

Type	Description
list of dict	The "fields" entry of extract_config_schema.

Source code in Modeling_Tool/Pipeline/field_meta.py

def extract_schema(config_class_or_key: str | type | Any) -> list[dict[str, Any]]:
    """Compatibility helper returning just the list of field schemas.

    Parameters
    ----------
    config_class_or_key : str, type or object
        A registry key (for example ``"credit_model"``), the name of a Config, Pipeline or result class, one of those
        classes, or an instance of one of them.

    Returns
    -------
    list of dict
        The ``"fields"`` entry of ``extract_config_schema``.
    """

    return extract_config_schema(config_class_or_key)["fields"]

config_to_dict

config_to_dict(config: Any, *, include_non_serializable: bool = False, exclude_none: bool = False) -> dict[str, Any]

Convert a Pipeline Config dataclass to a plain dict.

Non-serializable object fields (DataFrame, callable, sqlrunner, artifacts) are skipped by default, which is the safest behavior for GUI/YAML export.

Parameters:

Name	Type	Description	Default
config	dataclass instance	A Pipeline Config object.	required
include_non_serializable	bool	When False, fields flagged as not YAML-serializable and values that cannot be converted are skipped, and the other values are converted to plain data (tuples and ranges become lists, paths become strings). When True, every field is kept: convertible values are stored unchanged and the others are replaced by their repr().	False
exclude_none	bool	Skip fields whose value is None.	False

Returns:

Type	Description
dict	Field name to value.

Raises:

Type	Description
TypeError	If config is not a dataclass instance.

Source code in Modeling_Tool/Pipeline/field_meta.py

def config_to_dict(
    config: Any,
    *,
    include_non_serializable: bool = False,
    exclude_none: bool = False,
) -> dict[str, Any]:
    """Convert a Pipeline Config dataclass to a plain dict.

    Non-serializable object fields (DataFrame, callable, sqlrunner, artifacts)
    are skipped by default, which is the safest behavior for GUI/YAML export.

    Parameters
    ----------
    config : dataclass instance
        A Pipeline Config object.
    include_non_serializable : bool, default False
        When False, fields flagged as not YAML-serializable and values that cannot be converted are skipped, and the
        other values are converted to plain data (tuples and ranges become lists, paths become strings). When True,
        every field is kept: convertible values are stored unchanged and the others are replaced by their ``repr()``.
    exclude_none : bool, default False
        Skip fields whose value is None.

    Returns
    -------
    dict
        Field name to value.

    Raises
    ------
    TypeError
        If ``config`` is not a dataclass instance.
    """

    if not is_dataclass(config):
        raise TypeError("config_to_dict expects a dataclass config instance")
    meta = get_config_field_meta(config.__class__)
    payload: dict[str, Any] = {}
    for f in fields(config):
        value = getattr(config, f.name)
        if exclude_none and value is None:
            continue
        field_meta = meta.get(f.name)
        if field_meta and not field_meta.yaml_serializable and not include_non_serializable:
            continue
        if not include_non_serializable:
            try:
                payload[f.name] = _to_serializable(value)
            except TypeError:
                continue
        else:
            payload[f.name] = value if _is_serializable(value) else repr(value)
    return payload

config_from_dict

config_from_dict(config_class_or_key: str | type, payload: dict[str, Any], *, strict: bool = True) -> Any

Instantiate a Pipeline Config from a dict or GUI/YAML payload.

Parameters:

Name	Type	Description	Default
config_class_or_key	str or type	A registry key, a Config, Pipeline or result class name, or one of those classes.	required
payload	dict	Field values. A mapping that has a "config" key (the layout written by config_to_yaml) is read from that key.	required
strict	bool	When True, unknown field names raise KeyError. When False, they are dropped silently.	True

Returns:

Type	Description
object	An instance of the Config class; fields that are not in payload keep their defaults.

Raises:

Type	Description
KeyError	For an unknown reference, or for unknown field names when strict is True.

Source code in Modeling_Tool/Pipeline/field_meta.py

def config_from_dict(
    config_class_or_key: str | type,
    payload: dict[str, Any],
    *,
    strict: bool = True,
) -> Any:
    """Instantiate a Pipeline Config from a dict or GUI/YAML payload.

    Parameters
    ----------
    config_class_or_key : str or type
        A registry key, a Config, Pipeline or result class name, or one of those classes.
    payload : dict
        Field values. A mapping that has a ``"config"`` key (the layout written by ``config_to_yaml``) is read from
        that key.
    strict : bool, default True
        When True, unknown field names raise ``KeyError``. When False, they are dropped silently.

    Returns
    -------
    object
        An instance of the Config class; fields that are not in ``payload`` keep their defaults.

    Raises
    ------
    KeyError
        For an unknown reference, or for unknown field names when ``strict`` is True.
    """

    entry = _resolve_entry(config_class_or_key)
    config_class = entry.config_class
    values = dict(payload.get("config", payload))
    valid_fields = {f.name for f in fields(config_class)}
    unknown = sorted(set(values) - valid_fields)
    if unknown and strict:
        raise KeyError(f"Unknown fields for {config_class.__name__}: {unknown}")
    values = {key: value for key, value in values.items() if key in valid_fields}
    return config_class(**values)

config_to_yaml

config_to_yaml(config: Any, *, pipeline_key: str | None = None, include_non_serializable: bool = False) -> str

Serialize a Pipeline Config dataclass to a GUI-friendly YAML payload.

Parameters:

Name	Type	Description	Default
config	dataclass instance	A Pipeline Config object.	required
pipeline_key	str or None	Registry key of the Pipeline. When None it is inferred from the class of config.	None
include_non_serializable	bool	Passed to config_to_dict.	False

Returns:

Type	Description
str	YAML text with the keys pipeline, pipeline_class, config_class, smf_version and config.

Raises:

Type	Description
ImportError	If PyYAML is not installed.

Source code in Modeling_Tool/Pipeline/field_meta.py

def config_to_yaml(
    config: Any,
    *,
    pipeline_key: str | None = None,
    include_non_serializable: bool = False,
) -> str:
    """Serialize a Pipeline Config dataclass to a GUI-friendly YAML payload.

    Parameters
    ----------
    config : dataclass instance
        A Pipeline Config object.
    pipeline_key : str or None, default None
        Registry key of the Pipeline. When None it is inferred from the class of ``config``.
    include_non_serializable : bool, default False
        Passed to ``config_to_dict``.

    Returns
    -------
    str
        YAML text with the keys ``pipeline``, ``pipeline_class``, ``config_class``, ``smf_version`` and ``config``.

    Raises
    ------
    ImportError
        If PyYAML is not installed.
    """

    yaml = _yaml_module()
    entry = _entry_for_config(config, pipeline_key)
    try:
        import Modeling_Tool

        smf_version = getattr(Modeling_Tool, "__version__", None)
    except Exception:
        smf_version = None
    payload = {
        "pipeline": entry.key,
        "pipeline_class": entry.pipeline_class.__name__,
        "config_class": entry.config_class.__name__,
        "smf_version": smf_version,
        "config": config_to_dict(config, include_non_serializable=include_non_serializable),
    }
    return yaml.safe_dump(payload, allow_unicode=True, sort_keys=False)

config_from_yaml

config_from_yaml(config_class_or_key: str | type | None, yaml_text: str, *, strict: bool = True) -> Any

Deserialize a Pipeline Config from a YAML payload.

Parameters:

Name	Type	Description	Default
config_class_or_key	(str, type or None)	A registry key or a Config class. When None, the pipeline (or config_class) entry of the YAML payload selects the class.	required
yaml_text	str	YAML text, for example the output of config_to_yaml.	required
strict	bool	When True, unknown field names raise KeyError; when False, they are dropped.	True

Returns:

Type	Description
object	An instance of the Config class.

Raises:

Type	Description
ImportError	If PyYAML is not installed.
TypeError	If the YAML text is not a mapping.
KeyError	If no Pipeline can be determined, or for unknown field names when strict is True.

Source code in Modeling_Tool/Pipeline/field_meta.py

def config_from_yaml(config_class_or_key: str | type | None, yaml_text: str, *, strict: bool = True) -> Any:
    """Deserialize a Pipeline Config from a YAML payload.

    Parameters
    ----------
    config_class_or_key : str, type or None
        A registry key or a Config class. When None, the ``pipeline`` (or ``config_class``) entry of the YAML
        payload selects the class.
    yaml_text : str
        YAML text, for example the output of ``config_to_yaml``.
    strict : bool, default True
        When True, unknown field names raise ``KeyError``; when False, they are dropped.

    Returns
    -------
    object
        An instance of the Config class.

    Raises
    ------
    ImportError
        If PyYAML is not installed.
    TypeError
        If the YAML text is not a mapping.
    KeyError
        If no Pipeline can be determined, or for unknown field names when ``strict`` is True.
    """

    yaml = _yaml_module()
    payload = yaml.safe_load(yaml_text) or {}
    if not isinstance(payload, dict):
        raise TypeError("YAML payload must be a mapping")
    target = config_class_or_key or payload.get("pipeline") or payload.get("config_class")
    if target is None:
        raise KeyError("YAML payload must include pipeline/config_class when config_class_or_key is None")
    return config_from_dict(target, payload, strict=strict)

validate_pipeline_config

validate_pipeline_config(pipeline_key: str, values: dict[str, Any] | Any) -> list[str]

Lightweight config-only validation for GUI forms.

The Pipeline run methods remain the authoritative runtime validation, but this helper catches common form mistakes before code generation.

Parameters:

Name	Type	Description	Default
pipeline_key	str	A registry key (or the name of a Config, Pipeline or result class).	required
values	dict or Config instance	The form values. A key that is left out counts as unset: required keys are reported as empty and the others fall back to the Config default. Unknown keys are not reported.	required

Returns:

Type	Description
list of str	Error messages first, then warnings, each warning prefixed with "WARNING: ". An empty list means that nothing was found.

Raises:

Type	Description
KeyError	If pipeline_key does not match a registered Pipeline.

Source code in Modeling_Tool/Pipeline/field_meta.py

def validate_pipeline_config(pipeline_key: str, values: dict[str, Any] | Any) -> list[str]:
    """Lightweight config-only validation for GUI forms.

    The Pipeline ``run`` methods remain the authoritative runtime validation,
    but this helper catches common form mistakes before code generation.

    Parameters
    ----------
    pipeline_key : str
        A registry key (or the name of a Config, Pipeline or result class).
    values : dict or Config instance
        The form values. A key that is left out counts as unset: required keys are reported as empty and the others
        fall back to the Config default. Unknown keys are not reported.

    Returns
    -------
    list of str
        Error messages first, then warnings, each warning prefixed with ``"WARNING: "``. An empty list means that
        nothing was found.

    Raises
    ------
    KeyError
        If ``pipeline_key`` does not match a registered Pipeline.
    """

    entry = _resolve_entry(pipeline_key)
    if is_dataclass(values):
        vals = config_to_dict(values, include_non_serializable=True)
    else:
        vals = dict(values or {})
    errors: list[str] = []
    warnings: list[str] = []

    def missing(name: str) -> bool:
        return vals.get(name) in (None, "", [])

    if entry.key == "credit_model":
        if missing("target_col"):
            errors.append("target_col must not be empty.")
        if vals.get("warm_start_enabled") and missing("warm_start_score_col"):
            errors.append("warm_start_score_col is required when warm_start_enabled is on.")
        if vals.get("backward_enabled", True) and str(vals.get("backward_model", "lgb")).strip().lower() not in {"lgb", "xgb"}:
            errors.append("backward_model must be 'lgb' or 'xgb'.")
        if int(vals.get("optuna_n_trials", 5) or 0) < 1:
            errors.append("optuna_n_trials must be >= 1.")
        if int(vals.get("optuna_n_trials", 5) or 0) < 5:
            warnings.append("optuna_n_trials should be at least 5; very few search trials are unstable.")
        allowed_lr_search_params = {
            "objective", "primary_set", "gap_ref_sets", "metric", "refit", "verbose"
        }
        unknown_lr_search_params = sorted(
            set(vals.get("lr_search_params") or {}) - allowed_lr_search_params
        )
        if unknown_lr_search_params:
            errors.append(
                f"Unsupported lr_search_params keys: {unknown_lr_search_params}; "
                f"allowed keys are {sorted(allowed_lr_search_params)}."
            )
    elif entry.key == "feature_validation":
        has_batch_config = bool(vals.get("feature_batches")) or vals.get("feature_batch_size") is not None
        if vals.get("feature_batch_size") is not None and int(vals["feature_batch_size"]) <= 0:
            errors.append("feature_batch_size must be a positive integer.")
        if vals.get("enable_batch") and not has_batch_config:
            errors.append("feature_batch_size or feature_batches is required when enable_batch=True.")
        if vals.get("enable_batch") is False and has_batch_config:
            warnings.append("With enable_batch=False, feature_batch_size/feature_batches do not trigger CSV batching.")
        if vals.get("batch_corr_mode") == "block_pairwise":
            method = str((vals.get("corr_params") or {}).get("method", "pearson")).lower()
            if method == "kendall":
                errors.append("CSV block_pairwise correlation does not support kendall yet.")
    elif entry.key == "reject_inference":
        if missing("approved_col"):
            errors.append("approved_col must not be empty.")
        if missing("target_col"):
            errors.append("target_col must not be empty.")
        if missing("ri_methods"):
            errors.append("Select at least one entry in ri_methods.")
        if vals.get("train_prescore") is False and missing("score_col"):
            errors.append("score_col is required when train_prescore=False.")
        if vals.get("train_prescore", True) is not False and vals.get("ri_score_direction") == "high_good":
            errors.append(
                "ri_score_direction='high_good' needs your own score with train_prescore=False: "
                "the pre-score the Pipeline trains is the probability of bad."
            )
        if vals.get("ri_approved_frac") is not None and vals.get("ri_approved_n") is not None:
            errors.append("ri_approved_frac and ri_approved_n cannot both be set.")
    elif entry.key == "score_comparison":
        if missing("target_col"):
            errors.append("target_col must not be empty.")
        if missing("score_cols") and missing("base_score"):
            warnings.append("Without score_cols/base_score, the Pipeline will auto-detect the score columns.")
        if vals.get("group_specs") is not None:
            try:
                from ._common import normalize_group_specs

                normalize_group_specs(vals["group_specs"])
            except (TypeError, ValueError) as exc:
                errors.append(f"Invalid group_specs: {exc}")
        for name, spec in (vals.get("cross_metrics") or {}).items():
            if not isinstance(spec, (list, tuple)) or len(spec) != 2:
                errors.append(
                    f"cross_metrics[{name!r}] must be a two-item [column, aggregation] pair."
                )
        pairwise_agg = vals.get("pairwise_cross_agg_dict")
        if pairwise_agg is not None and not isinstance(pairwise_agg, dict):
            errors.append("pairwise_cross_agg_dict must be a {column: aggregation(s)} mapping.")
    elif entry.key == "score_consistency_uat":
        if missing("main_model_score_col"):
            errors.append("main_model_score_col must not be empty.")
        if vals.get("numeric_coercion_mode") not in (None, "safe", "aggressive", "off"):
            errors.append("numeric_coercion_mode must be one of safe/aggressive/off.")
        if int(vals.get("comparison_block_size", 128) or 0) <= 0:
            errors.append("comparison_block_size must be a positive integer.")
    elif entry.key == "sample_analysis":
        if missing("target_cols"):
            errors.append("target_cols must not be empty.")
        if missing("time_col"):
            errors.append("time_col must not be empty.")
        if vals.get("materialize_split") and missing("id_col"):
            errors.append("id_col is required when materialize_split=True.")
    elif entry.key == "mock_sample":
        n_samples = int(vals.get("n_samples", 80000) or 0)
        if n_samples < 1:
            errors.append("n_samples must be a positive integer.")
        if n_samples < 1000:
            warnings.append("n_samples should be at least 1000; smaller samples have limited statistical value.")
        if vals.get("applied_sample", 1) not in {0, 1}:
            errors.append("applied_sample must be 1 (all applications) or 0 (approved samples).")
        num_features = int(vals.get("num_features", 20) or 0)
        min_types = int(vals.get("min_num_feature_business_type", 5) or 0)
        if min_types > min(num_features, 10):
            errors.append("min_num_feature_business_type must not exceed min(num_features, 10).")
    return errors + [f"WARNING: {msg}" for msg in warnings]

generate_pipeline_code

generate_pipeline_code(pipeline_key: str, values: dict[str, Any]) -> str

Generate a minimal Python snippet for a configured Pipeline.

Parameters:

Name	Type	Description	Default
pipeline_key	str	A registry key (or the name of a Config, Pipeline or result class).	required
values	dict	Field name to value. Each value is written with repr(), so it must be a literal that Python can read back.	required

Returns:

Type	Description
str	Code that imports the Pipeline and Config classes, builds the Config from values, and calls run(data=your_dataframe) (or run() for a Pipeline that takes no data).

Raises:

Type	Description
KeyError	If pipeline_key does not match a registered Pipeline.

Source code in Modeling_Tool/Pipeline/field_meta.py

def generate_pipeline_code(pipeline_key: str, values: dict[str, Any]) -> str:
    """Generate a minimal Python snippet for a configured Pipeline.

    Parameters
    ----------
    pipeline_key : str
        A registry key (or the name of a Config, Pipeline or result class).
    values : dict
        Field name to value. Each value is written with ``repr()``, so it must be a literal that Python can read back.

    Returns
    -------
    str
        Code that imports the Pipeline and Config classes, builds the Config from ``values``, and calls
        ``run(data=your_dataframe)`` (or ``run()`` for a Pipeline that takes no data).

    Raises
    ------
    KeyError
        If ``pipeline_key`` does not match a registered Pipeline.
    """

    entry = _resolve_entry(pipeline_key)
    config_cls = entry.config_class.__name__
    pipeline_cls = entry.pipeline_class.__name__
    lines = [
        f"from {entry.import_path} import {pipeline_cls}, {config_cls}",
        "",
        f"cfg = {config_cls}(",
    ]
    for key, value in values.items():
        lines.append(f"    {key}={value!r},")
    lines.extend([")", "", f"pipeline = {pipeline_cls}(cfg)"])
    if entry.run_requires_data:
        lines.append("result = pipeline.run(data=your_dataframe)")
    else:
        lines.append("result = pipeline.run()")
    return "\n".join(lines)