We Put Lyubishchev on PyPI with interesting benchmarks

Last week I published a library called lyubishchev on PyPI. The previous article explained why — Alexander Lyubishchev developed multivariate classification methods in 1943, twenty years before the researchers whose names ended up in scipy.spatial.distance. The library was a citation as much as a tool

Then I ran actual benchmarks. The results were more interesting than I expected

The Setup

Three standard sklearn datasets. Four classifiers. Five-fold stratified cross-validation

from sklearn.datasets import load_iris, load_wine, load_breast_cancer
from sklearn.model_selection import cross_val_score, StratifiedKFold
from sklearn.discriminant_analysis import QuadraticDiscriminantAnalysis
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.naive_bayes import GaussianNB
from lyubishchev import LyubishchevClassifier

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)

No hyperparameter tuning. Default settings for everything except LyubishchevClassifier(standardize=True) — standardization on because the method requires features on comparable scales, same as any distance-based approach

The Results

Run It Yourself

Three commands. You need Python installed, nothing else unusual

# Install the libraries
pip install lyubishchev scikit-learn

# Run the benchmark
python3 << 'EOF'
import warnings
import numpy as np
from sklearn.datasets import load_iris, load_wine, load_breast_cancer
from sklearn.model_selection import cross_val_score, StratifiedKFold
from sklearn.discriminant_analysis import QuadraticDiscriminantAnalysis, LinearDiscriminantAnalysis
from sklearn.naive_bayes import GaussianNB
from lyubishchev import LyubishchevClassifier

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)

datasets = {
    "Iris          ": load_iris(return_X_y=True),
    "Wine          ": load_wine(return_X_y=True),
    "Breast Cancer ": load_breast_cancer(return_X_y=True),
}

classifiers = {
    "LyubishchevClassifier": LyubishchevClassifier(standardize=True),
    "QDA (sklearn)        ": QuadraticDiscriminantAnalysis(),
    "LDA (sklearn)        ": LinearDiscriminantAnalysis(),
    "GaussianNB (sklearn) ": GaussianNB(),
}

for ds_name, (X, y) in datasets.items():
    print(f"\n{ds_name.strip()}")
    for clf_name, clf in classifiers.items():
        try:
            with warnings.catch_warnings():
                warnings.simplefilter("ignore")
                scores = cross_val_score(clf, X, y, cv=cv, scoring="accuracy")
            print(f"  {clf_name} {scores.mean():.4f} ± {scores.std():.4f}")
        except Exception:
            print(f"  {clf_name} CRASH")
EOF

You will see QDA complete all three datasets on sklearn 1.6+. The numbers should match the table above exactly — same random seed, same splits

What Is Actually Happening on Breast Cancer

Breast Cancer is the interesting row. 30 features, 569 samples, many correlated measurements. This is where real data lives

On this dataset all three discriminant classifiers score within two tenths of a percent of each other on accuracy. The difference worth noting is that LyubishchevClassifier has the lowest standard deviation — ± 0.0035 versus QDA's ± 0.0055 and LDA's ± 0.0200. More consistent predictions across folds

More importantly: this dataset exposes a version-dependent fragility in sklearn's own QDA

from sklearn.discriminant_analysis import QuadraticDiscriminantAnalysis
qda_unsafe = QuadraticDiscriminantAnalysis(reg_param=0.0)

On sklearn 1.6, QDA applies a small silent regularization and runs. On sklearn 1.9+, it raises ValueError: covariance matrix is not full rank and crashes entirely. The behavior depends on which version you have installed — and you may not know which behaviour you are relying on until it breaks in production

LyubishchevClassifier applies reg_covar explicitly and by design. The regularization is not a patch — it is part of the method. Lyubishchev's 1943 formulation required a well-conditioned covariance matrix, so the diagonal ridge was structural from the start. The result is deterministic behaviour across sklearn versions: it fits, it predicts, and it does not change behaviour silently when you upgrade

What This Is and Is Not

LyubishchevClassifier is mathematically equivalent to regularized QDA. It is not magic. On Iris and Wine — clean, low-feature datasets — it produces identical results to sklearn's QDA down to four decimal places, because the two methods are doing the same thing when the covariance is non-singular

The difference is robustness. Real datasets are not Iris. They have correlated features, more features than samples in some classes, and measurement scales that vary by orders of magnitude. LyubishchevClassifier was built for that reality, because that is the reality Lyubishchev was working in — flea beetle measurements with complex correlation structure, identified by a field entomologist with a ruler

The API

It is fully sklearn-compatible. Passes check_estimator. Works inside Pipeline and GridSearchCV

from lyubishchev import LyubishchevClassifier
import numpy as np

clf = LyubishchevClassifier(standardize=True, reg_covar=1e-6)
clf.fit(X_train, y_train)

labels = clf.predict(X_test)
proba  = clf.predict_proba(X_test)
score  = clf.score(X_test, y_test)

# Works in a Pipeline
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('clf', LyubishchevClassifier()),
])
pipe.fit(X_train, y_train)

The lower-level functions from the original release are still there:

from lyubishchev import divergence_coefficient, scatter_ellipse, classify, transgression

Install

pip install lyubishchev

Source: github.com/AkzhanBerdi/lyubishchev

Primary source: Lyubishchev, A.A. (1943). Programma obshchey sistematiki. Manuscript, 22 November 1943. zin.ru/animalia/coleoptera/rus/lyubis05.htm

What Is Next

The library is listed under scikit-learn-contrib (#81) — the official sklearn ecosystem for compatible packages. An R version has been submitted to CRAN. A proposal to add lyubishchev_divergence to scipy.spatial.distance is open at scipy #25335, tagged by a SciPy maintainer

If you have used the library, hit an issue, or want to contribute — the repo is open

Yours, Bad Dog