Skip to content

eda

eigenkit.datasets.eda

Helpers for the visual analysis of a dataset, and for reproducible summaries.

This module has one main entry point, DatasetReport. It converts built-in and custom datasets to the same format, and then:

  • it draws quick overview plots of the distributions, the correlations, and the relation between the features and the target;
  • it draws diagnostics that are specific to time series;
  • it writes a reproducible summary, as text and as a dictionary, without using a language model.

FONTSIZE_AXES = 20 module-attribute

The font size, in points, that rc_params() gives the axis labels and titles.

FONTSIZE_LEGEND = 18 module-attribute

The font size, in points, that rc_params() gives the legend entries.

FONTSIZE_TICKS = 16 module-attribute

The font size, in points, of the tick labels, which is also the base font.size.

STYLE_COLORS = ('#CE2173', '#AA5EFF', '#7D4ECF', '#43CDFF', '#3333FB', '#11E57A', '#E6C541', '#94A3B8') module-attribute

The plot palette of the library, in the order of the colour cycle.

rc_params() sets it as axes.prop_cycle, so a plot drawn inside that context uses these colours without naming them. Index it when a series needs a fixed colour, and not the next one in the cycle. For example, use STYLE_COLORS[0] for a training curve and STYLE_COLORS[1] for validation, so that the two curves keep their colours across figures. It is a tuple, so that an append by mistake cannot change how every later plot in the process looks.

DatasetReport dataclass

An exploratory data analysis (EDA) report for a built-in or custom dataset.

Parameters:

Name Type Description Default
X ndarray

The feature matrix, of shape (n_samples, n_features).

 required
y ndarray

The target vector, of shape (n_samples,).

 required
feature_names list[str]

Readable names of the features, in the order of the columns of X.

 required
target_name str

The name of the target in the plots and the summary.

 required
task str

The type of task, usually "classification", "regression" or "timeseries".

 required
metadata Any

The metadata of the dataset, for example a DatasetInfo.

 None
time_index Any

The time axis for the time-series plots. It can be numeric (array-like) or a pd.DatetimeIndex.

 None
theme str

The theme of the plots, "light" or "dark".

 'light'
Notes

The recommended way to build a report is from_data(), which accepts the dictionaries returned by load_dataset(...) as well as your own X and y.

Example 
from eigenkit.datasets import DatasetReport, load_dataset

data = load_dataset("iris", split=False)
report = DatasetReport.from_data(data)
figs = report.plot_overview()
summary = report.summarize()

from_data(data=None, *, X=None, y=None, feature_names=None, target_name='target', task=None, time_index=None, theme='light') classmethod

Build a report from a dataset dictionary, or from your own arrays or tensors.

Parameters:

Name Type Description Default
data dict[str, Any]

A dataset dictionary in the format of eigenkit, either {"X", "y"} or {"X_train", "y_train", "X_test", "y_test"}. The training and test sets are combined.

 None
X Any

Your own feature matrix, as a NumPy array, a pandas object, a PyTorch tensor or another array-like.

 None
y Any

Your own target vector, in the same formats as X.

 None
feature_names list[str]

The names of the features. If they are not given, the names come from the metadata, or they are feature_0, feature_1 and so on.

 None
target_name str

The name of the target in the plots and the summary.

 'target'
task str

The type of task. If it is not given, it is taken from the metadata, or guessed from y.

 None
time_index Any

The time axis for the time-series analysis.

 None
theme str

The theme of the plots, "light" or "dark".

 'light'

Returns:

Type Description
'DatasetReport'

The report, ready for plotting and for the summary.

Raises:

Type Description
ValueError

If the inputs are missing or do not agree, for example if the shapes do not match, or if feature_names does not have one name per feature.

plot(kind, **kwargs)

Build one plot, chosen by its kind.

Parameters:

Name Type Description Default
kind str

The kind of plot: "data_health", "target", "distributions", "correlation", "feature_target" or "timeseries_target".

 required
**kwargs Any

Options of the plot, such as bins, max_features and max_points.

 {}

Returns:

Type Description
Any

The matplotlib figure.

Raises:

Type Description
ValueError

If kind is not one of the kinds above.

plot_overview(*, max_features=None, bins=30, max_points=2000, path=None)

Build the default set of plots of the exploratory data analysis.

Parameters:

Name Type Description Default
max_features int

The largest number of features shown in the panels about features. None shows every feature.

 None
bins int

The number of histogram bins in the distribution plots.

 30
max_points int

The largest number of points in the scatter plots. Larger datasets are subsampled.

 2000
path str | Path

A directory where every overview plot is saved as a PNG file. With None, the plots are not saved.

 None

Returns:

Type Description
dict[str, Any]

A dictionary from each kind of plot to its matplotlib figure. The "timeseries_target" plot is included only for a time-series task.

summarize()

Build a reproducible summary of the dataset, with recommendations.

Returns:

Type Description
dict[str, Any]

A dictionary with statistics about the shape and the quality of the data, signals specific to the task when there are any, and a readable summary under the "text" key.

rc_params(theme='light')

Return the matplotlib settings of the library for one plot, as rcParams.

Every figure that the library draws uses these settings: DatasetReport.plot_overview(), the benchmark curves and the plots in the worked examples. This is what makes them look like one set. Pass the result to matplotlib.rc_context() and draw as usual. The settings apply only inside that block, because the returned dictionary is a new copy and the global matplotlib.rcParams is never changed.

The two themes differ only in what has to change for a dark page. "dark" makes the backgrounds of the figure, the axes and the legend transparent, and makes the text, the ticks and the spines white. The sizes, the spines, the grid and STYLE_COLORS are the same in both themes, so a figure saved with each theme differs only in its background and in the colour of its text and lines.

Parameters:

Name Type Description Default
theme str

"light" or "dark". The case and any surrounding spaces are ignored.

 'light'

Returns:

Type Description
dict

The rcParams to draw with, ready for matplotlib.rc_context().

Raises:

Type Description
ValueError

If theme is neither "light" nor "dark".
Example 
import matplotlib
import matplotlib.pyplot as plt
from eigenkit.datasets import STYLE_COLORS, rc_params

with matplotlib.rc_context(rc_params(theme="light")):
    fig, ax = plt.subplots(figsize=(7, 4))
    ax.plot(history, color=STYLE_COLORS[0], label="train loss")
    ax.legend()
    fig.savefig("loss.png", bbox_inches="tight")