benchmark

Reset-online benchmark orchestrator for whole-timeseries metrics.

class pysatl_cpd.benchmark.online.reset.benchmark.OnlineResetBenchmark(dataset, registry, *, n_jobs=1)[source]

Benchmark subclass specialised for reset-online detectors.

Wraps the generic Benchmark and provides a convenience method for evaluating multiple detectors against a whole-timeseries metric.

Parameters:

dataset (Dataset[TypeVar(DataT), TimeseriesAnnotation]) – Labeled dataset whose providers serve as detector inputs.
registry (BenchmarkRegistry[TypeVar(DataT), OnlineDetectionTrace[Any]]) – Registry that caches per-detector execution results.
n_jobs (int) – Number of parallel worker processes (default 1). Must be non-zero.

__init__(dataset, registry, *, n_jobs=1)[source]

Parameters:

Return type:

None

get_metrics_for(entries, metric, *, force_recompute=False, n_jobs=None, backend='loky')[source]

Run a whole-timeseries metric for a collection of detector entries.

Creates an OnlineResetWholeTimeseriesMetricScenario from the supplied entries and metric, then delegates to run_scenario.

Parameters:

entries (Sequence[OnlineResetBenchmarkEntry]) – Detector entries to benchmark.
metric (IMultipleRunMetric[OnlineDetectionTrace[Any], LabeledData[TypeVar(DataT), TimeseriesAnnotation], TypeVar(ResultT)]) – Metric that evaluates multiple detection runs collectively.
force_recompute (bool) – If True, re-executes detectors even when cached results exist.
n_jobs (int | None) – Worker count override; falls back to instance n_jobs when None.
backend (str) – Joblib parallel backend identifier (default "loky").

Returns:

Metric results keyed by detector description.

Return type: