scenarios

Benchmark scenario definitions.

class pysatl_cpd.benchmark.scenarios.BenchmarkJob(detector, providers)[source]

Bases: Generic

A frozen dataclass binding a detector to its assigned providers.

Variables:

  • detector (ChangePointDetector) – Detector configuration to benchmark.

  • providers (Sequence[LabeledData]) – Input data providers the detector will be executed against.

Parameters:
detector: ChangePointDetector
providers: Sequence[LabeledData[DataT, TimeseriesAnnotation]]
__init__(detector, providers)
Parameters:
Return type:

None

class pysatl_cpd.benchmark.scenarios.BenchmarkScenario[source]

Bases: ABC, Generic

Abstract definition of a benchmark scenario.

A scenario encapsulates the lifecycle of a benchmark campaign: preparing detector-provider jobs from a dataset, orchestrating their execution through a registry, and producing analysis results from the accumulated registry data.

abstractmethod prepare_benchmark_jobs(dataset)[source]

Produce the sequence of detector-provider jobs for this scenario.

Parameters:

dataset (Dataset[TypeVar(DataT), TimeseriesAnnotation]) – Dataset whose providers and detector configurations are used to build the benchmark jobs.

Returns:

Jobs to be registered and executed against the registry.

Return type:

Sequence[BenchmarkJob[TypeVar(DataT)]]

abstractmethod analyze(registry)[source]

Derive scenario results from the populated registry.

Parameters:

registry (BenchmarkRegistry[TypeVar(DataT), TypeVar(TraceT, bound= DetectionTrace)]) – Registry containing execution results for all jobs previously prepared by this scenario.

Returns:

Scenario-specific analysis results keyed by detector description.

Return type:

dict[ChangePointDetectorDescription, TypeVar(ResultT)]

handle_benchmark_error(job, exc)[source]

Handle a ValueError raised during benchmark job execution.

Default implementation re-raises the exception. Override in subclasses to implement custom error recovery or logging.

Parameters:
Raises:

ValueError – Always re-raises the supplied exception.

Return type:

None