simulstream.metrics.scorers.latency

Functions

register_latency_scorer(name)

Decorator for registering a latency scorer class.

Classes

LatencyScorer(args)	Abstract base class for all latency scorers.
LatencyScores(ideal_latency[, ...])	Data structure representing a latency score.
LatencyScoringSample(audio_name, hypothesis)	Data structure representing a single evaluation sample.

class simulstream.metrics.scorers.latency.LatencyScorer(args: Namespace)

Abstract base class for all latency scorers.

A latency scorer evaluates system hypotheses against references and returns a LatencyScores object that represents the latency scores.

Subclasses must implement the abstract methods defined here and should be registered via register_latency_scorer().

Parameters:

args (argparse.Namespace) – Parsed command-line arguments.

class simulstream.metrics.scorers.latency.LatencyScores(ideal_latency: float, computational_aware_latency: float | None = None)

Data structure representing a latency score.

ideal_latency

The latency score in ideal conditions, which do not include computational costs.

Type:

float

computational_aware_latency

The latency score in computational aware conditions, which include computational costs.

Type:

Optional[float]

class simulstream.metrics.scorers.latency.LatencyScoringSample(audio_name: str, hypothesis: OutputWithDelays, reference: List[ReferenceSentenceDefinition] | None = None)

Data structure representing a single evaluation sample.

audio_name

The identifier of the audio file.

Type:

str

hypothesis

The system-generated hypothesis text.

Type:

str

reference

One or more reference sentences, including the text, start time and duration, or None if not required.

Type:

Optional[List[ReferenceSentenceDefinition]]

simulstream.metrics.scorers.latency.register_latency_scorer(name)

Decorator for registering a latency scorer class.

Parameters:

name (str) – The unique identifier for the scorer.

Raises:

TypeError – If the decorated class is not a subclass of LatencyScorer.

Example

>>> @register_latency_scorer("stream_laal")
... class StreamLAALScorer(LatencyScorer):
...     ...