simulstream.metrics.scorers.quality.mwersegmenter.MWERSegmenterBasedQualityScorer

class simulstream.metrics.scorers.quality.mwersegmenter.MWERSegmenterBasedQualityScorer(args)

Bases: QualityScorer

Abstract base class for scorers that require aligned system outputs and references through MWER Segmenter alignment.

This class wraps a quality scorer and applies the MWER Segmenter alignment by “Effects of automatic alignment on speech translation metrics” to hypotheses before scoring.

Subclasses must implement _do_score(), which receives ResegmentedQualityScoringSample instances, where output and references are aligned.

Example

>>> class CustomQualityScorer(MWERSegmenterBasedQualityScorer):
...     def _do_score(self, samples):
...         # Compute a custom quality score
...         return ...

__init__(args)

Methods

`__init__`(args)
`add_arguments`(parser)	Add scorer-specific arguments to the CLI parser.
`requires_reference`()	Indicate whether this scorer requires reference translations.
`requires_source`()	Indicate whether this scorer requires the source text.
`score`(samples)	Compute a quality score over a list of samples.

abstractmethod classmethod add_arguments(parser: ArgumentParser) → None

Add scorer-specific arguments to the CLI parser.

Parameters:: parser (argparse.ArgumentParser) – The parser to extend.

requires_reference() → bool

Indicate whether this scorer requires reference translations.

Returns:: True if references are required, False otherwise.
Return type:: bool

abstractmethod requires_source() → bool

Indicate whether this scorer requires the source text.

Returns:: True if source sentences are required, False otherwise.
Return type:: bool

score(samples: List[QualityScoringSample]) → float

Compute a quality score over a list of samples.

Parameters:: samples (List[QualityScoringSample]) – Samples to be evaluated.
Returns:: The computed quality score.
Return type:: float