Services Layer

services.text_normalization

Text normalization utilities for scoring comparisons.

class lingua_loop.services.text_normalization.DutchNormalizer[source]

Bases: TextNormalizer

Normalizer for Dutch text.

normalize_special_characters(text: str) → str[source]: Remove combining characters via NFKD normalization.

class lingua_loop.services.text_normalization.GenericNormalizer[source]

Bases: TextNormalizer

Normalizer that performs no special character normalization.

normalize_special_characters(text: str) → str[source]: Return text unchanged.

class lingua_loop.services.text_normalization.GermanNormalizer[source]

Bases: TextNormalizer

Normalizer for German text.

normalize_special_characters(text: str) → str[source]: Replace German special characters with ASCII equivalents.

class lingua_loop.services.text_normalization.TextNormalizer[source]

Bases: ABC

Normalizes text for comparison/scoring purposes.

The normalization is intentionally lossy and removes distinctions such as: - (language-specific) accent variations - case differences - punctuation

As a result, texts like “Er hat gesagt” and “er hat gesagt” are considered equivalent after normalization.

form: Literal['NFC', 'NFKC', 'NFD', 'NFKD'] = 'NFKD'

normalize(text: str) → str[source]: Normalize text through all normalization steps.

normalize_case(text: str) → str[source]: Normalize text to lowercase.

normalize_punctuation(text: str) → str[source]: Remove punctuation from text.

abstractmethod normalize_special_characters(text: str) → str[source]

Replace or remove special characters (return text if no change).

In some languages, like German, special characters like “ß” can be noramalized to written as “ss” and this does not change meaning. However, for languages like Italian, there is a difference between “è” and “e”, so no special characters normalization is implemented for that.

normalize_whitespace(text: str) → str[source]: Collapse whitespace and strip.

class lingua_loop.services.text_normalization.TextNormalizerFactory[source]

Bases: object

Factory for creating appropriate TextNormalizer instances.

services.transcript

Service layer for transcript scoring.

async lingua_loop.services.transcript.compute_score(video_id: str, segment_indices: list[int], user_text: str, language_code: SupportedLanguageCodes, session: AsyncSession) → Tuple[float, str][source]: Compute a score comparing user text to reference text.

async lingua_loop.services.transcript.get_or_create_transcript_with_segments(video_id: str, language_code: SupportedLanguageCodes, session: AsyncSession) → Transcript[source]: Pass through for DB logic for now.