Recognition result normalization
Normalization is the transformation (post-processing) of a text generated by a Yandex SpeechKit speech recognition model. It uses the parameters you specify in your API requests.
Note
Normalization is not available in auto language recognition mode (auto language code).
You can use the following normalization levels:
- Representing dates and time in numeric format.
- Converting text into numbers.
- Capitalizing proper nouns and the first letter in a sentence.
- Profanity masking.
- Formatting phone numbers, e.g.,
7(999)123-45-67instead of7 999 123 45 67. - Adding punctuation marks.
Warning
The text normalization rules may change. Please consider this during integration and stay tuned for updates.
Managing the normalization parameters
API v1
| Normalization level | API parameter |
|---|---|
| Profanity masking | profanityFilter=true |
| Converting text into numbers | This parameter is enabled by default. To disable, set rawResults=true. |
API v2
| Normalization level | API parameter |
|---|---|
| Profanity masking | config.specification.profanityFilter=true |
| Converting text into numbers | This parameter is enabled by default. To disable, set config.specification.rawResults=true. |
| Normalization level | API parameter |
|---|---|
| Profanity masking | config.specification.profanityFilter=true |
| Converting text into numbers | This parameter is enabled by default. To disable, set config.specification.rawResults=true |
| Capitalizing proper nouns and the first letter in a sentence. Adding punctuation marks | config.specification.literature_text=true |
API v3
To enable normalization, set the text_normalization=TEXT_NORMALIZATION_ENABLED parameter in the API. You need to use additional parameters to enable certain options after setting text_normalization=TEXT_NORMALIZATION_ENABLED.
| Normalization level | API parameter | Supported languages |
|---|---|---|
| Representing dates and time in numeric format | text_normalization=TEXT_NORMALIZATION_ENABLED |
ru, uz, kk, en, fr, de, es, it, tr |
| Converting text into numbers | text_normalization=TEXT_NORMALIZATION_ENABLED |
ru, uz, kk, en, fr, de, es, it, tr |
| Formatting phone numbers | text_normalization=TEXT_NORMALIZATION_ENABLED. To disable this level only, additionally specify phone_formatting_mode=PHONE_FORMATTING_MODE_DISABLED. |
ruNot performance-tested: en, fr, de, es, it, tr. |
| Profanity masking | profanity_filter=true with text_normalization enabled |
ruNot performance-tested: en, fr, de, es, it, tr. |
| Capitalizing proper nouns and the first letter in a sentence. Adding punctuation marks | literature_text=true with text_normalization enabled |
ru |
Normalization is performed only if there is one language in the list of allowed languages. Normalization is not supported for the following languages: pt, pl, nl, fi, sv, he, auto.
SDK
Here is a sample normalization configuration using an SDK:
import os
from yandex_ai_studio_sdk import AIStudio
# Authentication via environment variables:
# export YC_API_KEY=<API_key>
# export YC_FOLDER_ID=<folder_ID>
sdk = AIStudio(
auth=os.environ['YC_API_KEY'],
folder_id=os.environ['YC_FOLDER_ID'],
)
# Normalization is enabled; formatting of phone numbers is enabled by default.
stt = sdk.speechkit.speech_to_text(
audio_format='wav',
language_codes='ru-RU',
text_normalization=True,
)
with open('speech.wav', 'rb') as f:
audio_data = f.read()
result = stt.run(audio_data)
print(result.text)
# Normalization without formatting of phone numbers.
stt = stt.configure(
text_normalization=stt.TextNormalization(phone_formatting=False)
)
result = stt.run(audio_data)
print(result.text)
# Normalization with a profanity filter.
stt = stt.configure(
text_normalization=stt.TextNormalization(profanity_filter=True)
)
result = stt.run(audio_data)
print(result.text)
# Normalization with adding punctuation marks and capital letters.
stt = stt.configure(
text_normalization=stt.TextNormalization(literature_text=True)
)
result = stt.run(audio_data)
print(result.text)