Speech and Audio Processing

NeMo-Speech

Speech and audio processing refers to a system that processes audio signals, such as speech, music, and environmental sounds. This collection includes models for speech enhancement, restoration and extraction.

We will illustrate details in the following sections.

Models

Encoder-Mask-Decoder Model

Predictive Model

Score-Based Generative Model

Schrödinger Bridge Model

Converting NeMo manifest to Lhotse

Creating Lhotse shar dataset

Checkpoints

Local Checkpoints

Pretrained Checkpoints

Audio Models

NeMo Audio Configuration Files

NeMo Dataset Configuration

Lhotse Dataset Configuration

Model Architecture Configuration

Finetuning Configuration

Resources and Documentation

Tutorial notebooks can be found under

the audio tutorials folder

. If you are just starting with NeMo, consider trying out the tutorials of

NeMo Primer

and

NeMo Model

. These tutorials can be run on Google Colab by specifying the link to the notebooks’ GitHub pages on Colab.

If you are looking for information about a particular model, or would like to find out more about the model architectures available in the directory of

nemo.collections.audio

, refer to the

Models

section.

Information about how to load model checkpoints (either local files or pretrained ones from NGC), as well as a list of the checkpoints available on NGC are located on the

Checkpoints

section.

Documentation regarding the configuration files specific to the NeMo audio models can be found on the

Configuration Files

section.