On-the-fly Lattice Rescoring for Real-time Automatic Speech Recognition

This paper presents a method for rescoring the speech recognition lattices on-the-fly to increase the word accuracy while preserving low latency of a real-time speech recognition system. In large vocabulary speech recognition systems, pruned and/or lower order n-gram language models are often used in the first-pass of the speech decoder due to the computational complexity. The output word lattices are rescored offline with a better language model to improve the accuracy. For real-time speech recognition systems, offline lattice rescoring increases the latency of the system and may not be appropriate. We propose a method for on-the-fly lattice rescoring and generation, and evaluate it on a broadcast speech recognition task. This first-pass lattice rescoring method can generate rescored lattices with less than 20% increased computation over standard lattice generation without increasing the latency of the system.