Listen to the Latents: Self-Correcting Speech Recognition in Large Audio Language Models Through Hidden-State Interactions

The proposed decoding strategy reuses a preserved pre-adaptation language model to guide a speech-recognition model during inference.

Source: E007, E015

It uses hidden-state interactions to identify tokens associated with semantic dependence and applies language-model correction selectively to those tokens.

Source: E007

In the reported comparison with beam decoding, targeted hybrid search improves named-entity recognition while maintaining comparable overall transcription error.

Source: E001

The reported improvements include subset-specific gains and statistical-test results for both named-entity and overall transcription outcomes.

Source: E001

End-to-end correction experiments cover only a single speech-language architecture, so full-method generalization across architectures remains unestablished.

Source: E013

Feature distributions, layer aggregation, and decision thresholds may require recalibration across models and domains, with challenging settings potentially needing further adaptation.

Source: E013

The proposed decoding strategy reuses a preserved pre-adaptation language model to guide a speech-recognition model during inference.

Source: E007, E015

It uses hidden-state interactions to identify tokens associated with semantic dependence and applies language-model correction selectively to those tokens.

Source: E007

The work asks whether a speech-recognition language model can improve during inference by reusing its original language model for external guidance.

Source: E007

The proposed decoding strategy reuses a preserved pre-adaptation language model to guide a speech-recognition model during inference.

Source: E007, E015

It uses hidden-state interactions to identify tokens associated with semantic dependence and applies language-model correction selectively to those tokens.

Source: E007

The method pairs speech-conditioned hidden states with text-only hidden states from the underlying language model under matched decoded context.

Source: E014, E016

It derives matched-layer directional similarity and relative representation-magnitude features from those paired trajectories.

Source: E004, E010, E014

Named-entity tokens are used as a proxy for semantic dependence, while other tokens are treated as a proxy for stronger acoustic grounding.

Source: E006, E012

Feature calibration uses a speech corpus, a multimodal speech-language model, reference-conditioned decoding, and an entity-recognition tagger.

Source: E012

Selection is determined from a density-ratio decision region estimated for the token-feature groups, with the threshold controlling how permissive selection is.

Source: E017

Language-model probability fusion is restricted to targeted tokens, and expanded search is used only while processing targeted words before returning to greedy decoding after beam convergence.

Source: E009, E020

The reported decoding setup fixes the beam, interpolation, and selection-threshold settings.

Source: E018

Evaluation uses a multimodal speech-language model across a public speech-recognition benchmark with diverse subsets and an additional speech corpus.

Source: E018

The evaluation reports overall transcription error and named-entity error.

Source: E018

The comparison landscape includes prior language-model-assisted speech-recognition correction systems.

Source: E008

Baselines cover greedy and beam decoding, global correction approaches, a variant without language-model correction, and a feature-free targeted-word variant.

Source: E002, E003, E011

In the reported comparison with beam decoding, targeted hybrid search improves named-entity recognition while maintaining comparable overall transcription error.

Source: E001

The reported improvements include subset-specific gains and statistical-test results for both named-entity and overall transcription outcomes.

Source: E001

Under greedy decoding, the method recovers much of the named-entity benefit associated with beam search while selectively expanding only a limited share of tokens and retaining a lower amortized search cost than full beam decoding.

Source: E011

Global late fusion degrades overall transcription error, and naive rescoring also degrades that outcome despite possible named-entity benefits.

Source: E003

The feature-free targeted-word baseline does not retain beam-decoding robustness, supporting the reported need for targeted correction guided by intermediate hidden-state features.

Source: E002, E003

The approach is described as self-corrective, requiring no added parameters or further training; its demonstrated implementation is for speech recognition, while broader task use remains proposed.

Source: E019

Feature extraction is reported to add negligible overhead relative to a model forward pass.

Source: E005

End-to-end correction experiments cover only a single speech-language architecture, so full-method generalization across architectures remains unestablished.

Source: E013

Feature distributions, layer aggregation, and decision thresholds may require recalibration across models and domains, with challenging settings potentially needing further adaptation.

Source: E013