Multi-Head Attention as Ensemble Nadaraya-Watson Estimation: Variance Reduction, Decorrelation, and Optimal Head Diversity
arXiv 2605.20271•85bf889e39197c8abd5cef64755246785ac9cb1f3ecdf9465a9e25e28dbb8d31
GAMEHead-Diversity-IndexNadaraya-WatsonVC-dimensionarXivbanditsbayesian-inferencebias-variance-decompositioncontradiction-graphsgaussian-processesgroup-aware-matrix-estimationimportance-samplingintegrated-laplacelatent-gaussian-modelsmachine-learningmatrix-estimationmulti-head-attentionoptimal-transportpseudo-marginalrecommender-systemsscaling-lawsspectral-banditstheorytransfer-learningtransformers
Paper metadata
- arXiv ID
- 2605.20271
- Version
- Not specified by this published record
- Category
- Statistics — Machine Learning (stat.ML)
The PDF link points to arxiv.org. Baitaphish does not expose a private stored PDF.
Evidence and limitations
- Source ID
- arxiv_stat_ml
- Record identifier
- 85bf889e39197c8abd5cef64755246785ac9cb1f3ecdf9465a9e25e28dbb8d31
- Enrichment time
- 2026-05-21T07:23:54Z
- AI-assisted enrichment
- Yes
This record may overlap with other records. Its enrichment can be incomplete or wrong, and machine assistance was used. Validate consequential decisions against the linked source and your own environment.