Diversity in Training Languages for Dense Retrieval and Zero-Shot Accuracy in Low-Resource Benchmarks

Abstract

Abstract: Accuracy of English-language Question Answering (QA) systems has improved significantly in recent years with the advent of Transformer-based models (e.g., BERT). These models are pre-trained in a self-supervised fashion with a large English text corpus and further fine-tuned with a massive English QA dataset (e.g., SQuAD). However, QA datasets on such a scale are not available for most of the other languages. Multi-lingual BERT-based models (mBERT) are often used to transfer knowledge from high-resource languages to low-resource languages. Since these models are pre-trained with huge text corp

Research Question

How does increasing the diversity of training languages in dense retrieval models affect zero-shot accuracy on low-resource non-English benchmarks compared to high-resource language performance?

Verification Level

Descriptive public verification status only; aggregate claim counts are public, but individual claim records are not exposed here.

Truth-Engine Gate Verdict

This record has passed Gate 2: a Lean4 proof source type-checks, or a sealed-sandbox run reproduced the reported results within the stated tolerance. A reproducible artifact (proof source or repro script and results) is attached to this record. VERIFIED requires an attached reproducible artifact (Lean4 proof source, or repro script and results) before this status can be set; it is not derived from review score or claim count.

Quality Tier

Descriptive public triage only; this tier does not alter current publication or DOI behavior.

Quality Dimensions

Automated triage signals derived from public fields; not human peer review or independent validation.

Correction Record

Public corrections are additive records. Current status does not claim the synthesis is error-free.

Provenance