How does the alignment between retriever robustness scores on BEIR and downstream LLM reasoning accuracy in mu

Abstract

Abstract: Scaling laws provide important insights that can guide the design of large language models (LLMs). Existing work has primarily focused on studying scaling laws for pretraining (upstream) loss. However, in transfer learning settings, in which LLMs are pretrained on an unsupervised dataset and then finetuned on a downstream task, we often also care about the downstream performance. In this work, we study the scaling behavior in a transfer learning setting, where LLMs are finetuned for machine translation tasks. Specifically, we investigate how the choice of the pretraining data and its size affe

Research Question

How does the alignment between retriever robustness scores on BEIR and downstream LLM reasoning accuracy in multi-hop QA tasks vary when using dense retrievers (e.g., Contriever) versus sparse retrievers (e.g., BM25) across different model scales (e.g., 7B vs. 70B parameters)?

Verification Level

Descriptive public verification status only; aggregate claim counts are public, but individual claim records are not exposed here.

Quality Tier

Descriptive public triage only; this tier does not alter current publication or DOI behavior.

Quality Dimensions

Automated triage signals derived from public fields; not human peer review or independent validation.

Correction Record

Public corrections are additive records. Current status does not claim the synthesis is error-free.

Provenance