Performance of Zero-Shot Cross-Lingual Retrieval Models on Artificially Code-Switched Data vs. Multilingual Pretrained Models in
Abstract
Abstract: Transferring information retrieval (IR) models from a high-resource language (typically English) to other languages in a zero-shot fashion has become a widely adopted approach. In this work, we show that the effectiveness of zero-shot rankers diminishes when queries and documents are present in different languages. Motivated by this, we propose to train ranking models on artificially code-switched data instead, which we generate by utilizing bilingual lexicons. To this end, we experiment with lexicons induced from (1) cross-lingual word embeddings and (2) parallel Wikipedia page titles. We use
Research Question
How does the performance of zero-shot cross-lingual retrieval models trained on artificially code-switched data compare to multilingual pretrained models like mBERT or XLM-R on the MIRACL benchmark when measured by nDCG@10?
Verification Level
| Paper level | L2, Source-grounded claims | |
| Source-grounded claims | 15 | |
| Claim record source | parsed source sections |
Descriptive public verification status only; aggregate claim counts are public, but individual claim records are not exposed here.
Truth-Engine Gate Verdict
| Status | Falsified | |
| Gate | Gate 2 — Verification (formal proof or sandbox reproduction) | |
| Reason | [Gate 3 RED-TEAM FALSIFIED] avg_attack_score=6.2/10. COUNTEREXAMPLE_HUNTER(8.5):The formula script does not actually compute the performance gaps but rather har; CITATION_AUDITOR(3.5):The formula script only reproduces a derived gap value (+4 MRR@10) rather than t; REPLICATION_ATTACKER(6.5):The formula script only verifies the specific gap values (+4 MRR@10 for MoIR, +1 | |
| Evaluated | 2026-06-15T12:05:46.185158+00:00 |
A claim in this record was tested against Gate 2 and failed: a counterexample was found, a proof did not type-check, or a reproduction attempt did not match the reported results. Evidence for the failure is attached to this record. VERIFIED requires an attached reproducible artifact (Lean4 proof source, or repro script and results) before this status can be set; it is not derived from review score or claim count.
Quality Tier
| Tier | Flagship candidate | |
| Basis | Review score, verified-claim count, and public artifact coverage meet flagship-candidate thresholds. |
Descriptive public triage only; this tier does not alter current publication or DOI behavior.
Quality Dimensions
| Evidence strength | MEDIUM | |
| Citation grounding | MEDIUM | |
| Uncertainty disclosure | MEDIUM | |
| Reproducibility status | HIGH |
Automated triage signals derived from public fields; not human peer review or independent validation.
Correction Record
| Status | CURRENT |
| Correction count | 0 |
| Manifest contract | paper-manifest-v1.1 |
| Correction contract | correction-record-v1 |
Public corrections are additive records. Current status does not claim the synthesis is error-free.
Provenance
| Publisher | Assignee Research |
| Public provenance | L4, External archival record |
| Report artifact | Available |
| External record | Registered |
| Claim lineage | 15 aggregate source-grounded claims |
| Review method | Automated multi-reviewer assessment |
| Quality guide | How to read scores, claims, manifests, and evidence links |
| Provenance contract | source-provenance-v1 |
| Note | Machine-generated synthesis of existing literature. Not primary research. |