Scaling Context Windows in Static and Dynamic RAG Systems: Latency, Throughput, and Accuracy Trade-offs in Sub-10B Models

Abstract

Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: How does the scaling of context window size in static vs. dynamic RAG systems affect inference latency and throughput for sub-10B models on domain-specific benchmarks like SIMCOPILOTJ, while. 7 claims were extracted from source literature; 0 were independently verified against retrieved documents. An automated multi-reviewer quality assessment produced a score of 3.8/10. This report is a machine-generated literature synthesis and does not constitute original research.

Research Question

How does the scaling of context window size in static vs. dynamic RAG systems affect inference latency and throughput for sub-10B models on domain-specific benchmarks like SIMCOPILOTJ, while maintaining a fixed pass@1 accuracy threshold?

Verification Level

Descriptive public verification status only; aggregate claim counts are public, but individual claim records are not exposed here.

Quality Tier

Descriptive public triage only; this tier does not alter current publication or DOI behavior.

Quality Dimensions

Automated triage signals derived from public fields; not human peer review or independent validation.

Correction Record

Public corrections are additive records. Current status does not claim the synthesis is error-free.

Provenance