How does the predictive expert caching latency and token scheduling overhead affect end-to-end tokens-per-seco

Abstract

Abstract: We present DeepSeek-VL2, an advanced series of large Mixture-of-Experts (MoE)nVision-Language Models that significantly improves upon its predecessor,nDeepSeek-VL, through two key major upgrades. For the vision component, wenincorporate a dynamic tiling vision encoding strategy designed for processingnhigh-resolution images with different aspect ratios. For the languagencomponent, we leverage DeepSeekMoE models with the Multi-head Latent Attentionnmechanism, which compresses Key-Value cache into latent vectors, to enablenefficient inference and high throughput. Trained on an improved vi

Research Question

How does the predictive expert caching latency and token scheduling overhead affect end-to-end tokens-per-second throughput on multimodal reasoning benchmarks (MMMU, MathVista) for MoE-LLaVA compared to dense model baselines at 7B and 13B parameter scales?

Verification Level

Descriptive public verification status only; aggregate claim counts are public, but individual claim records are not exposed here.

Quality Tier

Descriptive public triage only; this tier does not alter current publication or DOI behavior.

Quality Dimensions

Automated triage signals derived from public fields; not human peer review or independent validation.

Correction Record

Public corrections are additive records. Current status does not claim the synthesis is error-free.

Provenance