What is the generalization capability of SMoES on out-of-domain VQA benchmarks like VQA-CP v2 and A-OKVQA, and

Abstract

Abstract: Mixture-of-Experts architectures have become the standard for scaling large language models due to their superior parameter efficiency. To accommodate the growing number of experts in practice, modern inference systems commonly adopt expert parallelism to distribute experts across devices. However, the absence of explicit load balancing constraints during inference allows adversarial inputs to trigger severe routing concentration. We demonstrate that out-of-distribution prompts can manipulate the routing strategy such that all tokens are consistently routed to the same set of top-\$k\$ experts,

Research Question

What is the generalization capability of SMoES on out-of-domain VQA benchmarks like VQA-CP v2 and A-OKVQA, and how does this compare to fixed routing strategies in terms of accuracy drop and expert utilization metrics?

Verification Level

Descriptive public verification status only; aggregate claim counts are public, but individual claim records are not exposed here.

Quality Tier

Descriptive public triage only; this tier does not alter current publication or DOI behavior.

Quality Dimensions

Automated triage signals derived from public fields; not human peer review or independent validation.

Correction Record

Public corrections are additive records. Current status does not claim the synthesis is error-free.

Provenance