Multimodal Knowledge Distillation for Robust Small Language Models in Code Generation

Abstract

Abstract: This report synthesises findings from 5 peer-reviewed papers addressing the following research question: To what extent can multimodal knowledge distillation from code-text pairs improve the robustness of small language models in code generation tasks, as measured by pass@k and latency metrics on. Recently, ChatGPT, along with DALL-E-2 and Codex,has been gaining significant attention from society. As a result, many individuals have become interested in related resources and are seeking to uncover the background and secrets behind its impressive performance. 11 claims were extracted from source literature; 10 were independently verified against retrieved documents. An automated multi-reviewer quality assessment produced a score of 7.2/10. This report is a machine-generated literature synthesis and does not constitute original research.

Research Question

To what extent can multimodal knowledge distillation from code-text pairs improve the robustness of small language models in code generation tasks, as measured by pass@k and latency metrics on HumanEval and MULTICODE benchmarks under adversarial perturbations?

Verification Level

Descriptive public verification status only; aggregate claim counts are public, but individual claim records are not exposed here.

Quality Tier

Descriptive public triage only; this tier does not alter current publication or DOI behavior.

Quality Dimensions

Automated triage signals derived from public fields; not human peer review or independent validation.

Correction Record

Public corrections are additive records. Current status does not claim the synthesis is error-free.

Provenance