Scaling Image-Text Pairs and Adversarial Transferability in CLIP-Based Models

Abstract

Abstract: As a general-purpose vision-language pretraining model, CLIP demonstrates strong generalization ability in image-text alignment tasks and has been widely adopted in downstream applications such as image classification and image-text retrieval. However, it struggles with fine-grained tasks such as object detection and semantic segmentation. While many variants aim to improve CLIP on these tasks, its robustness to adversarial perturbations remains underexplored. Understanding how adversarial examples transfer across tasks is key to assessing CLIP's generalization limits and security risks. In th

Research Question

What is the effect of scaling the number of one-to-many image-text pairs during training on the adversarial transferability (measured via attack success rate) of CLIP-based models across different visual-language benchmarks (e.g., COCO, VCR)?

Verification Level

Descriptive public verification status only; aggregate claim counts are public, but individual claim records are not exposed here.

Truth-Engine Gate Verdict

This record has not completed Gate 2 of the verification pipeline (a type-checked Lean4 proof for mathematical claims, or a sealed-sandbox reproduction for empirical claims). It is a literature synthesis only. VERIFIED requires an attached reproducible artifact (Lean4 proof source, or repro script and results) before this status can be set; it is not derived from review score or claim count.

Quality Tier

Descriptive public triage only; this tier does not alter current publication or DOI behavior.

Quality Dimensions

Automated triage signals derived from public fields; not human peer review or independent validation.

Correction Record

Public corrections are additive records. Current status does not claim the synthesis is error-free.

Provenance