How do the test-time compute scaling curves (accuracy vs. inference FLOPs) for DeepSeek-R1 and o1-preview diff

Abstract

Abstract: Recent advances in test-time scaling of large language models (LLMs), exemplified by DeepSeek-R1 and OpenAI's o1, show that extending the chain of thought during inference can significantly improve general reasoning performance. However, the impact of this paradigm on legal reasoning remains insufficiently explored. To address this gap, we present the first systematic evaluation of 12 LLMs, including both reasoning-focused and general-purpose models, across 17 Chinese and English legal tasks spanning statutory and case-law traditions. In addition, we curate a bilingual chain-of-thought dataset

Research Question

How do the test-time compute scaling curves (accuracy vs. inference FLOPs) for DeepSeek-R1 and o1-preview differ across legal sub-domains (e.g., contract interpretation vs. criminal law) on the multilingual benchmark, and do cross-lingual transfer effects emerge?

Verification Level

Descriptive public verification status only; aggregate claim counts are public, but individual claim records are not exposed here.

Quality Tier

Descriptive public triage only; this tier does not alter current publication or DOI behavior.

Quality Dimensions

Automated triage signals derived from public fields; not human peer review or independent validation.

Correction Record

Public corrections are additive records. Current status does not claim the synthesis is error-free.

Provenance