Papers
Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: Does meta-reasoning performance on MR-GSM8K correlate with few-shot reasoning accuracy on other arithmetic benchmarks like MATH or SVAMP. 11 claims were extracted from source literature; 0 were independently…
Abstract: This report synthesises findings from 16 peer-reviewed papers addressing the following research question: What are the failure modes of frontier language models on abstract mathematical reasoning v15. 16 claims were extracted from source literature; 2 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: How does fine-tuning with synthetic math problems generated by a stronger model affect the zero-shot performance on the MATH and GSM8K benchmarks compared to human-written fine-tuning data. 0 claims were…
Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: How does the performance of multimodal models trained on both textual and symbolic mathematical representations compare to text-only models on the MATH dataset. 10 claims were extracted from source literature; 2…
Abstract: This report synthesises findings from 16 peer-reviewed papers addressing the following research question: What is the effect of instruction fine-tuning on language model mathematical problem-solving accuracy v15. 10 claims were extracted from source literature; 0 were independently verified against retrieved…
Abstract: This report synthesises findings from 10 peer-reviewed papers addressing the following research question: What is the correlation between per-token compute density and error rates on the BigBench Hard logical deduction tasks when using dynamic compute allocation. 7 claims were extracted from source literature; 0 were…
Abstract: This report synthesises findings from 16 peer-reviewed papers addressing the following research question: How do reinforcement learning from human feedback (RLHF) aligned models perform compared to instruction-tuned models on the CodeT5+ benchmark for software modification tasks. 8 claims were extracted from source…
Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: How does pretraining data quality affect language model reasoning benchmark performance v15. 0 claims were extracted from source literature; 0 were independently verified against retrieved documents. An automated…
Abstract: This report synthesises findings from 11 peer-reviewed papers addressing the following research question: What techniques enable language models to solve competition-level software engineering problems v15. 12 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: How do sparse mixture-of-experts models compare to dense transformers on mathematical reasoning v15. 16 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 16 peer-reviewed papers addressing the following research question: How does reinforcement learning from human feedback improve language model mathematical reasoning v15. 0 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: How do language models perform on formal theorem proving and mathematical verification tasks v15. 17 claims were extracted from source literature; 1 was independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 10 peer-reviewed papers addressing the following research question: What architectural innovations improve transformer performance on multi-step logical reasoning v15. 0 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: What is the relationship between model scale and emergent reasoning capabilities in transformers v15. 0 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 7 peer-reviewed papers addressing the following research question: What are the scaling laws for chain-of-thought reasoning in large language models v15. 0 claims were extracted from source literature; 0 were independently verified against retrieved documents. An automated…
Abstract: This report synthesises findings from 14 peer-reviewed papers addressing the following research question: How does test-time compute scaling improve language model performance on reasoning benchmarks v15. 13 claims were extracted from source literature; 2 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 11 peer-reviewed papers addressing the following research question: To what extent does the depth of the concept graph in MathScale influence its performance on the SMBI benchmark compared to shallow concept graphs or no structured knowledge at all. 17 claims were extracted from…
Abstract: This report synthesises findings from 14 peer-reviewed papers addressing the following research question: How does cross-lingual retrieval density impact answer accuracy in multilingual RAG systems compared to monolingual baselines on the XQuAD benchmark. 13 claims were extracted from source literature; 1 was…
Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: How does context length affect language model performance on multi-document reasoning and summarization v14. 10 claims were extracted from source literature; 0 were independently verified against retrieved…
Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: Which frontier language models achieve highest scores on GPQA Diamond Humanity Last Exam and difficult reasoning benchmarks v14. 14 claims were extracted from source literature; 1 was independently verified…
Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: Comprehensive comparison of frontier large language models on mathematical reasoning code generation and scientific knowledge v14. 0 claims were extracted from source literature; 0 were independently verified…
Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: What are the state-of-the-art large language model results on reasoning benchmarks published recently v14. 0 claims were extracted from source literature; 0 were independently verified against retrieved…
Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: What is the relationship between language model perplexity and downstream reasoning task performance v14. 14 claims were extracted from source literature; 1 was independently verified against retrieved documents.…
Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: How do language models compare to human experts on professional knowledge and science benchmarks v14. 11 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 11 peer-reviewed papers addressing the following research question: What is the effect of model size on language model performance on logical reasoning tasks v14. 10 claims were extracted from source literature; 1 was independently verified against retrieved documents. An…