Papers
Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: What are the scaling laws for chain-of-thought reasoning in large language models v16. 0 claims were extracted from source literature; 0 were independently verified against retrieved documents. An automated…
Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: What architectural innovations improve transformer performance on multi-step logical reasoning v16. 11 claims were extracted from source literature; 9 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 14 peer-reviewed papers addressing the following research question: What is the relationship between model scale and emergent reasoning capabilities in transformers v16. 0 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: How do sparse mixture-of-experts models compare to dense transformers on mathematical reasoning v16. 11 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: How does context length affect language model performance on multi-document reasoning and summarization v15. 16 claims were extracted from source literature; 0 were independently verified against retrieved…
Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: What are the state-of-the-art large language model results on reasoning benchmarks published recently v15. 14 claims were extracted from source literature; 1 was independently verified against retrieved documents.…
Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: What are the limitations of current language model evaluation benchmarks for measuring reasoning v15. 0 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: How do language models compare to human experts on professional knowledge and science benchmarks v15. 15 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 2 peer-reviewed papers addressing the following research question: How do language models handle multi-hop reasoning chains in scientific question answering v15. 15 claims were extracted from source literature; 2 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: How does retrieval augmentation improve language model performance on knowledge-intensive tasks v15. 0 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: Does meta-reasoning performance on MR-GSM8K correlate with few-shot reasoning accuracy on other arithmetic benchmarks like MATH or SVAMP. 11 claims were extracted from source literature; 0 were independently…
Abstract: This report synthesises findings from 16 peer-reviewed papers addressing the following research question: What are the failure modes of frontier language models on abstract mathematical reasoning v15. 16 claims were extracted from source literature; 2 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: How does fine-tuning with synthetic math problems generated by a stronger model affect the zero-shot performance on the MATH and GSM8K benchmarks compared to human-written fine-tuning data. 0 claims were…
Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: How does the performance of multimodal models trained on both textual and symbolic mathematical representations compare to text-only models on the MATH dataset. 10 claims were extracted from source literature; 2…
Abstract: This report synthesises findings from 16 peer-reviewed papers addressing the following research question: What is the effect of instruction fine-tuning on language model mathematical problem-solving accuracy v15. 10 claims were extracted from source literature; 0 were independently verified against retrieved…
Abstract: This report synthesises findings from 10 peer-reviewed papers addressing the following research question: What is the correlation between per-token compute density and error rates on the BigBench Hard logical deduction tasks when using dynamic compute allocation. 7 claims were extracted from source literature; 0 were…
Abstract: This report synthesises findings from 16 peer-reviewed papers addressing the following research question: How do reinforcement learning from human feedback (RLHF) aligned models perform compared to instruction-tuned models on the CodeT5+ benchmark for software modification tasks. 8 claims were extracted from source…
Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: How does pretraining data quality affect language model reasoning benchmark performance v15. 0 claims were extracted from source literature; 0 were independently verified against retrieved documents. An automated…
Abstract: This report synthesises findings from 11 peer-reviewed papers addressing the following research question: What techniques enable language models to solve competition-level software engineering problems v15. 12 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: How do sparse mixture-of-experts models compare to dense transformers on mathematical reasoning v15. 16 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 16 peer-reviewed papers addressing the following research question: How does reinforcement learning from human feedback improve language model mathematical reasoning v15. 0 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: How do language models perform on formal theorem proving and mathematical verification tasks v15. 17 claims were extracted from source literature; 1 was independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 10 peer-reviewed papers addressing the following research question: What architectural innovations improve transformer performance on multi-step logical reasoning v15. 0 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: What is the relationship between model scale and emergent reasoning capabilities in transformers v15. 0 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 7 peer-reviewed papers addressing the following research question: What are the scaling laws for chain-of-thought reasoning in large language models v15. 0 claims were extracted from source literature; 0 were independently verified against retrieved documents. An automated…