Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: What is the impact of varying client participation rates on federated malware detection model convergence speed and final test accuracy within resource-constrained IoT environments. The growing importance of data…

Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: Benchmark archaeology: investigate HumanEval score discrepancy for GPT-4o — reported 27.7\%–86.2\% (spread 58.5pp) across 2 papers. Sources: 'HumanEval-V: Benchmarking High-Level Vis' (27.7\%);. Prompt…

Abstract: This report synthesises findings from 5 peer-reviewed papers addressing the following research question: How does federated transfer learning performance on N-BaIoT generalize to other IoT malware datasets when measured by cross-domain accuracy and F1 scores across different device types. This work investigates the…

Abstract: This report synthesises findings from 11 peer-reviewed papers addressing the following research question: What is the impact of differential privacy noise levels on the F1-score degradation of federated malware detection models when deployed on resource-constrained IoT devices. This work investigates the…

Abstract: This report synthesises findings from 8 peer-reviewed papers addressing the following research question: How do supervised versus unsupervised federated learning approaches compare in terms of model accuracy trade-offs when detecting zero-day malware variants in cross-device IoT deployments. This work investigates…

Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: How does domain adaptation (e.g., fine-tuning on legal or biomedical text) influence the robustness of Baichuan-2's hallucination detection with different TAE misalignment thresholds on the FactCC. In the era of…

Abstract: This report synthesises findings from 8 peer-reviewed papers addressing the following research question: How does the alignment score sensitivity of Baichuan 2 and Vicuna-13B vary when evaluated on low-resource language multimodal benchmarks with constrained inference budgets. Multimodal LLMs are evolving from…

Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: How does the TAE token misalignment threshold affect the trade-off between hallucination rates and response coherence in Vicuna-13B and Baichuan-2 when evaluated on the TruthfulQA benchmark. Since the…

Abstract: This report synthesises findings from 14 peer-reviewed papers addressing the following research question: What is the comparative inference latency and throughput efficiency of 34B versus 70B Code Llama models when generating complex multi-file code solutions under large context window constraints. We release Code…

Abstract: This report synthesises findings from 7 peer-reviewed papers addressing the following research question: How does the scaling of Code Llama Python-specialized models from 7B to 70B parameters impact pass@1 accuracy on cross-library function composition tasks in BigCodeBench compared to general-purpose. Task…

Abstract: This report synthesises findings from 8 peer-reviewed papers addressing the following research question: How does the temperature parameter during CoT generation affect the syntax error rate in code for structured data tasks on BigCodeBench across different model sizes (7B, 13B, 30B). CHARMM (Chemistry at HARvard…

Abstract: This report synthesises findings from 14 peer-reviewed papers addressing the following research question: How does cross-domain fine-tuning (e.g., pre-training on Python vs. Java) impact the CoT step quality for code generation on BigCodeBench, evaluated using functional correctness scores. Abstract The rapid…

Abstract: This report synthesises findings from 3 peer-reviewed papers addressing the following research question: How do Llama3-70B and Codestral-34B compare in terms of cross-domain generalization when evaluated on SIMCOPILOTJ (Java) versus SIMCOPILOTP (Python) for code completion tasks. We introduce LLaMA, a collection of…

Abstract: This report synthesises findings from 2 peer-reviewed papers addressing the following research question: What is the inference efficiency difference between Codestral-7B and Llama3-70B when fine-tuned on C/C++ security vulnerability detection tasks. Software vulnerabilities pose significant risks to the security and…

Abstract: This report synthesises findings from 10 peer-reviewed papers addressing the following research question: What is the effect of multi-agent context engineering workflows on the reasoning accuracy of LLMs in niche domain code generation tasks measured by ReCode. Large Language Models (LLMs) have garnered remarkable…

Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: How robust are fine-tuned Codestral-7B and Llama3-70B models when evaluated on cross-domain code generation tasks in low-resource languages. Pre-trained models for Natural Languages (NL) like BERT and GPT have…

Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: How does the cross-domain generalization accuracy of fine-tuned Codestral-7B compare to Llama3-70B on unseen programming languages beyond Python for security vulnerability classification. Finetuning language…

Abstract: This report synthesises findings from 11 peer-reviewed papers addressing the following research question: How does semantic retrieval augmentation impact pass@k scores for LLMs on multi-file code generation benchmarks compared to standard context window extension. Large Language Models (LLMs) showcase impressive…

Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: What is the impact of different inference optimization techniques on the latency and accuracy trade-off between Llama3-70B and Codestral-34B for SIMCOPILOT's infill tasks. Deep ensemble learning has been shown to…

Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: Can fine-tuning Llama3-70B with retrieval augmentation on a synthetic multi-file vulnerability dataset improve its classification performance on the CodeXGLUE security subset, and how does this. A detailed study…

Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: How does the retrieval-augmented performance of Llama3-70B on the CodeXGLUE security subset compare to other state-of-the-art LLMs like Claude 3 Opus when evaluated on precision, recall, and F1-score. Anomaly…

Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: What is the accuracy difference between retrieval-augmented Gemini 1.5 Pro and Llama3-70B on the CodeXGLUE security subset when evaluated with few-shot learning versus zero-shot learning. The rapid expansion of…

Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: What is the impact of quantization on the throughput-accuracy trade-off for fine-tuned SecLM models deployed on resource-constrained hardware. As the rapid scaling of large language models (LLMs) poses…

Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: How does the inference latency of SecLM variants scale with model size when processing multimodal inputs on edge devices compared to cloud GPUs. With the breakthroughs in deep learning, the recent years have…

Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: What is the human evaluation accuracy score for code correctness of Mistral-Large-2 generated solutions on the MBPP benchmark compared to reference implementations. We introduce self-invoking code generation, a…