Abstract: This report synthesises findings from 10 peer-reviewed papers addressing the following research question: What is the inference efficiency trade-off when applying cross-domain finetuning to DeepSeek-V3 on GPQA Diamond tasks. We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total…

Abstract: This report synthesises findings from 10 peer-reviewed papers addressing the following research question: To what extent does cross-domain finetuning improve DeepSeek-V3's robustness to distribution shifts in GPQA Diamond questions. Abstract The rapid evolution of large language models (LLMs) has driven a…

Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: How does Llama-3.1-8B's code generation performance on MBPP compare to other open-source 8B models like Falcon-8B and Mistral-8B in terms of pass@1 accuracy. Romanized Nepali, the Nepali language written in the…

Abstract: This report synthesises findings from 9 peer-reviewed papers addressing the following research question: What is the impact of vulnerability taxonomy alignment on the code repair success rates of DeepSeek R1 versus Codestral on the Big-Vul dataset. Many ML-based approaches have been proposed to automatically detect,…

Abstract: This report synthesises findings from 16 peer-reviewed papers addressing the following research question: What are the trade-offs between latency overhead and semantic preservation when applying GDPR-compliant anonymization techniques to Llama-3.1-8B inference pipelines. Large language models (LLMs) have achieved…

Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: How do different code generation models scale in inference efficiency when evaluated on multilingual programming benchmarks like CodeMixBench. Large Language Models (LLMs) have achieved remarkable success in code…

Abstract: This report synthesises findings from 11 peer-reviewed papers addressing the following research question: How does the diversity-weight parameter in Vendi-RAG influence the robustness of FLAN-T5-xl against adversarial attacks (e.g., ANLI) in knowledge-intensive QA, and what is the correlation between. Machine…

Abstract: This report synthesises findings from 10 peer-reviewed papers addressing the following research question: How does the Performance-Efficiency Ratio vary across different inference budget thresholds for code generation tasks using models ranging from 0.5B to 13B parameters on HumanEval and MBPP benchmarks. We…

Abstract: This report synthesises findings from 1 peer-reviewed paper addressing the following research question: How does the diversity-weight parameter in Vendi-RAG affect its performance on the ELI5 dataset when using a sparse retriever versus a dense retriever, measured by ROUGE-L scores. Questa tesi affronta il problema…

Abstract: This report synthesises findings from 14 peer-reviewed papers addressing the following research question: How does adaptive diversity-weight tuning in Vendi-RAG affect throughput on the TriviaQA benchmark compared to fixed-weight retrieval for FLAN-T5-xxl, and what is the optimal efficiency-accuracy.…

Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: How does the manifold-aware distance metric in DPR compare to Euclidean and cosine distance in terms of Recall@10 on Natural Questions (NQ) when the context window is limited to 512 tokens. The advent of…

Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: Can Vendi-RAG's diversity-aware retrieval approach improve robustness against adversarial or out-of-domain questions in the ELI5 benchmark compared to BM25 and dense retrieval baselines. Large Language Models…

Abstract: This report synthesises findings from 3 peer-reviewed papers addressing the following research question: How does Vendi-RAG's performance scale with increasing document corpus size in terms of EM score and latency on the TriviaQA benchmark compared to traditional RAG. The rapid evolution of natural language…

Abstract: This report synthesises findings from 7 peer-reviewed papers addressing the following research question: How robust is Vendi-RAG's diversity optimization to domain shifts when evaluated on cross-domain benchmarks like TyDiQA and DROP with F1 score comparisons. Aligned large language models (LLMs) demonstrate…

Abstract: This report synthesises findings from 11 peer-reviewed papers addressing the following research question: How does the robustness of Llama-2-7B and Llama-3-8B in handling out-of-domain retrieval tasks compare when evaluated on MuSiQue with a constrained context window of 1024 tokens. Prompt engineering has emerged as…

Abstract: This report synthesises findings from 14 peer-reviewed papers addressing the following research question: How does the retrieval accuracy of Contriever and DPR encoders compare on the Natural Questions benchmark when the context window size is increased to 2048 tokens. Retrieval-Augmented Generation (RAG) has…

Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: What is the impact of context window size reduction (4096 to 1024 tokens) on the throughput of Llama-3-8B when performing retrieval augmented generation on SQuAD 2.0. Retrieval-Augmented Generation (RAG) has…

Abstract: This report synthesises findings from 16 peer-reviewed papers addressing the following research question: How does the inference latency of DPR with manifold-aware distance metrics compare to standard DPR when scaling to 1M passages on the MS MARCO benchmark. The ice arches that usually develop at the northern and…

Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: What is the impact of incorporating manifold-aware distance metrics on the retrieval accuracy of DPR for out-of-domain (OOD) datasets like TriviaQA when the embedding dimension is reduced by 50\%. Dense Passage…

Abstract: This report synthesises findings from 8 peer-reviewed papers addressing the following research question: How does the addition of DPO to DONOD affect performance on adversarial safety benchmarks like AdvBench and WildHacks compared to SFT-only models. Predicting the trajectories of surrounding objects is a critical…

Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: Does scaling the model size from LLaMA-2-7B to larger variants (e.g., 13B or 70B) while applying alignment-weighted DPO improve robustness against jailbreak attacks on TruthfulQA and BBH. Recent advances in…

Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: How does scaling the model size (e.g., increasing the number of parameters) of fine-tuned multilingual models affect their performance on Arabic-SQuAD compared to monolingual models in terms of exact. In an era…

Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: How does varying the DON threshold in DONOD affect the trade-off between training efficiency (measured in training tokens reduced) and cross-domain generalization (measured by performance on MBPP and. Ad-hoc…

Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: How do multi-modal lightweight Transformers perform relative to text-only models on mixed code-generation and reasoning benchmarks (e.g., MBPP + MMLU) when evaluated for alignment with human. Large language…

Abstract: This report synthesises findings from 11 peer-reviewed papers addressing the following research question: What is the cross-domain robustness of fine-tuned multilingual models on Arabic QA when evaluated across multiple Arabic datasets (e.g., ArabiQA, ArSQuAD) compared to monolingual models. The rapid expansion of…