Training Arabic Language Models: From Data Collection to (PT + SFT + DPO)

This workshop aims to provide an integrated understanding of the stages of building and developing Large Language Models (LLMs), moving from the pre-training phase to model optimization based on human preferences. Participants will explore the full development cycle, starting with Pre-training (PT) to build the model’s foundational knowledge using vast datasets, followed by Supervised Fine-Tuning (SFT) to specialize the model’s behavior through structured instructional data, and concluding with Direct Preference Optimization (DPO), which enhances response quality based on human preferences and the trade-offs between different outputs.