Preference Data

DPO optimizes the preference data signal. Learn how to check for pairs, length bias, lexical shortcuts, tokenizer limits, and critical slices before fine-tuning.