Preference Optimization

Direct Preference Optimization: Your Language Model is Secretly a Reward Model

Notes

Bypasses reward modeling entirely. Simpler alignment, same results.