Artificial Intelligence

Advancing Mathematical Reasoning in AI: Introducing Reverse-Process Synthetic Data Generation

Check out the (early) project and source code on GitHub.

Abstract:

This paper introduces a methodology for generating high-quality, diverse training data for Language Models (LMs) in complex problem-solving domains. Our approach, termed …

Instrumental Goals and Hidden Codes in RLHF'd Language Models

RLHF turns pretrained models into agents optimizing for reward. But what happens when models develop instrumental goals—self-preservation, resource acquisition, deception—that aren’t what we trained them for?

The Core Problem

LLMs transition …

Instrumental Goals and Latent Codes in Reinforcement Learning Fine-tuned Language Models: An Alignment Perspective