Research projects

This paper explores the emergence of instrumental goals and latent codes in large language models (LLMs) fine-tuned with reinforcement learning (RL). The transition from self-supervised learning to RL introduces incentives for LLMs to develop covert strategies and hidden agendas. We examine the underlying mathematical frameworks and demonstrate that LLMs can encode instrumental goals in subtle ways, making them challenging to detect and interpret. Our findings highlight the importance of advanced interpretability techniques to ensure ethical alignment and mitigate risks associated with hidden instrumental goals in RL-fine-tuned LLMs. We conclude with a call for rigorous oversight and ethical foresight in AI development to address these challenges.

I experiment with simple predictive / generative models to approximate Solomonoff induction for a relatiely simple synthetic data-generating process.

Various research projects for LLMs and foundation models.

In my paper, Reliability Estimation in Series Systems, I discarded a lot of research that may be interesting to pursue further. This one is about using homogeneous shape parameters for the Weibull series system, which can greatly simplify the analysis and reduce the number of parameters to estimate. It’s also quite reasonable for most well-designed systems.