← Back to gallery
Open on X
Continual Pretraining Results
Author: Jason WestonModel: nanoBanana-ProPublished: 1/30/2026, 3:04:27 AM
Categories
Tags
#Safety#Pretraining#LLM Assessment#Factuality#Generation Quality#MMLU Tasks
Copy prompt
Continual pretraining results: We see strong gains in: - factuality across a suite of tasks - safety across a suite of tasks - generation quality judged by GPT-OSS - standard tasks like MMLU etc We can optimize for different things depending on the LLM-as-a-judge prompt (factuality, safety, etc). 🧵3/5