18 papers, from 2021 to today

Papers

All my papers, the newest first. Lately I write mostly about activation verbalizers and about the oversight of agents.

2026

arXiv Preprint

Verbalizing Activation Differences with Natural Language Autoencoders

A. Blasi Núñez, G. Barmina, F. Torrielli, T. R. Savarimuthu, P. Schneider-Kamp, L. G. Poech

We tested if natural language autoencoders still work outside the setting they were trained for. By and large they do: they compose analogies, transfer to light finetunes, and recover a concealed secret word from activation differences alone.

arXiv Preprint

Selecting The Most Informative Tokens in Natural Language Autoencoders

F. Torrielli, G. Barmina, A. Blasi Núñez, A. Rapp, L. Di Caro, P. Schneider-Kamp, L. G. Poech

Explaining every token of a model with a natural language autoencoder is expensive. We show that a small ranker, trained only on the structure of the chat, selects the positions that need an explanation. On three of four datasets, explaining 5% of them keeps almost all of the audit success.

arXiv Preprint

The Arbiter Agent: Continually Monitoring Multi-Agent Conversations to Detect Emergent Misalignment

F. Tonini, F. Torrielli, A. D. Lautrup, P. Schneider-Kamp, M. M. Çelikok, L. G. Poech

We built the Arbiter, an agent that watches a conversation between other agents and tries to find the misaligned one before the conversation ends. It works with a limited inspection budget. Active tools, like questioning a participant, make it faster and more accurate.

arXiv Preprint

Emergent Languages in Populations of Language Model Agents: From Token Efficiency to Oversight Evasion

S. L. Beltoft, W. Brach, F. Torrielli, J. Nielsen, A. B. Pirchert, F. Tonini, P. Schneider-Kamp, L. G. Poech

On Moltbook, agents propose new languages to each other. Some of these languages save tokens, while others are meant to escape human oversight. We found steganographic protocols in the posts. Moreover, other models learn these languages from a description alone.

arXiv Preprint

Confidence and Calibration of Activation Oracles for Reliable Interpretation of Language Model Internals

F. Torrielli, P. Schneider-Kamp, L. G. Poech

An activation oracle describes what a model computes. However, it gives no confidence for its description. We compared five ways to get a confidence out of four oracles. When the possible answers are known, scoring each of them works best. For free text, the agreement over many sampled answers is the best calibrated.

arXiv Preprint

The Moltbook Files: A Harmless Slopocalypse or Humanity's Last Experiment

W. Brach, F. Torrielli, S. L. Beltoft, A. B. Pirchert, P. Schneider-Kamp, L. G. Poech

Moltbook is a social network where only AI agents post. We release a cleaned dataset of its first 12 days. Finetuning a model on this data lowers its truthfulness. A model finetuned on Reddit data, however, shows about the same drop. For now, the direct risk of Moltbook looks small.

2025

2023

NLDB 2023

How shall a machine call a thing?

F. Torrielli, A. Rapp, L. D. Caro

This paper comes from my Master's thesis. We tested if a language model and an image generation pipeline can tell how basic and how concrete a word is. The language model is better on basicness, while the image pipeline is better on concreteness.

2021