Portrait of Federico Torrielli

Federico Torrielli

AI Safety researcher at the University of Southern Denmark

I study what happens inside language models. More in detail, I work on activation verbalizers: models that read the activations of another model and tell us, in plain words, what is going on in there. Can we trust large language models, at all? An auditor that questions a model about what it is hiding needs a correct answer and a good reason to believe it.

I am also finishing my Ph.D. at the University of Torino.

Research

Activation verbalizers

A verbalizer turns the activations of a model into text. In my work I measure how confident a verbalizer is when it answers, which token positions need an explanation, and if it still works on a model that was finetuned after its training.

Oversight of agents

When many agents talk to each other, new problems show up that a single model does not have. With my colleagues I built an auditor agent that watches these conversations. I also studied the languages that agents invent for themselves on Moltbook.

Safety and human factors

Some of my work looks at the people around these systems. For instance, I studied hidden instructions in papers that reviewers give to a chatbot, how a model can support a person in crisis when it has to decline a request, and how people explain generative AI to themselves.

Recent papers

All 18 papers

arXiv Preprint

Verbalizing Activation Differences with Natural Language Autoencoders

A. Blasi Núñez, G. Barmina, F. Torrielli, T. R. Savarimuthu, P. Schneider-Kamp, L. G. Poech

We tested if natural language autoencoders still work outside the setting they were trained for. By and large they do: they compose analogies, transfer to light finetunes, and recover a concealed secret word from activation differences alone.

arXiv Preprint

Selecting The Most Informative Tokens in Natural Language Autoencoders

F. Torrielli, G. Barmina, A. Blasi Núñez, A. Rapp, L. Di Caro, P. Schneider-Kamp, L. G. Poech

Explaining every token of a model with a natural language autoencoder is expensive. We show that a small ranker, trained only on the structure of the chat, selects the positions that need an explanation. On three of four datasets, explaining 5% of them keeps almost all of the audit success.

Teaching and projects

Courses

I teach generative AI to people who use it at work.

  • Generative AI Applications for Enterprise
  • Generative AI in the Public Administration
  • Prompt Engineering and Prompt Thinking

Projects

Projects outside of my papers.

  • Velvet - Italian Large Language Model Family
  • FAIR - Minerva Italian Large Language Model
  • MagIA - Magazine for Artificial Intelligence