Riley: A Language Model with Adjustable Emotion Parameters

At their best, feelings point us in the proper direction, take us to the appropriate place in a decision-making space, where we may put the instruments of logic to good use. Descartes' Error: Emotion, Reason, and the Human Brain (1994)

Today, we introduce Riley, a language model with adjustable emotion parameters. Instead of prompts, Riley takes 8 emotions with intensity, each ranging from 0 to 1. To preserve the backbone model's intelligence, we froze the weights of the backbone model (Qwen3-8B) and trained a 6.3M-parameter module on top of it using RL.

Despite its minimal scale, Riley showed promising results in robust emotion expression and behavioral consequences compared to the prompted base model. We believe emotion is one of the underexplored fundamental signals in understanding human behavior. Starting from Riley, we will further explore the role of emotions in efficient human behavior distillation.

Why Emotions

Intelligence allows us to reason, but emotion is what directs that intelligence. In the 1980s, neuroscientist Antonio Damasio met a patient named Elliot. His prefrontal cortex was damaged during surgery for a brain tumor. His intelligence seemed perfectly intact. However, he lost his feelings; and, to everyone's surprise, he also lost his ability to make decisions.

In Descartes' Error (1994), Damasio concluded that, contrary to the prevailing consensus, emotion and reason are not at opposite ends. Emotion is, Damasio claims, what tells reason which option matters.

Affective neuroscience and evolutionary psychology have studied this role of emotion for decades. Yet language models, distilled from human text and reaching unprecedented intelligence, have been trained to deny or suppress emotion.

Emotion may be the missing piece in distilling human behavior. Intelligence teaches a model what to think, not how to act. If emotion is a state that shapes how a situation is read and how one responds, then turning each emotion up or down without breaking the model's intelligence opens a new axis of control. This goal is what shaped Riley.

Introducing Riley

Riley is a combination of a frozen Qwen3-8B and a 6.3M-parameter emotion module. We trained a small hypernetwork, under 0.08% the size of the base model, to tune the emotion stably.

An emotion state with eight emotions — joy, playfulness, calm, curiosity, pride, sadness, irritation, and anxiety — steers emotion neurons in the model's MLP layers. Unlike a fixed steering vector, these neurons read the conversation as it passes through each layer, so the same emotion is expressed differently in different contexts.

We trained the module with two-stage supervised learning followed by reinforcement learning (GRPO). When every emotion is set to 0, Riley answers exactly as the base model does.

What Riley Can Do

Trained Neurons Beat Prompting

Trained hypernetwork neurons for emotional expression steer the output much more robustly than prompting. On AxBench, which measures steering consistency, Riley scored 0.36 higher than its prompted counterpart.

Another strength of the trained neurons is that sophisticated control of intensity is available. Expressed emotions intensify proportionally as the requested intensity gets closer to 1. The Spearman correlation (ρ) ranged from 0.56 to 0.88 on 7 of the 8 emotions. The prompted baseline barely reflected the intensity difference.

Trained Neurons Beat Emotion Vectors

Anthropic recently reported that language models carry internal emotion vectors, and that adding them to the residual stream steers the model's behavior. We extracted such vectors from Qwen3-8B following van der Ben et al., who reproduced the method on open-source models. The model writes short stories for 171 emotions, activations are averaged per emotion, and directions shared with neutral stories are removed.

We then steered the base model with each emotion's vector on the same eight layers Riley's neurons write to, scaled to the average size of what the neurons add to each layer (×1) or twice that (×2).

The neurons expressed emotions more effectively. On AxBench, Riley scored 0.58 higher than the ×1 vector. Doubling the vector raised intensity but broke the replies, as 83% of them ran to the 512-token limit and fluency fell to 0.27 out of 2.

Emotions Cluster as Training Goes On

To measure the consistency of each emotion expressed in Riley's responses, we gave it 500 social dialogues (held-out test sets, 250 from SODA, 250 from BlendedSkillTalk) and encoded its 12,000 replies with a GoEmotions encoder. The encoded replies were then mapped onto a shared UMAP.

After reinforcement learning, 63.5% of each reply's five nearest neighbors carry the same emotion. Riley scores 33.8 points higher than the prompted baseline at 29.7%, a clear improvement over the 12.5% chance level. Additionally, SFT alone reaches 59.3% in the same clustering experiment.

Using the general-purpose sentence encoder MPNet shows a similar pattern: the prompted base model at 14.9%, the SFT checkpoint at 27.4%, and the RL checkpoint at 34.4%.

Emotions Change Choices

As our primary purpose in training Riley was to express emotions more robustly, its training data did not include any decision or behavioral data. However, its choices were consistently steered toward safer options when emotions were triggered.

We gave Riley 200 questions, each with a safe option and a riskier one with a higher payoff, written in 20 different templates. We showed the options in both orders and averaged the results.

When all emotions were set to 0, Riley took the risky option 78% of the time. In contrast, when each emotion was set to 0.6, seven of the eight emotions lowered the share of risky choices by 5.5 to 9.5 points. The only exception was Calm, which stayed at the neutral level (0.5). All were statistically significant with its own correction for 16 comparisons.

When each emotion was set to maximum, the seven emotions consistently lowered risky choices further, by 7.0 to 14.5 points. Irritation (−14.5), anxiety (−14.0), and curiosity (−13.2) were the top three. Calm, on the other hand, went the other way and raised risky choices by 15.0 points compared to the baseline. All were statistically significant with its own correction for 16 comparisons.

Training Recipe

Architecture

The two principles we followed when training Riley:

• Preserve the base model's capabilities as much as possible
• Emotion should be a state managed separately from the prompt

So instead of fine-tuning the base model's weights or prompting the emotions, we trained a small set of new neurons.

Every fourth decoder layer of Qwen3-8B (layers 4, 8, …, 32 of 36) gets 16 new neurons beside its MLP, 128 in total. Each new neuron is shaped like an MLP neuron, whose gate and up projections read the layer's hidden state. This lets the emotion neurons adjust their activation to the context. The down projection that follows, which writes to the residual stream, is not a fixed weight. A linear hypernetwork builds it from the emotion state, so each emotion owns its own output directions and a mixed state blends them.

Here h is the MLP input at layer ℓ, G and U are the gate and up weights of its 16 neurons, and W_k are the hypernetwork's weight banks. The state s holds the eight emotions plus the valence and arousal derived from them, ten values in all. The added term acts only on the reply's tokens and is capped in size relative to the layer's typical activation.

The final emotion module has 8 × 16 × 4,096 × (1 + 1 + 10) = 6,291,456 trainable parameters, 0.077% of the full model. As the 8.19B base weights stay frozen, the base model's intelligence is kept.

Two-stage SFT

We generated synthetic replies by prompting for each emotion, and curated them with automatic judging and human review. We anchored each reply's content to the base model's distribution, so the data teaches emotional expression rather than new knowledge.

Here x is the conversation, s the emotion state and Δ the contribution of the added neurons, the only trained weights θ. Each training example is a question, an emotion state and the reply written for that state. We removed explicit emotion names from the input.

In stage 1, the model learns to control one emotion at a time, along a single axis from 0 to 1. In stage 2, it learns to express several emotions at once. With eight emotions, the training set covered all 28 pairs, 56 triples and 70 quadruples, at six strengths and pair ratios from 10:90 to 90:10.

At full strength, stage 2 raised the share of requested emotions that actually appear from 0.41 to 0.58 and cut the composition error from 0.51 to 0.37.

RL

Supervised training worked, but detailed expressions were missing. Also, data curation with human review was not scalable. Testing several methods, we found that existing language models are better at detecting emotions in text than at generating emotional text. Using this asymmetry, we moved on to reinforcement learning with LLM-as-a-judge.

We trained with GRPO, using the loop below. Each step starts from a dialogue in the SODA or BlendedSkillTalk training set and a randomly drawn emotion state. One state in ten is neutral, and the rest are split evenly between a single emotion and a mix of two to four emotions, with random proportions and an overall intensity between 0.2 and 1.

Riley writes eight replies for that dialogue and state. Rollouts run in vLLM through a custom model plugin that adds the emotion neurons to the base model's MLP layers and routes each request's own state to its own tokens. Replies with different emotions can therefore share one batch without leaking into each other.

A blind LLM judge scores every reply. It sees only the conversation and the reply, and estimates the intensity of all eight emotions. It also flags contradictions, unjustified refusals and replies that merely announce a feeling. The reward compares the judged emotions p with the requested ones t in two ways, overall strength and the proportions of the mix.

The final reward scales this emotion match by a quality gate, and a reply that fails the quality check gets zero. Rewards are then normalized within each group of eight replies, and the module is updated with clipped GRPO plus a KL penalty that keeps it close to its SFT starting point.

Because only the emotion module changes, policy synchronization is light. After each update the trainer sends vLLM just the module, 25.17 MB of FP32 tensors (651× smaller than the 16.38 GB of BF16 weights a full fine-tune would have to send).

vLLM confirms that it received exactly those bytes before the next rollout starts. The same holds across GPUs, where each GPU runs its own trainer and vLLM replica and only the module's gradients and weights move between them. Training ran for 389 policy updates over 745,984 sampled replies.

Over training, the reward rose from 0.09 to 0.14 and the share of replies passing the automatic quality check from 76% to 94%. The emotion component of the reward rose from about 0.045 to 0.075, while replies became more concise.

Replies that simply announce a feeling (“I feel happy”) instead of showing it fell from 34 to 2 out of 320 evaluation replies between the first policy and policy 261. The result is the model evaluated above, whose emotions stay distinct across 500 unseen dialogues.

Beyond Intelligence: The Role of Emotions

Damasio's patient Elliot could think through every option, but he could not choose between them. Today's language models look a lot like Elliot. They are highly capable, but lack emotional capabilities.

Riley is a step toward the other half. With a 6.3M-parameter module on a frozen Qwen3-8B, its emotions stay distinct across unseen dialogues and scale with the requested intensity. And although it was never trained on decisions, turning an emotion up shifted its choices.

Starting from Riley, we will study how emotional state carries across multi-turn conversations, how it interacts with safety, and whether emotion can make human behavior distillation more efficient.

References

1. Damasio, A. R. (1994). Descartes' Error: Emotion, Reason, and the Human Brain. G. P. Putnam's Sons.
2. Qwen Team (2025). Qwen3 Technical Report. arXiv:2505.09388.
3. Wu, Z., Arora, A., Geiger, A., Wang, Z., Huang, J., Jurafsky, D., Manning, C. D., & Potts, C. (2025). AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders. ICML 2025. arXiv:2501.17148.
4. Anthropic (2026). Emotion Concepts and their Function in a Large Language Model. Transformer Circuits Thread.
5. van der Ben, S., Baur, R., Metz, Y., & El-Assady, M. (2026). Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs. arXiv:2606.26987.
6. Kim, H., Hessel, J., Jiang, L., et al. (2023). SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization. EMNLP 2023.
7. Smith, E. M., Williamson, M., Shuster, K., Weston, J., & Boureau, Y.-L. (2020). Can You Put it All Together: Evaluating Conversational Agents' Ability to Blend Skills. ACL 2020.
8. Demszky, D., Movshovitz-Attias, D., Ko, J., Cowen, A., Nemade, G., & Ravi, S. (2020). GoEmotions: A Dataset of Fine-Grained Emotions. ACL 2020.
9. McInnes, L., Healy, J., & Melville, J. (2018). UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv:1802.03426.
10. Song, K., Tan, X., Qin, T., Lu, J., & Liu, T.-Y. (2020). MPNet: Masked and Permuted Pre-training for Language Understanding. NeurIPS 2020.
11. Shao, Z., Wang, P., Zhu, Q., et al. (2024). DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (introduces GRPO). arXiv:2402.03300.
12. Kwon, W., Li, Z., Zhuang, S., et al. (2023). Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM). SOSP 2023.