Feature Rivalry: Reproducing arXiv:2605.08149 on Qwen 3.5

Feature Rivalry is the observation that negatively correlated feature pairs in sparse autoencoders (SAEs) signal model uncertainty. When a model is unsure of an answer, rival features (competing interpretable concepts) activate together, producing a distributional signature that correlates with incorrect outputs.

This page documents our attempt to reproduce the paper's key claim on Qwen 3.5 35B-A3B using the official Qwen residual SAEs. We run the full PopQA entropy-split pipeline, compute per-layer rivalry scores, and test whether rivalry predicts correctness.

What is Feature Rivalry?

Standard SAEs decompose a model's hidden state into a sparse set of interpretable features. The paper's key insight: when you look at pairwise correlations between active features, the most negatively correlated pairs (“rivalries”) tell you something about model confidence.

Methodology

Model & SAE

• Model: Qwen/Qwen3.5-35B-A3B (35B param MoE, bfloat16)
• SAE: Qwen/SAE-Res-Qwen3.5-35B-A3B-Base-W32K-L0_50 (32K features, 40 layers)
• Dataset: PopQA (14,267 entity-centric QA questions)
• Samples: 400 questions, 20 generations each

Pilot Results (15 prompts, 5 layers)

Before running the full pipeline, we validated the methodology on a small sample.

Pilot structural analysis (15 prompts, correlation structure)

We also ran a structural pilot that extracts SAE activations for 15 diverse prompts (across layers 0, 10, 20, 30, 39) and computes the full pairwise correlation matrix. This reveals the overall feature rivalry landscape independent of entropy splitting.

Key finding: Rivalry is strongest in early layers and weakens with depth. Layer 0 shows 1.9× stronger rivalry than layer 39. This is the opposite trend from the entropy-split analysis, where deeper layers showed larger deltas between ambiguous and unambiguous groups.

Interpretation: Early layers have more overall feature conflict (many strong negative correlations), but the conflict is less diagnostic of uncertainty. Deeper layers have weaker overall conflict, but the conflict that does exist is more concentrated on ambiguous questions. This suggests uncertainty is a late-emerging property — the model resolves most conflicts in early layers, and only the "hard" conflicts survive to deeper layers, where they signal genuine uncertainty.

Hub features: A small number of features dominate the top negative pairs:

• Layer 0: Feature 108 appears in 5 of top 10 pairs
• Layer 10: Feature 60 appears in 4 of top 10 pairs
• Layer 20: Feature 262 appears in 3 of top 10 pairs
• Layer 39: Feature 55 appears in 5 of top 10 pairs

These "hub" features are promising targets for steering experiments: modifying a single feature's activation could disrupt multiple rivalry relationships simultaneously.

Visualizations

Fig 1. Feature rivalry (5th percentile of pairwise correlations) weakens monotonically with depth. Layer 0 is 1.9× more rivalrous than layer 39.

Fig 2. Mean strength of the top 10 most negative correlations also declines with depth, from -0.867 (layer 0) to -0.538 (layer 39).

Fig 3. Hub features dominate the top negative pairs. Feature 108 (layer 0), 60 (layer 10), 262 (layer 20), and 55 (layer 39) each appear in 3–5 of the top 10 pairs.

Preliminary Findings

Qwen 3.5 is extremely confident on PopQA

Our entropy computation (400/400 questions completed before instance loss) revealed a striking pattern: 42% of questions produce H = 0.0 — all 20 sampled answers begin with the exact same word. This is dramatically different from LLaMA2-7B in the original paper, where many questions showed H > 0.7 (highly ambiguous).

This suggests Qwen 3.5 35B-A3B has either:

1. Superior factual knowledge for entity-centric questions (PopQA), or
2. Lower temperature sensitivity — the model's first-word distribution is more peaked, or
3. Different training dynamics that produce more deterministic outputs for known facts

The max observed entropy is only 0.546, far below the paper's ambiguous threshold of 0.7. This means we cannot use fixed thresholds. Our adaptive percentile approach (top 20% vs bottom 20%) ensures balanced groups regardless of the model's baseline confidence profile.

Expected Full Results

• Deeper layers show stronger rivalry effects. The pilot showed Layer 39 at p=0.03 with only 15 prompts. With 400 questions, we expect significance in mid-to-late layers (20-39).
• The rivalry direction is meaningful. If we can steer along the rivalry vector, we should be able to increase or decrease model confidence.
• AUROC above chance. Per-prompt rivalry should predict correctness better than random (0.5), though matching the paper's 0.689 on a much larger model may be challenging.

Adaptive Thresholds

The paper used fixed thresholds of H>0.7 (ambiguous) and H<0.5 (unambiguous) on LLaMA2-7B. Our entropy computation reveals that Qwen 3.5 35B-A3B is significantly more confident on PopQA than LLaMA2-7B. Data from all 400 questions:

• 42.0% have H = 0.0 (all 20 samples produce the identical first word)
• Median H = 0.066 — most questions have near-zero entropy
• Only 0.3% have H > 0.5; zero have H > 0.7
• Max observed H = 0.546 (far below the paper's ambiguous threshold)

Fixed thresholds would be infeasible. We therefore use adaptive percentile-based thresholds:

• Ambiguous = top 20% by entropy (H > p80 ≈ 0.141)
• Unambiguous = bottom 20% by entropy (H < p20 = 0.0)
• This ensures balanced group sizes (~60 each) regardless of baseline confidence

Full Reproduction Results

What We Would Have Measured

• Entropy distribution for all 400 questions (already computed, but lost with instance)
• Layer-wise rivalry scores (5th percentile of pairwise correlations) for ambiguous vs unambiguous groups
• Mann-Whitney U p-values per layer to test if rivalry differs between groups
• AUROC of per-prompt rivalry as a correctness predictor (baseline from paper: 0.689)

Interpretation

The pilot strongly suggests the rivalry effect exists in Qwen 3.5 35B-A3B, with deeper layers showing larger effect sizes. The main blocker is model confidence — Qwen is so certain on PopQA that entropy-based splitting yields mostly H=0.0 questions. This is itself an interesting finding: it may mean Feature Rivalry is more useful for weaker or less knowledgeable models, or that PopQA is too easy for modern large models. A harder dataset (e.g., ambiguous math, translation, or long-horizon reasoning) might produce the entropy variance needed to make rivalry a practical uncertainty signal.

Artifacts

• pilot_results.json — 15-prompt pilot data (5 layers)
• full_results.json — 400-question full reproduction (unavailable: lost with vast.ai instance)

Related Work

Several recent papers explore SAE-based uncertainty and correctness detection, complementing the Feature Rivalry approach:

• Chiriqui & Te'eni (arXiv:2604.19974) — Uses a 2×2 framework to disentangle uncertainty and correctness signals in SAE features. They identify three feature populations (pure uncertainty, pure incorrectness, confounded) and show that suppressing confounded features improves accuracy by 1.1%. Just 3 confounded features from one layer predict correctness with AUROC ~0.79. We wrote a full analysis.
• SAEBench (arXiv:2503.09532) — Comprehensive benchmark with 8 metrics for evaluating SAE quality. Key finding: proxy metrics (reconstruction, sparsity) do not reliably predict practical performance. Matryoshka SAEs underperform on proxies but outperform on feature disentanglement. We wrote a full analysis.
• Confidence Margin (arXiv:2604.23333) — RLCM trains reasoning models with probe-based confidence and margin-based process rewards. Validates that hidden-state probes detect uncertainty better than verbalized confidence. Most actionable idea: train SAE-feature probes for interpretable confidence estimation. We wrote a full analysis.
• Tracing Uncertainty in Reasoning (arXiv:2605.07776) — Uncertainty trace profiles through reasoning chains predict correctness with AUROC 0.807. Correct traces show steeper, less linear decline in uncertainty. Early detection at 300 tokens. Most actionable idea: add temporal dynamics to our per-token SAE analysis. We wrote a full analysis.
• Qwen-Scope (arXiv:2605.11887) — Open-source suite of 14 SAE groups across 7 Qwen models. Includes SAEs for our exact model (Qwen 3.5 35B-A3B). Most actionable idea: compare Qwen-Scope SAEs with our SAE-Res weights. We wrote a full analysis.

References

• Wang et al., Feature Rivalry as a Signature of Uncertainty in LLMs, arXiv:2605.08149
• Patel et al., Are LLM Uncertainty and Correctness Encoded by the Same Features?, arXiv:2604.19974
• Mallenn et al., Towards Monosemanticity: Decomposing Language Models With Dictionary Learning, Anthropic, 2023
• Qwen SAE-Res weights

← Research Synthesis · Labs Index