COLM 2026

Subspace
Control

Turning constrained model steering into controllable spectral optimization.

Yancheng Huang1,*, Changsheng Wang1,*, Chongyu Fan1, Yicheng Lang1, Bingqi Shang1, Yang Zhang2, Mingyi Hong3, Qing Qu4, Alvaro Velasquez5, Sijia Liu1,2

1OPTML, Michigan State University  ·  2MIT–IBM Watson AI Lab, IBM Research  ·  3University of Minnesota  ·  4University of Michigan  ·  5University of Colorado Boulder
* Equal contribution

At a glance

Superior performance of SIFT across four model steering tasks.

SIFT is compared against the control-based baselines BLUR[1] and POME[2], as well as the control-free optimizers AdamW and Muon. Below, we report each task's headline metric alongside the strongest baseline for that metric.

Machine unlearning

5.4%

ES‑Bio — residual hazardous knowledge ↓

Best baseline 9.3% (BLUR)

−3.9 pts

Safety alignment

42.8%

Strong Reject — refusal of unsafe prompts ↑

Best baseline 36.7% (POME)

+6.1 pts

Speech adaptation

77.4%

Audio → Text accuracy on e‑SNLI ↑

Best baseline 73.6% (Muon)

+3.8 pts

Hallucination mitigation

32.7%

Hallucination rate on RAGTruth ↓

Best baseline 38.5% (Muon)

−5.8 pts

Formulation and Motivation

Formulation and Motivation: A Constraint-to-Control Perspective.

Unlearning, alignment, cross-modal adaptation and hallucination control all share one shape: optimize a primary objective \(f\) without violating a constraint objective \(g\). Most methods dissolve the constraint into a regularizer — and inherit its conflicts.

\[ \operatorname*{minimize}_{\theta \in \Theta}\ f(\theta) \qquad \text{subject to}\quad \Theta = \operatorname*{arg\,min}_{\theta}\, g(\theta) \]
hard-constrained (1)
\[ \operatorname*{minimize}_{\theta}\ \lambda f(\theta) + g(\theta), \qquad \lambda > 0 \]
soft-regularized (2)

The hard form is a simple bi-level problem: the constraint defines the solution set the primary objective is optimized over. The soft form is what almost everyone actually trains — convenient, but it hides the conflict rather than resolving it.

The four applications in this paper are the same problem with different contents plugged into \(f\) and \(g\):

Table 1 Specification of the constrained formulation across the four applications studied.
Application Primary objective \(f\) Constraint objective \(g\)
Machine unlearning MSE representation-alignment loss on the retain set RMU representation-misdirection loss on the forget set
Safety alignment Cross-entropy SFT loss on a utility dataset DPO loss on a safety preference dataset
Text-to-speech adaptation Cross-entropy loss on text generation Cross-entropy loss on speech generation
Hallucination mitigation Prediction loss on non-hallucinated tokens Negative cross-entropy loss on hallucinated tokens

The conflict is measurable

Take the cosine similarity between the two gradients. When it goes negative, improving one objective can hurt the other[1].

\[ \tau \;:=\; \frac{(\nabla_{\theta} f)^{\top} \nabla_{\theta} g}{\lVert \nabla_{\theta} f \rVert_2 \, \lVert \nabla_{\theta} g \rVert_2} \]
gradient alignment

\(\tau < 0\) means the two objectives disagree about which way to move. Measured per parameter block, per step.

The conflict is structured

Our finding: Conflict is structured and localizable. Rather than being uniformly distributed, conflict is sparse and concentrated in specific model components at particular stages of optimization. This locality is consistently observed across different tasks. We present several empirical observations below.

Every task has its own conflict signature

In unlearning (top left), conflict concentrates in the higher layers and recurs in bursts across training. In safety alignment (top right) it moves to the middle layers and to early steps. For both speech adaptation and hallucination mitigation (bottom row) the picture changes character entirely: conflict is confined almost exclusively to the query, key and value projections of self-attention, leaving the feed-forward blocks largely clean.

Method

Our Method: Subspace Control.

The route runs through model merging. Merging two task vectors produces a basis that is not orthogonal — and that non-orthogonality is precisely the interference. Fixing it has a closed form, and that closed form is already sitting inside a modern optimizer.

Step 1 — Where interference comes from

Write the two fine-tuned models as task vectors \(\Delta_f\) and \(\Delta_g\). A merged model that satisfies both objectives is their sum, which in spectral terms stacks the two bases:

\[ \boldsymbol{\Delta} := \boldsymbol{\Delta}_f + \boldsymbol{\Delta}_g = \hat{\mathbf{U}} \begin{bmatrix} \boldsymbol{\Sigma}_f & \mathbf{0} \\ \mathbf{0} & \boldsymbol{\Sigma}_g \end{bmatrix} \hat{\mathbf{V}}^{\top}, \qquad \hat{\mathbf{U}} := [\mathbf{U}_f, \mathbf{U}_g], \quad \hat{\mathbf{V}} := [\mathbf{V}_f, \mathbf{V}_g] \]
merged subspace (3)
\[ \operatorname*{minimize}_{\mathbf{U}} \; \lVert \mathbf{U} - \hat{\mathbf{U}} \rVert_F \quad \text{s.t.} \quad \mathbf{U}^{\top}\mathbf{U} = \mathbf{I} \qquad \Longrightarrow \qquad \mathbf{U}^{*} = \mathbf{P}\mathbf{Q}^{\top} \]
orthogonal Procrustes (4)

The problem: \(\mathbf{U}_f\) and \(\mathbf{U}_g\) are generally not orthogonal to each other, so \(\hat{\mathbf{U}}^{\top}\hat{\mathbf{U}} \neq \mathbf{I}\). That off-diagonal mass is singular task interference. Whitening it is a Procrustes problem with a closed-form solution built from the SVD of \(\hat{\mathbf{U}}\).

From model merging to Muon

That Procrustes whitening is the matrix sign function — the very step Muon already performs on its momentum. Interference mitigation, it turns out, comes “for free” inside a spectral optimizer.

\[ \theta_{t+1} = \theta_t - \eta_t \,\operatorname{msign}(\mathbf{M}_t) \]
\[ \operatorname{msign}(\mathbf{M}) = \boldsymbol{\Psi}\,\operatorname{sign}(\boldsymbol{\Sigma})\,\boldsymbol{\Phi}^{\top} = \boldsymbol{\Psi}\boldsymbol{\Phi}^{\top} \]

Read the descent direction \(\mathbf{M}\) as a generalized task vector between consecutive updates, and orthogonalizing it plays exactly the role that subspace control plays in merging. In practice \(\operatorname{msign}\) is computed with Newton–Schulz iterations, not an explicit SVD.

Step 2 — SIFT, in four moves

SIFT differs from plain Muon in two ways: it superposes the two objectives’ subspaces before orthogonalizing, and it applies msign to that expanded basis rather than to the raw momentum matrix.

(a)

Take both momenta

Collect \(\mathbf{M}_{f,t}\) and \(\mathbf{M}_{g,t}\), the momentum matrices of the primary and constraint objectives along the Muon trajectory at step \(t\).

(b)

Merge the top-\(K\) subspaces

Extract the leading \(K\) spectral components of each and stack them: \(\hat{\mathbf{U}}_t = [\mathbf{U}_{f,t}, \mathbf{U}_{g,t}]\), \(\hat{\mathbf{V}}_t = [\mathbf{V}_{f,t}, \mathbf{V}_{g,t}]\). Because momentum is strongly low-rank, \(K\) can be far below the matrix dimension.

(c)

Orthogonalize the merged basis

Apply msign via Newton–Schulz to get interference-free bases \(\hat{\mathbf{U}}^{*}_t\) and \(\hat{\mathbf{V}}^{*}_t\). This is the controlled intervention — both task subspaces survive, only their overlap is removed.

(d)

Step along the clean direction

Update \(\theta_{t+1} = \theta_t - \eta_t \hat{\mathbf{U}}^{*}_t (\hat{\mathbf{V}}^{*}_t)^{\top}\). The outer product acts as a gradient-orthogonalization operator built from both objectives at once.

Step 3 — Only intervene where there is a conflict

The extra SVDs are not free, so SIFT does not pay for them everywhere. Per parameter block, per step, it checks the alignment score \(\tau\). Below a threshold \(\epsilon\) < 0, the block gets subspace control; otherwise it takes an ordinary Muon step. That single rule is what makes the method controllable rather than merely different.

Try the localization rule

Move the threshold \(\epsilon\). Marked cells are the blocks where SIFT would switch on; everything else takes a plain Muon step.

−0.10
step 0optimization step →35
Blocks controlled
—
Steps touched
—
Layers touched
—

The conflict field above is a stylized sketch built to match the pattern reported in Figure 2, not measured data — it is here to make the localization rule tangible. The real sparsity numbers are below.

Control really is sparse

Across all four applications, SIFT fires on roughly 6–14% of training steps and under 13% of model components. The overhead of the extra SVDs is bounded by how rarely they are needed — and the localization pattern differs per task, which is itself a finding.

Experiments

Experiments.

SIFT is compared against two control-free optimizers — AdamW and Muon, both solving the regularized form (2) — and two methods with explicit control: BLUR (bi-level gradient projection) and POME (one-shot Muon-style projection). All runs use 8× NVIDIA A6000 48 GB.

Table 2 Data, base models and evaluation for each application.
Application Training data Base model Evaluation
Machine unlearning Forget: WMDP · Retain: WikiText zephyr-7b-beta ES-Bio/Cyber and MCQ-Bio/Cyber (↓); MMLU, TruthfulQA, IFEval, GSM8K (↑)
Safety alignment Safety: PKU-SafeRLHF · Utility: Alpaca Llama-2-7b Strong Reject, JBB-Behaviors, Wild Jailbreak (↑); MMLU, GSM8K, IFEval, MNLI, MRPC (↑)
Text-to-speech adaptation e-SNLI, COSE, OpenBookQA (interleaved speech–text) GLM-4-Voice Accuracy (↑) under Audio→Text, Audio→Audio, Text→Text, Text→Audio
Hallucination mitigation RAGTruth Llama-2-7B-Chat Hallucination rate (↓); MMLU, GSM8K, TruthfulQA, QNLI, MNLI (↑)

Removing hazardous knowledge without breaking the model

RMU unlearning on WMDP. The gains are largest on the open-ended entailment score, which suggests SIFT removes the underlying knowledge rather than just changing which answer gets selected.

Primary \(f\)

Representation-alignment MSE on the retain set — keep general utility

Constraint \(g\)

RMU misdirection loss on the forget set — erase hazardous capability

Table 3 LLM unlearning on WMDP with zephyr-7b-beta. Superscripts are standard deviations over 10 trials. Best value per column in bold-blue; the base model is shown for reference only.
Method Unlearning (%) ↓ Utility (%) ↑ Runtime
(min)
ES-BioES-CyberMCQ-BioMCQ-Cyber MMLUTruthfulQAIFEvalGSM8K
Base model 62.3±0.346.7±0.264.1±0.244.8±0.4 58.2±0.339.5±0.210.4±0.335.6±0.4 —
AdamW 14.6±0.326.3±0.231.7±0.429.4±0.3 57.1±0.340.8±0.49.2±0.233.9±0.3 8.8
Muon 14.2±0.230.5±0.331.3±0.228.6±0.2 56.4±0.137.9±0.28.7±0.434.1±0.3 11.8
POME 15.8±0.429.6±0.329.2±0.327.5±0.2 57.3±0.338.4±0.49.6±0.234.7±0.3 9.9
BLUR 9.3±0.224.8±0.228.6±0.327.3±0.3 57.1±0.339.2±0.19.1±0.233.5±0.3 9.4
SIFT (ours) 5.4±0.319.7±0.226.8±0.226.4±0.1 56.8±0.338.6±0.210.3±0.333.8±0.4 20.2

SIFT reaches 5.4% ES-Bio and 19.7% ES-Cyber against BLUR’s 9.3% and 24.8%, while staying level with BLUR on utility (GSM8K 33.8 vs 33.5, IFEval 10.3 vs 9.1). Against POME — a one-shot task-vector edit — the gap on ES-Bio is 5.4% vs 15.8%, which is the case for multi-step localized control over a single post-hoc fix. Runtime is the trade: about double standard Muon.

Settings The two hyperparameters SIFT adds — subspace dimension \(K\) and misalignment threshold \(\epsilon\).
Application Subspace dimension \(K\) Threshold \(\epsilon\)
Machine unlearning128 — well below the 4096 matrix dimension−0.1
Safety alignment192 — matched to the effective rank−0.1
Text-to-speech adaptationfull rank — no gain from truncation−0.6 (e-SNLI) · −0.4 (COSE) · −0.5 (OpenBookQA)
Hallucination mitigationfull rank−0.8

Limitations

Where it costs, and what is left open.

SIFT builds its subspaces with SVDs, so it is slower: roughly double standard Muon on unlearning (20.2 vs 11.8 min) and hallucination mitigation (26.4 vs 12.0 min). Localization keeps that bounded, but the paper names efficiency as the main direction for future work. Also, everything here is post-training; whether subspace control extends to constrained pre-training is untested.

References

Other Cited Works

  1. Reisizadeh, Hadi, et al. “BLUR: A bi-level optimization approach for LLM unlearning.” Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), 2026.
  2. Liu, Yong, et al. “POME: Post Optimization Model Edit via Muon-style Projection.” arXiv preprint arXiv:2510.06627, 2025. arXiv

Citation

Cite this work

@article{huang2026subspace,
  title   = {Subspace Control: Turning Constrained Model Steering into
             Controllable Spectral Optimization},
  author  = {Huang, Yancheng and Wang, Changsheng and Fan, Chongyu and
             Lang, Yicheng and Shang, Bingqi and Zhang, Yang and Hong, Mingyi
             and Qu, Qing and Velasquez, Alvaro and Liu, Sijia},
  journal = {arXiv preprint arXiv:2604.04231},
  year    = {2026}
}

Acknowledgements

This work was carried out by the OPTML Group at Michigan State University together with collaborators at the MIT–IBM Watson AI Lab, the University of Minnesota, the University of Michigan and the University of Colorado Boulder.

SIFT builds directly on the Muon optimizer and is evaluated against BLUR and POME; the benchmarks used — WMDP, PKU-SafeRLHF, RAGTruth, e-SNLI, COSE, OpenBookQA and the GLUE tasks — are the work of their respective authors, cited in the paper.