> ## Documentation Index
> Fetch the complete documentation index at: https://staging.docs.trywhitney.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Distillation on Whitney

> Train a student against a frozen teacher's logprobs, without a managed recipe.

Every distillation algorithm on Whitney shares one mechanism: a single run's
sampler only ever serves its own *current* weights, so a genuine frozen
teacher needs a **second Whitney run** — synced once, and never trained —
answering `logprobs` on the student's own generated tokens. There is no
external reward; the training signal is purely how much the teacher would
have preferred the student's own choices.

| You have                                                        | Use                        |
| --------------------------------------------------------------- | -------------------------- |
| A separate, usually larger teacher model                        | [OPD](/distillation/opd)   |
| Only the base model, plus golden-answer demonstrations          | [OPSD](/distillation/opsd) |
| Only the base model, plus feedback on the student's own attempt | [SDPO](/distillation/sdpo) |

## The two-run loop

1. Create the student session/run as usual (see [Quickstart](/quickstart)).
2. Create a second session/run for the teacher — same model for OPSD/SDPO, a
   different (typically larger) model for OPD.
3. Sync the teacher's sampler exactly once with `save_weights_for_sampler`.
   Never call `optim_step` on the teacher run again — it is a frozen
   reference for the rest of the loop.
4. Each cycle: sample from the student, then ask the teacher for `logprobs`
   on the student's exact generated tokens (prompt-conditioned differently
   per algorithm — see each page).
5. Build the loss from `teacher_logprob - student_logprob` per completion
   token, `forward_backward` and `optim_step` the student only.
6. `save_state` / `export_lora` / `finish` / `close` both sessions.

```python theme={null}
# Sync the teacher once — it never trains again.
teacher_sampler = client.run_operation(teacher_run_id, "save_weights_for_sampler", {})
teacher_sampler_id = result_sampler_id(teacher_sampler)

for cycle in range(cycles):
    student_sampler = client.run_operation(student_run_id, "save_weights_for_sampler", {})
    student_sampler_id = result_sampler_id(student_sampler)
    student_sample = client.sampler_operation(student_run_id, student_sampler_id, "sample", sample_request)

    # ctx bundles both runs' handles: ctx.teacher_run_id, ctx.teacher_sampler_id,
    # ctx.student_run_id, ctx.student_sampler_id, ctx.client.
    loss_payload = build_loss(student_sample, cycle, sample_request, ctx)
    client.run_operation(student_run_id, "forward_backward", loss_payload)
    client.run_operation(student_run_id, "optim_step", {"optimizer": adamw_optimizer(learning_rate=1e-6)})

client.run_operation(student_run_id, "save_state", {})
client.run_operation(student_run_id, "export_lora", {})
client.finish(student_run_id)
client.close_session(student_session_id)
client.finish(teacher_run_id)
client.close_session(teacher_session_id)
```

Fetching one sequence's teacher logprobs, conditioned on whatever
demonstration or feedback prompt your algorithm prepends:

```python theme={null}
def teacher_logprobs_for_completion(client, teacher_run_id, teacher_sampler_id, prompt_tokens, sequence):
    completion_tokens = sequence["tokens"]
    full_tokens = list(prompt_tokens) + list(completion_tokens)
    result = client.sampler_operation(
        teacher_run_id, teacher_sampler_id, "logprobs", {"input": {"token_ids": full_tokens}}
    )
    logprobs = result["logprobs"]
    # The result is N-1 causal predictions for an N-token input; the last
    # len(completion_tokens) of those are exactly the completion span.
    return [float(value) for value in logprobs[-len(completion_tokens):]]
```

On failure or interruption: `cancel` both runs, poll each to terminal, then
close both sessions. Never mix teacher and student weight versions on the
same sampler without explicit versioning.

Distillation loops build on the same [datum and advantage helpers](/cookbooks/helpers)
as every other algorithm — `rl_datum()` and `forward_backward_payload()` in
particular.