Supervised fine-tuning

Overview

Supervised fine-tuning trains on labeled token sequences. Your code builds each typed JSON batch and submits Whitney primitives in order.

Prerequisites

GET /v1/training/capabilities — confirm the model advertises forward_backward, optim_step, save_state, and export_lora.
Create a session and LoRA run (see Quickstart).

Training loop

For each step:

forward_backward(batch) → optim_step(explicit AdamW)

Build a ForwardPayload with your token batches, dense loss-input tensors, and loss configuration. This is the whole computation — right-shift the sequence by one position, and only weight the completion span:

# source: examples/training/http/whitney_datums.py — sft_datum()
def sft_datum(prompt_tokens, completion_tokens):
    full = prompt_tokens + completion_tokens
    input_tokens, target_tokens = full[:-1], full[1:]
    pad_len = len(prompt_tokens) - 1
    weights = [0.0] * pad_len + [1.0] * len(completion_tokens)
    return {
        "input": {"token_ids": input_tokens},
        "loss_inputs": {
            "target_tokens": {"dtype": "int64", "shape": [len(target_tokens)], "values": target_tokens},
            "weights": {"dtype": "float32", "shape": [len(weights)], "values": weights},
        },
    }


batch = {
    "data": [sft_datum(prompt_tokens, completion_tokens) for prompt_tokens, completion_tokens in examples],
    "loss_function": "cross_entropy",
    "loss_config": {},
}

python examples/training/http/sl_loop.py \
  --provider modal --model Qwen/Qwen3.5-0.8B \
  --forward-backward-json batch.json

The real implementation — sft_datum() in whitney_datums.py — is a stdlib port of tinker_cookbook’s create_rightshifted_model_input_and_leftshifted_targets. sl_loop.py is a complete, runnable loop over this: it reads a batch from --forward-backward-json, submits forward_backward/optim_step in order, and finishes. Submit forward_backward and optim_step as ordered run operations. Poll each operation to a terminal state before advancing seq_id.

Checkpoints

After training:

save_state → export_lora → finish → close session

save_state persists resume state, including the optimizer when requested.
export_lora produces a portable LoRA adapter artifact.

Optional sampling

Omit sampling for artifact-only SFT so Modal does not provision a separate sampler GPU. If you need generations during SFT:

save_weights_for_sampler(version N) → sample(version N)

Sampling never synchronizes newer trainer weights implicitly. Always sync the weight version you intend to read.

Resume

Resume is provider-bound and model-bound. Create a new session and run, then either bind the checkpoint at run creation or execute load_state as the first ordered operation — not both for the same run.

Cleanup

On failure: cancel the run, poll until terminal, then close the session.

Get started

How it works

Cookbooks

Coding agents

Supervised fine-tuning

Overview

Prerequisites

Training loop

Checkpoints

Optional sampling

Resume

Cleanup

​Overview

​Prerequisites

​Training loop

​Checkpoints

​Optional sampling

​Resume

​Cleanup

Overview

Prerequisites

Training loop

Checkpoints

Optional sampling

Resume

Cleanup