Documentation Index
Fetch the complete documentation index at: /llms.txt
Use this file to discover all available pages before exploring further.
Teacher = the same base model, conditioned on feedback about the student’s own attempt.
advantage = teacher_logprob - student_logprob — following
“Reinforcement Learning via Self-Distillation”
(Hübotter et al., 2026): “the current model conditioned on feedback as a
self-teacher… without any external teacher or explicit reward model.”
def feedback_prompt_tokens() -> list[int]:
"""Your own tokenized question + feedback/critique prompt."""
...
def build_loss(student_sample, teacher_logprobs, cycle, request, ctx):
del teacher_logprobs # recomputed below against the feedback-conditioned prompt instead
prompt_tokens = request["prompt"]["token_ids"]
teacher_prompt = feedback_prompt_tokens()
data = []
for sequence in student_sample["sequences"]:
completion_tokens = sequence["tokens"]
full_tokens = teacher_prompt + completion_tokens
result = ctx.client.sampler_operation(
ctx.teacher_run_id, ctx.teacher_sampler_id, "logprobs", {"input": {"token_ids": full_tokens}}
)
teacher_seq_logprobs = [float(v) for v in result["logprobs"][-len(completion_tokens):]]
student_seq_logprobs = sequence["logprobs"]
per_token_advantage = [
teacher_lp - student_lp
for student_lp, teacher_lp in zip(student_seq_logprobs, teacher_seq_logprobs, strict=True)
]
data.append(rl_datum(prompt_tokens, completion_tokens, student_seq_logprobs, per_token_advantage))
return forward_backward_payload(data, "importance_sampling")
rl_datum and forward_backward_payload are the shared datum helpers.