Researchers Fine-Tune YandexGPT with Rejection Sampling and DPO to Generate Russian Text at CEFR A1, A2 and B1

Researchers Fine-Tune YandexGPT with Rejection Sampling and DPO to Generate Russian Text at CEFR A1, A2 and B1

On October 21, 2026, arXiv posted paper 2610.27237v1, "CEFR-oriented Russian Text Simplification Using Reinforcement Learning," in which the authors fine-tune YandexGPT with rejection sampling and Direct Preference Optimization so that a single base model can produce Russian text at CEFR levels A1, A2 and B1 on demand. The work treats reading difficulty as an input parameter rather than an emergent property of prompting, and it is one of the first systematic attempts to apply preference

The full paper is not yet public, and the abstract concentrates on method and experimental design. The task is controlled simplification of Russian: given a source passage, the model must return a semantically equivalent version whose vocabulary and syntax sit inside a specified CEFR band, with A1, A2 and B1 as the three targets. The base model is YandexGPT, and training runs in two stages. Rejection sampling first filters the model's own generations for samples that match the target level, and those samples become preference pairs; DPO then fine-tunes on those pairs, turning difficulty from an unreliable prompt effect into part of the training signal. The authors compare several training configurations that differ in dataset composition, text type, and one further setting truncated in the abstract. No simplification accuracy, readability score or human-evaluation figure is reported,

The timing is not accidental. Since 2023, DPO has moved from an alternative to RLHF into a default post-training step, largely because it removes the separate reward model and online sampling loop, which puts preference alignment within reach of small teams. In parallel, the Russian LLM ecosystem is anchored by two domestic models, YandexGPT and GigaChat, giving researchers a viable base for domain fine-tuning. CEFR, the Council of Europe framework, has long been used across Russian language teaching and testing, so labels from A1 to C1 already function as shared vocabulary in textbooks, graded readers and exams; that makes level a natural candidate for a generation parameter. Academic work

Liu Gong's take: within twelve months, reading level will become a standard controllable parameter in LLM APIs alongside length and tone, and CEFR-graded simplification is the scenario most likely to get there first — but the hard problem in Russian is not the algorithm, it is who defines where each level boundary actually sits.

From a technical standpoint, the notable choice here is not DPO itself but the fact that the preference data is self-generated. Rejection sampling lets the model produce at volume and then filters for the target band, which is bootstrapping: the ceiling on preference-pair quality is the ceiling on the base model. That explains why dataset composition appears as a comparison axis — in graded simplification, better data organization usually beats a larger base model. The limitation is that DPO only teaches which output is harder, never why. The gap between A1 and A2 may be a handful of high-frequency words and one subordinate clause; the gap between A2 and B1 is sentence architecture. One preference signal struggles to represent both. Reference core vocabulary runs from roughly 500 words at A1 to over 2,000 at B1, a four-to-five-fold span, while preference pairs are typically built only between adjacent bands. Cross-band generalization is the number Liu Gong would check first in the full text.

On the business side, the buyers are easy to name: language-learning apps, graded-reading publishers, and public institutions that must issue plain-language versions of official documents. The Russian market is split between YandexGPT and GigaChat, and whichever ships a stable level-control interface first becomes the default for education workloads. Ticket sizes are modest, but volume is high and switching costs are real, which makes this an infrastructure play rather than an application play.

For the industry at large, Liu Gong has seen the same shape of problem in industrial visual inspection: defects also get bucketed into discrete grades, because a continuous score looks elegant but cannot be acted on, while a decidable grade lets downstream automation proceed. Text difficulty follows the same logic. A readability score of 62 gives an editor nothing to do; "this is B1" directly determines whether a text is pushed to a learner. Compressing continuous difficulty into finite bands and generating to the band will spread from language learning into government notices and patient-facing medical documents.

  • The study covers 3 CEFR levels (A1, A2, B1) and 2 preference-optimization components: rejection-sampling pair construction plus DPO fine-tuning.
  • Reference core vocabulary sizes run about 500 words at A1, 1,000-1,500 at A2 and 2,000-2,500 at B1, a 4-5x span a single model must cover continuously.
  • DPO drops the separate reward model and online sampling used in PPO-style RLHF, freeing memory for larger batches, a practical reason recent preference-optimization papers default to it.
  • Russian nouns inflect across 6 cases and verbs carry aspect pairs, giving a single lemma a dozen or more surface forms, one extra constraint layer versus equivalent English simplification.
  • The abstract

For the industry, the near-term (3 months) action is reproduction and transfer once the full paper lands: porting the rejection-sampling-plus-DPO recipe to languages with thinner resources, such as Italian or Indonesian, is technically cheap, and the real bottleneck is who labels the levels. Over the medium term (1 year), if the method holds across languages, CEFR grading is likely to be packaged as a standalone scoring-and-generation interface, become a standard component for education technology vendors, and force the creation of an automatic level-judging benchmark. Without such a benchmark, level control stays a marketing claim.

For ordinary users, near-term change is limited; the

Questions worth exploring:

  • Whether the quality ceiling of self-bootstrapped preference pairs is set by base-model capability or by the level-judging standard itself
  • Whether CEFR bands can be judged automatically and reliably, and how much systematic bias a classifier or LLM judge introduces
  • Whether transferring the method to Chinese or Japanese, with larger register gaps, raises or lowers data-construction cost

Further reading: 2026 Frontier LLM Comparison: A Practical Guide — a durable, evergreen reference. Bookmark it.

This item comes from arXiv cs.CL (published 2026-10-21 04:00:00). Wenfeng Radar provides a Chinese/English summary and commentary on overseas AI developments; the original content is owned by its author.


Daily aggregation of overseas AI developments with in-depth analysis. Bookmark us to never miss an important signal; back to homepage.

Republishing: feel free to quote or repost with a link back to the original article on Wenfeng Radar. · Business & partnership: see our media kit
Daily briefing + Sunday review · Unsubscribe anytime by replying · Your email stays private

Leave a Reply

Your email address will not be published. Required fields are marked *

中文EN
Contact · Privacy · Disclaimer · AI Disclosure · Weekly Review · Media & Business
Email: wenfengleida@wenfengleida.cn