AI ยท Feature
Synthetic training data can be useful, but it is not a risk-free substitute for diverse original data. Research into model collapse examines what can be lost through repeated learning from generated content. The key distinction is this: uncontrolled recursive use of data is different from examples that are deliberately generated, checked and combined with original data.
Why rare content can disappear
Generative models only approximate the distribution of their training data. If their outputs are then used as the basis for further training, deviations can be amplified over several generations. The Nature paper by Shumailov and colleagues examines this effect under the term model collapse. A central problem is the loss of rare parts of the original distribution. The system then increasingly learns from a narrowed representation of its own earlier results. The publication is therefore a warning against uncritical recursive use of data, not proof that every use of synthetic examples inevitably fails. How data is generated, selected and combined with original material is crucial. Equally important are the class of model being studied and the specific conditions under which the observed loss of quality arises and is measured.
Provenance becomes an indicator of quality
When human-created and machine-generated content is mixed, its provenance becomes harder to trace. This distinction nevertheless matters to training teams because it affects the composition and diversity of the collection. The idea behind Datasheets for Datasets offers a useful framework for this: documentation should cover creation, processing and intended use, among other things. For synthetic material, it should also include generation methods, selection criteria and checks. An automatically worded practice sentence with a known solution needs to be assessed differently from an entirely invented explanation that has not been checked. Even examples that appear correct can be systematically one-sided. Quality checks must therefore consider not only individual errors but also the distribution of the material as a whole, including rare cases, unusual wording and difficult counterexamples that can easily disappear through superficial selection.
Do not just monitor the average
For practical development, comparison with reference data kept unchanged is advisable. This data should not itself come from the models currently being investigated. After every major change to the data, checks should establish whether performance has deteriorated on rare tasks, languages or text patterns. A stable average is not enough if quality declines in certain marginal areas. It should also remain possible to trace which synthetic examples serve which purposes and what human or automated checks they have undergone. The research provides no universal threshold for a proportion of generated data that is safe in every setting. Rather, it shows why provenance, diversity and repeated evaluation belong together. Synthetic content is thus a tool whose usefulness depends on its design, not a resource that can be multiplied at will without repercussions for the model trained on it.
Sources & further reading
- Shumailov et al.: AI models collapse when trained on recursively generated datanature.com
- Gebru et al.: Datasheets for Datasetsdl.acm.org
Written in our own words with AI assistance, based on the sources listed above. The text distinguishes research findings, technical prototypes and established knowledge. This feature is not a breaking-news report.