In addition to manual curation, Alchemist offers an automatic curation feature powered by a proprietary algorithm.
This algorithm can select samples for your dataset based on various criteria, providing a quick way to build a comprehensive dataset.
The automatic curation process is designed to create a well-rounded dataset, but it doesn’t limit your control. After
the algorithm has run, you still have the ability to fine-tune the dataset by adding or removing samples as you see fit.
This combination of automatic selection and manual refinement allows for both efficiency and precision in dataset creation.
Automatic curation can be particularly useful when dealing with large volumes of samples or when you need a starting point
for your dataset. It can help identify valuable samples that might be overlooked in a purely manual process, ensuring a
diverse and representative dataset. By offering both manual and automatic curation options, Alchemist provides a flexible
and powerful toolkit for creating high-quality datasets. Whether you prefer hands-on control, algorithmic assistance,
or a combination of both, Alchemist supports your dataset curation needs, setting the stage for effective instruction
fine-tuning of language models.