FineWeb

huggingface.co · Datasets

Open source

Large, carefully filtered and deduplicated English web dataset for LLM pretraining, documented with ablations.

#pretraining#web#filtered#hugging-face

Visit FineWeb ↗

About the Datasets category

Open corpora and dataset tooling for pretraining and evaluation. Data quality and documentation matter as much as model architecture.

FineWeb is one of 5 datasets tools indexed on Axiomi. Facts on this page come from the tool's official site and public APIs; prices and features change, so confirm details on the official website before you commit.

// related

Related tools

All Datasets →

Dolmagithub.comAI2's open pretraining corpus and toolkit used to train OLMo, with tools for curating large text datasets.DatasetsOpen source★ 1.6k

Common Crawlcommoncrawl.orgNonprofit that maintains a free, open archive of web crawl data used in most large-scale pretraining corpora.DatasetsFreeHN 2

Hugging Face Datasetshuggingface.coHub of hundreds of thousands of datasets plus a library for loading and streaming them efficiently.DatasetsFree★ 22k

LAIONlaion.aiNonprofit producing open multimodal datasets, models and tools for democratising machine learning research.DatasetsFreeHN 2

Transformershuggingface.coHugging Face's library defining thousands of model architectures, the de facto reference implementation for open models.ML FrameworksOpen source★ 167k

Megatron-LMgithub.comNVIDIA's research codebase for training transformer models at scale with tensor, pipeline and sequence parallelism.Training & Fine-tuningOpen source★ 18.1k