Axiomi.si
Submit a tool

Megatron-LM

github.com · Training & Fine-tuning

Open source
#5 in Training & Fine-tuning

NVIDIA's research codebase for training transformer models at scale with tensor, pipeline and sequence parallelism.

#distributed-training#nvidia#parallelism#pretraining
Visit Megatron-LM ↗GitHub ↗

About the Training & Fine-tuning category

Distributed-training libraries, parameter-efficient fine-tuning methods and post-training toolkits for adapting foundation models to new tasks and preferences.

Megatron-LM is one of 9 training & fine-tuning tools indexed on Axiomi. Facts on this page come from the tool's official site and public APIs; prices and features change, so confirm details on the official website before you commit.

// related

Related tools

All Training & Fine-tuning →
DeepSpeeddeepspeed.aiLibrary for large-scale distributed training and inference, known for the ZeRO memory-optimisation family.Training & Fine-tuningOpen source★ 43.2knanoGPTgithub.comAndrej Karpathy's minimal repository for training and fine-tuning medium-sized GPTs, a classic learning resource.Training & Fine-tuningOpen source★ 63.6kUnslothunsloth.aiFine-tuning toolkit that speeds up LoRA and full training while cutting GPU memory use, with free notebooks.Training & Fine-tuningOpen source★ 77.2kPEFThuggingface.coParameter-efficient fine-tuning methods such as LoRA, letting large models adapt on modest hardware.Training & Fine-tuningOpen source★ 21.8kTRLhuggingface.coHugging Face library for post-training models with supervised fine-tuning, DPO, GRPO and other preference-optimisation methods.Training & Fine-tuningOpen source★ 19.4kAxolotlaxolotl.aiConfig-driven tool for fine-tuning open models across many architectures and training methods.Training & Fine-tuningOpen source★ 12.5k