Tinygrad vs PyTorch performance is not a single winner. PyTorch is usually faster out of the box because eager dispatch is heavily optimised and the open-model stack is built on it; wrap a stable graph in torch.compile and you often gain more. tinygrad deliberately keeps Python dispatch slow, then asks you to put @TinyJit on a pure step so it can replay kernels and, with BEAM, search for fast ones—after that warmup it can match or beat PyTorch on the same hardware, especially when driver enqueue cost dominates. Treat "performance" here as compile time, steady-state execution, kernel quality, memory behaviour and hardware reach, not one published scoreboard.
Tinygrad vs PyTorch performance at a glance
| Item | PyTorch | tinygrad |
|---|---|---|
| Default execution | Eager (runs each op immediately) | Lazy (runs on realize / .numpy()) |
| Path to peak speed | Optional torch.compile (TorchInductor / Triton) | Required @TinyJit; optional BEAM kernel search |
| Dispatch philosophy | Fast Python/C++ dispatch without a JIT | Slow without JIT; JIT replays captured kernels |
| Compile story | Graph capture via TorchDynamo; backends include Inductor | UOp rewrites + codegen; first compile ~on par with torch.compile |
| Neural-net API | torch.nn.Module + forward | Plain classes + __call__ (no nn.Module) |
| Hardware focus | Broad: CUDA, ROCm, MPS, Intel, CPU | Many runtimes: CUDA, NV, AMD, Metal, CPU, OpenCL, WebGPU, QCOM |
| Ecosystem for open models | Default (Transformers, fine-tuning, most research) | Small; interoperability helpers with PyTorch tensors |
| Licence / maturity | Mature, Foundation-backed | Pre-1.0; API described as stable but still evolving |
API and design notes below come from the tinygrad documentation and the PyTorch torch.compiler guide. Runtime names come from the tinygrad runtimes page.
What "performance" means here
People searching for tinygrad vs pytorch speed usually mix four different clocks:
- Time to first useful step — Python compile / graph capture before steady state.
- Steady-state step time — after JIT /
torch.compilewarmup, often with CUDA graphs or equivalent. - Kernel quality — how close matmuls and convolutions get to hardware peak.
- End-to-end product speed — training a real model through a library stack, or serving tokens through an engine that may not use either framework's Python loop.
A fair tinygrad vs pytorch benchmark names which of those you measured, the device (CUDA vs METAL vs AMD), whether PyTorch was eager or compiled, and whether tinygrad used TinyJit and BEAM. Comparing eager PyTorch to non-jitted tinygrad, or a Triton-tuned PyTorch model to tinygrad without BEAM, is not a fair race.
There is no single official head-to-head scoreboard that settles the comparison for every model. What we can do honestly is use each project's own docs about how they get fast.
How PyTorch gets fast
Eager PyTorch runs ops as you call them. That is why debugging feels natural, and why a lot of engineering goes into making dispatch cheap. When you need more throughput on a stable shape, PyTorch 2.x adds torch.compile: TorchDynamo captures graphs, AOT Autograd can capture the backward pass ahead of time, and TorchInductor (the default backend) generates fast code. On NVIDIA, AMD and Intel GPUs, Inductor leans on OpenAI Triton as a key building block. Other backends (CUDA graphs, TensorRT, OpenVINO, TVM, IPEX) exist for specialised cases.
Important implications for a speed comparison:
- Compilation is optional. Most tutorials and research code still start eager.
- Warmup matters. First compile can be expensive; later runs reuse compiled graphs when shapes allow.
- Ecosystem kernels matter. Serving and fine-tuning tools often ship hand-tuned attention and communication kernels that sit on top of PyTorch, so "framework speed" and "product speed" diverge.
For a broader framework-choice view (not just speed), see PyTorch vs JAX and TensorFlow vs PyTorch vs Keras.
How tinygrad gets fast
tinygrad's docs are blunt about the trade-off. Ops are lazy: a + b builds a graph until you realize. There is no special nn.Module; a network is an ordinary class with __call__. And the homepage difference list says tinygrad requires @TinyJit to be fast, because unlike PyTorch it does not invest heavily in fast dispatch—the decorator replays the kernels used in the decorated function.
The MNIST tutorial walks through the practical pattern: write a step function (often including the optimiser), wrap it in TinyJit, accept that the first two calls capture, then later calls replay tinygrad ops and skip Python overhead. The same tutorial notes that with Context(BEAM=2) searches many kernel implementations for what runs fastest on your hardware, that results are cached, and that after that search tinygrad is usually speed competitive with PyTorch.
The developer Speed page splits performance into four layers you should keep separate when you measure:
| Layer | What it is | tinygrad's stance (from their docs) |
|---|---|---|
| Compile speed (Python) | First-run UOp rewrites | Currently a bit slow, but on par with torch.compile; slower with BEAM |
| Execution speed (driver) | Submitting work after compile | Claims best-in-class by bypassing the GPU driver and prebuilding the command queue; often faster than normal CUDA and competitive with CUDA Graph |
| Model speed (scheduler) | Fusing ops and choosing what hits memory | Called a big bottleneck for training today |
| Kernel speed (codegen) | How fast each kernel runs | Improved by BEAM search over equivalent kernels |
So when tinygrad looks "slow," check whether you forgot TinyJit, whether you are timing the first compile, or whether the scheduler chose a chatty fusion. When it looks "impossibly fast," check whether the timer synced the GPU—the MNIST tutorial warns that its JIT timings do not sync, so GPU time may be slower than the printed Python times.
Tinygrad vs PyTorch speed: qualitative verdicts
Use these as starting hypotheses, then measure your own model.
When tinygrad tends to look strong
- Stable, jitted training or inference loops where Python dispatch and driver enqueue would dominate.
- Hardware where tinygrad's NV/AMD userspace paths matter, and you are willing to tune with
BEAMand debug withDEBUG. - Learning how compilers work—the stack is small enough that reading kernels is realistic; that is a productivity win even when absolute FLOPS are similar.
- Apple Metal / multi-backend experiments where you want one Tensor API across CUDA, Metal, AMD and CPU (see runtimes). On Macs for local LLMs, also compare MLX vs GGUF; MLX is often the more natural Apple-first stack.
When PyTorch tends to win
- Anything that depends on the open-weight ecosystem—fine-tuning stacks, Transformers, most published research code. See how to fine-tune an LLM locally.
- First iteration and interactive debugging, where eager mode starts immediately and
print/ breakpoints Just Work. - Workloads already tuned with Triton, FlashAttention-style kernels, or specialised
torch.compilebackends. - Teams that need production maturity, packaging, and a large hiring pool.
Compile time vs run time
Both frameworks ask you to pay compile cost for later speed. tinygrad's Speed docs put first-run compile on par with torch.compile, and slower when BEAM explores many kernels. PyTorch's Inductor path has the same shape: pay once (or once per new shape), then amortise. If your job is a short notebook cell or constantly changing shapes, eager PyTorch usually feels snappier. If your job is thousands of identical steps, JIT/torch.compile dominate the story.
Hardware backends and memory
PyTorch remains the safe default on NVIDIA CUDA and has first-class or solid paths for AMD ROCm, Apple MPS, Intel and CPU.
tinygrad lists many accelerators on GitHub and in docs: OpenCL, CPU, Metal, CUDA, AMD, NV, QCOM and WebGPU, with the claim that a new accelerator needs on the order of ~25 low-level ops. The runtimes table documents concrete requirements (for example Metal on M1+ Macs, NV aimed at newer NVIDIA architectures, AMD on CDNA/RDNA generations). For NVIDIA and AMD it also documents lower-level interfaces that can talk more directly to the device than a classic CUDA/HIP userspace path—that is the background for the "execution speed" claims above, not a promise that every model is faster.
Memory behaviour differs by design: tinygrad's lazy graph and scheduler decide when intermediates materialise; PyTorch eager allocates as you go unless you compile or use activation checkpointing. Neither doc set replaces profiling your own peak VRAM. For LLM sizing context, see how much VRAM you need for an LLM.
tinygrad also documents CUDA/Metal interoperability via Tensor.from_blob, so you can keep weights or activations in a PyTorch tensor and run a tinygrad subgraph without a full copy—useful if you are evaluating speed, not rewriting an entire stack.
Ecosystem reality check
Speed only matters if you can express the model. For open-weight LLMs in 2026, PyTorch is still the centre of gravity: Hugging Face Transformers, Unsloth, Axolotl, most GRPO/PPO trainers and most public research land there. tinygrad ships a small nn library and examples, and it is excellent for understanding a full stack, but it is not a drop-in replacement for that ecosystem.
If your question is really "which framework should I standardise on?", start with PyTorch vs JAX. If it is "I want to train or fine-tune models people already publish," stay on PyTorch unless you have a concrete reason to leave. Google Colab is a convenient place to try tinygrad (the MNIST tutorial even gives a one-liner install) without buying new hardware.
Pros and cons (performance lens)
PyTorch
- Pros: fast eager path; mature
torch.compile+ Triton; huge pool of optimised kernels and tools; easy debugging. - Cons: peak performance often needs compile or custom kernels; large API surface; TPU story still more awkward than JAX (see the JAX guide).
tinygrad
- Pros:
TinyJitcan remove Python/driver overhead; BEAM searches kernels for your chip; multi-backend runtimes; small, readable compiler stack. - Cons: slow without JIT; compile/BEAM warmup; scheduler still a training bottleneck per their own Speed docs; small LLM ecosystem; pre-1.0.
Which should you choose?
- You need the open-model / fine-tuning ecosystem: PyTorch.
- You have a fixed-shape loop and will invest in JIT + BEAM: try tinygrad and measure.
- You are learning compilers or bringing up unusual hardware: tinygrad is purpose-built for that.
- You want Apple-first local LLMs: look at MLX (and our Mac guide) before forcing either framework.
- You only care about a published training recipe: use whatever the recipe uses—almost always PyTorch today.
Who this is for
Engineers comparing frameworks on speed rather than API taste, students who saw tinygrad on GitHub and wondered if it replaces PyTorch, and anyone designing a fair microbenchmark.
FAQ
Is tinygrad faster than PyTorch?
Sometimes, after TinyJit (and often BEAM) on a stable loop—tinygrad's own docs claim strong execution speed and usual parity with PyTorch after BEAM search. Eager or ecosystem-heavy workloads usually favour PyTorch. Always measure your model.
What is a fair tinygrad vs pytorch benchmark?
Same model, same dtype, same device, synced GPU timers, warmup excluded from the reported mean, PyTorch tested both eager and torch.compile, tinygrad tested with and without TinyJit/BEAM, and a clear statement of whether you measured compile time or steady state.
Does tinygrad replace torch.compile?
Not as a drop-in. torch.compile accelerates existing PyTorch programs. TinyJit is tinygrad's capture-and-replay tool inside tinygrad's own stack. They solve a similar "stop paying Python every step" problem with different compilers.
Why is my tinygrad code slow?
Common causes: no @TinyJit, timing the first compile, dynamic Python values frozen incorrectly across JIT, or expecting PyTorch-level eager dispatch. Follow the MNIST tutorial pattern before concluding the runtime is the problem.
Can I use tinygrad and PyTorch together?
Yes for tensors on CUDA/Metal via tinygrad's from_blob interop (sync the PyTorch stream first). That is for experiments and hybrid pipelines, not a full training-stack replacement.
Should beginners learn tinygrad or PyTorch first?
PyTorch first if your goal is jobs, papers and open models. tinygrad second (or in parallel) if you want to see a full compiler. Keras 3 can also sit on a PyTorch backend if you prefer a higher-level API.
