gpt-oss vs Gemma 4: Which Should You Run?

In the gpt-oss vs Gemma 4 choice, pick Gemma 4 for most local setups in late 2026: it is newer, reads images on every size, offers longer context (256K tokens on the 12B, 26B A4B and 31B models) and scores higher on the benchmarks both vendors report. Pick gpt-oss when you want a text only reasoning model with adjustable effort levels and built in browsing and Python tool formats, or when you already run the 120b model on an 80 GB GPU. Both families are free to download under Apache 2.0 and run in Ollama, llama.cpp, LM Studio and vLLM.

This guide puts the two families side by side using only the official model cards and runtime pages, checked on November 10, 2026. For setup commands, see our dedicated guides to running gpt-oss locally and running Gemma 4 locally.

gpt-oss vs Gemma 4 at a glance

The short version: Gemma 4 covers far more hardware, from phones to a single workstation GPU, while gpt-oss offers two points on the curve, one for 16 GB machines and one for data center class memory.

Model sizes and architecture

gpt-oss comes in two sizes. According to OpenAI's launch post, gpt-oss-120b has 36 layers, 117B total parameters and 5.1B active per token, using 4 of 128 experts. gpt-oss-20b has 24 layers, 21B total and 3.6B active, using 4 of 32 experts. The expert weights were post trained in the 4 bit MXFP4 format, so the official download is already compact, and OpenAI ran all of its evaluations at that precision.

Gemma 4 offers five instruction tuned sizes on the Gemma 4 31B model card:

• E2B and E4B: small dense models with Per Layer Embeddings. They behave like 2.3B and 4.5B parameter models (5.1B and 8B counting embeddings) and target phones and laptops.
• 12B Unified: a dense 11.95B model with no separate vision or audio encoder. Images and audio are projected straight into the language model.
• 26B A4B: the mixture of experts model, with 25.2B total and 3.8B active parameters (8 of 128 experts plus 1 shared).
• 31B: the largest dense model, at 30.7B parameters.

This matters for speed. gpt-oss-20b (3.6B active) and Gemma 4 26B A4B (3.8B active) do a similar amount of work per token, so you can expect broadly similar generation speed on the same hardware, even though the Gemma model holds more total weights. Memory bandwidth and your runtime still decide the final numbers. Gemma 4 31B is dense, so every token touches all 30.7B parameters and decoding is slower than either mixture of experts model.

Memory and hardware

Download sizes come from the Ollama library pages for gpt-oss and gemma4, checked today. OpenAI says gpt-oss-20b runs within 16 GB of memory and gpt-oss-120b fits a single 80 GB GPU such as an H100 or MI300X. Context adds memory on top of the weights, so leave headroom. Our guide to how much VRAM you need for an LLM shows how to estimate it, and KV cache quantization can cut the cache cost when you raise context.

The awkward spot for gpt-oss is the middle. If you have 24 to 48 GB, the 20b model leaves memory unused and the 120b model does not fit without heavy offloading to system RAM. Gemma 4 has two strong options in exactly that range.

Context length and inputs

gpt-oss supports up to 128K tokens of context and reads text only. OpenAI's post says it was trained on a mostly English, text only dataset focused on STEM, coding and general knowledge.

Gemma 4 doubles that on its larger sizes: 256K tokens on 12B, 26B A4B and 31B, and 128K on the edge models. Every size accepts images, with a configurable visual token budget (70 to 1120 tokens per image) so you can trade detail for speed. E2B, E4B and 12B also accept audio clips up to 30 seconds for speech recognition and translation, and all sizes can read short videos as frames. Google's card says the models were pretrained on over 140 languages with support for 35+ out of the box.

So if your work involves screenshots, scanned documents, charts or non English text, Gemma 4 is the clear choice. One caveat: Ollama's gemma4 tags currently list text and image input, so audio may need Transformers or another runtime that exposes it.

Reasoning and tool use

Both families think before answering, but they expose it differently.

gpt-oss has three reasoning efforts (low, medium and high) that you set with one line in the system prompt, such as Reasoning: high. Higher effort means longer chains of thought and better scores on hard problems, at the cost of latency. The models were trained on OpenAI's harmony response format and the gpt-oss-20b card warns they will not work correctly without it. Most runtimes apply harmony for you through the chat template. gpt-oss also has native formats for function calling, web browsing and Python execution, and supports Structured Outputs.

Gemma 4 thinking is a switch rather than a dial. You turn it on by placing the <|think|> token at the start of the system prompt, and Transformers exposes it as enable_thinking=True. Gemma 4 adds native function calling and, for the first time in the family, a proper system role. Google recommends temperature 1.0, top_p 0.95 and top_k 64 for every use case.

In practice, gpt-oss gives you finer control over how long the model thinks. Gemma 4 gives you a simpler on or off choice. On E2B and E4B, turning thinking off also drops the empty thought block that the larger sizes still print.

Benchmarks: what the vendors report

Neither vendor ran the other's model, and the two cards use different benchmark suites and settings. Only a few tests overlap, and even those may differ in harness and prompt. Treat this table as a rough signal, not a head to head result.

The gpt-oss figures come from the gpt-oss model card on arXiv; the Gemma figures come from the Gemma 4 31B card. On GPQA Diamond both Gemma 4 workstation models report higher scores than gpt-oss-120b, which is a much larger model. On Humanity's Last Exam the picture is mixed: gpt-oss-120b beats Gemma 4 26B A4B, and Gemma 4 31B leads both.

gpt-oss also reports 85.3% (20b) and 90.0% (120b) on MMLU, while Gemma 4 reports MMLU Pro (85.2% for 31B), a harder variant, so those are not comparable. If benchmarks drive your choice, run your own prompts on both. Our shortlist of open source LLM models to run locally lists the Ollama tags to start with.

Licence and commercial use

Both are Apache 2.0 on their Hugging Face pages, which allows commercial use, modification and redistribution. gpt-oss adds OpenAI's gpt-oss usage policy on top. Gemma 4 moved to Apache 2.0, a cleaner grant than the Gemma Terms of Use that covered earlier Gemma releases. Neither publishes its training data, so both are open weight rather than open source in the strict sense; our explainer on open weight vs open source covers the difference.

Fine tuning

Both families can be fine tuned locally, using the steps in our guide to fine tuning an LLM locally. The gpt-oss-20b card says the 20b model can be fine tuned on consumer hardware and the 120b on a single H100 node. Unsloth publishes run and fine tuning guides for gpt-oss and Gemma 4. Gemma 4 gives you more small sizes to train cheaply, and the 12B Unified model can be fine tuned in one pass across text, image and audio because it has no separate encoders. With gpt-oss, keep the harmony format in your training data, and see our guide to fine tuning a thinking or instruct model for the template traps. Tools like Unsloth and Transformers handle both.

Which should you choose?

• You have 8 GB or less, or want a phone model: Gemma 4 E2B or E4B. gpt-oss has nothing this small.
• You have a 16 GB GPU or Mac and want text reasoning: gpt-oss-20b is a fair pick, especially with high reasoning effort for maths and code. Gemma 4 12B is the better pick if you also want images or audio.
• You have 24 to 32 GB: Gemma 4 26B A4B for speed or 31B for quality. This is where Gemma 4 pulls ahead most clearly.
• You have an 80 GB GPU and need a text agent with browsing and Python formats: gpt-oss-120b is still a reasonable choice, though Gemma 4 31B reports higher GPQA Diamond and HLE scores with under a third of the download size.
• You need long documents, screenshots or many languages: Gemma 4.

To try both quickly, install Ollama and run ollama run gpt-oss:20b and ollama run gemma4:26b on the same prompt. If Gemma 4 sizes feel too large, compare smaller options in SmolLM vs Gemma.

FAQ

Is Gemma 4 better than gpt-oss?

On the benchmarks both vendors report, Gemma 4 31B scores higher than gpt-oss-120b on GPQA Diamond and Humanity's Last Exam, and Gemma 4 adds image input and longer context. gpt-oss still offers adjustable reasoning effort and native browsing and Python tool formats. For most local users, Gemma 4 is the stronger general choice.

Is gpt-oss-20b or Gemma 4 26B A4B faster?

They activate a similar number of parameters per token (3.6B and 3.8B), so speed is broadly similar on the same hardware. The Gemma model needs more memory (16 to 19 GB in Ollama versus 14 GB), so it can be slower if it spills out of VRAM.

Can gpt-oss read images?

No. gpt-oss is text only. Every Gemma 4 size accepts images, and the E2B, E4B and 12B models also accept short audio clips.

Are gpt-oss and Gemma 4 free for commercial use?

Yes. Both are published under Apache 2.0, which permits commercial use. gpt-oss also comes with OpenAI's usage policy, so read it before shipping a product.

How does Gemma 4 compare with Gemma 3 and gpt-oss?

Gemma 4 is a large step over Gemma 3: on Google's card, Gemma 4 31B scores 84.3% on GPQA Diamond against 42.4% for Gemma 3 27B without thinking. gpt-oss-20b and gpt-oss-120b sit between them on that test at 71.5% and 80.1%.

Which runs on a Mac?

Both. gpt-oss-20b fits a 16 GB Mac, and Gemma 4 has MLX builds in Ollama for every size. For the runtime tradeoffs on Apple Silicon, see MLX vs GGUF on Mac, and to pick a file size, see GGUF quantization explained.