How to Fine Tune an LLM with a Knowledge Graph

To fine tune an LLM with a knowledge graph, export the graph as triples (subject, relation, object), turn each triple into several differently worded training examples, and train a LoRA adapter on them with a standard supervised fine tuning tool. The bigger decision comes first, though: research shows models pick up new facts slowly and unreliably through fine tuning, so the most dependable pattern is to keep the graph as a live data source and fine tune the model to read and reason over the graph facts you retrieve for it. Use fine tuning to teach the skill and the format; let the graph supply the facts.

This guide covers both routes, plus a third job people often mean by the phrase: training a model to build or query a graph. For the general setup (hardware, LoRA settings, exporting to GGUF), start with our guide to fine tuning an LLM locally.

What a knowledge graph fine tune can and cannot do

A knowledge graph stores facts as nodes and labeled edges: "Aspirin, treats, headache" or "Acme Corp, headquartered in, Denver". When people ask how to fine tune an LLM with a knowledge graph, they usually want one of three things.

1. Put the facts into the model's weights, so it answers questions about your domain with no lookup.
2. Teach the model to use graph facts you hand it, so it reasons over retrieved triples or paths and explains its answer.
3. Teach the model to work on the graph, by extracting triples from text or writing graph queries (Cypher for Neo4j, SPARQL for RDF stores).

The first goal is the hardest. In Fine Tuning or Retrieval?, Microsoft researchers compared fine tuning with retrieval for adding knowledge to Llama 2 7B, Mistral 7B and Orca 2 7B. Retrieval beat fine tuning across their tasks, both for facts the models had partly seen and for genuinely new current events. On the new events set, Mistral 7B scored 0.481 with no help, 0.504 after plain fine tuning, 0.588 after fine tuning on ten paraphrases of each passage, and 0.875 with retrieval alone.

A Google and Technion study, Does Fine Tuning LLMs on New Knowledge Encourage Hallucinations?, is even more relevant here because its training data came from Wikidata triples turned into questions. The authors found that examples carrying facts the model did not already know were learned much more slowly than familiar ones, and that as the model finally fit them, it got worse at answering questions it used to get right. Their practical advice: stop training early, filter out facts the model clearly does not know, or relabel those examples with "I don't know".

So goals two and three are where fine tuning pays off most reliably. Goal one works best for a small, stable set of facts that rarely changes, and only with heavy paraphrasing.

Fine tuning vs graph retrieval: pick the pattern first

If you have not tried plain retrieval yet, do that first. Microsoft's GraphRAG is one documented way to do it: it uses an LLM to build a knowledge graph from your documents, groups entities into communities, writes summaries for them, and adds those structures to the prompt at question time. No training is involved, which makes it the baseline any fine tune has to beat.

Step 1: export the graph as clean triples

Whatever your store, get the facts out as flat rows of subject, relation and object with human readable labels.

• Neo4j: a Cypher query such as MATCH (s)-[r]->(o) RETURN s.name, type(r), o.name returns every edge.
• RDF files or triple stores: rdflib in Python parses Turtle, N-Triples and other formats and runs SPARQL locally.
• Public facts: the Wikidata Query Service answers SPARQL queries if you want to enrich your own graph with general knowledge.

Then clean the export. Replace internal IDs and URIs with readable names (a model learns "Acme Corp" far better than "Q4829304"), turn relation names like headquartered_in into plain phrases, drop duplicates, and decide how to handle facts with dates attached, such as a CEO who changed in 2025. Keep the source of each fact so you can trace mistakes.

Step 2: turn triples into varied training examples

A single template per relation is the most common mistake. Research on how models store facts shows why it fails. In the Physics of Language Models Part 3.1 study, researchers trained small models from scratch on synthetic biographies. With one fixed biography per person, question answering on held out people after fine tuning reached 9.7%. With five differently worded biographies per person and shuffled sentence order, it reached 96.6%. That experiment varied pretraining data, not a fine tuning set, but it agrees with the paraphrase result above: a model needs each fact phrased many ways.

For each triple, generate several kinds of text:

• Statements in different wordings: "Acme Corp is headquartered in Denver." "Denver is home to Acme Corp's head office."
• Questions in both directions: "Where is Acme Corp based?" and "Which company has its headquarters in Denver?" The Reversal Curse paper showed that models trained on "A is B" often fail to answer "B is A", so write the reverse explicitly.
• Multi hop questions from graph paths: if Acme is based in Denver and Denver is in Colorado, ask "Which state is Acme's head office in?" with an answer that names both steps.
• Refusals for gaps: questions about entities or relations that are not in the graph, answered with a plain statement that the information is not available.

An instruct model can write the paraphrases; keep each one tied to its triple so you can check no fact changed. Store each example in the prompt and completion format that TRL's SFTTrainer accepts:

{"prompt": [{"role": "user", "content": "Which company has its headquarters in Denver?"}], "completion": [{"role": "assistant", "content": "Acme Corp is headquartered in Denver."}]}

Split by entity, not by row. If every paraphrase of a fact can land in either the training or the test set, your test score only measures memorized wording.

Step 3: teach the model to reason over retrieved graph paths

This is the route most teams should take. Instead of asking the model to remember the graph, you put the relevant triples or paths in the prompt and train it to answer from them, cite the path it used, and ignore the noise.

The Reasoning on Graphs (RoG) paper from Monash and Griffith universities is the best documented example. It instruction tuned Llama 2 Chat 7B on two tasks: generating relation paths (a plan such as "child of, then has son") that point to the answer in the graph, and answering a question from the reasoning paths retrieved by following that plan. The training sets came from Freebase question answering benchmarks, with 216,006 planning examples and 30,465 reasoning examples, and training took 38 hours on two A100 80 GB GPUs. The authors report it beat their baselines on the WebQSP and CWQ benchmarks, and their code and data are public.

You can apply the same idea at a smaller scale. Each training example holds:

1. The question.
2. A block of retrieved paths, written one per line, such as Acme Corp -> headquartered_in -> Denver -> located_in -> Colorado, mixed with a few irrelevant paths so the model learns to filter.
3. An answer that names the result and the path behind it.

Because the facts arrive in the prompt, you can update the graph without retraining.

Step 4: train the adapter

Any standard SFT stack works. TRL with PEFT is the most direct; Unsloth cuts memory use on a single GPU, and Axolotl runs the same job from a YAML file. A minimal TRL run on the prompt and completion file from Step 2 looks like this:

from datasets import load_dataset
from peft import LoraConfig
from trl import SFTConfig, SFTTrainer

train = load_dataset("json", data_files="kg_train.jsonl", split="train")

trainer = SFTTrainer(
    model="Qwen/Qwen3-4B-Instruct-2507",
    train_dataset=train,
    args=SFTConfig(output_dir="kg-lora", num_train_epochs=2, learning_rate=2e-4),
    peft_config=LoraConfig(r=16, lora_alpha=32, target_modules="all-linear", task_type="CAUSAL_LM"),
)
trainer.train()

With prompt and completion data, TRL computes the loss on the completion only by default, so the model is not trained to repeat your questions. A few settings matter more for knowledge work than for style tuning:

• Evaluate every epoch and keep the best checkpoint. The hallucination study found the damage from unfamiliar facts shows up late in training, so early stopping on a held out set protects you.
• Mix in general instruction data if the model will also chat about other things, so it does not forget how.
• Start with LoRA. If memory is tight, our LoRA vs QLoRA comparison explains the tradeoff, and our Google Colab fine tuning guide shows a free Colab setup if you have no local GPU.

Step 5: train a model to build or query the graph

The third goal turns the arrow around. To extract triples from documents, train on pairs of a passage and the JSON list of triples it contains, using a fixed set of relation names. To let users search the graph in plain English, train on pairs of a question and the Cypher or SPARQL query that answers it.

Both tasks have a big advantage: you can check outputs mechanically. Parse the JSON, validate relation names against your schema, and run every generated query against a copy of the database to confirm it executes and returns the expected rows. The RoG authors note that query generation methods fail when the produced query cannot run, so put that check in your evaluation, not just your production code. Our guide to fine tuning an LLM for code generation covers execution based evaluation in more detail.

How to test a knowledge graph fine tune

• Held out entities: questions about facts whose paraphrases never appeared in training.
• Reversed and multi hop questions: these show whether the model learned the fact or the sentence.
• Questions about things not in the graph: the model should decline rather than invent.
• Questions it answered correctly before training: compare the base and tuned model on general facts to catch the regression the hallucination study describes.
• A retrieval baseline: run the base model with the same triples in the prompt. If the fine tune does not beat it, ship the retrieval system instead.

Common mistakes

• One template per relation. The model memorizes the sentence and cannot answer a reworded question.
• Random row splits. Paraphrases leak into the test set and inflate scores.
• Training too long. Unfamiliar facts get forced in late, and accuracy on other questions drops.
• Expecting the weights to stay current. A graph that changes weekly belongs in a retrieval step, not in the adapter.

FAQ

Can you fine tune an LLM directly on a knowledge graph?

Not on the graph structure itself. You convert it into text first: verbalized triples, questions and answers, or prompts that contain graph paths. The model then trains on that text like any other supervised dataset.

Is GraphRAG the same as fine tuning with a knowledge graph?

No. GraphRAG builds a graph and community summaries from your documents and feeds them into the prompt at question time, with no change to the model's weights. Fine tuning changes the weights. You can combine them by fine tuning a model to read graph context better.

How do I fine tune an LLM for domain knowledge?

Decide whether the model must recall the facts unaided or can look them up. For lookup, retrieval plus a light fine tune for format and reasoning is the more dependable option in published comparisons. For recall, write each fact in many wordings and both directions, train briefly, and test on held out entities.

How many paraphrases per fact do I need?

There is no fixed number. The Microsoft study used ten paraphrases per passage for its new knowledge set and saw accuracy rise steadily as the count went up. Treat that as a starting point and measure on your own held out questions.

Can I run a local LLM with a knowledge graph?

Yes. The graph can live in Neo4j, an RDF file read by rdflib, or even a CSV of triples (no graph database is needed just to build training data), and the model can run in Ollama or llama.cpp after you merge and export the adapter. Our local fine tuning guide covers the export step.