legal-it-8b: Ministral-3-8B fine-tuned for Italian law, trained on a European supercomputer

[u/primoco]

We took Mistral's Ministral-3-8B (Apache 2.0) and trained it to answer questions on Italian law from the legal texts a retrieval system puts in the prompt, citing them. Supervised fine-tuning on ~19k open-book examples written from public legislation (Normattiva), then GRPO with programmatic rewards.

Trained on EuroHPC Leonardo (CINECA, Italy). European end to end: European base model, European compute, public Italian law as data.

• 89.0% correct on 429 held-out questions, vs 80.9% for the base Ministral-3-8B with the same retrieval
• Q4_K_M GGUF, 5.2 GB, runs locally (llama.cpp, Ollama, EuLLM Engine)
• It beats our Qwen3-4B-based sibling (85.1%) and matches or beats every Qwen3.5-9B we have trained since with the same recipe: the Mistral base is still our best

It is built for RAG. In a plain chat, without the legal texts in the prompt, it will confidently invent articles. A version that says when the texts are missing is in training.

Next we'd like to distil from a larger Mistral teacher into Ministral-3-3B and 8B, all on EuroHPC. Ideas and criticism welcome.

Try it and tell us where it fails. The model card has the prompt format and the retrieval setup we use. We're especially interested in:

• questions where it gets the law wrong or cites the wrong article
• cases where it answers although the right text isn't in the prompt
• how it behaves in your own RAG stack

Italian lawyers and law students: your feedback is gold. Comment here or open a discussion on the Hugging Face page.

https://huggingface.co/EuLLM/legal-it-8b

[u/EconomyBitter1439]

This is a model I could fine train on a single 5090. No cluster needed. Weird flex.

[u/primoco → u/EconomyBitter1439]

Fair question, and you may well be right about the training step itself. It never needs a teacher and a student in memory at the same time. Here’s what we did: Open-book SFT, LoRA rank 32, one epoch on 18,972 examples. A 30B model wrote question-answer pairs from single articles, told not to add anything the text doesn’t say. But the training prompt holds everything retrieval returns for that question, so the model learns to find the right article among the wrong ones. 15% are questions about articles that don’t exist, where the right answer is “it isn’t there”.

GRPO with programmatic rewards, 250 steps, 8 answers per prompt. The questions ask for time limits (by article number or by topic), for articles that don’t exist, and some have their own article taken out of the texts. A program scores the answers, no model involved: the reward goes to the right time limit, or to saying “not in the texts” when the answer isn’t there.

Both LoRAs merged into the weights, then a Q4_K_M GGUF, which scored the same as bf16 (437 vs 435 of 472). On 429 held-out questions with the same retrieval, the base scores 80.9%, SFT alone 84.6% and SFT + GRPO 89.0%.

We haven’t tried it on a 32 GB card, so I can’t promise it fits. On Leonardo the SFT ran on one 64 GB A100 and GRPO on three, mostly to get the sampling done in time. The 30B that wrote the data and graded the exams takes two of those GPUs in bf16. Where a cluster really is needed is the next step: distilling from a larger Mistral teacher, which has to sit in memory next to the student.

[u/viitorfermier]

How much did it cost? How much data was it?

[u/primoco → u/viitorfermier]

Data: 18,972 SFT examples built from public Italian legislation (Normattiva). About 85% are question-answer pairs a 30B model wrote from a single article; 15% are questions about articles that don’t exist. Then 250 GRPO steps with programmatic rewards. No court decisions, no personal data.
On cost, do you mean what it took to train, or what it costs to run? Training ran on a EuroHPC research allocation, so there was no cloud bill. The model itself is free under Apache 2.0, and running it means a 5.2 GB file on your own hardware. If you’re thinking of running it in an office or a firm, happy to talk details.

[u/cosimoiaia]

Release the dataset as well, then it would be a good effort.

An 8B model can be fine tuned on a potato with a few clicks, there's nothing special in that recipe. I know law firms that did that themselves years ago.

A good Italian law dataset on the other hand can be useful.

I hope you expand your efforts to broader scopes and bigger models since you have that amount of compute available.

[u/C080]

Was Cineca really necessary for this? If you have access to that amount of compute pls aim higher or you'll never going to build "sovereign LLM"

[u/FamouslySqueaky → u/C080]

The real question is what else were they supposed to use, some rented gpu cluster in a random cloud region that undercuts the whole sovereign angle from the start

the entire point is the stack stays within european borders, compute included. doesn't matter if it's a modest 8b model, the infrastructure choice is the message just as much as the final weights

also cineca's not just sitting there waiting for the one big breakthrough model, they're running dozens of smaller experiments like this all the time. that's how you actually build the expertise and tooling for when you do go bigger

[u/primoco → u/FamouslySqueaky]

Right, this is our direction!

[u/arkham00 → u/FamouslySqueaky]

But what is the point in the sovereign angle here? It is a model trained on publicly available data anyway...

[u/primoco → u/arkham00]

The training data is public on purpose: no court decisions and no personal data went in. The point is where the model runs. A lawyer’s files are confidential, and a model this size runs on their own machine or on their own server, so the documents never leave it. Open weights under Apache 2.0, like the Mistral base, trained on EuroHPC: nobody can switch it off or change its terms of use for you.

[u/Soifon99 → u/arkham00]

that money spend stays in the EU? and does not make our china or US Neighbors fatter and money.

[u/primoco → u/C080]

Fair point, but a “sovereign LLM” isn’t what we’re after. Mistral already builds Europe’s general-purpose models, and this one is built on top of theirs. We’re working at the other end: small models specialised in one domain that run on modest hardware, with no supercomputer and no cloud API. legal-it-8b is a 5.2 GB GGUF (Q4_K_M) that runs locally, next to your own documents.
Leonardo is where the training happened (SFT, then GRPO), not where the model is meant to run. As for aiming higher: the next step is a larger Mistral teacher to distil Ministral-3 3B and 8B. The teacher gets bigger, but what you download stays small.

[u/gized00]

Let's hope AI succeeds where humans failed /s

[u/ComprehensiveJury509]

How does it compare to larger models ootb? Is it better than, say, Qwen3.8 27B? Or larger Mistral models if it being European is a concern.

[u/primoco → u/ComprehensiveJury509]

We haven’t published numbers for models in that size class yet. Two comparisons we have run:
a larger Mistral: Ministral-3-14B, trained with exactly the same recipe, scored 436 on our 472-question development exam, against 437 for the 8B. On this task, going up in size within the same family bought nothing.
Qwen3.5-9B: on our exams the 8B Mistral ties or wins. Counting only fully correct answers, the difference isn’t statistically significant; counting partial answers too, it is.
Against its own base, Ministral-3-8B with the same retrieval, it’s 89.0% vs 80.9% on 429 held-out questions. The goal here is a 5 GB file that runs on the lawyer’s own machine, so a bigger model that scores the same isn’t the one we’d ship.

[u/jcrestor]

Is it perceived as problematic for real world deployment that it still gets one in ten questions wrong?

[u/primoco → u/jcrestor]

Yes, if you used it on its own, and the honest picture is worse than “one in ten”. The 89% is on questions generated from the text of the articles, with the right text usually in the prompt. On harder, practical questions the scores are lower, and an external test on criminal-law questions with case law went clearly worse. The grading has a margin too: our automatic judge agreed with an independent blind review 68-85% of the time, which is why the card also reports program-graded results.
The rate matters less than the kind of error. A wrong answer you can spot is manageable. A confident one, with the wrong number or a false premise accepted, is the dangerous kind, and that’s mostly what ours are. So the model isn’t meant to be the last word: it’s a reader behind retrieval, and what makes it deployable is what surrounds it. Answers should cite the passage they come from so a lawyer can check in seconds, there should be a check that the governing law is actually in the context, and the numbers and quotes should be verified against the source. A version that says when the texts aren’t enough is in training.

[u/jcrestor → u/primoco]

I will not pretend to understand your approach completely, but this is interesting work. Thank you for sharing!

[u/Robert__Sinclair]

I'm Italian. I read the full thing and I can tell you it's rubbish. It can be done way better with LESS.

[u/primoco → u/Robert__Sinclair]

And ...

[u/FalseRegister]

Is the training data the law only or also decisions taken by judges (that set a precedence)?

[u/primoco → u/FalseRegister]

Statutes only. Training used public legislation from Normattiva (the civil and criminal codes and their procedure codes, the consumer code, the administrative procedure code, the Constitution, and a couple of key laws). No court decisions and no personal data, in either training stage.
Italy is a civil law system, so judgments aren’t binding precedent the way they are in common law, although Cassation rulings carry a lot of weight in practice. The model answers from the legal texts retrieval puts in its prompt, and case law isn’t in that collection. We haven’t evaluated it on judgments.
Case law is the part we’re working on now (Cassazione and Consiglio di Stato rulings, pseudonymised), but nothing trained on it has been released.