Which AI Model Saves Recipes Best? 21 Models Tested on Polish and English Recipes

Every recipe saved in ZapiszPrzepis goes through an AI model. The model pulls out the ingredients, quantities, steps, temperatures and times, puts them in order by the app’s rules, and translates a recipe from a foreign site on top of that. It started with the translation, because recipes from English blogs came out half translated: „1 funt” (1 pound) next to the grams, „mięso mielone z indyka (94% chude)” (94% lean ground turkey), „mały makaron (np. ditalini, alfabet)” (alphabet pasta, word for word) and the step „Umieścić mięso mielone z indyka w misce”, a stiff calque of “Place the turkey in a bowl”. I fixed the prompt and the code, and then checked which model should do the job: first 11 models on the app’s format, then 21 models on whether the saved recipe matches the original.

ZapiszPrzepis is a Polish app, so the examples below are in Polish, with a gloss where it matters.

In short:

• The top is a tie. Claude Haiku 5.5, Gemini 3.5 Flash, Gemini 3.1 Flash-Lite, gpt-4.1-mini and gpt-5.4-mini fall within the same confidence intervals.
• The app keeps gpt-5.4-mini. With the full input it has 0.09 critical errors per recipe, about one in eleven recipes, and answers in 4.4 seconds.
• The more expensive model was not better. Claude Sonnet 5.5 cost about ten times as much as Claude Haiku 5.5 and did not do better.
• The host matters more than the model size. The same gpt-oss via OpenRouter made anywhere from zero to over 12 critical errors per output, depending on who ran it.
• More reasoning did not help gpt-5.4-mini.
• Polish does not separate the cloud models. In my blind rating all 20 outputs from different models got 5 out of 5.
• The local models do worst, above all on English recipes, where qwen3:14b writes broken Polish.

How ZapiszPrzepis extracts a recipe

ZapiszPrzepis saves a recipe from a link: it fetches the page, and a language model turns its text into JSON with the title, ingredients, steps, times and category. An ingredient has four fields: amount, unit, name and a household measure, for example 480 ml with „2 szklanki” (2 glasses) beside it. The prompt has about 40 rules, from categories to the form of the steps: the infinitive and no narration, which is how Polish cookbooks write them.

For a recipe from a foreign site, the same prompt also has to translate it and fit it to a Polish kitchen. It had one sentence for that: translate to Polish and convert US units to metric (1 cup ≈ 240 ml, 1 tbsp ≈ 15 ml). It said nothing about pounds, ounces, degrees Fahrenheit or brands.

Why not DeepL

The first reflex is to add DeepL. But translation is not localization. DeepL would turn “finely grated” into „drobno starte”, and a glossary could pin “ground turkey” to the right Polish phrase. It would not do three things:

• convert pounds to grams or °F to °C,
• move a note (“sliced”, “juiced”) out of the measure field into the ingredient name,
• swap a US brand used as an ingredient for a product you can buy in a Polish shop.

It would also be a second paid service and a second pass that translates short JSON fields without context, and a lone “splash” easily becomes the Polish word for a splash of water. Four of the six reported problems would remain.

What I changed

The change has three parts.

1. Code detects the source language before it calls the model. It counts Polish diacritics and common Polish and English words in the page text. The lang attribute is only a hint, because many Polish blogs ship lang="en-US" in their theme.
2. A foreign source gets an extra block of rules, appended to the shared prompt: convert pounds, ounces, sticks of butter and °F (in the steps too), translate every word, write the steps the way a Polish cookbook does, name products the way Polish shops do (alphabet pasta is „literki”), and drop US lean grades. A brand stays when it is the dish itself (homemade SpaghettiOs) and goes when it is an ingredient. I did not write a separate English prompt, because it would drift from the shared one with the first rule change.
3. Checks in code, because the model ignored some rules. Two mistakes survived dry runs on gpt-4.1-mini despite the rules in the prompt, so the code now enforces them. The household-measure field keeps only a Polish measure („półtorej szklanki”, „płaska łyżka”, „1 saszetka”). A note moves into the name, a US weight is dropped when grams are there, and a measure that repeats the amount is dropped too. This check runs for every source: of the 53 measures stored in the database, it changed 14, and each one was a note or a US unit. For foreign sources the code also strips a percentage before „chude” (lean) and keeps a cup or spoon measure only when the original gives one.

Before and after, on the three howsweeteats.com recipes that started it:

Two things are still not perfect. The model wrote “Spaghettios” instead of the original’s “SpaghettiOs”. And the cup check looks at the whole recipe, not at each ingredient, so a made-up „1/2 szklanki” gets through if the original says “1/2 cup” for some other ingredient.

The first test: does the model keep the format

The first test checked whether a model follows the app’s rules: 26 recipes (25 Polish and one English), 11 models (5 from OpenAI, 3 on free Groq and 3 local), each recipe run twice. The app’s own code scored the raw JSON, before any checks run.

The numbers are totals over 52 extractions per model. A format error is one of three kinds:

gpt-5.4-mini made the fewest format errors, almost never got the step form wrong and was the fastest of the OpenAI models, so after this test it replaced gpt-4o-mini in the app. Bielik did worst: in over 200 ingredients it put the measure in the wrong field, and it did not write steps in the infinitive.

That test only said whether a model keeps the format, though. It did not say whether the saved recipe is true: whether the quantities and temperatures match the original and nothing is missing. That needed a second test.

The second test: does the saved recipe match the original

Recipes and reference

The set is 50 recipes saved in the app: 25 from Polish sites and 25 from English blogs in the US, the UK, Australia and India. Three dropped out because the saved page held no recipe (a paywall or an empty page). Two more dropped out of the markdown-only variant only, because the recipe started after the first 10,000 characters.

Each recipe got a reference: ingredients with quantities, steps, temperatures, times, servings and photos matched to steps. Two models drafted it independently: Claude Fable 5.1 (in Claude Code) and Gemini 3.5 Flash at high reasoning effort. Where they agreed, the reference was done. I settled every difference myself, blind, without knowing which model wrote what. I also checked 31 randomly sampled points where they agreed against the original, and all 31 were right. I set three rules too: a value within a stated range is correct, a recipe must keep both oven modes (top-and-bottom and fan), and a computed or estimated total time is not an error.

One caveat, stated plainly: the reference was drafted by models from the same families as some of the contestants (Claude and Gemini). My review of the differences reduces that risk but does not remove it, and the Claude and Gemini models happen to be at the top.

What counts as an error

A model is scored only on what it got as input. Confidence intervals are 95%, bootstrapped over recipes (2,000 resamples). Each recipe went through each model twice, with the same prompt as in production.

I also rated the Polish myself: 20 outputs from different models, blind. All of them got 5 out of 5, so fluency does not separate these models, and an AI judge would have had nothing to learn from me. That is why I did not use one.

Two input variants

In production the model gets the page text as markdown plus its HTML, each up to 20,000 characters. Free Groq will not take that: the limit is 8,000 tokens per minute, and that is also the largest request you can send, while a full request is 10–13 thousand tokens. That is why the test has two variants: full (markdown and HTML, as in production) and markdown only (the first 10,000 characters). Every model ran the markdown-only variant, so that is where the comparison is complete.

The models

• 5 from OpenAI, called directly: gpt-4o-mini, gpt-4.1-mini, gpt-5-mini, gpt-5-nano and gpt-5.4-mini, the last one also at a higher reasoning effort (low);
• 10 through OpenRouter: Claude Haiku 5.5 and Sonnet 5.5, Gemini 3.5 Flash and 3.1 Flash-Lite, Gemma 4 31B, GLM 5.3, Qwen 3.7 Plus, DeepSeek v4 Flash, Mistral Medium 3.1 and Llama 4 Maverick;
• 3 on free Groq: gpt-oss-120b, gpt-oss-20b and qwen3.8-27b. I also ran both gpt-oss models through OpenRouter to compare hosts;
• 3 local, through Ollama: Bielik 11B v2.3 and v3.0 (Q4_K_M quantization) and qwen3:14b.

Results

Two exceptions to the full set. Free Groq rejects any request over 8,000 tokens, and six English recipes do not fit within that even as markdown only (three for gpt-oss-20b). The Groq rows carry a †: they are scored without those longest recipes, which matters in every comparison. I capped the local models’ output at 4,096 tokens, because qwen3:14b looped on one recipe for over 17 thousand tokens (a normal recipe is about 1,300). An output cut by the cap counts as broken JSON. The “Recipes PL/EN” column says how many recipes each model received.

A bar’s colour shows how I called the model, not who made it: through the OpenAI API, through OpenRouter, on free Groq or locally through Ollama. That is why gpt-oss, an OpenAI model, takes the colour of Groq or OpenRouter.

Correctness, markdown only

Critical, major and minor are mean errors per recipe; the brackets hold the 95% confidence interval. Critical PL and EN count recipes from Polish and English sites separately. Polish is LanguageTool errors per 1,000 words. † free Groq rejected the longest English recipes, so these rows are scored without them.

Cost and reliability, markdown only

Median is the time of one extraction; an asterisk marks OpenRouter times measured separately on 6 recipes. Cost is the amount per recipe from OpenRouter’s accounting; OpenAI and Groq did not report it in their responses and the local models cost nothing, hence the dash. Rejected counts the requests free Groq refused (an 8,000-token limit). Broken JSON counts outputs that could not be read, including those cut by the 4,096-token cap on the local models. Recipes PL/EN says how many recipes each model received.

Correctness, full input

Critical, major and minor are mean errors per recipe; the brackets hold the 95% confidence interval. Critical PL and EN count recipes from Polish and English sites separately. Polish is LanguageTool errors per 1,000 words. † free Groq rejected the longest English recipes, so these rows are scored without them.

Cost and reliability, full input

Median is the time of one extraction; an asterisk marks OpenRouter times measured separately on 6 recipes. Cost is the amount per recipe from OpenRouter’s accounting; OpenAI and Groq did not report it in their responses and the local models cost nothing, hence the dash. Rejected counts the requests free Groq refused (an 8,000-token limit). Broken JSON counts outputs that could not be read, including those cut by the 4,096-token cap on the local models. Recipes PL/EN says how many recipes each model received.

What kind of errors

Critical errors by kind, counted over all outputs in the markdown-only variant:

What the numbers say

The top is a tie

On markdown alone the fewest critical errors go to Claude Haiku 5.5 (0.01), Gemini 3.5 Flash (0.02), Gemini 3.1 Flash-Lite (0.03) and gpt-4.1-mini (0.04). Their confidence intervals overlap with gpt-5.4-mini (0.08), though, so with 50 recipes there is no honest way to say which of them is best. The difference you can measure separates the top from the rest: the local models, gpt-5-nano, gpt-oss via OpenRouter and Llama 4 Maverick make many times as many critical errors.

The app keeps gpt-5.4-mini

In the setup the app uses, with the page HTML, gpt-5.4-mini has 0.09 critical errors per recipe, the fewest major (0.41) and minor (0.56) errors of the models in that variant, and a median of 4.4 seconds. Switching to another top model would bring no measurable gain, and gpt-5.4-mini also won the first test, the one on format.

The more expensive model was not better

Claude Sonnet 5.5 cost $0.0215 per recipe through OpenRouter and had 0.06 critical errors per recipe. Claude Haiku 5.5, at $0.0022, had 0.01, and Gemini 3.1 Flash-Lite, at $0.0065, had 0.03. Over a thousand recipes that is about $21.50 against $2.20, with no better result for the higher price.

Same model, different host, different result

This is the biggest surprise of the test. OpenRouter spread the gpt-oss requests over more than ten companies that run the model themselves, with the same request settings. The result depended on the company more than on the model size:

That explains why gpt-oss-120b called directly on Groq made no critical errors on the Polish recipes, while through OpenRouter it had 0.80 per recipe. If you use OpenRouter for a task where accuracy matters, name the hosts you have checked in the request. Otherwise you get a random one.

More reasoning did not help

gpt-5.4-mini at reasoning effort low had 0.24 critical errors per recipe, and at the lowest (none) 0.08. The intervals overlap at the edge, so I do not claim that reasoning hurts, but it certainly did not help: the model produced more tokens, answered more slowly (7.2 instead of 4.6 seconds) and dropped ingredients more often.

HTML does not reduce critical errors

The OpenAI models made about as many critical errors on markdown alone as with the HTML, with about half the input: gpt-5.4-mini 0.08 against 0.09, gpt-4o-mini 0.20 against 0.19. The HTML stays in the app all the same, because the step photos come from it.

Polish: a blind rating sees no difference

In the blind rating I gave every output 5 out of 5. LanguageTool does find differences between the models: with the full input, from 0.9 errors per 1,000 words for Bielik 11B v2.3 and 1.1 for gpt-4o-mini up to 6.8 for gpt-5-nano. Given its hit rate, that is a ranking signal, not a verdict.

The exception is qwen3:14b, at 18.5 errors per 1,000 words. On English recipes it writes broken Polish, and its hits were checked by hand and are real: „pudr”, „vanilię”, „nacisknij”, „wyciąnij”, „mikrowarce”, and „auberginy” instead of the Polish word for aubergines.

Bielik and the local models

The local models do worst, and worse still once the English recipes are in. On markdown alone Bielik 11B v2.3 has 0.62 critical errors per recipe, v3.0 0.64 and qwen3:14b 1.00. With the full input it is 0.82, 1.68 and 1.17, so there the newer Bielik does worse than 2.3.

They handle Polish recipes far better than English ones. Bielik 11B v2.3 has 0.24 critical errors per recipe on Polish sources and 1.02 on English ones (markdown only). qwen3:14b hardly errs on Polish sources (0.02) and has 2.13 on English ones. Bielik most often drops ingredients and gets temperatures wrong, and it does not keep the app’s format either: v2.3 has 37.9 minor errors per recipe. Its Polish is sound, with the fewest errors on full input (0.9 per 1,000 words), which fits the first test: it knows Polish but does not keep the format.

Locally a recipe takes just under a minute on Bielik and a minute and a half on qwen3:14b with markdown only, longer with the HTML. qwen3:14b with a 24-thousand-token context does not fit in the card’s 12 GB of memory, so part of the model runs on the CPU from system RAM. That explains its times; it would be faster on a machine with a bigger GPU.

Other observations

• DeepSeek v4 Flash returned broken JSON in 19 outputs, generated over 12 thousand tokens per output on average and needed more than two minutes.
• Mistral Medium 3.1 and Llama 4 Maverick drop ingredients most often: 20 and 30 times.
• gpt-5-nano, OpenAI’s cheapest model, has the most critical errors of the cloud models on markdown alone, mostly missing ingredients (51).

What this test does not tell you

• Models drafted the reference. I checked every difference between them and a sample of their agreements, but if both models misread the same passage outside the sample, I would not have noticed. Fable and Gemini belong to the same families as some of the contestants.
• Free Groq rejected the longest English recipes (an 8,000-token limit per request), so its rows (†) leave them out and are not fully comparable with the rest.
• The local models’ output was capped at 4,096 tokens. An output cut by the cap counts as broken JSON (qwen3:14b: 2 with the full input, 5 with markdown only).
• 50 recipes and two runs are enough to separate the top from the bottom of the table, but not to rank the top.
• One person rated the Polish, on 20 outputs, and LanguageTool hits a real error in about 57% of cases.
• OpenRouter times (asterisk) were measured separately on 6 recipes and depend on the host and the time of day.
• Cost is shown where OpenRouter billed it. Prices are from 25 and 26 October 2026.
• I wrote no prompt for Bielik or any other model. All of them got the prompt that grew up with the OpenAI models.

Summary

• Several models save recipes with almost no critical errors, and the differences between them are within the measurement’s uncertainty. Choosing within that group comes down to price, speed and whose API you already use.
• ZapiszPrzepis keeps gpt-5.4-mini: about one critical error in eleven recipes and an answer in 4.4 seconds.
• Neither a more expensive model nor more reasoning improved the result.
• On OpenRouter, check the host, not just the model.
• Local models, Bielik included, are free in money but slow, and with this prompt they make more mistakes.

How do you choose a model for your own app: on benchmarks, on your own data, or do you just take the one you already know? I would like to hear what works for you.