# Which AI Model Saves Recipes Best? 21 Models Tested on Polish and English Recipes

21 AI models, 50 Polish and English recipes, a hand-verified reference. The top models tie, and the host can ruin a good model.

**Canonical:** https://spoko.space/blog/choosing-ai-model-zapiszprzepis/  
**Language:** en  
**Published:** 2026-10-26  
**Updated:** 2026-10-27  
**Tags:** AI, LLM, OpenAI, OpenRouter, Groq, Ollama, Bielik, ZapiszPrzepis  
**Category:** AI

---
**Every recipe saved in ZapiszPrzepis goes through an AI model.** The model pulls out the ingredients, quantities, steps, temperatures and times, puts them in order by the app's rules, and translates a recipe from a foreign site on top of that. It started with the translation, because recipes from English blogs came out half translated: „1 funt” (1 pound) next to the grams, „mięso mielone z indyka (94% chude)” (94% lean ground turkey), „mały makaron (np. ditalini, alfabet)” (alphabet pasta, word for word) and the step „Umieścić mięso mielone z indyka w misce”, a stiff calque of "Place the turkey in a bowl". I fixed the prompt and the code, and then checked which model should do the job: first 11 models on the app's format, then 21 models on whether the saved recipe matches the original.

ZapiszPrzepis is a Polish app, so the examples below are in Polish, with a gloss where it matters.

In short:

- **The top is a tie.** Claude Haiku 5.5, Gemini 3.5 Flash, Gemini 3.1 Flash-Lite, gpt-4.1-mini and gpt-5.4-mini fall within the same confidence intervals.
- **The app keeps gpt-5.4-mini.** With the full input it has 0.09 critical errors per recipe, about one in eleven recipes, and answers in 4.4 seconds.
- **The more expensive model was not better.** Claude Sonnet 5.5 cost about ten times as much as Claude Haiku 5.5 and did not do better.
- **The host matters more than the model size.** The same gpt-oss via OpenRouter made anywhere from zero to over 12 critical errors per output, depending on who ran it.
- **More reasoning did not help** gpt-5.4-mini.
- **Polish does not separate the cloud models.** In my blind rating all 20 outputs from different models got 5 out of 5.
- **The local models do worst**, above all on English recipes, where qwen3:14b writes broken Polish.

## How ZapiszPrzepis extracts a recipe {#how-zapiszprzepis-extracts-a-recipe}

[ZapiszPrzepis](https://spoko.space/zapiszprzepis/) saves a recipe from a link: it fetches the page, and a language model turns its text into JSON with the title, ingredients, steps, times and category. An ingredient has four fields: amount, unit, name and a household measure, for example 480 ml with „2 szklanki” (2 glasses) beside it. The prompt has about 40 rules, from categories to the form of the steps: the infinitive and no narration, which is how Polish cookbooks write them.

For a recipe from a foreign site, the same prompt also has to translate it and fit it to a Polish kitchen. It had one sentence for that: translate to Polish and convert US units to metric (1 cup ≈ 240 ml, 1 tbsp ≈ 15 ml). It said nothing about pounds, ounces, degrees Fahrenheit or brands.

## Why not DeepL {#why-not-deepl}

The first reflex is to add DeepL. But translation is not localization. DeepL would turn "finely grated" into „drobno starte”, and a glossary could pin "ground turkey" to the right Polish phrase. It would not do three things:

- convert pounds to grams or °F to °C,
- move a note ("sliced", "juiced") out of the measure field into the ingredient name,
- swap a US brand used as an ingredient for a product you can buy in a Polish shop.

It would also be a second paid service and a second pass that translates short JSON fields without context, and a lone "splash" easily becomes the Polish word for a splash of water. Four of the six reported problems would remain.

## What I changed {#what-i-changed}

The change has three parts.

1. **Code detects the source language before it calls the model.** It counts Polish diacritics and common Polish and English words in the page text. The `lang` attribute is only a hint, because many Polish blogs ship `lang="en-US"` in their theme.
2. **A foreign source gets an extra block of rules**, appended to the shared prompt: convert pounds, ounces, sticks of butter and °F (in the steps too), translate every word, write the steps the way a Polish cookbook does, name products the way Polish shops do (alphabet pasta is „literki”), and drop US lean grades. A brand stays when it is the dish itself (homemade SpaghettiOs) and goes when it is an ingredient. I did not write a separate English prompt, because it would drift from the shared one with the first rule change.
3. **Checks in code, because the model ignored some rules.** Two mistakes survived dry runs on gpt-4.1-mini despite the rules in the prompt, so the code now enforces them. The household-measure field keeps only a Polish measure („półtorej szklanki”, „płaska łyżka”, „1 saszetka”). A note moves into the name, a US weight is dropped when grams are there, and a measure that repeats the amount is dropped too. This check runs for every source: of the 53 measures stored in the database, it changed 14, and each one was a note or a US unit. For foreign sources the code also strips a percentage before „chude” (lean) and keeps a cup or spoon measure only when the original gives one.

Before and after, on the three howsweeteats.com recipes that started it:

| Before | After |
| --- | --- |
| 450 g · mielone indyka · beside the grams: 1 funt (1 pound) | 450 g · mięso mielone z indyka (ground turkey), no household measure |
| szalotka · beside the grams: finely grated | szalotka drobno starta (shallot, finely grated) |
| mięso mielone z indyka (94% chude) | mięso mielone z indyka (najlepiej chude), "preferably lean" |
| mały makaron (np. ditalini, alfabet) | drobny makaron do zup, np. małe rurki (ditalini), literki: soup pasta, the shapes Polish shops sell |
| 225 g · makaron · beside the grams: ½ funta (½ pound), after the first fix 1/2 szklanki (½ glass) | 225 g · makaron, no household measure |
| Umieścić mięso mielone z indyka w misce (calque of "Place … in a bowl") | Przełożyć mięso mielone z indyka do miski (how a Polish cookbook says it) |
| internal temperature 175 to 180 degrees (°F read as °C) | 75–80°C |

Two things are still not perfect. The model wrote "Spaghettios" instead of the original's "SpaghettiOs". And the cup check looks at the whole recipe, not at each ingredient, so a made-up „1/2 szklanki” gets through if the original says "1/2 cup" for some other ingredient.

## The first test: does the model keep the format {#the-first-test-does-the-model-keep-the-format}

The first test checked whether a model follows the app's rules: 26 recipes (25 Polish and one English), 11 models (5 from OpenAI, 3 on free Groq and 3 local), each recipe run twice. The app's own code scored the raw JSON, before any checks run.

The numbers are totals over 52 extractions per model. A format error is one of three kinds:

| Column | What counts as an error |
| --- | --- |
| measure beside grams | The app has a separate field for the household measure it shows beside the grams: „2 szklanki” (2 glasses), „1 łyżka” (1 tablespoon). An error is anything else the model puts there: a note („pokrojone”, sliced; „do dekoracji”, for garnish), a US unit („1 funt”, 1 pound, beside 450 g of meat) or a repeat of the amount („3 łyżki” beside 3 tablespoons). |
| step form | A step has to be in the infinitive, the way Polish cookbooks write it: „Dodać mąkę” (to add flour). An error is the imperative („Dodaj mąkę”, add flour) or narration („Teraz dodajemy mąkę”, now we add the flour). |
| wrong field | A measure put in another field of the ingredient: „320 g” as the amount, instead of 320 as the amount and „g” as the unit, or „2 szklanki” as the unit. I added this column during the test, because the measure-beside-grams check never sees such an ingredient: its household-measure field is empty. |
| total | The sum of the three columns above. |

**Format errors per model, markdown only (first test)**

| Model | Where | Errors |
| --- | --- | --- |
| gpt-5.4-mini | OpenAI API (paid) | 26 |
| qwen3.8-27b | Groq (free plan) | 48 |
| gpt-oss-120b | Groq (free plan) | 80 |
| gpt-4o-mini | OpenAI API (paid) | 80 |
| gpt-4.1-mini | OpenAI API (paid) | 96 |
| gpt-5-mini | OpenAI API (paid) | 108 |
| gpt-oss-20b | Groq (free plan) | 150 |
| gpt-5-nano | OpenAI API (paid) | 234 |
| qwen3:14b | local (Ollama) | 434 |
| Bielik 11B v3.0 | local (Ollama) | 562 |
| Bielik 11B v2.3 | local (Ollama) | 577 |

Lower is better. The table above explains the kinds of error.

gpt-5.4-mini made the fewest format errors, almost never got the step form wrong and was the fastest of the OpenAI models, so after this test it replaced gpt-4o-mini in the app. Bielik did worst: in over 200 ingredients it put the measure in the wrong field, and it did not write steps in the infinitive.

That test only said whether a model keeps the format, though. It did not say whether the saved recipe is true: whether the quantities and temperatures match the original and nothing is missing. That needed a second test.

## The second test: does the saved recipe match the original {#the-second-test-does-the-saved-recipe-match-the-original}

### Recipes and reference {#recipes-and-reference}

The set is 50 recipes saved in the app: 25 from Polish sites and 25 from English blogs in the US, the UK, Australia and India. Three dropped out because the saved page held no recipe (a paywall or an empty page). Two more dropped out of the markdown-only variant only, because the recipe started after the first 10,000 characters.

Each recipe got a reference: ingredients with quantities, steps, temperatures, times, servings and photos matched to steps. Two models drafted it independently: Claude Fable 5.1 (in Claude Code) and Gemini 3.5 Flash at high reasoning effort. Where they agreed, the reference was done. I settled every difference myself, blind, without knowing which model wrote what. I also checked 31 randomly sampled points where they agreed against the original, and all 31 were right. I set three rules too: a value within a stated range is correct, a recipe must keep both oven modes (top-and-bottom and fan), and a computed or estimated total time is not an error.

One caveat, stated plainly: the reference was drafted by models from the same families as some of the contestants (Claude and Gemini). My review of the differences reduces that risk but does not remove it, and the Claude and Gemini models happen to be at the top.

### What counts as an error {#what-counts-as-an-error}

| Severity | What it means |
| --- | --- |
| critical | A cook following the saved recipe gets a different dish or fails: a missing ingredient, a quantity more than 5% off the original, a lost step, a wrong or missing oven temperature (±5°C tolerance). |
| major | The dish works, but something is off: a wrong time, a missing step photo or the cover used as a step photo, US units left unconverted. |
| minor | An optional ingredient, English words left in, the app-format slips from the first test. |
| Polish errors | LanguageTool 6.6 (local, pl-PL) on the title, ingredient names and steps: grammar, syntax, word order, and misspellings of words that are not on the source page. Of 80 hits checked by hand, 46 were real errors („Piecić”, „ostudzć”, „drożdże suchy”); the rest were mostly dictionary gaps that hit every model alike. So differences under 1 error per 1,000 words mean nothing. |

A model is scored only on what it got as input. Confidence intervals are 95%, bootstrapped over recipes (2,000 resamples). Each recipe went through each model twice, with the same prompt as in production.

I also rated the Polish myself: 20 outputs from different models, blind. All of them got 5 out of 5, so fluency does not separate these models, and an AI judge would have had nothing to learn from me. That is why I did not use one.

### Two input variants {#two-input-variants}

In production the model gets the page text as markdown plus its HTML, each up to 20,000 characters. Free Groq will not take that: the limit is 8,000 tokens per minute, and that is also the largest request you can send, while a full request is 10–13 thousand tokens. That is why the test has two variants: **full** (markdown and HTML, as in production) and **markdown only** (the first 10,000 characters). Every model ran the markdown-only variant, so that is where the comparison is complete.

### The models {#the-models}

- **5 from OpenAI**, called directly: gpt-4o-mini, gpt-4.1-mini, gpt-5-mini, gpt-5-nano and gpt-5.4-mini, the last one also at a higher reasoning effort (`low`);
- **10 through OpenRouter**: Claude Haiku 5.5 and Sonnet 5.5, Gemini 3.5 Flash and 3.1 Flash-Lite, Gemma 4 31B, GLM 5.3, Qwen 3.7 Plus, DeepSeek v4 Flash, Mistral Medium 3.1 and Llama 4 Maverick;
- **3 on free Groq**: gpt-oss-120b, gpt-oss-20b and qwen3.8-27b. I also ran both gpt-oss models through OpenRouter to compare hosts;
- **3 local, through Ollama**: Bielik 11B v2.3 and v3.0 (Q4_K_M quantization) and qwen3:14b.

| Item | Settings |
| --- | --- |
| Laptop | NVIDIA RTX PRO 3000 GPU (12 GB VRAM), 64 GB DDR5-5600 RAM, Intel Core Ultra 9 275HX CPU |
| Ollama | 0.40.5, `num_ctx` 24,576, `format: json`, temperature 0.3, KV cache q8_0, flash attention |
| Groq, free plan | 8,000 tokens per minute, 1,000 requests and 200,000 tokens a day per model (25 Oct 2026) |
| OpenRouter | OpenRouter picked the host; cost taken from its accounting of each request |
| Reasoning effort | the lowest each model offers; gpt-5.4-mini also at `low` |

## Results {#results}

Two exceptions to the full set. Free Groq rejects any request over 8,000 tokens, and six English recipes do not fit within that even as markdown only (three for gpt-oss-20b). The Groq rows carry a †: they are scored without those longest recipes, which matters in every comparison. I capped the local models' output at 4,096 tokens, because qwen3:14b looped on one recipe for over 17 thousand tokens (a normal recipe is about 1,300). An output cut by the cap counts as broken JSON. The "Recipes PL/EN" column says how many recipes each model received.

A bar's colour shows how I called the model, not who made it: through the OpenAI API, through OpenRouter, on free Groq or locally through Ollama. That is why gpt-oss, an OpenAI model, takes the colour of Groq or OpenRouter.

**Critical errors per recipe, markdown only**

| Model | Where | Critical per recipe | 95% interval |
| --- | --- | --- | --- |
| Claude Haiku 5.5 | OpenRouter (paid) | 0.01 | 0.00–0.03 |
| Gemini 3.5 Flash | OpenRouter (paid) | 0.02 | 0.00–0.07 |
| Gemini 3.1 Flash-Lite | OpenRouter (paid) | 0.03 | 0.00–0.09 |
| gpt-4.1-mini | OpenAI API (paid) | 0.04 | 0.00–0.10 |
| Gemma 4 31B | OpenRouter (paid) | 0.06 | 0.00–0.13 |
| Claude Sonnet 5.5 | OpenRouter (paid) | 0.06 | 0.00–0.14 |
| gpt-5-mini | OpenAI API (paid) | 0.06 | 0.00–0.14 |
| GLM 5.3 | OpenRouter (paid) | 0.07 | 0.01–0.13 |
| gpt-5.4-mini | OpenAI API (paid) | 0.08 | 0.02–0.16 |
| qwen3.8-27b † | Groq (free plan) | 0.09 | 0.01–0.17 |
| Qwen 3.7 Plus | OpenRouter (paid) | 0.11 | 0.01–0.26 |
| DeepSeek v4 Flash | OpenRouter (paid) | 0.15 | 0.00–0.42 |
| gpt-4o-mini | OpenAI API (paid) | 0.20 | 0.07–0.37 |
| gpt-oss-120b (Groq) † | Groq (free plan) | 0.24 | 0.05–0.56 |
| gpt-5.4-mini @low | OpenAI API (paid) | 0.24 | 0.08–0.44 |
| Mistral Medium 3.1 | OpenRouter (paid) | 0.26 | 0.09–0.43 |
| Llama 4 Maverick | OpenRouter (paid) | 0.37 | 0.18–0.56 |
| gpt-oss-20b (Groq) † | Groq (free plan) | 0.40 | 0.12–0.73 |
| gpt-oss-20b (OpenRouter) | OpenRouter (paid) | 0.59 | 0.31–0.87 |
| Bielik 11B v2.3 | local (Ollama) | 0.62 | 0.36–0.93 |
| Bielik 11B v3.0 | local (Ollama) | 0.64 | 0.36–0.92 |
| gpt-oss-120b (OpenRouter) | OpenRouter (paid) | 0.67 | 0.21–1.24 |
| gpt-5-nano | OpenAI API (paid) | 0.72 | 0.36–1.30 |
| qwen3:14b | local (Ollama) | 1.00 | 0.55–1.59 |

Lower is better. The line under each bar is the 95% confidence interval. † free Groq rejected the longest English recipes (an 8,000-token limit per request), so these rows are scored without them.

**Correctness, markdown only**

| Model | Where | Critical [95%] | Major | Minor | Polish |
| --- | --- | --: | --: | --: | --: |
| Claude Haiku 5.5 | OpenRouter (paid) | 0.01 [0.00–0.03] | 0.39 | 0.52 | 2.8 |
| Gemini 3.5 Flash | OpenRouter (paid) | 0.02 [0.00–0.07] | 0.38 | 1.46 | 1.5 |
| Gemini 3.1 Flash-Lite | OpenRouter (paid) | 0.03 [0.00–0.09] | 0.61 | 1.30 | 1.5 |
| gpt-4.1-mini | OpenAI API (paid) | 0.04 [0.00–0.10] | 0.52 | 2.00 | 1.5 |
| Gemma 4 31B | OpenRouter (paid) | 0.06 [0.00–0.13] | 0.33 | 0.49 | 3.0 |
| Claude Sonnet 5.5 | OpenRouter (paid) | 0.06 [0.00–0.14] | 0.40 | 0.98 | 2.7 |
| gpt-5-mini | OpenAI API (paid) | 0.06 [0.00–0.14] | 1.28 | 2.36 | 2.9 |
| GLM 5.3 | OpenRouter (paid) | 0.07 [0.01–0.13] | 0.44 | 1.36 | 4.3 |
| gpt-5.4-mini | OpenAI API (paid) | 0.08 [0.02–0.16] | 0.46 | 0.68 | 2.1 |
| qwen3.8-27b † | Groq (free plan) | 0.09 [0.01–0.17] | 0.38 | 1.04 | 2.3 |
| Qwen 3.7 Plus | OpenRouter (paid) | 0.11 [0.01–0.26] | 0.30 | 1.83 | 2.7 |
| DeepSeek v4 Flash | OpenRouter (paid) | 0.15 [0.00–0.42] | 0.40 | 0.24 | 2.0 |
| gpt-4o-mini | OpenAI API (paid) | 0.20 [0.07–0.37] | 0.67 | 1.70 | 0.8 |
| gpt-oss-120b (Groq) † | Groq (free plan) | 0.24 [0.05–0.56] | 1.26 | 1.47 | 2.8 |
| gpt-5.4-mini @low | OpenAI API (paid) | 0.24 [0.08–0.44] | 0.54 | 0.36 | 1.5 |
| Mistral Medium 3.1 | OpenRouter (paid) | 0.26 [0.09–0.43] | 0.36 | 0.64 | 2.5 |
| Llama 4 Maverick | OpenRouter (paid) | 0.37 [0.18–0.56] | 0.79 | 2.56 | 4.4 |
| gpt-oss-20b (Groq) † | Groq (free plan) | 0.40 [0.12–0.73] | 0.55 | 3.50 | 4.1 |
| gpt-oss-20b (OpenRouter) | OpenRouter (paid) | 0.59 [0.31–0.87] | 0.91 | 2.69 | 3.5 |
| Bielik 11B v2.3 | local (Ollama) | 0.62 [0.36–0.93] | 3.36 | 37.88 | 2.9 |
| Bielik 11B v3.0 | local (Ollama) | 0.64 [0.36–0.92] | 3.09 | 22.29 | 1.9 |
| gpt-oss-120b (OpenRouter) | OpenRouter (paid) | 0.67 [0.21–1.24] | 1.66 | 1.34 | 3.7 |
| gpt-5-nano | OpenAI API (paid) | 0.72 [0.36–1.30] | 1.54 | 3.37 | 4.5 |
| qwen3:14b | local (Ollama) | 1.00 [0.55–1.59] | 1.31 | 7.02 | 17.0 |

Critical, major and minor are mean errors per recipe; the brackets hold the 95% confidence interval. Critical PL and EN count recipes from Polish and English sites separately. Polish is LanguageTool errors per 1,000 words. † free Groq rejected the longest English recipes, so these rows are scored without them.

**Cost and reliability, markdown only**

| Model | Where | Median | Cost | Rejected | Broken JSON | Recipes PL/EN |
| --- | --- | --: | --: | --: | --: | --: |
| Claude Haiku 5.5 | OpenRouter (paid) | 15.2 s* | $0.0022 | 0 | 0 | 23/22 |
| Gemini 3.5 Flash | OpenRouter (paid) | 8.2 s* | $0.0189 | 0 | 0 | 23/22 |
| Gemini 3.1 Flash-Lite | OpenRouter (paid) | 8.7 s* | $0.0065 | 0 | 0 | 23/22 |
| gpt-4.1-mini | OpenAI API (paid) | 7.3 s | – | 0 | 0 | 23/22 |
| Gemma 4 31B | OpenRouter (paid) | 26.6 s* | $0.0012 | 0 | 0 | 23/22 |
| Claude Sonnet 5.5 | OpenRouter (paid) | 9.9 s* | $0.0215 | 0 | 0 | 23/22 |
| gpt-5-mini | OpenAI API (paid) | 10.6 s | – | 0 | 0 | 23/22 |
| GLM 5.3 | OpenRouter (paid) | 8.2 s* | $0.0038 | 0 | 0 | 23/22 |
| gpt-5.4-mini | OpenAI API (paid) | 4.6 s | – | 0 | 0 | 23/22 |
| qwen3.8-27b † | Groq (free plan) | 2.9 s | – | 12 | 0 | 23/16 |
| Qwen 3.7 Plus | OpenRouter (paid) | 24.3 s* | $0.0027 | 0 | 0 | 23/22 |
| DeepSeek v4 Flash | OpenRouter (paid) | 133.5 s* | $0.0025 | 0 | 19 | 18/21 |
| gpt-4o-mini | OpenAI API (paid) | 9.2 s | – | 0 | 0 | 23/22 |
| gpt-oss-120b (Groq) † | Groq (free plan) | 3.4 s | – | 12 | 0 | 23/16 |
| gpt-5.4-mini @low | OpenAI API (paid) | 7.2 s | – | 0 | 0 | 23/22 |
| Mistral Medium 3.1 | OpenRouter (paid) | 8.5 s* | $0.0034 | 0 | 0 | 23/22 |
| Llama 4 Maverick | OpenRouter (paid) | 51.2 s* | $0.0011 | 0 | 0 | 23/22 |
| gpt-oss-20b (Groq) † | Groq (free plan) | 1.8 s | – | 6 | 0 | 23/20 |
| gpt-oss-20b (OpenRouter) | OpenRouter (paid) | 17.4 s* | $0.0002 | 0 | 2 | 23/22 |
| Bielik 11B v2.3 | local (Ollama) | 53.2 s | – | 0 | 0 | 23/22 |
| Bielik 11B v3.0 | local (Ollama) | 56.5 s | – | 0 | 0 | 23/22 |
| gpt-oss-120b (OpenRouter) | OpenRouter (paid) | 9.2 s* | $0.0006 | 0 | 2 | 23/22 |
| gpt-5-nano | OpenAI API (paid) | 8.7 s | – | 0 | 0 | 23/22 |
| qwen3:14b | local (Ollama) | 85.8 s | – | 0 | 5 | 23/20 |

Median is the time of one extraction; an asterisk marks OpenRouter times measured separately on 6 recipes. Cost is the amount per recipe from OpenRouter's accounting; OpenAI and Groq did not report it in their responses and the local models cost nothing, hence the dash. Rejected counts the requests free Groq refused (an 8,000-token limit). Broken JSON counts outputs that could not be read, including those cut by the 4,096-token cap on the local models. Recipes PL/EN says how many recipes each model received.

**Critical errors per recipe, full input (as in the app)**

| Model | Where | Critical per recipe | 95% interval |
| --- | --- | --- | --- |
| gpt-5-mini | OpenAI API (paid) | 0.07 | 0.01–0.16 |
| gpt-5.4-mini | OpenAI API (paid) | 0.09 | 0.01–0.17 |
| gpt-4.1-mini | OpenAI API (paid) | 0.09 | 0.02–0.16 |
| gpt-4o-mini | OpenAI API (paid) | 0.19 | 0.03–0.38 |
| gpt-5-nano | OpenAI API (paid) | 0.66 | 0.27–1.26 |
| gpt-oss-120b (OpenRouter) | OpenRouter (paid) | 0.77 | 0.37–1.22 |
| Bielik 11B v2.3 | local (Ollama) | 0.82 | 0.39–1.46 |
| qwen3:14b | local (Ollama) | 1.17 | 0.71–1.77 |
| gpt-oss-20b (OpenRouter) | OpenRouter (paid) | 1.31 | 0.55–2.11 |
| Bielik 11B v3.0 | local (Ollama) | 1.68 | 0.84–2.71 |

Lower is better. Free Groq does not take requests this large, and only the gpt-oss models ran this variant through OpenRouter.

**Correctness, full input**

| Model | Where | Critical [95%] | Critical PL | Critical EN | Major | Minor | Polish |
| --- | --- | --: | --: | --: | --: | --: | --: |
| gpt-5-mini | OpenAI API (paid) | 0.07 [0.01–0.16] | 0.04 | 0.10 | 2.22 | 2.90 | 3.2 |
| gpt-5.4-mini | OpenAI API (paid) | 0.09 [0.01–0.17] | 0.07 | 0.10 | 0.41 | 0.56 | 1.5 |
| gpt-4.1-mini | OpenAI API (paid) | 0.09 [0.02–0.16] | 0.11 | 0.06 | 0.82 | 2.68 | 1.5 |
| gpt-4o-mini | OpenAI API (paid) | 0.19 [0.03–0.38] | 0.33 | 0.06 | 0.79 | 2.46 | 1.1 |
| gpt-5-nano | OpenAI API (paid) | 0.66 [0.27–1.26] | 0.37 | 0.94 | 2.77 | 4.38 | 6.8 |
| gpt-oss-120b (OpenRouter) | OpenRouter (paid) | 0.77 [0.37–1.22] | 0.43 | 1.08 | 1.88 | 1.65 | 4.1 |
| Bielik 11B v2.3 | local (Ollama) | 0.82 [0.39–1.46] | 0.35 | 1.27 | 5.05 | 44.40 | 0.9 |
| qwen3:14b | local (Ollama) | 1.17 [0.71–1.77] | 0.04 | 2.25 | 1.88 | 7.30 | 18.5 |
| gpt-oss-20b (OpenRouter) | OpenRouter (paid) | 1.31 [0.55–2.11] | 0.35 | 2.23 | 1.27 | 2.55 | 4.1 |
| Bielik 11B v3.0 | local (Ollama) | 1.68 [0.84–2.71] | 1.70 | 1.67 | 4.77 | 22.72 | 2.0 |

Critical, major and minor are mean errors per recipe; the brackets hold the 95% confidence interval. Critical PL and EN count recipes from Polish and English sites separately. Polish is LanguageTool errors per 1,000 words. † free Groq rejected the longest English recipes, so these rows are scored without them.

**Cost and reliability, full input**

| Model | Where | Median | Cost | Rejected | Broken JSON | Recipes PL/EN |
| --- | --- | --: | --: | --: | --: | --: |
| gpt-5-mini | OpenAI API (paid) | 11.9 s | – | 0 | 0 | 23/24 |
| gpt-5.4-mini | OpenAI API (paid) | 4.4 s | – | 0 | 0 | 23/24 |
| gpt-4.1-mini | OpenAI API (paid) | 7.7 s | – | 0 | 0 | 23/24 |
| gpt-4o-mini | OpenAI API (paid) | 10.8 s | – | 0 | 0 | 23/24 |
| gpt-5-nano | OpenAI API (paid) | 8.9 s | – | 0 | 0 | 23/24 |
| gpt-oss-120b (OpenRouter) | OpenRouter (paid) | 9.2 s* | $0.0008 | 0 | 2 | 23/24 |
| Bielik 11B v2.3 | local (Ollama) | 59.6 s | – | 0 | 0 | 23/24 |
| qwen3:14b | local (Ollama) | 125.8 s | – | 0 | 2 | 23/24 |
| gpt-oss-20b (OpenRouter) | OpenRouter (paid) | 17.4 s* | $0.0004 | 0 | 4 | 23/24 |
| Bielik 11B v3.0 | local (Ollama) | 66.8 s | – | 0 | 0 | 23/24 |

Median is the time of one extraction; an asterisk marks OpenRouter times measured separately on 6 recipes. Cost is the amount per recipe from OpenRouter's accounting; OpenAI and Groq did not report it in their responses and the local models cost nothing, hence the dash. Rejected counts the requests free Groq refused (an 8,000-token limit). Broken JSON counts outputs that could not be read, including those cut by the 4,096-token cap on the local models. Recipes PL/EN says how many recipes each model received.

### What kind of errors {#what-kind-of-errors}

Critical errors by kind, counted over all outputs in the markdown-only variant:

| Model | missing ingredient | wrong quantity | lost step | wrong temperature |
| --- | --: | --: | --: | --: |
| Claude Haiku 5.5 | 0 | 1 | 0 | 0 |
| Gemini 3.5 Flash | 0 | 2 | 0 | 0 |
| Gemini 3.1 Flash-Lite | 2 | 1 | 0 | 0 |
| gpt-4.1-mini | 0 | 4 | 0 | 0 |
| Gemma 4 31B | 2 | 3 | 0 | 0 |
| Claude Sonnet 5.5 | 4 | 0 | 0 | 1 |
| gpt-5-mini | 1 | 2 | 0 | 2 |
| GLM 5.3 | 1 | 5 | 0 | 0 |
| gpt-5.4-mini | 3 | 0 | 0 | 4 |
| qwen3.8-27b † | 3 | 4 | 0 | 0 |
| Qwen 3.7 Plus | 6 | 3 | 1 | 0 |
| DeepSeek v4 Flash | 11 | 0 | 1 | 0 |
| gpt-4o-mini | 4 | 6 | 0 | 8 |
| gpt-oss-120b (Groq) † | 11 | 2 | 0 | 6 |
| gpt-5.4-mini @low | 11 | 7 | 0 | 4 |
| Mistral Medium 3.1 | 20 | 3 | 0 | 0 |
| Llama 4 Maverick | 30 | 2 | 0 | 1 |
| gpt-oss-20b (Groq) † | 17 | 12 | 0 | 5 |
| gpt-oss-20b (OpenRouter) | 33 | 9 | 2 | 9 |
| Bielik 11B v2.3 | 29 | 9 | 0 | 18 |
| Bielik 11B v3.0 | 8 | 21 | 0 | 29 |
| gpt-oss-120b (OpenRouter) | 30 | 1 | 5 | 13 |
| gpt-5-nano | 51 | 7 | 0 | 7 |
| qwen3:14b | 49 | 20 | 0 | 9 |

**Median response time, markdown only**

| Model | Where | Median |
| --- | --- | --- |
| gpt-oss-20b (Groq) | Groq (free plan) | 1.8 s |
| qwen3.8-27b | Groq (free plan) | 2.9 s |
| gpt-oss-120b (Groq) | Groq (free plan) | 3.4 s |
| gpt-5.4-mini | OpenAI API (paid) | 4.6 s |
| gpt-5.4-mini @low | OpenAI API (paid) | 7.2 s |
| gpt-4.1-mini | OpenAI API (paid) | 7.3 s |
| Gemini 3.5 Flash * | OpenRouter (paid) | 8.2 s |
| GLM 5.3 * | OpenRouter (paid) | 8.2 s |
| Mistral Medium 3.1 * | OpenRouter (paid) | 8.5 s |
| Gemini 3.1 Flash-Lite * | OpenRouter (paid) | 8.7 s |
| gpt-5-nano | OpenAI API (paid) | 8.7 s |
| gpt-4o-mini | OpenAI API (paid) | 9.2 s |
| gpt-oss-120b (OpenRouter) * | OpenRouter (paid) | 9.2 s |
| Claude Sonnet 5.5 * | OpenRouter (paid) | 9.9 s |
| gpt-5-mini | OpenAI API (paid) | 10.6 s |
| Claude Haiku 5.5 * | OpenRouter (paid) | 15.2 s |
| gpt-oss-20b (OpenRouter) * | OpenRouter (paid) | 17.4 s |
| Qwen 3.7 Plus * | OpenRouter (paid) | 24.3 s |
| Gemma 4 31B * | OpenRouter (paid) | 26.6 s |
| Llama 4 Maverick * | OpenRouter (paid) | 51.2 s |
| Bielik 11B v2.3 | local (Ollama) | 53.2 s |
| Bielik 11B v3.0 | local (Ollama) | 56.5 s |
| qwen3:14b | local (Ollama) | 85.8 s |
| DeepSeek v4 Flash * | OpenRouter (paid) | 133.5 s |

Log scale. * timed separately on 6 recipes, because the main OpenRouter run only measured the time to the response headers.

## What the numbers say {#what-the-numbers-say}

### The top is a tie {#the-top-is-a-tie}

On markdown alone the fewest critical errors go to Claude Haiku 5.5 (0.01), Gemini 3.5 Flash (0.02), Gemini 3.1 Flash-Lite (0.03) and gpt-4.1-mini (0.04). Their confidence intervals overlap with gpt-5.4-mini (0.08), though, so with 50 recipes there is no honest way to say which of them is best. The difference you can measure separates the top from the rest: the local models, gpt-5-nano, gpt-oss via OpenRouter and Llama 4 Maverick make many times as many critical errors.

### The app keeps gpt-5.4-mini {#the-app-keeps-gpt-54-mini}

In the setup the app uses, with the page HTML, gpt-5.4-mini has 0.09 critical errors per recipe, the fewest major (0.41) and minor (0.56) errors of the models in that variant, and a median of 4.4 seconds. Switching to another top model would bring no measurable gain, and gpt-5.4-mini also won the first test, the one on format.

### The more expensive model was not better {#the-more-expensive-model-was-not-better}

Claude Sonnet 5.5 cost $0.0215 per recipe through OpenRouter and had 0.06 critical errors per recipe. Claude Haiku 5.5, at $0.0022, had 0.01, and Gemini 3.1 Flash-Lite, at $0.0065, had 0.03. Over a thousand recipes that is about $21.50 against $2.20, with no better result for the higher price.

### Same model, different host, different result {#same-model-different-host-different-result}

This is the biggest surprise of the test. OpenRouter spread the gpt-oss requests over more than ten companies that run the model themselves, with the same request settings. The result depended on the company more than on the model size:

**gpt-oss via OpenRouter: critical errors by host**

| Host | Model | Critical per output |
| --- | --- | --- |
| CoreWeave | gpt-oss-120b | 4.47 |
| AkashML | gpt-oss-120b | 0.28 |
| BaseTen | gpt-oss-120b | 0.25 |
| Crusoe | gpt-oss-120b | 0.22 |
| Near AI | gpt-oss-120b | 0.19 |
| Novita | gpt-oss-120b | 0.17 |
| DeepInfra | gpt-oss-120b | 0.00 |
| DekaLLM | gpt-oss-120b | 0.00 |
| Mancer 2 | gpt-oss-120b | 0.00 |
| Parasail | gpt-oss-120b | 0.00 |
| SiliconFlow | gpt-oss-20b | 12.20 |
| DekaLLM | gpt-oss-20b | 1.67 |
| AkashML | gpt-oss-20b | 0.77 |
| Darkbloom | gpt-oss-20b | 0.44 |
| CoreWeave | gpt-oss-20b | 0.38 |
| Parasail | gpt-oss-20b | 0.32 |
| DeepInfra | gpt-oss-20b | 0.00 |

Hosts with at least 5 outputs. There are few outputs per host, so read the order, not the decimals.

That explains why gpt-oss-120b called directly on Groq made no critical errors on the Polish recipes, while through OpenRouter it had 0.80 per recipe. If you use OpenRouter for a task where accuracy matters, name the hosts you have checked in the request. Otherwise you get a random one.

### More reasoning did not help {#more-reasoning-did-not-help}

gpt-5.4-mini at reasoning effort `low` had 0.24 critical errors per recipe, and at the lowest (`none`) 0.08. The intervals overlap at the edge, so I do not claim that reasoning hurts, but it certainly did not help: the model produced more tokens, answered more slowly (7.2 instead of 4.6 seconds) and dropped ingredients more often.

### HTML does not reduce critical errors {#html-does-not-reduce-critical-errors}

The OpenAI models made about as many critical errors on markdown alone as with the HTML, with about half the input: gpt-5.4-mini 0.08 against 0.09, gpt-4o-mini 0.20 against 0.19. The HTML stays in the app all the same, because the step photos come from it.

### Polish: a blind rating sees no difference {#polish-a-blind-rating-sees-no-difference}

In the blind rating I gave every output 5 out of 5. LanguageTool does find differences between the models: with the full input, from 0.9 errors per 1,000 words for Bielik 11B v2.3 and 1.1 for gpt-4o-mini up to 6.8 for gpt-5-nano. Given its hit rate, that is a ranking signal, not a verdict.

The exception is qwen3:14b, at 18.5 errors per 1,000 words. On English recipes it writes broken Polish, and its hits were checked by hand and are real: „pudr”, „vanilię”, „nacisknij”, „wyciąnij”, „mikrowarce”, and „auberginy” instead of the Polish word for aubergines.

### Bielik and the local models {#bielik-and-the-local-models}

The local models do worst, and worse still once the English recipes are in. On markdown alone Bielik 11B v2.3 has 0.62 critical errors per recipe, v3.0 0.64 and qwen3:14b 1.00. With the full input it is 0.82, 1.68 and 1.17, so there the newer Bielik does worse than 2.3.

They handle Polish recipes far better than English ones. Bielik 11B v2.3 has 0.24 critical errors per recipe on Polish sources and 1.02 on English ones (markdown only). qwen3:14b hardly errs on Polish sources (0.02) and has 2.13 on English ones. Bielik most often drops ingredients and gets temperatures wrong, and it does not keep the app's format either: v2.3 has 37.9 minor errors per recipe. Its Polish is sound, with the fewest errors on full input (0.9 per 1,000 words), which fits the first test: it knows Polish but does not keep the format.

Locally a recipe takes just under a minute on Bielik and a minute and a half on qwen3:14b with markdown only, longer with the HTML. qwen3:14b with a 24-thousand-token context does not fit in the card's 12 GB of memory, so part of the model runs on the CPU from system RAM. That explains its times; it would be faster on a machine with a bigger GPU.

### Other observations {#other-observations}

- **DeepSeek v4 Flash** returned broken JSON in 19 outputs, generated over 12 thousand tokens per output on average and needed more than two minutes.
- **Mistral Medium 3.1** and **Llama 4 Maverick** drop ingredients most often: 20 and 30 times.
- **gpt-5-nano**, OpenAI's cheapest model, has the most critical errors of the cloud models on markdown alone, mostly missing ingredients (51).

## What this test does not tell you {#what-this-test-does-not-tell-you}

- **Models drafted the reference.** I checked every difference between them and a sample of their agreements, but if both models misread the same passage outside the sample, I would not have noticed. Fable and Gemini belong to the same families as some of the contestants.
- **Free Groq rejected the longest English recipes** (an 8,000-token limit per request), so its rows (†) leave them out and are not fully comparable with the rest.
- **The local models' output was capped at 4,096 tokens.** An output cut by the cap counts as broken JSON (qwen3:14b: 2 with the full input, 5 with markdown only).
- **50 recipes and two runs** are enough to separate the top from the bottom of the table, but not to rank the top.
- **One person rated the Polish**, on 20 outputs, and LanguageTool hits a real error in about 57% of cases.
- **OpenRouter times** (asterisk) were measured separately on 6 recipes and depend on the host and the time of day.
- **Cost** is shown where OpenRouter billed it. **Prices** are from 25 and 26 October 2026.
- **I wrote no prompt for Bielik or any other model.** All of them got the prompt that grew up with the OpenAI models.

## Summary {#summary}

- Several models save recipes with almost no critical errors, and the differences between them are within the measurement's uncertainty. Choosing within that group comes down to price, speed and whose API you already use.
- ZapiszPrzepis keeps gpt-5.4-mini: about one critical error in eleven recipes and an answer in 4.4 seconds.
- Neither a more expensive model nor more reasoning improved the result.
- On OpenRouter, check the host, not just the model.
- Local models, Bielik included, are free in money but slow, and with this prompt they make more mistakes.

How do you choose a model for your own app: on benchmarks, on your own data, or do you just take the one you already know? I would like to hear what works for you.

## Frequently Asked Questions {#faq}

### Which AI model saves recipes best? {#faq-which-ai-model-saves-recipes-best}

There is no single winner. On markdown-only input Claude Haiku 5.5, Gemini 3.5 Flash, Gemini 3.1 Flash-Lite and gpt-4.1-mini have the fewest critical errors, but their confidence intervals overlap with gpt-5.4-mini, so it is a tie. ZapiszPrzepis keeps <strong>gpt-5.4-mini</strong>: with the full input the app uses, it has 0.09 critical errors per recipe, the fewest major and minor errors and a median of 4.4 seconds.

### Does a more expensive model save recipes better? {#faq-does-a-more-expensive-model-save-recipes-better}

Not in this test. Claude Sonnet 5.5 cost about $0.02 per recipe through OpenRouter and had 0.06 critical errors per recipe; Claude Haiku 5.5, at about $0.002, had 0.01. The confidence intervals overlap, so Sonnet did not do better despite costing roughly ten times as much.

### Does it matter which OpenRouter host serves the request? {#faq-does-it-matter-which-openrouter-host-serves-the-request}

Yes, a lot. The same gpt-oss-120b made no critical errors on several hosts and averaged 4.47 per output on CoreWeave. gpt-oss-20b on SiliconFlow averaged 12.2. If accuracy matters, name the hosts you have checked in the request.

### Is DeepL not enough to translate recipes? {#faq-is-deepl-not-enough-to-translate-recipes}

No. DeepL translates text, but it does not convert pounds to grams or °F to °C, does not move a note out of the measure field into the ingredient name, and does not swap a US brand for a product sold in Polish shops. It would also be a second paid service and a second pass that translates short fields without context.

### Is Bielik suitable for saving recipes? {#faq-is-bielik-suitable-for-saving-recipes}

Not well, with my prompt. On markdown alone Bielik 11B v2.3 had 0.62 critical errors per recipe and the newer v3.0 had 0.64. It does better on Polish recipes (0.24 and 0.43) and worse on English ones. Its Polish is good, but it drops ingredients, gets temperatures wrong and does not keep to a strict JSON format. I did not try a prompt written for Bielik, so this is a result for this task, not a verdict on the model.

### Should the model get the page HTML along with the text? {#faq-should-the-model-get-the-page-html-along-with-the-text}

Not for saving the recipe itself: the OpenAI models made about as many critical errors on markdown alone as with the HTML, with about half the input. The app also uses the HTML for step photos, though, so it stays.

### Is a local model free? {#faq-is-a-local-model-free}

In money, yes; in time, no. On a laptop with an RTX PRO 3000 (12 GB), Bielik took just under a minute per recipe and qwen3:14b about a minute and a half. gpt-5.4-mini does the same in a few seconds.

### How did you check that a saved recipe is correct? {#faq-how-did-you-check-that-a-saved-recipe-is-correct}

For every recipe two models, Claude Fable 5.1 and Gemini 3.5 Flash, drafted a reference from the original page independently. I settled every difference between them blind and checked 31 of 31 randomly sampled points where they agreed against the original. Code then compared each model's output with that reference.


## Sources {#sources}

- [Recipe extraction model evaluation (ZapiszPrzepis)](https://github.com/spokospace/zapiszprzepis/blob/master/context/foundation/extraction-model-eval-2026-10.md) — The first test: five OpenAI models, method, results and limitations.
- [Localizing recipes from foreign sites (PR #525)](https://github.com/spokospace/zapiszprzepis/pull/525) — Source-language detection, the rules for foreign sites and the household-measure check.
- [Polish product names and phrasing (PR #528)](https://github.com/spokospace/zapiszprzepis/pull/528) — The second round of fixes, with checks in code where the prompt was not enough.
- [Switch to gpt-5.4-mini (PR #531)](https://github.com/spokospace/zapiszprzepis/pull/531) — The one-line model change after the first test.
- [OpenRouter: provider selection](https://openrouter.ai/docs/guides/routing/provider-selection) — How to require or exclude the hosts OpenRouter routes a request to.
- [OpenAI API pricing](https://developers.openai.com/api/docs/pricing) — Model prices, checked on 25 October 2026.
- [Groq rate limits](https://console.groq.com/docs/rate-limits) — Tokens per minute and per day on the free plan.
- [LanguageTool](https://github.com/languagetool-org/languagetool) — The open-source proofreader I counted Polish errors with.
- [Ollama](https://github.com/ollama/ollama) — The server I ran the local models on.
- [Bielik 11B v2.3 Instruct](https://huggingface.co/speakleash/Bielik-11B-v2.3-Instruct) — The Polish language model by SpeakLeash on Hugging Face.
- [Bielik 11B v3.0 Instruct](https://huggingface.co/speakleash/Bielik-11B-v3.0-Instruct) — The newer Bielik, also tested.