A week of AI meetups in Bielsko-Biała

In one week I went to two meetups about artificial intelligence: on Thursday about brand communication, on Friday about models you run on your own hardware. Different audiences, different levels of abstraction, and yet both evenings circled the same question: how do you measure something that behaves differently every time. Only one measurement was missing, and it is the one that matters most to most companies.

Both took place in Bielsko-Biała, a city of about 170,000 in southern Poland, close to the Czech and Slovak borders.

Thursday: how three models score the same email

17 September, at the Rekord SI research centre. „Komunikacja w dobie promptów” (Communication in the age of prompts), part of the Upgrade Your Business series run by the Startup Podbeskidzie foundation. The premise: now that AI speeds up content production, telling one brand from another gets harder.

Paweł Sala, co-founder and former CEO of FreshMail, showed something I did not expect: how three models score the same messages, and how far apart they land.

• Claude came across as an analytical critic that knows who it is scoring for and adjusts its priorities to the recipient. It gives credit for a good subject line even when the body falls short technically. It favours mail that matters to that specific person.
• ChatGPT is the even-handed reviewer: consistent, predictable, the most conservative of the three. It rarely gives a five and rarely a one, and it judges the quality of the message more than its relevance.
• Gemini classifies rather than scores. The most binary of the three: a technical error is an instant one, great content an instant five. A strict examiner, pass or fail.

The same text gets three different scores, because each model is asking a different question. Tune an email for one and the other two may not notice at all. „Write better emails” does not cover that.

The second half went further: the model reads a message in the context of the whole inbox. A sudden change of tone or content from a sender you know looks unnatural and drags the score down. History works the other way too, because if you have already been getting mail on a topic, the model concludes it interests you and takes the next one more seriously.

Sales offers show this best. The classic „buy before 23:59, the promotion ends” is spotted immediately, because the model can see that a similar message goes out every week. Instead of an opportunity it gets labelled a routine offer, described less than honestly. Ordinary scams are just as easy to catch.

For the sender the conclusion is uncomfortable: everything you sent over the past few months feeds into the score of a single email.

That part does not have to be guesswork. Connectors now let you plug an agent straight into your own mailbox, so whatever a model does with your email at the other end, you can check on your own correspondence.

The rest of the talk stayed with classics that work regardless of AI: emotion as the foundation of decisions (after Antonio Damasio), the framing effect, where „save 200 zł a year” hits harder than „just 16 zł a month” even though the arithmetic is identical, and the consistency rule, where someone who has taken a small step is more likely to take the next one.

Olga Wilczyńska carried the same idea into copy. Her thesis: in a world where anyone can write like an expert, the winner is whoever people believe. Two rules stayed with me. Do not name the emotion, show the situation: instead of „your child struggles with maths”, describe the evening that ends in tears twenty minutes in. And do not write what the client gets, write what they get out of it: „the treatment takes 40 minutes” becomes „you can have it done on Friday and be back at work without explaining your face to anyone”.

The third talk, by Mateusz Miernikowski of Konceptika, ran under the heading „What if we do not need to say more, only ask more?”. Instead of further advice on writing we got a set of questions a company should put to itself and then run through a model along with its own documents: what are we really selling, where do we really make money, what can we credibly promise, where are we genuinely better than the competition, and where is the gap between our promise and the client’s experience.

The most interesting part was the discipline built into the prompts themselves. Each one tells the model to work only from the supplied material, to point at the document behind every conclusion, to separate fact from hypothesis, and to name missing data rather than guess it. That is the difference between asking a model for an opinion and using it for analysis, and it is a pattern worth copying whatever your line of work.

Friday: two ways of measuring models

The next day, same town, same Rekord, this time on Kasprowicza street. The second meetup of bielsko.ai, the regional AI community run by Tomasz Kielar and Michał Staśkiewicz. It was part of the BBDays4.IT festival programme, with a two-day hackathon on the schedule, and it filled the room.

The guest was Przemek Smyrdek, co-founder of 10xDevs, on measuring models and on 10xBench. It was not his first appearance in town: in April he spoke on the same subject at deBBug, the third edition of the local IT meetup run by Artur Dorda, under the title „How to find your own best AI coding agent?”. His thesis was simple and inconvenient:

Public benchmarks tell you nothing about your tasks.

A ranking where model X beats model Y was built on somebody else’s set of tasks. Your project has its own stack, its own conventions and its own traps, and only there can you see which model actually delivers.

For me this was not news, because I first saw 10xBench at that April deBBug, before I started the 10xDevs course. But this knowledge barely leaves a narrow circle. In the world I move in, measuring models on your own tasks is obvious; outside it, choosing a model still usually comes down to what someone wrote online or what happened to win somebody’s ranking. Which is why it is worth writing down.

10xBench exists to measure exactly that: you run the same set of tasks through several models and compare four things, namely time, number of mistakes, number of hits and cost. Not for free, since a meaningful run across a dozen or so models on OpenRouter costs somewhere around 100 to 200 dollars. You pay once, though, and live off the answer for months.

The Przeprogramowani team’s public results are at 10xbench.ai: sixteen model families, from GPT and Claude through Gemini to DeepSeek, Qwen and Kimi, each given several attempts at the same task, scored on whether the project built, whether it stuck to the required stack, whether it is responsive and whether the SEO is there. The reference task is building the Przeprogramowani.pl site in Astro, React and Tailwind on Cloudflare.

A funny coincidence, since that is almost exactly the stack behind the page you are reading. Przemek’s point holds anyway, because even a benchmark built on my stack measures somebody else’s project, with somebody else’s conventions and traps. A good starting point and nothing more.

One caveat about those April results: several models shipped days before the meetup, or on the day itself, so they simply did not make it into the test. Which is the problem with rankings in a nutshell: a comparison ages faster than it spreads.

Two things stuck with me from what the test did show. First, Codex handles a specific, named stack very well, Astro, React and Laravel among them, and writes a modern front-end straight away without being told in the prompt which version to target. Second, and more important: doing well on the front-end says nothing about the back-end. Those are two separate skills and they have to be measured separately.

Your own measurements turn up things no public ranking will show. A more expensive model is sometimes worse, and so is one with a larger context window. It happens that despite an explicit instruction in the prompt it fails to carry it out fully, or skips the tests. GPT-5.6 alone comes in several variants at different prices, and their average scores on the same set of tasks differ noticeably.

The age of training data hurts most

There is one more thing a ranking will not show you, and in practice it costs the most time: a model knows the version of a framework it was trained on. For me this shows up most clearly with Astro and FilamentPHP, because both change their APIs faster than training cycles close. The effect is always the same: the code looks reasonable and does not work, because it reaches for namespaces and functions the current version no longer has. Then come several rounds of fixing the same thing.

Rather than fighting that with prompts, I keep separate rule files for frameworks like these, for instance one for Filament 5 listing the namespace changes against versions 3 and 4. The model gets it as context before it starts writing and stops guessing from what it remembers of training. Knowledge written down properly once beats fixing the same mistake over and over.

Choosing a model therefore starts with the stack you have and ends with how much work you put into context and precise instructions. The ranking sits somewhere in between and means little. From what I see, GPT leads today on front-end work with a modern stack, while on more specific tasks it varies. Among the Chinese models Kimi keeps getting better reviews and is on my list to try.

How many tokens per second one box gives you

Filip Brodka from CSHARK demonstrated an NVIDIA DGX Spark: a Qwen3.8-27B model running live, on hardware standing in the room, with no API behind it. It is a dense model of 27.8 billion parameters, natively multimodal, under Apache 2.0.

The most important sentence of the evening was about arithmetic:

Ceiling on tokens per second = memory bandwidth ÷ size of the weights.

At 273 GB/s that gives a hard limit no configuration will beat. Same machine, same model, four quantisation variants:

More than a threefold difference in speed between the heaviest and the lightest variant, and it follows directly from how many bytes have to be read for every token.

That arithmetic turned out to be strikingly accurate. It predicted about thirteen tokens per second and thirteen is what the hardware delivered. Good news for anyone weighing up such a purchase, because you can estimate the throughput before spending the money. You do not even have to do the sums yourself, since llmfit is there for it.

The worse news is that the arithmetic only sets the floor. With one software technique enabled, the same model on the same hardware gave more than four times as much. So the calculation tells you what you can count on, and for the rest you have to ask the software and measure it yourself.

The full configuration, the scripts and every number from the talk are public: github.com/fifeek0/bielskoAI.

What this means for a business

One such machine is enough for simpler work. Larger jobs mean linking machines together, and then the cost stops adding up, unless you are putting it in a data centre and need a closed offline system because the data cannot leave. At that point it starts being a real option.

On top of that comes a trend that shifts the calculation more than the price lists suggest: the strongest models either get more expensive or get capped. Access to the best variant is increasingly rationed rather than simply costly, as the Codex 20x style plans show, where what you pay for is a higher usage ceiling.

That is where your own hardware starts to look different. For ordinary API use it is still cheaper to rent, but a machine of your own gives you two things no subscription sells: there is no meter on it and the data never leaves the building. If rationing of the strongest models tightens, that balance will tip towards owning the hardware faster than a comparison of rates would suggest.

What nobody measured

All four speakers were measuring essentially the same thing at different scales: an email against a whole inbox, a model against a specific project, hardware against memory bandwidth, a company against its own documents.

One measurement was missing, and it is the one every company runs into sooner or later: what these systems answer when someone outside asks about a firm like yours. The test takes five minutes. Ask an assistant about your service exactly the way your client would, and see whether you come up at all.

The result can be unpleasant, and the cause usually sits outside the website itself. Whether a company makes the list at all depends on whether anyone other than the company writes about it: directories, rankings, other people’s write-ups. The places you do not control, which is exactly why they count.

Olga said the winner is whoever people believe. I would add one sentence to that: before anyone can believe you, they have to see you first.

Good that this is happening here

Bielsko-Biała has had an IT scene for a long time. meet.js has been gathering people for years, BBDays4.IT comes round annually, and companies in the field are based here. What is new is meetups devoted entirely to AI, and enough of them that you can go regularly rather than once a quarter: deBBug started in spring, bielsko.ai in summer. I try to keep up with them, because that is where most of what ends up in my work comes from, rather than from rankings online.

The result of this one week: I heard how three models score the same emails, and I watched a model running on hardware standing two metres away. After hours, ten minutes from home.

Thanks to the Startup Podbeskidzie foundation for Thursday, to Tomasz Kielar and Michał Staśkiewicz for bielsko.ai, and to Artur Dorda for deBBug, where all of this started for me. Thanks to the speakers too, for bringing numbers instead of slides about a revolution.

If you are anywhere nearby: sign-ups for the next bielsko.ai meetups go through Luma, the community lives on Discord, and hackathons are on the way. Worth dropping in, if only to listen. The room is full of developers, but nobody checks your credentials at the door.