A colleague wanted to extract serial numbers from a batch of support files. The plan involved a language model, a prompt, and a retry loop for malformed output.
The serial numbers had a fixed format. The answer was a regular expression: instant, free, deterministic, and correct on every input rather than almost every input.
This is not an argument against language models. It is an argument for knowing which problem you have.
The test
Does the input have a reliable structure?
If yes, parse it. A log line with a fixed format, a filename convention, a CSV, a structured API response, an identifier with a defined shape. These are parsing problems and parsers solve them completely.
If no, and the meaning matters more than the form, a model is the right tool. Free text, inconsistent human writing, documents from many sources with no common structure.
The interesting cases are mixed, and the right answer is usually both: parse what is structured, use a model for the rest.
Why the model is often the worse choice here
It is not deterministic. The same input can produce different output. For a task with a single correct answer, that is a defect.
It fails silently and plausibly. A regex that does not match returns nothing, which is an obvious signal. A model returns something confident and wrong, which is not.
It costs, in money or time or memory. A regex costs microseconds.
It needs error handling you would not otherwise write. Validation, retries, fallbacks. All of that is code you maintain to compensate for non-determinism you chose to introduce.
It cannot be tested exhaustively. You can reason about a regex completely. You can only sample a model’s behaviour.
Where the instinct comes from
Models are genuinely remarkable, and having one available makes every problem look like a candidate. There is also something satisfying about solving a problem by describing it in English rather than by writing a parser.
But the satisfaction is the wrong signal. The right signal is whether the problem has a known grammar.
The hybrid that usually wins
Parse the structure, model the ambiguity. Extract the fields with code. Where a field contains free text that needs interpretation, hand that specific field to a model. Validate the model’s output against a schema, because you can.
This produces systems that are mostly deterministic with a small non-deterministic component that is bounded and checkable. Far easier to reason about than a pipeline where a model is doing everything including things a parser would do perfectly.
The version of this that matters
The same principle applies further up. Before building a retrieval system over your documents, check whether the answer is in a database somebody already maintains. Before training anything, check whether a lookup table would do.
The cheapest system that solves the problem is the one you should build, and in a field moving this fast the pull is constantly toward the most sophisticated available tool rather than the most appropriate one.
Being the person who says “this is a parsing problem” is not a fashionable position. It is usually the correct one, and it is always the one that still works in three years without maintenance.