The Reality of Prompt Engineering

By Sherwin Vishesh Jathanna. Published August 3, 2026. 8 min read.

In March 2024, IEEE Spectrum published an obituary. The headline read "AI Prompt Engineering Is Dead" [1]. Two years on, the model companies still publish prompt engineering guides, the most valuable document inside most AI products is still a prompt, and a coding agent will still burn an hour of compute on a badly written one.

So the obituary was half right. Something did die, and it deserved to. What replaced it is smaller, quieter, and considerably more important. That distinction is the whole story, and almost every take on the subject flattens it.

The Era of Magic Words

ChatGPT launched in November 2022 on GPT-3.5 [2]. It was astonishing for 2022. It is painfully unreliable by the standards of today.

It lost the thread halfway through an instruction. It drifted out of whatever format you asked for. It reasoned shallowly unless you pushed it. And the prompt was the only control surface you had. No tools. No structured outputs. No thinking budget. Just words in a box.

So a craft grew around holding the model exactly right. It was not folklore. Most of it came straight out of published research.

Few-shot prompting. The GPT-3 paper was literally titled "Language Models are Few-Shot Learners". You pasted worked examples above your question because the model was dramatically better with them than without [3].
Role prompting. "You are a senior tax accountant with twenty years of experience." A persona reshaped tone and framing, and on early models it often moved the quality of the answer too.
Chain-of-thought. Wei and colleagues showed that demonstrating step-by-step reasoning lifted performance on math and logic [4]. Kojima and colleagues found the shortcut: append "Let's think step by step" and skip the demonstrations entirely [5].
Delimiters. Triple backticks. XML tags. Anything that stopped the model from confusing your instructions with your data.
Format pleading. "Respond ONLY with valid JSON. No commentary." Repeated. Capitalized. Ignored often enough that production code shipped with retry logic around it.
Decomposition. Splitting one job across several calls, because a single call could not hold a multi-step task together.
Incentives and emotion. "This is very important to my career" produced measurable gains on one benchmark [6], and a follow-up study catalogued 26 prompting principles, politeness and tipping among them [7].

None of this was superstition. The gains were benchmarked and published. By 2024 a systematic survey had counted 58 distinct text-based prompting techniques [8].

Fifty-eight techniques. To ask a question.

That number tells you two things at once. The field was real. And it had become absurd.

What the Models Swallowed

Three things ended that era, and not one of them was a change in how people typed.

Instruction tuning matured. Models now parse casual, imprecise English and infer what you meant instead of answering what you said.

Reasoning moved inside. For the reasoning class of models, the deliberate thinking that chain-of-thought used to inject by hand now runs internally before the first visible token. Explicit reasoning cues still earn their keep on smaller models and unusual tasks, but you are no longer the one supplying the thinking.

The fragile parts became platform features. Structured outputs enforce a schema instead of hoping the model obeys [9]. Native tool calling replaced copy-pasted context. Long context windows retired the summarize-and-stitch dance.

What none of that removed is the work of deciding what belongs in the window in the first place. Hold that thought.

Here is where the old toolkit went.

Role prompting ("You are an expert..."): Rarely needed for competence. Still useful for scope and audience.
"Let's think step by step": Built into reasoning models that plan before they answer.
Stacks of few-shot examples: Zero-shot works. One example, for genuine ambiguity.
Format pleading ("ONLY valid JSON"): Structured outputs, enforced at the API level.
Tips, threats, emotional appeals: Effects shrank toward zero as models improved.

The practical result: for ordinary work, the gap between a lazy prompt and a crafted one has nearly closed. You type a vague sentence. You get a solid answer. Often the first response is good enough to use as it is.

That gap has not closed everywhere. Ambiguous analysis, transformations with a dozen constraints, anything where you hold context the model cannot see: all of it still rewards care. But the everyday experience is the first one, and the everyday experience is what every "prompt engineering is dead" post is actually describing.

What Actually Survived

The strategies did not go to zero. They consolidated. Strip the current vendor guidance down to its load-bearing parts [10] and five things are left.

Say what you want. The task, who it is for, and what a good result looks like.
Give clean context. Everything the model cannot know on its own, and nothing it does not need. Irrelevant context actively degrades the output.
Show one example, and only when the task is genuinely ambiguous. Skip the other four you would have pasted in 2023.
State the constraints. Length, tone, format, and what to avoid.
Check the result. Especially where you cannot judge correctness at a glance.

Read that list again. There is not a trick anywhere in it. That is how you would brief a capable colleague, which is exactly the point.

"Every technique that survived is a form of being clearer. Everything that died was a workaround for a model that could not meet you halfway."

Where It Got Serious

The interesting story was never chat. It is what happened inside software.

A GenAI product, whether a support bot, a document analyzer, or a code review tool, is a pipeline with a model call in the middle. At the center of that call sits a system prompt.

A system prompt is not a suggestion. It is a specification.

Its role, its rules, its tone, its refusal cases, its output schema, its edge case handling. Getting that prompt right is what prompt engineering means in production, and it looks nothing like typing magic words.

You draft it. You run it against a suite of real inputs. You study the failures. You tighten the wording. You run it again. You measure whether the new version fixed what was broken without breaking what already worked. You version it, you review changes to it, and you regression-test it before it ships, because a system prompt serving millions of requests turns one small ambiguity into a support queue. Frameworks like DSPy push this to its logical end and treat prompt optimization as a compilation problem, searching automatically for the wording that maximizes a metric [11].

Here is what that looks like with something at stake. A support agent answers 97 percent of routine questions correctly. It also reads "cancel my order" the same way whether the order has shipped or not. One sentence added to the system prompt, telling it to check fulfillment status before offering a cancellation, removes that entire class of failure. It may also quietly change how the agent handles refund questions. Which is why the evaluation suite runs on every edit.

Prompt changes behave like code changes. Small, targeted, and perfectly capable of breaking something three features away.

The modern twist is how short the loop has become. Frontier models are good enough that a clear first draft usually lands near the target, and iteration runs in a handful of passes instead of weeks of coaxing. You are not fighting the model any more. You are aligning it with a product, and most of the remaining effort goes into the evaluation set that proves the prompt does what you say it does.

In practice: a production system prompt is a product asset. It gets versioned, tested, and reviewed like code, because a one-line edit to it changes product behavior exactly the way a code change does.

The Agent Problem

The strongest evidence that prompt engineering is alive is the newest category of tool.

In an agent harness like Claude Code or Codex, a model works inside your repository for minutes or hours at a stretch. It reads files. It runs commands. It edits code. Nobody is watching each step.

Two layers of prompting govern every session. The first is the harness's own system prompt: thousands of words defining the agent's tools, its safety rules, and its working style, written and maintained by the vendor with the care of an API contract [12]. The second layer is yours. The task you type, plus the standing instructions you keep in files like CLAUDE.md or AGENTS.md that load into every session as durable context [13].

The second layer is where the old skill comes back, with higher stakes.

A human colleague fills the gaps in a vague request by asking you a question. An agent fills them with assumptions, at machine speed, forty tool calls deep before you notice the drift. "Fix the auth bug" and a paragraph naming the failing case, the suspected module, the constraint that the public API cannot change, and the command that proves the fix are not two versions of the same request. They are an hour of useful work and an hour of confident wrong turns. The industry's name for the wider discipline is context engineering: deciding what enters the model's context window at all, not just how the request is phrased [14].

And here is the inverse relationship at the heart of this whole story. The better models get, the less your wording matters on any single exchange, because the model repairs small ambiguities on its own. But better models also run longer without supervision, and over a long run the opening specification is most of the steering you get. Every ambiguity in it is multiplied by every step the agent takes.

Micro-prompting lost its leverage at the exact moment macro-prompting gained it.

Prompting stopped mattering for chat as it started compounding for builders.

So Is It Dead?

As a bag of incantations, yes. The models absorbed it, and nothing of value was lost.

As a job title, mostly. The role folded into ordinary engineering work the way "webmaster" dissolved into a dozen specializations.

As a discipline, no. It moved into system prompts, evaluation suites, agent instructions, and context design, where it is practiced every day by people who never use the term.

This is the familiar arc of an abstraction that works. The skill did not disappear. It moved up the stack, got renamed, and stopped being visible from the chat box.

The Bottom Line

The magic words are gone, and good riddance. Role prompts, step-by-step incantations, tips and threats. The models absorbed all of it, and nothing worth keeping went with them.
What survived is not a trick. It is clarity. Say what you want, give clean context, state the constraints, check the result. That is a brief, not a spell.
Casual prompting needs no engineering. Production prompting is nothing but engineering. A system prompt is a product asset, and it gets versioned, tested, and reviewed like code.
Agents raised the stakes rather than lowering them. Wording matters less per exchange and more per run, because every ambiguity in the opening specification is multiplied by every step the agent takes.
The real discipline is context, not phrasing. Deciding what enters the window at all is the part the models have not taken, and the part nobody can do for you.

If the prompt serves you, type plainly. If it serves a product or an agent, treat it like a spec.

References

IEEE Spectrum, "AI Prompt Engineering Is Dead", March 2024.
OpenAI, "Introducing ChatGPT", November 2022.
T. Brown et al., "Language Models are Few-Shot Learners", NeurIPS 2020.
J. Wei et al., "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models", NeurIPS 2022.
T. Kojima et al., "Large Language Models are Zero-Shot Reasoners", NeurIPS 2022.
C. Li et al., "Large Language Models Understand and Can Be Enhanced by Emotional Stimuli", 2023.
S. M. Bsharat, A. Myrzakhan, Z. Shen, "Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4", 2023.
S. Schulhoff et al., "The Prompt Report: A Systematic Survey of Prompting Techniques", 2024.
OpenAI, "Introducing Structured Outputs in the API", August 2024.
Anthropic, "Prompt Engineering Overview", Claude Documentation.
O. Khattab et al., "DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines", 2023.
Anthropic, "Building Effective Agents", December 2024.
Anthropic, "Claude Code: Best Practices for Agentic Coding", April 2025.
Anthropic, "Effective Context Engineering for AI Agents", September 2025.

Machine-readable version of this page: https://www.sherwinvishesh.com/blog/prompt-engineering-reality.md | site index: https://www.sherwinvishesh.com/llms.txt