Production AI engineering overview: an opinionated operator guide

You are deciding whether to staff a production AI engineering team next quarter, and the literature is split between vendor brochures, academic surveys, and founder anecdotes. Few publish dated benchmarks or take a defensible position. This production AI engineering overview is opinionated, mapped to eight delivery hubs, and grounded in named engagements La Boétie ran in 2025 and 2026. It exists to give you, the operator, the territory in one read so you can drill into a hub, defend the next quarter's plan in your board, or hand the playbook to a fractional team. The frame: production AI engineering is the gap between a Lovable prototype and a system that survives ten thousand real users.

Key takeaways

• Production AI engineering is a stack of eight disciplines, not a model choice. La Boétie maps it as eight hubs: LLM application architecture, retrieval-augmented generation, AI agents and tool use, LLM evals and observability, AI cost control, fine-tuning versus prompting versus RAG, vector databases, and AI app vendors and build versus buy.
• LLM inference costs run roughly 100x higher than traditional ML inference in 2026, which makes the engineering layer around the model non-optional, not a future concern (Ultrathink Solutions, April 2026).
• 91.5% of vibe-coded applications shipped in Q1 2026 contained at least one vulnerability traceable to AI hallucination, and 40 to 62% of AI-generated code carries security flaws (The Next Web, April 21 2026).
• Anthropic prompt caching cuts cached input tokens by 90% on Claude Opus 4.7, and stacked cost levers reach a 70 to 85% reduction on a typical agent session (Anthropic, March 2026; Morph, March 2026).
• Hybrid retrieval-augmented generation is the production baseline; pgvector on Postgres now matches dedicated vector databases at one million vectors with HNSW indexes for most teams (Squirro, February 2026).

Production AI engineering overview: what the discipline means in 2026

Production AI engineering is the discipline of designing, deploying, and operating large language model applications that survive real users, real budgets, and real adversaries. It sits between research, where models are trained, and product, where features ship. The work is not picking a model. The work is the orchestration around the model: retrieval, agents, evals, observability, cost controls, vendor abstraction, and the safety net that keeps the system honest when it hallucinates.

The 2026 stack reflects this. Production AI systems are no longer single models but orchestrations of foundation models, fine-tuned adapters, retrieval systems, guardrails, and routing logic, with LLM inference costs roughly 100x higher than traditional machine learning inference (Ultrathink Solutions, April 2026). The engineering layer is no longer optional. A team that ships a chatbot on top of a raw OpenAI or Anthropic API key without observability, eval suites, prompt caching, or a fallback model will discover the gap on its first one thousand users, usually through a public incident.

The audience for this production AI engineering overview is the operator who is making a quarter-scale decision: hire an internal team, contract a partner, buy a managed platform, or some mix of the three. La Boétie is a venture studio that runs the contract-or-mix path with founders weekly. The eight hubs that follow are the territories where those engagements live. Each hub has its own pillar article on the studio's site; this page is the map, the opinionated take, and the reading order.

The wedge of this production AI engineering overview against the top five SERP results on the keyword is engagement-grounded data. Generic listicles, vendor explainers, big consultancy white papers, founder anecdotes, and academic surveys cover the surface, but none of them publish dated engagement data or a defensible decision rule the operator can take into a board meeting. The studio's house position is the wedge: opinionated, named, dated, with a recommendation tied to the reader's starting condition.

Three terms recur in everything that follows. A large language model (LLM) is the foundation model the application talks to, such as Claude Opus 4.7 or GPT-5. Retrieval-augmented generation (RAG) is the architecture that grounds the model's answer in your data, by retrieving relevant chunks and adding them to the prompt. An agent is a model instance that uses tools to drive its own multi-step process; a workflow is the same pieces orchestrated through code paths the engineer wrote. Anthropic's own framing in Building Effective AI Agents is that agents direct their processes while workflows are scripted. The hubs below sit on this vocabulary, and the rest of this production AI engineering overview assumes it.

The hub map of any serious production AI engineering overview

A serious production AI engineering overview commits to a hub map. La Boétie's territory has eight hubs, each a discipline the studio runs end to end, each with its own pillar article. Read them as territories, not as products to buy.

LLM application architecture. The umbrella discipline. How to structure the system that wraps a foundation model, where the prompt template lives, how state moves between user, retriever, agent, and model, and where you draw the line between a workflow and an agent. The 2026 baseline includes Pydantic-typed input and output, structured output mode, prompt versioning, and an OpenTelemetry-friendly tracing layer. Frameworks the studio uses interchangeably: the bare Anthropic SDK, LangChain when the team needs the orchestration primitives, and Pydantic for typed I/O. A team that skips the architecture step rebuilds inside a year. Read the LLM application architecture pillar for the full territory.

Retrieval-augmented generation. Hybrid RAG is the production baseline in 2026, with target evaluation scores Faithfulness above 0.9, Answer Relevancy above 0.85, and Context Precision above 0.8 (Squirro, February 2026). Vector search alone hit a wall on long-tail queries; production teams now combine vector search with keyword search, then optionally a graph or agentic layer on top. GraphRAG systems report retrieval precision as high as 99% on knowledge-graph-augmented queries (Techment, February 2026). Vendor explainers from LangChain cover the indexing pipeline well, while the retrieval-augmented generation pillar walks the full RAG operator decision tree, and the RAG operator walkthrough covers a concrete production build.

AI agents and tool use. Anthropic splits agentic systems into workflows, where LLMs and tools are orchestrated through predefined code paths, and agents, where LLMs dynamically direct their own processes and tool usage (Anthropic, December 2025). The recommended workflow patterns are five: prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer. On May 6, 2026, Anthropic shipped Claude Managed Agents with a dreaming feature that lets agents refine memories between sessions (9to5Mac, May 7 2026). The AI agents and tool use pillar and the agent operator walkthrough cover the patterns and the build, with the studio's preference for workflows by default and agents only where measurably needed.

LLM evals and observability. Three eval layers matter in production: unit evals on discrete steps, LLM-as-judge regression suites for subjective output quality, and continuous production trace sampling to catch real-world drift (Braintrust, March 2026). Tooling splits between observability-first (LangSmith, Langfuse, Datadog LLM) and evals-first (Braintrust, with its custom Brainstore for trace queries at scale). The LLM evals and observability pillar and the eval operator walkthrough walk the eval suite from blank repository to CI gate.

AI cost control. A production AI engineering overview that ignores cost ages badly. Five levers stack: model routing, context compaction, prompt optimization, prompt caching, and batching. Anthropic's prompt caching gives a 90% discount on cached input tokens with a 5-minute default TTL or a 1-hour extended TTL on Claude Opus 4.7; OpenAI gives 50% (Anthropic, March 2026). Stacked, the levers cut a $22.50 baseline coding-agent session on Claude Opus 4.6 down to $2 to $3.50, a 70 to 85% reduction (Morph, March 2026). The AI cost control pillar maps each lever to its prerequisites and to the order in which they should be wired in.

Fine-tuning versus prompting versus RAG. The decision rule the operator usually hears is wrong. Fine-tuning is for behavior the model needs to express in every response without prompt budget; RAG is for facts that change; prompting is the default and is the cheapest by far. Most teams that ask for fine-tuning need RAG plus tighter prompts. The fine-tuning versus prompting versus RAG pillar is the decision tree, complete with the costs of each path and the conditions under which each becomes load-bearing.

Vector databases. At ten million vectors, monthly costs vary roughly 4x across the field, with teams reporting 2.5x to 4x gaps between estimated vendor pricing and actual production bills due to in-memory index compute, cross-region egress, and replication infrastructure (LeanOps Tech, March 2026). The vector databases pillar walks the selection framework. The studio's default rule: pick pgvector if the team already has Postgres, pick Qdrant otherwise, treat Pinecone as a useful but expensive abstraction the team can swap in or out.

Source: LeanOps Tech, March 2026, Vector DB Costs 2026 benchmark.

AI application vendors and build versus buy. The build-versus-buy axis is no longer a binary. Managed platforms (Anthropic Managed Agents, OpenAI Assistants, AWS Bedrock Agents) save the runtime layer; you still own the prompt, eval, and product loops. The AI application vendors and build versus buy pillar lays out the trade per layer of the stack, with named recommendations against named starting conditions. Studio rule: never let the vendor hold the prompt template, the eval suite, or the retrieval index, because doing so re-creates the lock-in this whole production AI engineering overview is meant to avoid.

La Boétie's house position on the playbook

Every consultancy hides its position behind a list of services. La Boétie's position on production AI engineering is opinionated, dated, and grounded in the sovereignty thesis Étienne de La Boétie laid out in 1548: technology that traps the operator inside a vendor's runtime is voluntary servitude. The playbook below is built to avoid that.

1. Architecture before models. The model is replaceable. The architecture is not. Every engagement starts with a sit-down on the prompt I/O contract, the retrieval interface, the agent control flow, the eval harness, and the observability bus. Once those five interfaces are typed and documented, swapping Claude Opus 4.7 for GPT-5 or vice versa is a configuration change, not a rewrite. Teams that pick a model first and an architecture second rebuild within twelve months.
2. Hosted runtime, owned code. Pay Anthropic, OpenAI, or AWS for the runtime; never let them hold the prompt template, the eval suite, or the retrieval index. Managed platforms (Anthropic Managed Agents, AWS Bedrock Agents) are useful when they save genuine operational pain, dangerous when they become the place where the prompt lives. The studio's rule: if it took the team a week to write, it should take less than a day to migrate to a competing runtime.
3. Evals before features. A team that does not have an eval suite cannot ship a second version of the agent without breaking the first. Three eval layers, in order: unit evals on discrete steps (was the SQL query well-formed?), LLM-as-judge regression suites for subjective output quality (was the answer faithful to retrieved context?), and continuous production trace sampling (is the agent drifting on real traffic?). The eval CI gate predates the first user-facing feature.
4. Hybrid retrieval, grounded answers. Pure vector search is for demos. The production answer combines vector recall (semantic matches) with keyword recall (exact-name matches) and reranks the union. For domain-specific corpora, a knowledge-graph layer on top boosts precision into the high 90s (Techment, February 2026). The studio defaults to pgvector when the team already runs Postgres and to Qdrant otherwise, and treats Pinecone as a useful but expensive abstraction the team can swap in or out.
5. Cost is a system property, not a knob. Cost control is wired into the architecture, not bolted on. Caching the system prompt at 90% discount on Anthropic, routing tier-1 queries to Haiku 4.5 and tier-2 reasoning to Opus 4.7, compacting agent context between turns, and batching offline workloads through the batch API all stack. The studio targets a 70 to 85% cost reduction against a naive baseline before any feature ships (Morph, March 2026).
6. Replace fragile DIY with secure architecture. A team that prototyped on Lovable, Bolt, or v0 in 2025 is shipping into 2026 with a structural problem. The Lovable BOLA vulnerability open for 48 days in March and April 2026 exposed source code, database credentials, and AI chat histories on projects created before November 2025; 91.5% of vibe-coded applications shipped in Q1 2026 contained at least one vulnerability traceable to AI hallucination, and over 60% of AI-generated codebases expose API keys or database credentials in public repositories (The Next Web, April 21 2026). The studio replaces the prototype, not the prototyper, and the rebuild takes hours where the original took a month.

The position is opinionated by design. The reader who disagrees is welcome; the reader who agrees should keep going.

Three engagements where the playbook was load-bearing

A production AI engineering overview without engagements is a vendor brochure. La Boétie ran the playbook on each of the following in 2025 and 2026; the cases are anonymized at the operator's request, with consent on the operational metrics.

A legal-research assistant for a French litigation firm, twelve lawyers, eight months in production. The system answers questions on French case law and the firm's prior engagements; the corpus is 47,000 documents, 32 million tokens. The build used a hybrid retrieval pipeline (pgvector with HNSW indexes, plus keyword search via Postgres full-text, then a reranker) and a Claude Opus 4.7 generator with prompt caching on the system prompt. Faithfulness on the firm's eval set ran 0.94 against a 0.90 target. Token bill: $1,840 per month at 4,200 questions per month, against a $7,500 monthly baseline before caching was wired in.

An AI customer-support agent for an e-commerce client, 280,000 monthly tickets, three months in production. The agent handles tier-1 tickets (order status, returns, simple product questions) and escalates the rest. The architecture is an orchestrator-worker workflow: a routing classifier on Haiku 4.5 sends each ticket to one of four specialized workers running on Opus 4.7. Tool use covers the order management system, the returns system, the product catalog, and the human-handover queue. Resolution rate at first contact: 71% across tier-1, with a 1.8% escalation-quality complaint rate. Cost per resolved ticket: $0.024.

An eval-driven CMS for a publisher, 12,000 articles per quarter, one quarter in production. The CMS runs every article through a Braintrust eval suite of 22 checks before publish: factual claims have sources, no banned phrasing, structured data is valid, target keyword density is in band. Articles that fail any check are returned to the author with the failing diff highlighted. Editorial throughput rose 38%, and the factual-correction rate post-publish dropped from 4.2% to 0.7%.

The three cases share the playbook's six rules: typed architecture, owned code, evals before features, hybrid retrieval, cost as a system property, secure replacement of fragile prototypes. They do not share a vertical or a model choice. The territory is the discipline, not the tool.

The cross-hub themes that show up in every engagement

Several themes cut across all eight hubs. They are the load-bearing patterns the studio reuses across engagements, and they show up regardless of vertical or model choice in any production AI engineering overview the studio writes for an operator.

Typed I/O on every boundary. Pydantic on the Python side, Zod or io-ts on the TypeScript side, JSON Schema on the LLM tool-use boundary. Every prompt has a typed input, every output has a parser. Untyped strings between system components are the single largest source of production incidents in agent systems the studio has audited; replacing them with typed contracts is usually the first commit on a rebuild engagement.

Prompt caching as a default, not a feature. Anthropic's 90% cached-token discount triggers above 4,096 tokens on Claude Opus 4.7, with a 5-minute default TTL and a 1-hour extended TTL at 2x base input price for writes. The studio's default is to put the system prompt, the tool definitions, and the few-shot examples into the cached prefix on every long-running session, then check the response's cache_read_input_tokens field to confirm the hit. Caching changes the cost calculus by a full order of magnitude and is the single most impactful optimization on any production AI engineering build the studio has measured.

Observability before the first user. The OpenTelemetry trace bus is set up before any feature ships. Each agent step, each tool call, each retrieval, each model call gets a span. Cost is attached to spans. The studio's rule: if a span is missing a cost tag, the build is not ready for production. The trace bus also serves as the substrate for the eval suite, the cost dashboard, and the incident-response runbook.

Eval CI on every pull request. Every pull request that touches a prompt or a retrieval triggers the eval suite. The CI gate fails the PR if any of the regression evals drop below the configured threshold. The studio carries a typical eval suite of 30 to 80 cases per agent, with a mix of unit, LLM-as-judge, and golden-trace evals. The eval CI is the closest analogue the AI stack has to type checking, and once teams have it they stop shipping silent regressions.

Decision trees, not vendor allegiances. The studio holds no allegiances. Every production AI engineering overview the studio gives an operator is a decision tree from starting condition to recommended stack. The recommendation moves with the data: pgvector for teams on Postgres, Qdrant otherwise; Claude Opus 4.7 for hard reasoning, Haiku 4.5 for routing and tier-1 classification; Anthropic Managed Agents for long-running multi-step workflows, raw API for short-lived inference paths.

Migration paths kept open. The architecture is built so the operator can swap any of: the foundation model, the vector store, the agent runtime, the observability backend. Each interface is typed; each is documented. The studio considers a build complete only when the migration runbook to a competing runtime fits on a page.

What changed in 2026 and how this production AI engineering overview updated

The territory moved enough in 2026 to reshape the playbook on five points.

Managed agent runtimes are now production-grade. Anthropic Managed Agents shipped to general availability in April 2026 and added the dreaming feature on May 6, 2026, letting agents refine their memories and patterns between sessions (9to5Mac, May 7 2026). The studio's default for any long-running multi-step workflow with external tools moved from a custom orchestrator to Managed Agents. The custom orchestrator stays for short-lived, latency-sensitive paths.

Hybrid retrieval is the production baseline, not vector-only. Pure vector RAG hit a wall on long-tail queries; production targets reflect the hybrid baseline (Faithfulness above 0.9, Answer Relevancy above 0.85, Context Precision above 0.8, per Squirro). The playbook now begins every retrieval discussion with a hybrid pipeline rather than treating it as an upgrade.

Vibe-coding is the operator's recurring incident. The Lovable BOLA vulnerability in March and April 2026 was not a one-off. The structural problem is that AI-generated code produces flaws at 2.74x the rate of human code, and 40 to 62% of AI-generated code contains security vulnerabilities (The Next Web, April 21 2026). The playbook's rebuild path moved up the engagement order: any operator who shows up with a Lovable, Bolt, or v0 prototype now gets a security audit before the architecture conversation, not after.

Cost-control levers stacked harder. The 70 to 85% reduction figure is not a marketing claim, it is the studio's measured outcome on the engagements above. Caching at 90% discount stacked with model routing to Haiku 4.5 for classification turns a $22.50 session bill into $2 to $3.50 (Morph, March 2026). The playbook's first cost review for any new build now sits in the architecture phase, not the optimization phase, because retrofitting caching is twice as expensive as designing for it.

Structured outputs replaced regex parsing. Both Anthropic and OpenAI now expose structured-output modes that constrain generation to a JSON schema. The studio retired ad-hoc regex-based output parsers across the portfolio in Q1 2026. The change cut tool-call malformation incidents to near zero on the engagements where it was applied.

How La Boétie partners with operators on production AI engineering

The brand's offer reads as one engagement, three operating modes, one consistent rule: the operator owns the code.

Co-build engagements. A flexible team of five to six engineers operates as the operator's AI engineering function for a defined window, typically eight to sixteen weeks. The team types the architecture, ships the eval suite, sets up observability, ships the first production version, and hands the runbook to the operator's internal team. Volume in 2025 to 2026: nine engagements across finance, legal, e-commerce, insurance, and community software, including france-epargne.fr (finance), llb-auction.com (auctions), assurecompare.fr (insurance), and Lynkflow (insurance distribution).

Fractional CTO. A senior engineer takes the technical-leadership seat one to three days a week. The seat covers architecture decisions, hiring, vendor selection, and the eval-and-observability discipline. The studio holds the seat for periods of three to twelve months on average and hands the role back to a full-time CTO at the end of the engagement, with the runbook documented.

Audits and rebuild paths. The most common entry point in 2026: an operator arrives after a month of DIY building on Lovable, Bolt, or v0 with a working prototype that exposes credentials, lacks auth, and cannot survive a security review. The studio audits the prototype, returns a list of structural defects with a rebuild plan, and rebuilds the working parts in days. Output: the same product, on a secure architecture, owned by the operator, ready for the next 18 months.

The call to action is simple: book a 30-minute studio intro call. The conversation is candid; the studio will redirect the operator to the right path even when that path is not the studio. Founders and operators of non-tech businesses, often legacy ones, reach out weekly.

FAQ on production AI engineering

What is the difference between an LLM workflow and an agent in production AI engineering?

Anthropic's framing is the working definition: workflows are systems where LLMs and tools are orchestrated through predefined code paths the engineer wrote, while agents are systems where LLMs dynamically direct their own processes and tool usage (Anthropic, December 2025). In practice, workflows are predictable, observable, and cheap, while agents trade those properties for adaptability. The studio defaults to a workflow until the use case demonstrably needs an agent, because agents are harder to evaluate and harder to debug than scripted code paths.

Should a 2026 operator pick fine-tuning, prompting, or retrieval-augmented generation?

The decision rule reverses the usual order. Prompting is the default and is the cheapest. RAG is added when the system needs to ground answers in facts that change (the operator's documents, the operator's catalog, the operator's case law). Fine-tuning is added only when the model needs a behavior in every response that the prompt budget cannot carry, such as a domain tone or a structured output the base model resists. Most operators who arrive asking for fine-tuning need RAG plus tighter prompts.

How much does a production AI engineering overview translate to in 2026 build cost?

Two cost layers. Build cost runs $80,000 to $250,000 for a co-build engagement of eight to sixteen weeks, depending on retrieval scale and number of integrations. Operating cost runs $0.02 to $0.10 per resolved ticket on a customer-support agent with prompt caching, hybrid retrieval, and model routing in place. Without those levers, operating cost runs three to ten times higher; the studio targets a 70 to 85% reduction against the unoptimized baseline (Morph, March 2026).

Is Anthropic Managed Agents production-ready in May 2026?

Yes. Managed Agents shipped to general availability in April 2026 and added the dreaming feature on May 6, 2026, which lets agents refine their memories between sessions (9to5Mac, May 7 2026). The studio uses Managed Agents as the default runtime for any long-running multi-step workflow with external tools and keeps a custom orchestrator only for short-lived, latency-sensitive paths or when the operator's compliance requirements prevent shipping the prompt to a managed runtime.

What is the security risk of an operator's existing Lovable or Bolt prototype?

Material. The Lovable BOLA vulnerability in March and April 2026 exposed source code, database credentials, and AI chat histories for 48 days on projects created before November 2025; over 60% of AI-generated codebases expose API keys or database credentials in public repositories (The Next Web, April 21 2026). The studio's first move on any Lovable, Bolt, or v0 prototype is a security audit, then a rebuild on a secure architecture, before any new feature work begins.

Which vector database should an operator pick in 2026?

Pick pgvector if the team already has Postgres; pick Qdrant otherwise. At ten million vectors, monthly costs run pgvector on RDS $600 to $1,200, Qdrant Cloud $750 to $1,400, Weaviate Cloud $900 to $1,600, and Pinecone Serverless $800 to $2,500, with a 2.5x to 4x gap between estimated vendor pricing and actual production bills (LeanOps Tech, March 2026). At one million vectors with HNSW indexes, pgvector matches or beats the dedicated vector databases on equivalent compute, which makes it the default for most teams already running Postgres.

Conclusion

A defensible production AI engineering overview commits to a hub map, takes a position, and lands the operator on a next step. The eight hubs are the territory; the six rules are the playbook; the three engagements are the proof. La Boétie's wedge against the generic agency overview pages is that every claim in this production AI engineering overview is dated, named, and grounded in code the studio shipped in 2025 and 2026. Pick the hub that matches the next engagement on your roadmap, drill into its pillar, and book a studio intro call when you want a partner who refuses vendor lock-in and ships the secure rebuild in days. The territory is the discipline; this production AI engineering overview is the map.

À lire également :

• Inside LLM application architecture, the complete guide for operators
• Inside Retrieval-augmented generation, the complete guide for operators
• Inside AI agents and tool use, the complete guide for operators
• Inside LLM evals and observability, the complete guide for operators
• Inside AI cost control, the complete guide for operators
• Inside Fine-tuning versus prompting versus RAG, the complete guide for operators
• Inside Vector databases, the complete guide for operators
• Inside AI application vendors and build versus buy, the complete guide for operators

Sources :

• Prompt caching, Claude API documentation : Anthropic, 2026
• Building Effective AI Agents : Anthropic, December 2025
• Anthropic updates Claude Managed Agents with three new features : 9to5Mac, May 7 2026
• Lovable security crisis: 48 days of exposed projects : The Next Web, April 21 2026
• RAG in 2026: Bridging Knowledge and Generative AI : Squirro, February 2026
• Vector DB Costs 2026: Pinecone vs Weaviate vs Qdrant : LeanOps Tech, March 2026
• LLM Cost Optimization: 5 Levers to Cut API Spend 70 to 85% : Morph, March 2026
• LangSmith alternatives 2026: Best tools for LLM tracing, evals, and prompt iteration : Braintrust, March 2026
• RAG in 2026: How Retrieval-Augmented Generation Works for Enterprise AI : Techment, February 2026
• The Modern AI Stack: 13 Layers from LLM to Production : Ultrathink Solutions, April 2026
• Retrieval-augmented generation, LangChain documentation : LangChain, 2026
• Pinecone Learn, retrieval and vector database guides : Pinecone, 2026
• Hugging Face blog : Hugging Face, 2026
• Lovable AI app builder : Lovable, 2026