ENTERPRISE AI ECONOMICS

Beyond token pricing: the real cost drivers of enterprise AI

The model may generate the answer, but the surrounding system determines what enterprise AI costs—and whether it creates value.

By Vijay Rengan, Founder & Chief AI Officer, EMLI

September 20, 2026

When leaders estimate the cost of an enterprise AI application, they often begin with the model’s API price. If a provider charges according to the number of tokens processed, the calculation appears straightforward: estimate the number of questions, multiply by the average input and output length, and arrive at a monthly cost.

That calculation is useful, but incomplete. A production AI system is rarely just a user interface wrapped around a single model call. The model may perform the most visible part of the work, but the surrounding data, software, security, and operational systems determine whether its answer is accurate, authorized, and useful.

The better question is not simply, “How much does the model cost?” It is: what resources are required to turn a model into a dependable business system?

These costs are not hidden from technical teams. They are better understood as cost drivers that are often overlooked when an organization compares AI initiatives using token prices alone.

The model is only one component

Consider an internal human-resources assistant. An employee asks how many vacation days California employees can carry into the next year. From the employee’s perspective, the application receives a question and generates an answer. Behind the interface, the system may need to verify the employee’s identity, enforce permissions, search the current policy library, select relevant passages, prepare the model’s instructions, validate the answer, preserve citations, and create an audit record.

The model charge covers its processing. Depending on the provider and model, that may include the employee’s question, system instructions, retrieved policy passages, conversation history, generated answer, and potentially reasoning tokens. It does not automatically cover the cost of maintaining the policies, enforcing access rights, operating the application, monitoring its performance, or resolving an incorrect answer.

In a demonstration, most of the visible functionality may come from one model call. In a reliable enterprise product, that call sits inside a larger system.

Data readiness is an operating cost

An AI assistant cannot make effective use of company information merely because that information exists. Corporate data is commonly distributed across databases, shared drives, content-management systems, collaboration platforms, email archives, and business applications. Documents may be duplicated, outdated, poorly labeled, or stored in incompatible formats. Different employees may be entitled to see different parts of the same repository.

Before this information can support an AI application, companies may need to connect multiple sources, extract and clean their contents, identify obsolete versions, classify sensitive information, divide documents into searchable passages, attach metadata and permissions, and preserve citations and provenance.

This work continues after implementation. New files are created, policies change, employees move between roles, and access rights are revised. Unless the AI system follows those changes, it can produce outdated answers or expose information to the wrong users. Data preparation is therefore both an implementation cost and an ongoing operating cost.

Embeddings make information searchable; they do not perform the search

Embeddings are sometimes discussed as if they allow an AI model to absorb an entire corporate repository. They do not. An embedding converts a passage of text into a numerical representation of its meaning. Those vectors can be stored in an index and compared with the vector representation of a new question.

Suppose a company has 10 gigabytes of policy documents. During ingestion, the system parses and cleans the documents, divides them into smaller passages, creates an embedding for each passage, and stores the vectors together with the original text and metadata. This process makes the collection searchable. It does not determine which passages will be relevant to a question that has not yet been asked.

When an employee submits a question, the system must still embed the query, search the stored vectors, apply permission and metadata filters, rank the results, and send the most relevant passages to the model. The application may reduce millions of stored passages to five or ten useful ones.

Sending the entire repository would be impractical. It could exceed the model’s context limit, increase latency and token consumption, introduce irrelevant information, and violate access restrictions. Retrieval therefore creates its own cost drivers: embedding generation, vector storage, database capacity, search operations, reranking, and the engineering required to keep the system current.

One user request can create many billable operations

The number of interactions visible to users does not necessarily equal the number of operations performed behind the scenes. A single request could cause an application to classify the user’s intent, rewrite the question for retrieval, create a query embedding, search internal documents, rerank the results, generate an answer, check whether the answer is supported, apply a safety review, and summarize the interaction for storage.

Some of these steps can be handled by conventional software. Others may use a language model or a specialized model. Agent-based applications can make still more calls as they select tools, inspect results, and decide what to do next.

The useful unit of analysis is therefore not merely the number of user questions. It is the complete sequence of model calls, searches, and tool operations triggered by each question.

Hosted AI services add costs beyond model tokens

A company does not have to move its entire application to an AI provider in order to use the provider’s model. In a common hybrid architecture, the company continues to run the user experience, authentication, source data, permission systems, business logic, and audit records. Its software selects the information needed for a request and sends that limited context to the model API.

That arrangement can make the model charge look like the principal external cost. In practice, the application may also use separately priced services to prepare, find, or process the information the model needs. Embedding generation can carry a token charge. A hosted vector store can carry a recurring storage charge. File search or web search can carry a per-call charge. Code execution can require a paid container. The model then charges again to read the retrieved material and generate the answer.

OpenAI, for example, offers embeddings and hosted tools for file search, web search, and code execution. Its File Search service can search uploaded knowledge bases using semantic and keyword search, reducing the retrieval infrastructure a company must implement itself. The operational work becomes easier, but it does not disappear; part of it moves onto the provider’s invoice as storage, tool-call, and model-token charges.

As of September 20, 2026, OpenAI lists File Search at $0.10 per gigabyte of storage per day after the first free gigabyte and $2.50 per 1,000 tool calls through the Responses API. Its text-embedding-3-small model is listed at $0.02 per million input tokens. Model tokens and other paid tools are additional. A company budgeting only for answer-generation tokens could therefore omit several recurring cost categories. Prices and service terms can change, so current vendor documentation should always be used for budgeting.

What the provider charges can look like

The unit prices become more understandable when they are applied to a workload. Using OpenAI’s published rates as of the article date, the following examples isolate the provider charge for each capability. They do not include application engineering, data preparation, security, networking, or model tokens unless specifically stated.

The first example is especially important. Creating embeddings for even a large text collection can be inexpensive. The larger cost can emerge from storing and searching the index repeatedly, preparing and governing the source data, integrating retrieval with permissions, and operating the workflow over time. A low embedding price should not be interpreted as a low total cost for enterprise search.

The architectural choice is not simply between owning AI hardware and using every service from one provider. A company without specialized hardware can use an embedding API, store the resulting vectors in a managed database, and send only selected passages to a model. Alternatively, it can use a hosted retrieval service that manages more of the workflow. The first option shifts cost toward external infrastructure and internal engineering; the second shifts more cost onto the AI provider’s recurring service charges.

Hosted convenience and internal control have different cost profiles

With an embedding API and a managed vector database, the company pays for embedding generation, database capacity, search, integration, monitoring, backup, and operational support. This approach provides greater control over ranking, metadata, permissions, and portability between model providers.

With hosted file search, the company can shift much of the parsing, chunking, indexing, vector storage, and retrieval operation to the AI provider. The visible recurring charges may be higher, but the company writes and maintains less infrastructure.

Neither option is universally cheaper. At low usage volumes, a hosted service may save far more in employee time than it adds in platform charges. At high volumes, a company-managed retrieval system may lower recurring costs, particularly when the organization already operates suitable data infrastructure.

A valid comparison must include the full cost of both choices:

A comparison that excludes engineering and operations will make the self-managed option look artificially inexpensive. A comparison that ignores usage growth, customization, and vendor dependence may favor the hosted option too readily.

The largest cost drivers may sit outside the AI invoice

The AI provider’s invoice is measurable and therefore attracts attention. Several other cost drivers may be larger.

Integration. The application must connect with identity systems, databases, document repositories, and established business processes. Older systems may lack modern interfaces, making integration slow and expensive.

Security and access control. Retrieval must respect the permissions of the person making the request. An otherwise accurate assistant becomes a security failure if it reveals confidential compensation, legal, customer, or strategy information to an unauthorized employee.

Reliability engineering. Production systems need rate-limit management, retries, timeouts, caching, fallbacks, and incident monitoring. The organization must decide what happens when a model or external tool is unavailable.

Evaluation. AI systems produce variable responses, so companies need representative test sets and criteria for accuracy, relevance, grounding, safety, and consistency. Evaluations must be repeated when models, prompts, data, or workflows change.

Governance and compliance. Organizations may require audit logs, retention controls, data classifications, risk reviews, and evidence that automated decisions comply with internal policies and external requirements.

Human oversight. Consequential activities may require employee approval. Reviewing uncertain answers, investigating failures, and correcting source data all carry labor costs.

Adoption and process redesign. A technically successful system can still fail if employees do not trust it or cannot incorporate it into their work. Training and change management belong in the investment calculation.

Error costs. An inaccurate answer about a low-risk topic may be inconvenient. An inaccurate answer involving a contract, financial transaction, regulated process, or customer commitment can create material liability. The appropriate assurance effort depends on the consequence of failure.

What the organizational costs can look like

Provider prices are published; internal costs depend on the organization, its systems, and the consequence of failure. A transparent estimate can still make those costs visible. The following examples use assumed fully loaded labor rates solely to illustrate the calculation. They are not market benchmarks or EMLI project estimates.

These examples do not predict what a particular implementation will cost. They show why the provider invoice may be one of the smaller parts of an early enterprise deployment. Every organization should replace the assumed hours, rates, volumes, and review requirements with its own data and calculate the cost per successful business outcome.

Architecture determines the cost curve

AI economics change with scale. A pilot with a few hundred users may favor hosted services because they reduce implementation time and allow the company to learn quickly. The same architecture may become expensive when millions of searches or long agent workflows run every month.

Conversely, building a customized retrieval platform at the beginning may consume more money than the business problem justifies. An organization can spend heavily optimizing infrastructure before it knows whether employees will use the product.

Hosted capabilities are often attractive when the project is experimental, usage is modest or uncertain, the engineering team is small, and rapid deployment matters. A company-managed retrieval layer becomes more attractive when usage is large and predictable, the organization already operates suitable infrastructure, permissions or ranking require deep customization, or several model providers must share the same data layer.

Fully self-hosting a model offers still more control, but it does not remove the cost of AI. It replaces API charges with computing capacity, model-serving infrastructure, security, scaling, optimization, and specialist labor.

Estimate systems, not tokens

Token pricing remains important. At high volumes, inference can become a major operating expense, and inefficient prompts or unnecessarily large models can materially affect margins. But token optimization alone cannot make an unreliable system valuable.

The vacation-policy assistant succeeds only if it locates the current policy, applies the employee’s permissions, gives an answer supported by the source, and records enough information for the organization to investigate a problem. Each capability has a cost, whether it is purchased as a hosted service or built internally.

An effective AI business case should therefore estimate the complete lifecycle: data preparation, integration, model and tool usage, evaluation, governance, operations, adoption, and eventual migration—not merely the price of generating an answer.

OpenAI pricing checked for the September 20, 2026 publication date. Sources: OpenAI API pricing, File Search documentation, and text-embedding-3-small documentation. Verify current rates and service-specific terms before making purchasing or budgeting decisions.