AI Cost Estimation

A colleague asked how you’d know what an AI system costs to run, as if it were a black box. It isn’t. It’s inputs, outputs, and simple math.

The idea

Step one: pick the model. Costs vary enormously. At the time of filming, OpenAI’s flagship model cost about 84 times more than its mini model. A well-designed system uses the least expensive model that does the job well, which you find through testing.

Step two: read the API pricing. Two numbers matter: input tokens (what you send) and output tokens (what comes back), usually priced per million. Ignore cached-input pricing for simple estimates. Step three: count tokens. Write a prompt that mimics your system, run it on real sample data, then multiply word count by about 1.3, or use a tokenizer for exact counts.

Why it matters

A worked example. Extracting data from customer emails: 191 input tokens and 81 output tokens per call, times 10,000 emails a month, came to roughly $15 a month.
Don’t anchor on $15. Bigger inputs, outputs, and volumes get expensive fast, which is why early estimation matters.
Count the non-AI costs too. Vector databases and cloud hosting follow normal software pricing your team already knows how to estimate.

Not a black box

Price per token times tokens per call times calls per month. That’s most of it. My download in the video includes the spreadsheet and details.

Use it to plan your next AI system before the bill surprises you.

Join the conversation: see the comments on LinkedIn.

Read the full transcript

Hey, I’m Matt. A colleague asked me a great question the other day. She said, how do I know what an AI system would cost to run? She asked like it was some sort of mysterious black box and it was impossible to estimate. But the reality is that it’s pretty straightforward to estimate. So let’s walk through it and check out my download in the comments section. It has all the details from the video and more.

First step, decide which large language model you plan to use. Different models have very different costs. The API price for commercial models are available online. So for example, you’re looking for prices on OpenAI, just search OpenAI API cost. Nowadays, LLM usage cost is measured in tokens in and tokens out.

For now, think of a token as a word, but we’ll refine that in a few minutes. Here’s an important point. A well-designed AI system does not always utilize the top flagship model from a provider. From a cost perspective, there can be a huge difference in the flagship model and the mini model. As of this filming, for example, OpenAI’s flagship model is about 84 times more expensive than their mini model.

So you’ll want to use the least expensive model that does the job well, and you find that through experimentation and testing. Okay, second step is to look at the API pricing. Two numbers matter. Input tokens, which is what you send into the model, and output tokens, which is what comes back from the model. You might also see cached input. This is a cheaper price for repeated inputs the system can reuse, but because you cannot fully control when caching is used, I suggest you ignore it for simple estimations.

Most models are priced per thousand or per million tokens of input and output. Once you know how it’s priced, the math is pretty straightforward. You multiply the input tokens by the input price and you multiply the output tokens by the output price. You add them together and that’s your cost for one call and then you can extrapolate to figure your estimated monthly cost. Now you might be wondering, how do I calculate the number of tokens? For estimation, that’s actually pretty simple.

Let’s start by writing a typical prompt that mimics how your system will prompt the LLM. In my example, I’m extracting specific information from a customer email. So I’ve created my mimicking prompt and I can execute it interactively in ChatGPT against my target GPT model and see my output. I can experiment with different data, which in my case is different customer emails, and I can see how different data affects my output. Okay, but for step three, we want to use our mimicking prompt and the accompanying output to calculate the number of input and output tokens.

There are two approaches to calculate the tokens. A quick and dirty way is to estimate the number of words in your input and your output and multiply by 1.3 to get a rough token estimation. Or, more accurately, you can use a tokenizer tool to get the exact token count from your words. In my case, using the GPT 5.2 model and a tokenizer tool, I see my mimicking prompt results in 191 input tokens and my output as 81 output tokens. I do my calculations in the spreadsheet by plugging in the blue values, which are the input and output tokens and my anticipated number of calls per month. And that calculates the estimated LLM cost per individual LLM call and monthly for my system. The sheet simply uses model price per million tokens multiplied by tokens per call and then by my estimated 10,000 calls a month, which is something I estimated thinking in my example, there would be approximately 10,000 customer emails per month that I’d need to process.

And this calculates to about $3 for input and $11 for output, or a total of about $15 a month. A few notes on estimating. This is a very simple example with small input and output, and the calls per month are not very high. So don’t take away from this video that calling LLMs is going to only be $15 a month. It can get expensive, and that’s why early estimation is so critical. And there can also be additional costs outside of large language models.

For example, hosting a vector database with your own data or running a web app along with your AI feature in the cloud. But those are costs that follow normal software pricing and your team can easily estimate them. So AI cost is not a black box. It is just basic inputs, outputs, and simple math. I hope this helps you plan for your next AI system and gets your creative AI juices flowing.

Thanks for watching.