2025 playbook for enterprise AI success, from agents to evals

2025 is poised to be a pivotal year for enterprise AI. The past year has seen rapid innovation, and this year will see the same. This has made it more critical than ever to revisit your AI strategy to stay competitive and create value for your customers. From scaling AI agents to optimizing costs, here are the five critical areas enterprises should prioritize for their AI strategy this year.

1. Agents: the next generation of automation

AI agents are no longer theoretical. In 2025, they’re indispensable tools for enterprises looking to streamline operations and enhance customer interactions. Unlike traditional software, agents powered by large language models (LLMs) can make nuanced decisions, navigate complex multi-step tasks, and integrate seamlessly with tools and APIs.

At the start of 2024, agents were not ready for prime time, making frustrating mistakes like hallucinating URLs. They started getting better as frontier large language models themselves improved.

“Let me put it this way,” said Sam Witteveen, cofounder of Red Dragon, a company that develops agents for companies, and that recently reviewed the 48 agents it built last year. “Interestingly, the ones that we built at the start of the year, a lot of those worked way better at the end of the year just because the models got better.” Witteveen shared this in the video podcast we filmed to discuss these five big trends in detail.

Models are getting better and hallucinating less, and they’re also being trained to do agentic tasks. Another feature that the model providers are researching is a way to use the LLM as a judge, and as models get cheaper (something we’ll cover below), companies can use three or more models to pick out the best output to make a decision on. 

Another part of the secret sauce? Retrieval-augmented generation (RAG), which allows agents to store and reuse knowledge efficiently, is getting better. Imagine a travel agent bot that not only plans trips but books flights and hotels in real time based on updated preferences and budgets.

Takeaway: Businesses need to identify use cases where agents can provide high ROI — be it in customer service, sales, or internal workflows. Tool use and advanced reasoning capabilities will define the winners in this space.

2. Evals: the foundation of reliable AI

Evaluations, or “evals,” are the backbone of any robust AI deployment. This is the process of choosing which LLM — among the hundreds now available — to use for your task. This is important for accuracy, but also for aligning AI outputs with enterprise goals. A good eval ensures that a chatbot understands tone, a recommendation system provides relevant options, and a predictive model avoids costly errors.

For example, a company’s eval for a customer-support chatbot might include metrics for average resolution time, accuracy of responses, and customer satisfaction scores.

A lot of companies have been investing a lot of time into processing inputs and outputs so that they conform to a company’s expectations and workflows, but this can take a lot of time and resources. As models themselves get better, many companies are saving effort by relying more on the models themselves to do the work, so picking the right one gets more important.

And this process is forcing clear communication and better decisions. When you “get a lot more conscious of how to evaluate the output of something and what it is that you actually want, not only does that make you better with LLMs and AI, it actually makes you better with humans,” said Witteveen.  “When you can clearly articulate to a human: This is what I want, here’s how I want it to look like, here’s what I’m going to expect in it. When you get really specific about that, humans suddenly perform a lot better.” 

Witteveen noted that company managers and other developers are telling him: “Oh, you know, I’ve gotten much better at giving directions to my team just from getting good at prompt engineering or just getting good at, you know, looking at writing the right evals for models.”

By writing clear evals, businesses force themselves to clarify objectives — a win for both humans and machines.

Takeaway: Crafting high-quality evals is essential. Start with clear benchmarks: response accuracy, resolution time, and alignment with business objectives. This ensures that your AI not only performs but aligns with your brand’s values.

3. Cost efficiency: scaling AI without breaking the bank

AI is getting cheaper, but strategic deployment remains key. Improvements at every level of the LLM chain are bringing dramatic cost reductions. Intense competition among LLM providers, and from open-source rivals, is leading to regular price cuts.

Meanwhile, post-training software techniques are making LLMs more efficient.

Competition from new hardware vendors such as Groq’s LPUs, and improvements by the legacy GPU provider Nvidia, are dramatically reducing inference costs, making AI accessible for more use cases.

The real breakthroughs come from optimizing the way models are put to work in applications, which is the time of inference, rather than the time of training, when models are first built using data. Other techniques like model distillation, along with hardware innovations, mean companies can achieve more with less. It’s no longer about whether you can afford AI — you can do most projects much less expensively this year than even six months ago — but how you scale it.

Takeaway: Conduct a cost-efficiency analysis for your AI projects. Compare hardware options and explore techniques like model distillation to cut costs without compromising performance.

4. Memory personalization: tailoring AI to your users

Personalization is no longer optional — it’s expected. In 2025, memory-enabled AI systems are making this a reality. By remembering user preferences and past interactions, AI can deliver more tailored and effective experiences.

Memory personalization isn’t widely or openly discussed because users often feel uneasy about AI applications storing personal information to enhance service. There are privacy concerns, and the ick factor when a model spits out answers that show it knows a great deal about you — for example, how many kids you have, what you do for a living, and what your personal tastes are. OpenAI, for one, safeguards information about ChatGPT users in its system memory — which can be turned off and deleted, though it is on by default.

While businesses using OpenAI and other models that are doing this can not get the same information, what they can do is create their own memory systems using RAG, ensuring data is both secure and impactful. However, enterprises must tread carefully, balancing personalization with privacy.

Takeaway: Develop a clear strategy for memory personalization. Opt-in systems and transparent policies can build trust while delivering value.

5. Inference and test-time compute: the new efficiency and reasoning frontier

Inference is where AI meets the real world. In 2025, the focus is on making this process faster, cheaper and more powerful. Chain-of-thought reasoning — where models break down tasks into logical steps — is revolutionizing how enterprises approach complex problems. Tasks requiring deeper reasoning, like strategy planning, can now be tackled effectively by AI.

For instance, OpenAI’s o3-mini model is expected to be released later this month, followed by the full o3 model at a later date. They introduce advanced reasoning capabilities that decompose complex problems into manageable chunks, thereby reducing AI hallucinations and improving decision-making accuracy. These reasoning improvements work in areas like math, coding, and science applications where increased thought can help — though in other areas, like synthesizing language, advancements may be limited. 

However, these improvements will also come with increased computational demands, and so higher operational costs. The o3-mini is meant to provide a compromise offering to contain costs while keeping performance high.

Takeaway: Identify workflows that can benefit from advanced inference techniques. Implementing your own company’s special chain-of-thought reasoning steps, and selecting optimized models, can give you an edge here.

Conclusion: Turning insights into action

AI in 2025 isn’t just about adopting new tools; it’s about making strategic choices. Whether it’s deploying agents, refining evals, or scaling cost-efficiently, the path to success lies in thoughtful implementation. Enterprises should embrace these trends with a clear, focused strategy.

For more detail on these trends, check out the full video podcast between Sam Witteveen and myself here: