AI agent improvement every month, based on real conversations
An AI agent needs training after launch too. Every month we review real calls and chats, add flows, and update prompts and the knowledge base. Every change is checked against an eval suite. From $1,000 per month.
What hurts
At launch the agent passed every test. A month later customers ask about a new plan, a promotion or a delivery option that was never in the script. The agent makes up an answer or hands everything to a human.
Someone on the team edited the prompt directly to fix one complaint. Two other conversation types broke, and nobody noticed. There are no versions and nothing to roll back to.
AI sometimes makes mistakes, and that is expected. The real problem is when nobody collects those mistakes and turns them into tests. Then the same error repeats in every similar conversation.
What we build
- Weekly review of a sample of real calls and chats
- Error classification: facts, tone, flow, tool calls
- New flows and tools as your business changes
- Knowledge base updates: prices, promotions, delivery terms, schedules
- A test case for every error found, so it does not come back
- Prompt versioning with diffs and rollback to any earlier version
- Full regression run of the eval suite before each agent release
- Monthly report with changes and quality trends
What you get
- An eval suite that grows every month and stays yours
- Prompt version history with a note on every change
- An updated agent knowledge base
- Report on errors, fixes and test run results
- Flow plan for the next month
How we measure the result
- Share of conversations resolved by the agent without a human
- Repeat errors after a fix has shipped
- Share of test cases passing before release
- Conversation scores against a quality checklist
Systems and stack
Industries
Questions
How much does agent improvement cost?
From $1,000 per month for one agent. The price depends on the number of agents and channels, the volume of conversations to review, and how often your business changes. An agent with frequent promotions and new products needs more work than an appointment booking agent.
Do you fine-tune the model itself?
Usually not, and it is rarely needed. Most gains come from prompts, the knowledge base, tools and evals. Fine-tuning makes sense only with a large labelled dataset. If you need it, we estimate it separately.
How do you know a change did not break anything?
Every change goes into a draft of the agent. We run the full eval suite on it, including cases that came from past errors. Only a green run goes to production. Any version can be rolled back.
Does this work with agents not built on Happ?
Yes. We work with agents on any platform where we can access the prompt, conversation logs and a way to run tests. If there are no tests yet, the first month goes into a baseline suite. On the Happ platform, evals and versions are built in.
Who sees customer conversations?
Only engineers assigned to the project, under NDA and a GDPR compliant DPA. For call analysis we can use our own vLLM on our servers, so data does not go to third-party models. Personal data is masked in reports.
Related services
More in this section
Describe the task. We reply within one business day, the first call is with the founder.
Describe the task. The first call is with the founder.
We reply within one business day. 30 minutes to tell you whether it is worth doing and what it costs.