The First 30 Days With Edah AI: An Operator Playbook

Day 1 to 7: observation mode

You went live. Calls are coming in or going out. The temptation is to start tweaking the scenario immediately. Resist it for a week. The first week is for data, not changes. Listen to recordings. Read transcripts. Note where the agent hesitated, what callers asked that you did not anticipate, which intents the scenario missed.

Day 8 to 14: targeted tuning

By the end of week one, you have a list of the top friction points. Now you tune. Add the missing intents. Update the knowledge base with the questions the agent could not answer. Adjust the conversation flow where it felt unnatural in playback.

Tune the scenario, not the model

You do not change how the agent reasons. You change what it knows and what it is allowed to do. New intents, new tools, updated knowledge. The model handles the natural variation, the scenario handles the deterministic structure.

Watch the tool call success rate

Tools either succeed or fail. A high tool call failure rate usually means the CRM mapping is off, the knowledge base is stale, or the agent is being asked to do something the integration does not support. Each failure is a fixable signal, not a model problem.

Day 15 to 30: measure conversion

With the scenario tuned, look at what callers actually do. Did the booking conversation result in a booking? Did the lead capture conversation produce a lead with the right fields populated? Did the follow-up call result in the next step? Conversion is the only metric that matters at this stage.

A/B test small changes

A different greeting. A different qualifying question. A different escalation threshold. Run each change against a subset of calls and measure conversion. Keep what works, revert what does not. Small wins compound.

After day 30

The scenario is stable. The knowledge base is current. Conversion is trending where you want it. The agent runs without daily supervision. You check the dashboard weekly and tune when something drifts. The system that was a project becomes infrastructure.