Salesforce announced Agent Evals as part of updates to Agent Fabric, its system for managing and governing AI agents across platforms. The feature is designed to test agents before launch and check their performance in use, with possible interventions when results fall short. Salesforce says Agent Evals can automatically generate tests before deployment and evaluate agents in production against defined quality thresholds.
Salesforce lists Agent Evals as beta in November 2026 and generally available in January 2027. The company says below-threshold results can trigger predefined interventions, including the Agent Kill Switch, with approval where policy requires it.
For an appointment-based business, the useful idea is not to copy an enterprise platform. It is to keep checking a rebooking or win-back assistant after it goes live. An AI agent, for this use, is software that can respond to a client and take steps such as finding booking information or offering a next appointment. A reply that sounds polished is not enough if the assistant uses the wrong availability or breaks your contact rules.
Check the rebooking work, not just the wording
Start by writing down the tasks the assistant is allowed to handle. It might follow up after a first visit, remind a client that it is time to rebook or reach out to a regular who has not booked recently. Then consider what a correct response looks like in each situation.
For example, test whether the assistant uses the right booking details, offers only services and times you have approved and follows your rules about when and how to contact someone. Include cases where it should stop and send the conversation to staff, such as a request the assistant cannot answer or a booking detail it cannot verify. These are practical checks for your workflow, not a claim that any particular test will prevent every error.
Keep the information available to the assistant limited to what it needs for the booking task. For med spas, IV and wellness clinics, check whether privacy rules such as HIPAA apply before an assistant handles client information. Do not send client health or treatment details to ad platforms or general marketing tools.
Measure agent quality and retention separately
A useful evaluation has two distinct questions. First: did the assistant follow its rules, use the right booking information and hand off unusual cases appropriately? Second: did the follow-up work help more clients return?
The first question is about agent quality. The second is about retention. Passing the quality checks does not show that the assistant caused more repeat visits. Clients may rebook for many reasons, so compare the retention measures you already use—such as repeat visits, rebooking after a first appointment or returning after a lapse—rather than treating a correct response as proof of business impact.
If you want to assess whether the assistant is contributing to a change in rebooking, use attribution experiments—a controlled marketing test—to measure that work. Keep the reporting focused on booking and retention outcomes, not client health or treatment details.
Watch for changes after launch
A launch check is only a starting point. Salesforce notes that changes to the models, APIs, costs and systems an agent depends on can lead to problems that surface silently. That is why the company describes evaluation as an ongoing activity.
For your business, revisit the same kinds of checks when you change the assistant’s instructions, booking tools or approved information. Also review examples of conversations and staff handoffs on a regular schedule that suits your team. If you see a pattern of errors, pause the affected task or route more conversations to staff while you investigate. The aim is to catch a problem before it becomes a repeated part of the client experience.
Cost is another operating measure. Salesforce’s announcement describes hard limits that stop requests before spending exceeds a budget. That is a useful reminder to set a budget for an assistant and keep an eye on whether its costs are changing. For a small team, a simple monthly review of usage, cost and staff escalations may be enough to decide whether the assistant still fits its purpose.
What to do
- Define which rebooking and win-back tasks the assistant may handle.
- Write checks for contact rules, booking details and staff handoffs.
- Review the assistant again after changes to its instructions or connected tools.
- Track agent quality separately from repeat visits and rebooking outcomes.
- Check costs and privacy requirements before expanding its role.
The practical lesson from Agent Evals is to treat launch as the start of oversight, not the end of testing. A reliable assistant can support retention work, but only separate measurement can tell you whether that work is bringing clients back.
- ai evaluation
- client retention
- rebooking
- marketing measurement
- appointment businesses
Alan Jordan, founder and principal solutions architect. Data engineering, data science and business analytics for appointment-based businesses.
