What our biggest AI mistake taught me about customer expectations

At a digital-first agency, there is always pressure to use technology to do more, especially with AI. When a tool can answer questions, route calls, and speed up workflows, it’s easy to start thinking about how much of the customer experience you can hand off to automation.

I’ve learned to be careful with that instinct, because the client experience still rises or falls on human connection.

AI can do mundane, repetitive work extremely well, and I believe it should. But when customers want to talk to someone, they want an actual person. They want someone who can offer empathy, clarity, and connection. They want to feel heard.

My team and I at RightSure were reminded of that in a very direct way when we made AI too visible and put it between our customers and our “Famously Friendly Humans.”

When we put AI in the wrong place

Like many agencies, my team was looking for bottlenecks we could solve with our AI tools. Where were we creating friction for customers? Where were we tying up team members with repetitive work? Where could we improve speed without lowering the quality of the experience?

On paper, putting AI at the front of our phone system during business hours looked like a smart move. We thought it could route calls faster, answer simple requests immediately, and free up our team for the conversations that needed more judgment and care.

I let it run for three weeks to collect data to see if the trade-off was worth it.

Our customers made their opinion very clear. They hated it.

The people calling us during business hours expected to reach a person. They did not want a virtual assistant between them and our team. In our agency, that disconnect hit even harder because we’ve spent years building our brand around our “Famously Friendly Humans.” When customers called during work hours and got a machine instead of a person, they were not happy.

What was interesting is that the reaction changed once we used that same kind of AI in the right context. When we were closed and told callers clearly that our Famously Friendly Humans were out of the office and this was a virtual assistant, people were fine with it. They were also fine with AI working behind the scenes. They just did not want it standing between them and a person without explanation.

Where AI has worked

The phone system experiment was just one of many ways we’ve implemented AI in our agency. When AI removes friction without getting in the way of the relationship, it can create real value for both the agency and the customer. We’ve seen that most clearly in quote intake and after-hours service.

Faster quote intake

We used AI to cut intake time from 12 and a half minutes to two minutes and nine seconds. We also processed more than 18,000 quotes from speech to text, which means less rekeying and faster handoffs to our agents.

Customers may not know exactly what’s happening behind the scenes, but they know when the process feels slow. They know when they have to repeat themselves. And they know when getting a quote feels harder than it should. AI helps us smooth that out.

Better after-hours service

Some of our best results came after hours. Customers don’t care when you’re open. They care when they need you.

That’s exactly why the AI phone system worked after hours even though it failed during the workday. The audience was different, and so was the expectation. If someone called us at night, they understood a person might not be available. What they wanted was quick guidance and a clear next step.

Through automation, more than 3,000 policyholders got after-hours help with payments and reinstatements, and more than 2,000 got immediate support for small claims issues like towing and windshield damage. When someone is stranded or dealing with a cracked windshield, they do not want to wait until the next morning for the next step.

One of the more surprising numbers for me was that 12% of our after-hours calls were new business calls. We processed more than 17,000 new business quotes. That says a lot. People shop for insurance when it works for them, not when it works for us. Without a system in place to capture those calls and keep the process moving, we likely would have missed business that came in after hours.

That is exactly where I want AI working.

The lessons from our AI successes and failures

Agencies can get into trouble when they adopt AI just to say they’re using AI, or when they expect it to solve every problem. AI works best when you start with customer intent, not with the tool itself.

Start with what the customer needs

Ask what the customer is trying to do right now. Are they trying to make a payment? Completing a simple transaction? Calling after hours and hoping for immediate guidance? AI can be incredibly effective in those moments.

Then ask about the emotional temperature of the interaction. Is this person confused? Stressed? Looking for reassurance before deciding? Those are usually the moments when a human being matters most.

Put leadership in front of AI

I also think agency leaders have to be very clear with their teams when they introduce AI. If you announce a new AI initiative without context, some hear innovation while others hear job loss.

You can’t build a durable strategy that way.

Even if you have the best AI and the best tech stack, if your team isn’t on board, it’s a failure before it’s even out of the gate. If the new tech runs at odds with your culture and customers, it will not work. That’s why I say culture eats technology.

Get the team involved early. Ask where the pain points are and what’s slowing them down. Ask what customers keep struggling with. Then build from there. When people understand the purpose, adoption gets easier and your culture stays intact.

Keep the roles clear

I’m optimistic about what AI can do for agencies because I’ve seen it reduce quote times, support after-hours service, improve workflows, and create new business opportunities. I’ve also seen what happens when technology steps into a moment where the customer wanted a person.

That distinction guides how we move forward at RightSure. We use AI for the work machines do well. We rely on our Famously Friendly Humans for conversations that require trust, empathy, and connection.

When those roles are clear, AI becomes a powerful tool. It helps us serve people faster, stay available longer, and protect the part of the experience that matters most.