The AI Chatbot Developer Scorecard: 24 Checks Before You Hire

A 24-point scorecard for hiring an AI chatbot developer: channels, booking tools, dates, limits, monitoring and ownership, with a score that says hire or pass.

By Carlos Familia, CRM & AI automation expert. Updated October 16, 2026. 7 min.

Seven practitioners book in parallel in one New Jersey facial spa's Square account. The chat agent I built there had to find a real open slot among them, book it, and say "booked" only if Square agreed. The test I trust most on that build is the dullest one: book a real slot, find it in Square, cancel it.

Before the agent passed that test, it had matched one request through a name-synonym map that ran backwards and landed on a $350 service. It had also booked a real client on a Sunday, because one practitioner's schedule in Square still showed Sundays open. Neither problem showed in a chat transcript. Both showed up in the calendar.

Use the 24 checks below as a scorecard. A developer earns a point only for a check they can show you on a real build, with a log, a screenshot or a live run. At 21 or more, hire. Under 15, keep looking. The groups follow the places where chat agents break on web chat, SMS, Instagram, Facebook and WhatsApp.

Channels: does it follow each platform's rules?

• It answers on every channel you use. I once filtered the spa's agent to social conversations only, and leads who wrote by SMS got nothing.
• The developer can state the Instagram and Facebook reply window. Meta's messaging policy (updated September 28, 2026) says the window lasts 24 hours for most user actions, and up to 7 days when the person's first message comes from a Click-to-Messenger or Click-to-Direct ad. On Messenger only, a Human Agent tag lets a person reply manually within a separate 7-day period. Ask what happens to a follow-up on day 3.
• The developer can state the WhatsApp rules. Free-form replies can only go out inside an open 24-hour customer service window, and outside it you send approved templates. Since October 1, 2026, those in-window replies cost money too: Meta's pricing page (updated September 30, 2026) gives each business phone number 1,000 service messages a month at no charge and bills the rest. Ask for an estimate of your bot's WhatsApp bill and where it will show in WhatsApp Manager billing. On a credit repair build, I learned that templates made in one business account do not carry over to another, so ask who creates yours.
• One test message goes out on each channel before launch. Not one test on the easiest channel.

Booking tools: does the calendar decide?

• You can see a real tool result inside one run. The spa's first router was wired up from raw API calls, and every tool result reached the model blank. With nothing to read, it invented times, prices and bookings. Rebuilding it as a clone of a workflow that already worked cured it.
• "Booked" is said only after the booking system confirms. Every test booking on the spa build was verified in Square, then cancelled.
• The agent reads the system that holds the real booking. At a med spa that books in Boulevard, the bot checked the GoHighLevel calendar instead and could double-book, until a backfill made its free slots exclude Boulevard bookings.
• The developer knows the platform's limits. Square's booking list rejects date windows longer than 31 days, so the tool has to search in shorter spans.

Dates, names and policy: who does the thinking?

• Code resolves dates, not the model. A request for "tomorrow (Saturday)" once landed on the same day. The tool had no date parameter, and the prompt's clock was the server's, which runs on UTC.
• Services and staff come from the live catalog. Names are matched fuzzily against the booking system, the slot gets a hard check, and phone numbers are normalized to one format (E.164) before any lookup.
• Lookups use what the client typed. The support agent once invented an email address to search for a client. A lookup on a made-up value finds the wrong person or nobody.
• Business policy lives in the tool. The Sunday booking was fixed with a Sunday filter inside the booking tool. A bot obeys the calendar, not your policy, unless the policy is enforced where the offers are built.

Limits and handoff: what happens when it should stop?

• There is a written list of what it must never say. On the med spa agents, the rules include a wall around medical questions. For a roofing contractor in Kentucky, the warranty knowledge base came from the manufacturer's guide and states exactly what the bot may and may not claim.
• A person gets the conversation, and a fallback reply exists. One mid-booking message once went to an agent with no booking tool. It hit its maximum iterations and went silent. Sticky routing, a booking tool for every agent and a fallback reply fixed it.
• Limits that must hold are enforced in code. One bot kept pushing for an appointment time in every message, whatever the prompt said. A counter in code is what finally capped it.
• Test mode cannot reach a real lead. On the roofing build, a test-mode string leaked into a message to a lead. Ask how test traffic is fenced off.

Watching it after launch: who notices a failure?

• The double-message guard was tested with a case it must block. My first guard on the spa bot compared nothing, because a field was null.
• Follow-ups have caps and exits. The spa's re-engagement stops at 4 touches per lead and respects Meta's 24-hour window. On the roofing build, the nurture kept emailing clients who had already booked until I added an exit.
• Someone reads the first automated run by hand. The first tagger run on a med spa tagged 94 contacts, and 8 were wrong.
• An alert fires when the bot stops. An auto body shop's changelog said its chat router was live, but it was inactive with zero executions, and about 181 Facebook conversations went unanswered.

Ownership and cost: whose bot is it?

• Every account is in your name: the chat platform, the automation tool, the model account and the booking system.
• You get a usage estimate per 1,000 conversations, and you know who pays it. My breakdown is in AI chatbot cost.
• The prompt, knowledge base and tool code are exported at handover.
• The developer asked which messages need the bot at all. For an online English academy, the stock bot replied to every DM, emojis and "mi amor" included. A small free model on my own server now sorts each message into one of eight categories before the bot is allowed to answer.

How do you run these checks in a 30-minute call?

Ask the developer to share their screen. Open one past execution and read a tool result together. Have them book a test slot on a test calendar, find it in the booking system and cancel it. Type "tomorrow" late in the evening and see which date comes back. Ask for the last alert that fired and who received it, and which account each tool runs in.

You do not need to read code for any of this. You only need to see each thing happen once. A developer who has run chat agents for real businesses will have an execution log and an alert to show you. One who has only built demos will offer to send a recording later.

How do you score it?

Count the checks the developer showed you, with a screenshot, a log or a live run.

What does a passing score mean?

It means the failures will be visible and cheap to fix, not that there will be none. Every build in this post broke at least once, and these checks are the ones that made the breaks cheap.

For the n8n side of the same failures, read my n8n AI agent mistakes. If the agent will live inside GoHighLevel, see GoHighLevel Conversation AI setup, and for WhatsApp, n8n WhatsApp integration. The phone version of this scorecard is the AI voice agent developer interview, and if the agent will act on its own, the teardown in hire agentic AI developers applies too. To score a developer you are talking to, book a call and bring their answers.

About the author

Carlos Familia, CRM & AI automation expert. GoHighLevel, n8n, AI agents, servers. Full-stack developer.

I build CRM systems in GoHighLevel, n8n automations on my own servers and AI voice and chat agents for service businesses. Full-stack developer: I write the workflow, the API integration, the database and the server it runs on, and I test every build with real journeys before handoff.