How to Build an AI Chatbot From Scratch for Business

Knowing how to build an AI chatbot from scratch comes down to five decisions: what job it does, which model powers it, what data grounds it, which systems it can act on, and how you measure whether it is actually helping. Get those right and the code is the easy part. This guide walks through each decision in the order a real project makes them, with the trade-offs spelled out.

It is written for Shopify merchants, ops managers and CTOs who want a working assistant — not a demo that answers three questions and then hallucinates shipping times.

Step 1: Define the job the chatbot must do

A chatbot with no defined job becomes a support ticket generator. Before touching a model, write down one primary job and one fallback.

• Pre-sales questions. "Does this fit a 2019 model?", "When does it ship?", "Can I return it?"
• Order status. "Where is order #10432?"
• Internal ops. "What is our refund policy for wholesale accounts?"
• Lead qualification. "Collect company size, budget range and timeline, then book a call."

Write 20 real questions your team answers every week. Those 20 become your test set later. If you cannot list 20, you do not yet have a chatbot project — you have a curiosity.

Pick a channel before you pick a stack

A widget on a website, a WhatsApp number, an in-app help panel and a Slack bot are four different builds. Website widgets are the cheapest to ship and the easiest to iterate. Start there unless the business case is clearly elsewhere.

Step 2: Choose a model (OpenAI, Claude, Gemini, open-source)

You are choosing between hosted frontier models and self-hosted open-source ones. For most business chatbots, hosted wins on time-to-value.

Practical rule: prototype on a hosted model, measure real traffic and cost for a month, then decide whether open-source is worth the operational load. Most businesses never need to self-host.

Do not skip the system prompt

The system prompt defines tone, refusal rules and what the bot must never claim. Write it like an employee handbook page: short, specific, with examples of good and bad answers.

Step 3: Ground it in your data with RAG

Retrieval-augmented generation (RAG) is what stops the bot from inventing answers. Instead of hoping the model memorised your return policy, you retrieve the relevant passages and pass them into the prompt.

The pipeline is short:

1. Collect sources. Help-centre articles, PDFs, policy pages, product descriptions, past support tickets.
2. Chunk them. Split into 300–800 token passages with a little overlap.
3. Embed and store. Convert each chunk into a vector and store it in a vector database (pgvector on PostgreSQL, Pinecone, Weaviate or similar).
4. Retrieve at query time. Embed the user's question, pull the top 3–8 matching chunks.
5. Answer with citations. Instruct the model to answer only from the retrieved context and to say "I don't know" otherwise.

The step most teams skip: evaluation

Keep a spreadsheet of 50 question-and-expected-answer pairs. Every time you change the prompt, chunking or model, re-run it. Without this, "improvements" are guesses.

Step 4: Connect it to your CRM, store and helpdesk

A chatbot that can only talk is a FAQ page with extra steps. The value comes from acting on live data.

• Shopify. Order status, tracking links, stock levels, return eligibility — via the Admin API.
• Zoho CRM. Create a lead, log the conversation, update a deal stage when the visitor qualifies.
• Helpdesk. Escalate to a human with the full transcript attached.

Implement these as tools (function calls) the model can invoke, not as hard-coded if-statements. Each tool needs a strict schema, a permission scope and a timeout. Anything that writes data — creating a lead, issuing a refund — should require a confirmation step or a human approval queue.

That "AI proposes, the customer approves" pattern is how we build AI agents for clients, and it is the single biggest reason business chatbots get trusted internally rather than quietly switched off.

Authentication and rate limits

Use OAuth where the platform offers it, store refresh tokens server-side, and never expose API keys to the browser widget. Add per-session rate limits so one abusive visitor cannot burn your model budget overnight.

Step 5: Test, monitor and improve answers

Shipping is the midpoint, not the finish line.

• Log every conversation with the retrieved chunks and the model's answer.
• Track containment rate — the share of conversations resolved without a human.
• Track escalation quality — did the human get enough context to finish the job?
• Review failures weekly. Add missing documents to the index; tighten the system prompt.

Expect the first two weeks to produce most of your tuning. After that, a monthly review is usually enough.

A realistic build checklist

• 20 real questions written down and turned into a test set
• Model chosen and system prompt drafted
• Source documents collected, chunked and embedded
• Retrieval tested against the question set
• At least one live integration (order status or CRM) working
• Human handover path tested end to end
• Logging, rate limits and cost alerts in place

When to build in-house vs hire a team

Build in-house if you have a developer with API experience, a clear single use case, and time to iterate for a few months. Hire a team if you need multiple integrations, a review-and-approve workflow, or the bot to touch customer and order data from day one.

We build AI chatbots and AI agents that plug into Shopify, Zoho CRM and helpdesks, with the approval workflow built in from the start. If you would rather skip the trial-and-error phase, see our AI agent development service or compare the wider product and service range.

Related reading

• Zoho Consulting & Implementation
• How to Choose a Mobile App Development Company