Building Support Agent without a vector database

There is a reflex in machine-learning engineering that the moment you start working with embeddings, you must reach for a vector database. It is the architecturally “correct” answer, and so it gets reached for early, often before anyone has asked whether the problem actually demands it. I built Support Agent, my retrieval system for support tickets, and I deliberately did not reach for one. What follows is what I built, and why the absence is a feature rather than an oversight.

What Support Agent does

Every Tier 1 ticket type can now be classified, analyzed, and handed a suggested resolution through Support Agent. The system stands on roughly a thousand resolved Zendesk tickets drawn from the past two years. Each one is summarized using Azure OpenAI, embedded using sentence-transformers, and stored as a vector embedding in S3. When a new ticket arrives, the tool places it in a category, finds the historical tickets most like it, and uses them to draft a grounded reply and a set of resolution steps. That is the whole shape of it. The interesting parts are what happens to the data on the way in, and what I chose not to build to make it work.

Keeping PHI out of the pipeline

Before any ticket text leaves my pipeline, it is scrubbed, and the scrubbing is layered rather than trusted to a single pass. The first pass is regex, catching the predictable identifiers: emails, phone numbers, IDs, and the rest. The second is an LLM-based validation pass whose only job is to find what the regex missed. Summarization then runs on scrubbed content alone, and any ticket that fails scrubbing is dropped from the corpus entirely rather than being repaired and waved through. As a last line of defense, the embedding stage runs one more residual-PHI check and excludes anything still carrying sensitive content, reporting a phi_excluded count for every build. The principle is simple: when a ticket is in doubt, it does not make it into the index.

How retrieval works

Retrieval happens in two stages, and the separation matters. First, an incoming ticket is classified into a (product feature, issue type) category, either by mapping its existing Zendesk tags directly or, when tags are missing, by embedding the ticket text and finding the nearest historical ticket through cosine similarity. That classification is threshold-gated; when confidence is low, it does not guess, it falls back to "needs review" and lets a human take it. Only then does the second stage run, retrieving the five most similar historical tickets from within the identified category. Those five exemplars, together with the relevant playbook entries and KB articles, go into the prompt, and the model generates its draft from that grounded context rather than from thin air.

The artifacts themselves live in S3, under a versioned path for each build: the embeddings, the historical ticket manifest, and the corpus. The repository holds only the durable, human-edited pieces: the taxonomy in playbook.json, the Zendesk tag-to-feature mappings, and about sixteen curated KB articles, which will become embeddings of their own in a later phase. A GitHub Actions pipeline already ties this together. It pulls resolved tickets, scrubs and summarizes them, generates the embeddings, and publishes a new versioned bundle. Scheduled builds are switched off for now because they run on demand, but the schedule is one toggle away from giving me a continuously refreshed weekly index.

So where is the vector database?

There isn’t one. And the reason is worth stating plainly, because it runs against the reflex. At the end of the day, embeddings are just numpy arrays. A vector database is a storage and query layer optimized for searching very large numbers of those arrays, and at my scale of 916 vectors, roughly 3 MB in total, it would add complexity without buying me anything I need. I load the embeddings into memory from embeddings.npz and run brute-force cosine similarity directly in numpy. For a dataset this size that is fast, simple to reason about, and exact.

Why does brute force hold up here? Look at the size of the work. The full embedding matrix is 916 × 768 × 4 bytes, which is about 3 MB. A single query against the whole corpus is roughly 916 × 768, or about 700,000 multiply-add operations. That sounds like a lot until you set it against what a modern CPU is built to do. Through BLAS underneath, numpy clears it in well under a millisecond, tens of microseconds in practice. The compute time is effectively invisible next to ordinary API and network latency.

I am not using approximate nearest-neighbor search for the same reason. Pinecone, Weaviate, Qdrant, and FAISS earn their place once you are dealing with hundreds of thousands or millions of vectors, with multi-tenant systems where no single pod can hold every customer’s embeddings, with real-time inserts and deletes, or with indexes shared across many replicas. Those systems lean on ANN algorithms that trade a little accuracy for speed at scale. At my scale that trade does not pay; exact brute-force search is already fast enough, and it is both simpler and more accurate.

One implementation detail does most of the heavy lifting. I normalize the embeddings when I build the index, using vec = model.encode(text, normalize_embeddings=True), so that both stored and query vectors are L2-normalized. Once that is true, cosine similarity collapses into a dot product, cosine(a, b) = a · b, and the entire search becomes a single matrix-vector multiplication that BLAS can run on hardware-accelerated paths.

The lesson underneath the architecture

The broader point is that this approach scales much further than most tutorials let on. Plenty of production RAG systems run comfortably on FAISS-in-process, or plain numpy, well into the tens of thousands of vectors. Vector databases get recommended early because they sound correct, but for a small corpus, loading an .npz file into memory is the more practical answer.

None of this is an argument against vector databases. It is an argument for matching the machinery to the size of the problem in front of you. The day my corpus outgrows memory, the day the KB articles join the index and the weekly builds turn back on and the numbers climb, I will reach for the heavier tooling, and reach gladly. Until then, the simplest thing that gives exact answers is also the right thing. The reflex can wait.