In the previous post, we built a test strategy and found real bugs. The agent works and it is tested. Now we ship it. This post has three parts. First, what the evals cost and how I cut it. Second, three problems I found in a pre deployment check that would each have broken the live site. Third, the chat page itself, and how it ended up on my resume page. ...
In the previous post, we added guardrails to my resume agent. It grounds its answers, refuses off topic questions and resists prompt injection. At least, it seemed to. “Seemed to” is not good enough for a quality engineer. So this post is about proving it. We will build a test strategy for an AI agent, and then look at the bugs it found. Some of them I would never have spotted by hand. ...
In the previous post, we built the retrieval pipeline. A question goes in, and the right chunks of my resume come out. Now we add the language model. This is where things get risky. A model that writes fluent answers can also write fluent nonsense. And this agent speaks for me, on a public page. A wrong answer here is my reputation. So this post is about controls. We make the agent answer only from evidence. We make it refuse everything else. And we defend it against people who try to talk it into misbehaving. ...
A resume is a static document. A recruiter reading it has questions it cannot answer. Did you create AI agents? What did you actually do in your last role? Have you used Kafka? So I built an agent that answers those questions for me. You can try it on my resume page. Ask it anything about my background and it answers in the first person, with sources. This is the first of four posts about how I built it. Today we cover the architecture and the data pipeline. By the end you will know how a question turns into the right piece of my resume. ...