You give a coding agent a bug and say, “This is the problem. Fix it.” The agent works on it, changes a few files, and comes back with “Implemented.” Okay, good. But did it actually fix the problem? You can run the application, open your browser, and try the behavior yourself. You can also run automated tests. Another check we can stitch into that workflow is an LLM as a judge . Let’s reason this out. We give the LLM an output, a rubric describing what good looks like, and evidence. It assesses where the result meets those criteria, falls short, or still needs verification. 1. Start with the issue Describe the bug → coding agent → proposed fix. Say you enter the wrong password, click Log in , and the loading spinner keeps spinning. You ask the coding agent to fix it. The agent returns a change. That gives us something to assess. Define what “fixed” means Before accepting it, we put together what “fixed” should mean: A wrong password shows a clear error. The spinner stops when t...