How to Test and Evaluate AI Agents
Build a small agent evaluation dataset. Compare task success, tool correctness, permission outcomes, local timing and unknown cost with a Python fixture.
Read the articleA personal publication for curious developers
Practical articles and visual lessons on software, AI, and the systems behind everyday applications.
Read. Try. Understand.
Build a small agent evaluation dataset. Compare task success, tool correctness, permission outcomes, local timing and unknown cost with a Python fixture.
Read the articleSeparate read, draft and send permissions. Run a mock email workflow with exact-action approval, changed arguments, expiry, reuse checks and audit records.
Compare one assistant with a planner, researcher and reviewer. Run an offline Python example and evaluate contracts, failures, latency and token costs.
Trace a harmless document injection through tool restrictions, output checks and mock-action approval. Run a local Python fixture and learn its limits.
Build an inspectable context with instructions, evidence, tool results and recent state. Test a byte budget, reviewed excerpts and missing-evidence handoffs.
Compare product-document retrieval with saved user preferences. Run a support fixture that checks access, freshness, provenance and retrieval failures.
SkillsHats Visualize
Follow an algorithm step by step. Change an input. Watch a system respond. Build an intuition for what the code is doing.
Explore visual lessonsData structures & algorithms · System design
One comparison. Half the possibilities.
Looking for 23: 12 is too small. Keep the right half.