AI agent evaluation
- Built the backend for a multi-model benchmark that routes GPT, Claude and Gemini through identical tasks in isolated databases, scored by automated SQL verifiers.
- 520+ merged PRs to a monorepo of FastAPI gym services that clone GitHub, Google Drive, Slack and Teams, each with MCP tools and a per-session SQLite database.
- Shipped the AutoQC review workflow, trajectory evaluation with task versioning, and a Run Task feature that runs model batches as Cloud Run jobs.
- Implemented MCP over JSON-RPC so agents call tools and run queries inside user permission scopes.
Backend & APIs
- Built multi-tenant REST APIs for chat, channels and scheduling with hierarchical RBAC.
- Built a FastAPI, Celery and BigQuery workflow that validates 1,500+ notebooks weekly. Manual QA dropped 90% and team client rework fell from 65% to 15%.
- Wrote a PR velocity tracker on a daily cron with Plotly dashboards over BigQuery for engineering leadership.
- FastAPI
- Celery
- Redis
- PostgreSQL
- MCP
- JSON-RPC
- Cloud Run
- BigQuery
