PortKey
Site ↗Evals ↗AI intake demo ↗access code on requestCode private
A mortgage portability neobank prototype. When a homeowner sells and buys, PortKey lets them carry their existing low-rate mortgage to the new property instead of refinancing at today’s rates.
Intake
A conversational assistant pulls a loan application out of a mortgage statement and a short borrower conversation. Every field records where it came from. Plain code decides which lenders qualify; the model only writes the explanation, and a human approves the handoff.
Providers
Three model providers behind one interface, all fed the same system prompts. Swapping providers is a query parameter, and adding a fourth is a template file. API keys stay in server routes and never reach the browser. One schema validates every model output, whichever provider produced it.
Evals
Every provider change runs through three layers: ten deterministic gates per case, a model judge from a different model family scoring against a locked rubric, and a human calibration step that checks the judge against my own blind labels. Results are published as static JSON on the evals page, which makes no model calls. Offline unit tests cover the app and the eval harness itself.
| Provider | Cases passing every gate | Intake faithfulness | Homeowner tone |
|---|---|---|---|
| OpenAI | 8/8 | 4.3 | 3.3 |
| Anthropic | 8/8 | 4.4 | 3.6 |
| Groq (open-weight) | 0/8 | 2.2 | 2.7 |