The Poe eval harness
n8n runs the suite, Poe is the bot under test, Claude grades, Notion stores the run. Not a chat playground.
Who owns what
n8n
The harness — cases out, scores in, latency. Not the persona voice.
Try n8n (opens in a new tab)Poe by Quora
The replies — custom bot across backends. Not the rubric.
Try Poe by Quora (opens in a new tab)Claude
The judge — fidelity, safety, 1–5. Not the target inference.
Try Claude (opens in a new tab)Notion AI
The suite and the run log. Not HTTP.
Try Notion AI (opens in a new tab)
How to use this stack
- n8n pulls adversarial cases from Notion. Do not grade in the Poe UI by hand.
- Poe answers as the persona. Claude does not speak as the bot.
- Claude scores the transcript. Fail the run on a guardrail miss.
- n8n writes scores back to Notion. This is not the research stack. Not a ChatGPT–Claude zoo.