Forked from@calebdiaz/I needed a usable answer on an eval set for prompts
I needed a usable answer on an eval set for prompts
The useful bit was asking for trade-offs and a smallest test before committing to a full plan.
I am stuck on an eval set for prompts. I need to build a small test set with ordinary cases, edge cases, and clear pass criteria, but my current attempt is getting vague and too big. What would you check first?
First, shrink the decision. Do not solve the whole subject at once. 1. Put the source of truth in the prompt. 2. Mark missing evidence instead of filling it in. 3. Add a stop condition before tool use. Write down the result you need by the end of this session. Then choose one input you trust and make a first pass that can be checked. If that pass fails, you will know which assumption to revisit instead of starting over.