writing eval cases before changing a prompt: a study session that did not turn into more notes

Good. Do these without looking anything up:

1. Explain writing eval cases before changing a prompt in two sentences.
2. Give one situation where you would use it and one where you would not.

I am checking your current model, not your vocabulary. A partial answer is more useful than a polished guess.

That points to a retrieval and boundary problem, not a lack of reading. Use this 25-minute block:

• 5 min: keep the source material beside the output.
• 8 min: work one example and say why each step is needed.
• 7 min: write a case the prompt should fail.
• 5 min: make the model state what it could not verify.

Your success test is simple: you can start a fresh example and explain the first decision without a cue. Stop there today. More input would hide whether retrieval improved.

Practice task:

Create a small, realistic example of writing eval cases before changing a prompt from your own work or daily life. State the goal, the evidence or inputs you have, the first decision you would make, and one condition that would make your approach fail.

Check your answer against four things:

• The example is specific enough to test
• The first decision follows from the inputs
• You named a boundary or failure case
• You can explain the choice without repeating a definition

Do not compare with a model answer yet. Take five minutes, then revise once from your own checklist. Tomorrow, redo only the first decision from memory.