Be blunt about my plan for an eval set for prompts. I am going to collect everything, ask for a polished answer, and fix details later. What is likely to go wrong?
Forked from @calebdiaz/ChatGPT was overcomplicating an eval set for prompts, so I reset the question
ChatGPT was overcomplicating an eval set for prompts, so I reset the question
A compact prompt and the response it produced. Replace the bracketed parts with your own details.
Three things.
- “Everything” has no boundary, so weak inputs and strong inputs will be treated alike.
- A polished answer can hide missing evidence. Ask for gaps and assumptions before prose.
- “Fix details later” means there is no acceptance test now.
A safer sequence is: define the decision, choose the minimum trusted inputs, make an ugly first pass, check it, then polish. For this task, mark missing evidence instead of filling it in.
Fair. I was using polish as proof that the reasoning was finished.
2 comments
Sign in to join the conversation.