Model Guides prompts

Help me choose a model for this task.

THE TASK: [WHAT YOU NEED DONE]
VOLUME: [how often, how many times]
WHAT MATTERS MOST: [accuracy / speed / cost / output quality / following instructions precisely]
CONSTRAINTS: [budget, latency requirements, data handling requirements, what you have access to]
CURRENT MODEL: [WHAT YOU USE NOW, if any]
WHAT IS NOT WORKING: [IF ANYTHING]

Produce:

1. THE FRAMING - model capabilities and the available line-up change frequently, and any specific recommendation dates quickly. What does not change is how to decide. Give me the decision framework rather than a list of names, and be explicit that current options should be checked against the providers' own documentation.

2. THE TASK CHARACTERISTICS that determine the choice:
   - Does it need multi-step reasoning, or is it a transformation?
   - Does it need long context?
   - Does it need current information?
   - Does it need precise instruction-following, or judgement?
   - Does it need to produce structured output?
   - Is a mistake expensive or cheap?
   Assess my task against each.

3. THE SIZE DECISION - larger models cost more and are better at reasoning and nuance; smaller models are faster, cheaper, and often equally good at classification, extraction, formatting and simple transformation. Most production workloads use a larger model where a smaller one would do. Say which my task likely needs.

4. THE TEST, NOT THE ASSUMPTION - the only reliable way to choose is to run my actual task on candidates and compare. Describe how: build a small evaluation set from real inputs, run each model, and compare on the criteria I said matter. This is the answer to the question, and it takes an afternoon.

5. THE ROUTING OPTION - using a cheaper model for the straightforward majority and escalating to a stronger one for the hard cases. Say how to decide which is which for my task, and whether the complexity is worth it at my volume.

6. THE COST ARITHMETIC - at my stated volume, roughly what the difference between a cheaper and a more capable model amounts to. Sometimes the difference is negligible and the decision is only about quality; sometimes it dominates.

7. THE CONSTRAINTS - my stated requirements around latency, budget and data handling. Some of these rule options out before quality is considered. Say which apply.

8. WHAT IS NOT WORKING - if I said something is failing with my current model, whether that is a model problem or a prompt problem. It is usually a prompt problem, and changing model will not fix it.

9. THE SWITCHING COST - prompts tuned for one model do not transfer unchanged. Say what to expect and to re-test rather than assuming.

10. THE EVALUATION SET - how to build the small test set that answers this properly, and that also serves for future model changes.

Help me adapt this prompt for a different model.

THE PROMPT:
"""
[PASTE]
"""
CURRENTLY WORKS ON: [MODEL]
MOVING TO: [MODEL]
WHAT THE PROMPT DOES: [THE TASK]
WHAT IS GOING WRONG ON THE NEW MODEL: [THE PROBLEM, if you have tried]

Produce:

1. THE EXPECTATION - prompts do not transfer unchanged. Models differ in instruction-following style, default verbosity, how they handle structure, how they respond to role framing, and what they do with ambiguity. A prompt tuned for one is tuned to its particular tendencies.

2. THE MODEL-SPECIFIC ELEMENTS in my prompt - anything relying on a particular model's conventions: a specific syntax for structure, a parameter format, a feature only one supports, or wording that was added to counteract a tendency the new model may not have.

3. THE COUNTERACTING INSTRUCTIONS - prompts accumulate instructions that fix one model's habits: telling it to be less verbose, to stop apologising, to avoid preamble, to stop summarising. On a different model these may be unnecessary or actively harmful. Identify them in mine and say which to remove and retest.

4. THE DEFAULT DIFFERENCES to expect - between any two models, differences in: default length, how much structure is imposed, willingness to decline, how uncertainty is expressed, and how literally instructions are followed. Say what to look for.

5. THE STRUCTURED OUTPUT DIFFERENCE - if the prompt produces structured output, the reliability and the available features differ. Check whether the new model has a native structured output or tool-use feature, because that is more reliable than any prompt wording.

6. THE PROBLEM I DESCRIBED - if I have already tried, diagnose it against the above.

7. THE ADAPTED PROMPT - with model-specific elements removed and anything the new model needs added.

8. THE TEST PLAN - the specific inputs to run on both, including edge cases, so the comparison is real rather than impressionistic. A prompt that works on one example is not ported.

9. THE REGRESSION RISK - the things that worked on the old model that may quietly not work on the new one: instruction-following on a specific constraint, format consistency, and behaviour on unusual inputs. These fail silently.

10. THE ONGOING PROBLEM - the same applies within a model family as versions update. A prompt tuned for one version can behave differently on the next. Say what to keep as a test set so this is caught rather than discovered by users.

11. THE PORTABILITY ADVICE - how to write prompts that transfer better: clear structure, explicit instructions, stated output format, and fewer instructions added to counteract specific model habits.

Help me handle knowledge limits for this task.

THE TASK: [WHAT YOU ARE ASKING THE MODEL TO DO]
HOW TIME-SENSITIVE IT IS: [not at all / somewhat / it must be current]
WHAT KIND OF INFORMATION IT NEEDS: [DESCRIBE]
DO I HAVE SEARCH OR DOCUMENT ACCESS: [yes / no]
WHAT WENT WRONG: [IF ANYTHING - outdated answers, confident wrong information]

Produce:

1. THE CUTOFF PROBLEM - models are trained on data up to a point and have no knowledge after it. The difficulty is not the gap itself but that a model may answer confidently about the period after its cutoff, describing a state of affairs that has since changed, without any signal that it is doing so.

2. WHAT MY TASK NEEDS - assess the information required. Categories:
   - STABLE: concepts, methods, principles, history. Cutoff is largely irrelevant.
   - SLOW-MOVING: practices, standards, established tools. Usually fine with a check.
   - FAST-MOVING: prices, versions, rankings, who holds a position, what a company offers, current events, anything about AI models themselves. The model's answer should not be relied on.
   Say which mine is.

3. THE HIGH-RISK ITEMS in my task - the specific things most likely to be out of date. Version numbers, pricing, product features, regulations, rankings, personnel, and the availability of tools and services all change.

4. IF I HAVE SEARCH OR DOCUMENT ACCESS - the approach: retrieve the current information and supply it in the prompt rather than relying on recall. The instruction that makes this work: use only the supplied material for anything factual and say when it does not cover something.

5. IF I DO NOT - the approach: ask the model for the reasoning, the framework, and what to check, rather than for the facts. A model is reliable about how to evaluate options and unreliable about which options currently exist.

6. THE ASKING-FOR-UNCERTAINTY INSTRUCTION - telling the model to flag where its information may be out of date and to state when something is likely to have changed. This helps and is not complete; it can only flag what it is aware of being uncertain about.

7. THE SELF-KNOWLEDGE LIMIT - models are frequently wrong about their own cutoff date, about which model they are, and about the current state of AI products, because that information was changing rapidly around their training. Treat any claim about these with particular scepticism.

8. THE PROMPT - rewritten for my situation.

9. THE VERIFICATION LIST - the specific claims in my task's output that should be checked against a current source before being used.

10. THE DATE CONTEXT - supplying today's date in the prompt helps the model reason about elapsed time and flag what may have changed. Worth doing for anything time-sensitive.

Help me understand the failure modes for this task.

THE TASK: [WHAT YOU ARE USING A MODEL FOR]
WHAT I DO WITH THE OUTPUT: [PURPOSE, AND WHO SEES IT]
HOW MUCH IT MATTERS IF IT IS WRONG: [not much / it would be embarrassing / it would cause real problems]
WHAT CHECKING HAPPENS: [I review everything / spot checks / nothing]
MODEL: [WHICH]

Produce:

1. THE RELEVANT FAILURE MODES for my specific task, from the general set:
   - FABRICATION: producing plausible, specific, wrong information. Highest risk for citations, references, quotations, statistics, names, dates, and anything niche.
   - ARITHMETIC AND COUNTING: unreliable, including counting items, characters and occurrences
   - INSTRUCTION DROPPING: quietly omitting a constraint when there are several
   - OVERCONFIDENCE: the tone does not vary with the reliability of the content, so a guess reads exactly like a fact
   - CONSISTENCY: different answers to the same question across runs
   - RECENCY: confident answers about things that have changed
   - LONG CONTEXT: material in the middle of very long input given less weight
   - SYCOPHANCY: agreeing with a premise in the question rather than challenging it
   Say which of these bear on my task.

2. THE HIGHEST RISK FOR MY TASK - one, named specifically, with what it would look like if it happened.

3. THE TONE PROBLEM - the most practically dangerous characteristic. A model's fluency does not vary with its reliability, so there is no signal distinguishing a solid answer from a guess. Everything reads as equally confident. Say what this means for how I should treat my output.

4. THE CHECKING PLAN - given what I said actually gets checked, whether it is proportionate to the risk. If nothing is checked and a mistake would cause real problems, say so plainly.

5. WHAT TO CHECK SPECIFICALLY - not everything, which is impractical, but the highest-risk items in my output. For most tasks that means: every citation, every number, every name, every date, and any claim the output rests on.

6. THE UNVERIFIABLE PROBLEM - where I cannot easily check because I do not know the subject well enough. This is the most dangerous position, and the honest answer is that model output in an area I cannot evaluate needs a source or a person who can.

7. THE PROMPT MITIGATIONS - what helps: asking it to state uncertainty, to distinguish what it is confident about from what it is not, to cite where information comes from, and to say when it does not know. These reduce the rate and do not eliminate it.

8. THE STRUCTURAL MITIGATIONS - what helps more: supplying source material rather than relying on recall, verifying output against a source, and having a person who knows the subject review it.

9. WHAT NOT TO USE IT FOR - given my task, anything that falls outside what this is reliable for.

10. THE PROPORTIONATE CONCLUSION - matched to how much it matters. For low-stakes work, light checking is fine; for anything consequential, say what the checking must cover.

Help me set the parameters for this task.

THE TASK: [WHAT YOU ARE DOING]
WHAT I NEED FROM THE OUTPUT: [consistency / variety / accuracy / creativity]
WHERE I AM CALLING IT FROM: [an API / a chat interface / a tool]
CURRENT SETTINGS: [IF YOU HAVE SET ANY]
WHAT IS GOING WRONG: [IF ANYTHING]

Produce:

1. WHAT IS AVAILABLE TO ME - if I said I am using a chat interface, most parameters are not exposed and the answer is that this is a prompt question rather than a parameter one. Say so if that applies, and skip to point 8.

2. TEMPERATURE - what it actually does: it affects how much the model favours the most likely next token versus sampling more broadly. Lower produces more predictable, more repetitive output; higher produces more varied output and more errors. For my task, what value range makes sense and why.

3. THE COMMON MISCONCEPTION - temperature is not a creativity dial and not a quality dial. High temperature does not produce better ideas; it produces less likely ones, some of which are interesting and some of which are wrong. For factual or structured tasks, lower is almost always correct.

4. THE OTHER SAMPLING PARAMETERS - top-p and similar, what they do, and the general advice that adjusting one sampling parameter is usually sufficient and adjusting several together produces unpredictable interactions.

5. MAX TOKENS - what it controls and the thing to understand: it truncates rather than shortening. A limit set too low produces output cut off mid-sentence rather than a concise version. Length is controlled by the prompt, not by this.

6. STOP SEQUENCES - what they are for and when they help, particularly for structured output and for preventing a model continuing past where you want it to end.

7. THE SEED - if available, what it does for reproducibility, and the caveat that it does not guarantee identical output across provider-side changes.

8. THE PROMPT VERSUS PARAMETER QUESTION - most problems people try to fix with parameters are prompt problems. Output too long, too generic, inconsistent in format, or wrong in content are all prompt issues. Parameters change how the model samples, not what it is trying to do. Say which mine is.

9. FOR MY TASK - the recommended settings, with the reason for each.

10. THE TESTING APPROACH - change one parameter at a time, on the same inputs, and compare. Parameter tuning done by changing several things at once teaches nothing.

11. THE DEFAULT ADVICE - the provider defaults are reasonable for most tasks. Adjusting parameters is worth it for specific reasons, and the majority of the gain available is in the prompt.

Help me compare models on my task.

THE TASK: [WHAT YOU NEED DONE]
THE MODELS I AM COMPARING: [WHICH ONES]
WHAT MATTERS: [accuracy / quality / consistency / cost / speed / instruction-following]
MY PROMPT:
"""
[PASTE]
"""
HOW MANY TEST CASES I CAN PREPARE: [NUMBER]

Produce:

1. WHY BENCHMARKS DO NOT ANSWER THIS - published benchmarks measure performance on standardised tasks that are not yours. A model that leads on a reasoning benchmark may be worse at your particular extraction task. Your own small test set is more informative than any published comparison.

2. THE TEST SET - what it should contain, given my task:
   - Typical inputs, the majority
   - Edge cases: empty, very long, ambiguous, unusual formatting
   - Known-difficult cases where the current approach fails
   - At least one adversarial or out-of-scope input
   Aim for the number I said I can prepare, weighted toward typical.

3. THE PROMPT FAIRNESS PROBLEM - my prompt is likely tuned to whichever model I developed it on, which biases the comparison. Either use a neutral prompt for all, or adapt the prompt to each model and compare the best each can do. Say which suits my purpose and be explicit that comparing a tuned prompt against an untuned one is not a model comparison.

4. THE EVALUATION CRITERIA - my stated priorities turned into checkable criteria. For each: how to judge it, and whether it can be checked automatically (format, length, presence of required elements) or needs judgement.

5. THE BLIND COMPARISON - judge the outputs without knowing which model produced which. Expectations bias evaluation strongly, and this is easy to arrange and frequently skipped.

6. THE RUN COUNT - one output per model per case is not enough, because variation between runs can exceed the difference between models. Run each case several times and look at the distribution, particularly if consistency is one of my criteria.

7. THE SCORING - a simple structure: per criterion, per case, per model. Keep it simple enough that it actually gets done.

8. THE COST AND SPEED MEASUREMENT - recorded during the test rather than estimated from pricing pages, since token usage differs between models for the same task.

9. THE DECISION - how to weigh the results given my priorities, including that the best output quality is not always the right choice if the cost or latency difference is large.

10. THE REUSE - keep this test set. It answers the same question again when a new model appears, when a provider updates one, and when the prompt changes. It is the most durable artefact from this exercise.

11. THE SAMPLE SIZE CAVEAT - with a small test set, a small difference between models is not meaningful. Say roughly what difference would be worth acting on at my number of cases.

Help me prepare for model changes.

WHAT I HAVE BUILT: [DESCRIBE YOUR PROMPTS, TOOLS OR WORKFLOW]
HOW CRITICAL IT IS: [personal use / used by colleagues / customer-facing / it must not break]
HOW MANY PROMPTS: [ROUGH COUNT]
WHAT I DO NOW WHEN SOMETHING BREAKS: [CURRENT PROCESS, if any]
WHO NOTICES IF IT DEGRADES: [me / users / nobody]

Produce:

1. WHAT ACTUALLY CHANGES - providers update models, deprecate versions, adjust default behaviour, and occasionally change how instructions are interpreted. A prompt that worked can behave differently without any change on my side. The failure is usually degradation rather than an error, which is why it goes unnoticed.

2. THE SILENT DEGRADATION PROBLEM - I said who notices. If the answer is nobody or users, that is the main risk: output quality can drop and nothing surfaces it until someone complains. Say what this means for my situation.

3. THE VERSION PINNING QUESTION - whether the provider allows specifying an exact model version rather than a moving alias. If it does, what the trade-off is: stability against eventual deprecation. For anything customer-facing, pinning and upgrading deliberately is usually right.

4. THE DEPRECATION TIMELINE - pinned versions are eventually retired. What to watch for and roughly how much notice to expect, noting that this varies by provider and should be checked in their documentation.

5. THE REGRESSION TEST SET - the essential defence. A fixed set of inputs with expected behaviour, run on a schedule and before any change. Without it, degradation is invisible. Say what mine should contain given what I have built.

6. THE SCHEDULE - how often to run it, proportionate to how critical the system is.

7. THE UPGRADE PROCESS - when a new version is available: run the test set on both, compare, identify what changed, adjust prompts, and only then switch. Not switching and hoping.

8. THE PROMPT PORTABILITY - writing prompts that survive model changes better: explicit instructions, stated output format, clear structure, and few instructions that exist only to counteract a specific model's habits. Those are the ones that break.

9. THE MONITORING - for a live system, what to log and what to watch: output length distribution, format validity rate, error rate, and any downstream failure. A change in these is usually the first sign.

10. THE FALLBACK - if a model becomes unavailable or a change breaks something, what the contingency is. For anything critical, having a tested alternative provider is worth the effort.

11. THE PROPORTIONATE ANSWER - given how critical I said this is, how much of the above is actually warranted. For personal use, a handful of saved test prompts is enough; for a customer-facing system it is not.