What is the best way to optimize prompt length for cost?
Strategies for shorter prompts
Start by writing clear, direct instructions. For example, instead of 'I would like you to please answer the following question in a helpful manner,' use 'Answer helpfully.' Every word counts as tokens, so brevity saves money.
Use abbreviations or shorthand where unambiguous, and avoid unnecessary examples unless they significantly improve output. If you need to provide context, summarize it instead of pasting long documents.
- Remove polite filler like 'please' and 'thank you'.
- Use bullet points instead of paragraphs for instructions.
- Limit few-shot examples to the most essential.
- Summarize long context into a few key points.
- Set a system prompt once instead of repeating.
Balance length and quality
Very short prompts might lead to poor responses, requiring retries that cost more. Test different lengths to find the sweet spot where quality remains acceptable. For chatbots, keeping history to the last 2–3 exchanges often works well.
Consider using a cheaper model for initial drafts and a more expensive one only for complex queries. This tiered approach can optimize cost without sacrificing quality.
Common mistakes
- Cutting prompts so much that the model misunderstands, causing costly retries.
- Including entire conversation history when only the last message matters.
- Using verbose system prompts that repeat instructions unnecessarily.
