Guides

Building Prompt Engineering with open-source tools

This guide outlines a production-grade workflow for prompt engineering, moving beyond trial-and-error to a systematic approach that ensures reliability, cost-efficiency, and model portability. You will learn how to structure complex logic, implement regression testing, and manage prompt versions as application code.

4-6 hours7 steps
1

Define Structured System Instructions and Output Schemas

Start by defining a strict system prompt that establishes the persona, constraints, and output format. Use JSON schema to enforce structure, as this allows your application code to parse LLM responses reliably without regex or brittle string manipulation.

prompt-config.ts
const schema = {
  type: "object",
  properties: {
    analysis: { type: "string" },
    confidence_score: { type: "number" },
    tags: { type: "array", items: { type: "string" } }
  },
  required: ["analysis", "confidence_score", "tags"]
};

const systemPrompt = `You are a technical analyst. You must return your findings strictly in JSON format according to this schema: ${JSON.stringify(schema)}. Do not include markdown formatting or preamble.`;

⚠ Common Pitfalls

  • •Mixing instructions for formatting and logic in the same paragraph
  • •Failing to specify how the model should handle cases where it lacks sufficient information
2

Implement Few-Shot Examples for Pattern Matching

LLMs perform significantly better when provided with 3-5 high-quality examples of the desired transformation. These examples should cover common edge cases and the specific 'tone' or 'reasoning' required.

few-shot-examples.json
[
  { "role": "user", "content": "Process: User updated subscription to Pro." },
  { "role": "assistant", "content": "{ \"event\": \"billing_update\", \"tier\": \"pro\", \"action\": \"upgrade\" }" },
  { "role": "user", "content": "Process: User canceled trial early." },
  { "role": "assistant", "content": "{ \"event\": \"billing_update\", \"tier\": \"trial\", \"action\": \"churn\" }" }
]

⚠ Common Pitfalls

  • •Using synthetic examples that don't reflect real-world messy data
  • •Providing too many examples that consume unnecessary tokens and dilute the system instructions
3

Integrate Chain-of-Thought (CoT) for Complex Logic

For tasks requiring multi-step reasoning, explicitly instruct the model to think through the problem step-by-step. This reduces hallucinations by forcing the model to generate a logical bridge before arriving at the final answer.

cot-instructions.txt
To determine the correct classification:
1. Identify the primary intent of the user request.
2. Check for conflicting keywords in the request body.
3. Evaluate the request against the compliance checklist provided.
4. Output the final classification only after completing these steps.

⚠ Common Pitfalls

  • •Asking for CoT in the same field as the final JSON output, which can break parsers
  • •Not providing enough scratchpad space for the model to 'think'
4

Externalize and Version Prompt Templates

Do not hardcode prompts inside your application logic. Use a templating engine (like Mustache or Jinja) or a dedicated prompt management tool. This allows you to update prompts without redeploying the entire application and enables A/B testing.

prompt_manager.py
from jinja2 import Template

prompt_template = Template("Analyze the following log entry for {{ user_id }}: {{ log_content }}")
formatted_prompt = prompt_template.render(user_id="123", log_content="Timeout at 08:00")

⚠ Common Pitfalls

  • •Treating prompts as static strings rather than code artifacts
  • •Losing track of which prompt version was used for a specific production output
5

Establish an Evaluation Framework (LLM-as-a-Judge)

Automate the evaluation of prompt changes by using a more capable model (e.g., GPT-4o or Claude 3.5 Sonnet) to grade the outputs of your production model based on a rubric. This catches regressions that simple string matching cannot.

eval-suite.ts
const evalRubric = `Grade the response from 1-5 on:
1. Accuracy to the source text
2. Adherence to JSON schema
3. Conciseness`;

// Run this as part of your CI/CD pipeline
const score = await evaluatorModel.predict(evalRubric, targetResponse);

⚠ Common Pitfalls

  • •Relying solely on manual inspection of 1-2 outputs
  • •Using biased rubrics that don't penalize common model failures like verbosity
6

Optimize for Token Usage and Latency

Analyze the prompt to remove redundant instructions and shorten examples. Use tools like Helicone or LangSmith to monitor token usage per request and identify prompts that are unnecessarily 'heavy' for the task.

⚠ Common Pitfalls

  • •Including large chunks of documentation in the prompt that are rarely relevant
  • •Ignoring the impact of long system prompts on TTFT (Time to First Token)
7

Implement Provider-Specific Adapters

Different models respond differently to prompt structures (e.g., Anthropic prefers XML tags, while OpenAI excels with Markdown). Create an adapter layer that transforms your generic prompt template into the optimal format for the target provider.

adapters.ts
const wrapForClaude = (content: string) => `<instructions>${content}</instructions>`;
const wrapForGPT = (content: string) => `### Instructions\n${content}`;

⚠ Common Pitfalls

  • •Assuming a prompt that works perfectly for GPT-4 will work identically for Claude
  • •Failing to normalize parameters like 'temperature' or 'top_p' across different providers

What you built

By treating prompt engineering as a software development lifecycle—complete with versioning, structured schemas, and automated evaluations—you ensure that your AI features are reliable and maintainable. Always prioritize small, iterative changes backed by evaluation scores over massive, sweeping prompt rewrites.