Mastering the Art of LLM Prompts

Large Language Models (LLMs) like GPT-4 and Claude possess remarkable capabilities. However, unlocking their full potential requires effective communication through well-crafted prompts. This guide delves into the art of prompt engineering, offering a step-by-step approach – from fundamental principles to advanced techniques – to harness the true power of LLMs.

Step 1: Choosing the Optimal Model

  • Latest and Greatest: Newer models like GPT-4 Turbo offer significant advantages over predecessors like GPT-3.5 Turbo, including smoother natural language understanding. For simpler tasks, extensive prompt engineering may be less crucial.

  • Benchmarking: Utilize resources like LLM leaderboards and benchmark results to compare models and identify the best fit for your specific needs.

image

Examples:

  • For nuanced language translation, GPT-4 Turbo’s contextual understanding is likely superior to older models.
  • For tasks that require both capabilities and speed, the Llama-3-70b open-source model is an excellent option.

Step 2: Establishing Clear Communication

Clarity and Specificity

  • Explicit Instructions: Treat the LLM as a collaborator requiring clear direction. Define the task, desired outcome, format, style, and output length explicitly, avoiding ambiguity.

  • Contextual Grounding: Provide relevant background information and context to guide the LLM towards the desired response, considering the intended audience and purpose.

  • Separation of Concerns: Clearly separate instructions from context using ### or """.

  • Examples as Guides: Demonstrate the desired output format and style with well-chosen examples.

  • Cueing: Use short phrases like “Key points:”, “Summary:”, or “Code:” to indicate the desired output format.

Examples:

  • Instead of “Create a blog post,” specify “Write a 500-word blog post about renewable energy’s impact, targeting industry professionals.”
  • For a poem generation task, you might specify the desired tone, rhyme scheme, and theme.

Few-Shot Learning

Provide a few examples demonstrating the desired output format and style to reduce ambiguity and set clear expectations.

Examples:

  • Prompt: Translate these sentences to Spanish:

    • “The cat sat on the mat.” - “El gato se sentó en la alfombra.”
    • “The dog barked at the mailman.” - “El perro ladró al cartero.”
    • “The children played in the park.”

  • Expected Output: “Los niños jugaron en el parque.”

  • Prompt: Write product descriptions in a witty and engaging style:

    • Product: Noise-cancelling headphones
    • Description: “Silence the world and get lost in your own soundscape with these noise-cancelling headphones. Perfect for escaping the daily grind or focusing on your work.”
    • Product: Travel backpack
    • Description: “This travel backpack is your ultimate adventure companion. With its spacious compartments and durable design, it’s ready to explore the world with you.”
    • Product: Smartwatch
    • Description:

  • Expected Output: “This smartwatch is more than just a timepiece; it’s your personal assistant, fitness tracker, and notification hub, all wrapped up in one sleek design.”

Step 3: Guiding the LLM’s Thought Process

Chain-of-Thought (CoT) Prompting

For complex reasoning tasks, guide the LLM step-by-step through the thought process, encouraging it to break down the problem and generate a logical sequence of reasoning steps leading to the answer.

Examples:

  • Problem: John has 5 apples, gives 2 to Mary, then buys 5 more. How many apples does he have now?

  • CoT Prompt: “John starts with 5 apples. After giving 2 to Mary, he has 3 left. Buying 5 more gives him 3 + 5 = 8 apples.”

  • Problem: Who was the most decorated (maximum medals) individual athlete in the 2000 Sydney Olympic games? Take a step-by-step approach in your response, cite sources and give reasoning before sharing the final answer in the below format: ANSWER is: <name>

  • CoT Prompt:

Step 1: Determine the year of the Sydney Olympic games. The Sydney Olympic games were held in the year 2000.  
  
Step 2: Find a list of individual athletes who won multiple medals in the 2000 Sydney Olympics.  
  
According to the official website of the Olympic Games, there were several individual athletes who won multiple medals in the 2000 Sydney Olympics. Some of the notable names include:  
• Ian Thorpe (Australia) - 3 gold, 2 silver, 0 bronze  
• Pieter van den Hoogenband (Netherlands) - 2 gold, 0 silver, 2 bronze  
• Lenny Krayzelburg (USA) - 3 gold, 0 silver, 0 bronze  
• Inge de Bruijn (Netherlands) - 3 gold, 1 silver, 0 bronze  
  
Step 3: Determine the athlete with the most medals. After reviewing the list of individual athletes who won multiple medals in the 2000 Sydney Olympics, we can see that the athlete with the most medals is Ian Thorpe from Australia, who won a total of 5 medals (3 gold and 2 silver).  
  
Therefore, the ANSWER is: Ian Thorpe.

Self-Consistency

Generate multiple outputs and choose the most consistent or frequent response to mitigate biases and enhance reliability.

Examples:

  • When seeking historical facts, generate multiple explanations and select the most frequent or seemingly accurate one based on consensus.
  • Similarly, for creative tasks like story generation, generating multiple drafts and choosing the most engaging or consistent narrative can be beneficial.

Prompt:


Q: There are 15 trees in the grove. Grove workers will plant trees in the grove today. After they are done,
there will be 21 trees. How many trees did the grove workers plant today?
A: We start with 15 trees. Later we have 21 trees. The difference must be the number of trees they planted.
So, they must have planted 21 - 15 = 6 trees. The answer is 6.

Q: If there are 3 cars in the parking lot and 2 more cars arrive, how many cars are in the parking lot?
A: There are 3 cars in the parking lot already. 2 more arrive. Now there are 3 + 2 = 5 cars. The answer is 5.

Q: Leah had 32 chocolates and her sister had 42. If they ate 35, how many pieces do they have left in total?
A: Leah had 32 chocolates and Leah’s sister had 42. That means there were originally 32 + 42 = 74
chocolates. 35 have been eaten. So in total they still have 74 - 35 = 39 chocolates. The answer is 39.

Q: Jason had 20 lollipops. He gave Denny some lollipops. Now Jason has 12 lollipops. How many lollipops
did Jason give to Denny?
A: Jason had 20 lollipops. Since he only has 12 now, he must have given the rest to Denny. The number of
lollipops he has given to Denny must have been 20 - 12 = 8 lollipops. The answer is 8.

Q: Shawn has five toys. For Christmas, he got two toys each from his mom and dad. How many toys does
he have now?
A: He has 5 toys. He got 2 from mom, so after that he has 5 + 2 = 7 toys. Then he got 2 more from dad, so
in total he has 7 + 2 = 9 toys. The answer is 9.

Q: There were nine computers in the server room. Five more computers were installed each day, from
monday to thursday. How many computers are now in the server room?
A: There are 4 days from monday to thursday. 5 computers were added each day. That means in total 4 * 5 =
20 computers were added. There were 9 computers in the beginning, so now there are 9 + 20 = 29 computers.
The answer is 29.

Q: Michael had 58 golf balls. On tuesday, he lost 23 golf balls. On wednesday, he lost 2 more. How many
golf balls did he have at the end of wednesday?
A: Michael initially had 58 balls. He lost 23 on Tuesday, so after that he has 58 - 23 = 35 balls. On
Wednesday he lost 2 more so now he has 35 - 2 = 33 balls. The answer is 33.

Q: Olivia has $23. She bought five bagels for $3 each. How much money does she have left?
A: She bought 5 bagels for $3 each. This means she spent $15. She has $8 left.

Q: When I was 6 my sister was half my age. Now I’m 70 how old is my sister?
A:

Output 1:

When I was 6 my sister was half my age, so she was 3. Now I am 70, so she is 70 - 3 = 67. The answer is 67.

Output 2:

When the narrator was 6, his sister was half his age, which is 3. Now that the narrator is 70, his sister would be 70 - 3 = 67 years old. The answer is 67.

Output 3:

When I was 6 my sister was half my age, so she was 3. Now I am 70, so she is 70/2 = 35. The answer is 35.

Task Decomposition

Break down complex tasks into smaller, more manageable subtasks using techniques like prompt chaining.

Examples:

  • To solve a complex math problem, first ask the LLM to identify the problem type, then solve each part step-by-step, and finally, combine the parts to reach the solution.

  • Prompt Chaining for Code Generation:

    1. Prompt 1: “Write a Python function that takes a list of numbers as input.”
    2. Prompt 2 (based on output of Prompt 1): “Modify the function to calculate the sum of all numbers in the list.”
    3. Prompt 3 (based on output of Prompt 2): “Add functionality to the function to return the average of the numbers in the list.”

Step 4: Enhancing Capabilities and Control

System Prompt and Role

  • System Prompt: Define the LLM’s behavior, role, personality, and limitations.

  • Role Assignment: Assign a specific role to focus responses and tailor the language style.

Examples:

  • “You are a friendly nutritionist providing a balanced meal plan for someone with a gluten allergy.”
  • “You are a seasoned travel guide creating a one-week itinerary for exploring the historical sites of Rome.”

Retrieval Augmentation (RAG)

  • RAG: Combine LLMs with external knowledge sources like databases or APIs for improved accuracy and up-to-date information.

Examples:

  • “Using current stock market data from the XYZ API, analyze and predict next week’s trend for tech stocks.”
  • “Access the latest research papers on climate change from a scientific database and summarize the key findings and proposed solutions.”

Temperature and Top_p

  • Temperature: Control the randomness and creativity of the output.

  • Top_p: Adjust the probability distribution of the next token to influence the diversity of generated text.

Examples:

  • For creative writing, higher temperatures encourage more varied and unexpected plot twists. For factual reports, lower temperatures ensure more straightforward and consistent information.
  • For generating different creative text formats, like poems, scripts, or musical pieces, adjusting temperature and top_p can lead to diverse styles and structures within the chosen format.

References

https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/advanced-prompt-engineering
https://platform.openai.com/docs/guides/prompt-engineering
https://docs.anthropic.com/claude/docs/prompt-engineering
https://www.promptingguide.ai/