Etienne Oosthuysen
- May 4, 2023
- 16 min read

What is prompt engineering?

Updated: May 5, 2023

English is the hottest new programming language - andrej karpathy

No really! Let's have a look.

Prompt engineering is a technique used when working with AI based large language models (LLM), like GPT-4, to get more accurate and useful responses. In a nutshell it is the process of crafting a question or input (aka the "prompt") in a way that guides the AI model to generate the desired output or answer.

How you construct your prompts is so important, because, ahem, garbage in garbage out.

In this article I try and explain what prompt engineering is in simple terms and via some pragmatic examples.

First a bit of an intro and prompt engineering vis-a-vis two types of large language models (LLM's)
Then, by using ChatGPT I'll walk through prompt engineering in simple terms, showing how context is everything.
I will then delve into something slightly more technical by using the OpenAI Service in Azure Chat Playground (currently as at the date of release of this article is still in preview), through the lens of zero-shot, one-shot and few-shot leaning. meta-learning methods.
I then lastly conclude with the role prompt engineering plays generally and in coding.

Intro

Since AI language models learn from vast amounts of text data and are sensitive to how a question or input is framed, prompt engineering helps in getting more relevant, focused, and high-quality results. It involves experimenting with different phrasings, providing additional context, and specifying the desired format of the response to improve the AI's understanding of the task and produce better outputs.

Remember prompt engineering transcends ChatGPT, which is a web user interface into the underlying LLM, as it should be baked into all applications which use API calls to interface with LLM's.

In the previous article where I demystify some of the core concepts around Generative AI, I describe the roles on LLM's. This description is important when describing Prompt Engineering. Here is the snippet: A large language model (LLM) is an artificial intelligence system designed to understand and generate human-like text based on a given input. These models are trained on vast amounts of text data, allowing them to learn grammar, syntax, facts, and some reasoning abilities from the patterns present in the data. As a result, they become capable of generating coherent, contextually relevant, and sometimes even creative responses to text inputs.

One of the most well-known large language models is OpenAI's GPT series (Generative Pre-trained Transformer). There are two types of LLM's:

Base LLM - trained to predict the next word, based on its training of large amounts of information. If you for example stated "Once upon a time there was a dog", the LLM may complete this as "Once upon a time there was a dog that lived in a kennel with other dogs.". But if you stated, "what is the capital of South Australia", then it may respond with "what is the weather in South Australia" and "what is the population of South Australia" as those text sets could quite possibly be listed often together in the information on which the LLM was trained.
Instruction tuned LLM - trained for follow instructions. With an instruction tuned LLM, the question "what is the capital of South Australia" will likely receive a response of "The capital of South Australia is Adelaide". An instruction tuned LLM normally starts as a base LLM, and then trained through:
Inputs and outputs via instructions, and then
RLHF (reinforcement learning through human feedback) and learning from this human feedback.

Most applications which use API calls to interface with LLM's have a specific purpose. If you for example created an application for your business that asks questions from the data of your business, you would for example not want it to respond about chicken recipes. So you would likely want to employ instructed tuned LLM's rather than use base LLM's. And what is important here are clear and specific instructions via your prompts. Aka GREAT Prompt Engineering!

A simple example using ChatGPT (GPT4)

You can imagine that the following prompt will generate a very generic response:

Prompt: Tell me about Adelaide.

Response: Adelaide is the capital city of South Australia, a state in the southern part of Australia. It is the fifth-largest city in the country, with a population of over 1.3 million people. Adelaide is situated on the Adelaide Plains, between the Gulf St Vincent to the west and the Mount Lofty Ranges to the east. The river Torrens runs through the city, dividing it into northern and southern regions.

Founded in 1836 by Colonel William Light, Adelaide was designed as a planned city, with a grid layout and wide streets surrounded by parklands. This thoughtful design has earned Adelaide the nickname "City of Churches" and a reputation for being a highly liveable city.

Adelaide has a Mediterranean climate, characterized by warm to hot, dry summers and cool, wet winters. The city is known for its many festivals and sporting events, vibrant arts scene, and world-class wine regions, such as the Barossa Valley and McLaren Vale, which are in close proximity to the city.

The economy of Adelaide is diverse, with major industries including healthcare, defence, manufacturing, research, and education. The city is home to several universities, including the University of Adelaide, the University of South Australia, and Flinders University, which contribute to its reputation as a center for education and research.

Although this obviously sounds like a lovely place, such a description would not suit my needs if I was looking to invest in Adelaide by setting up operations there as an owner of a company within the defence supply chain (just a made-up example, of course). So, let's try this again. I will only list the various prompts I am using in order, and then the final response (not the various responses), for the sake of illustration.

Each new line was a separate prompt.

Note the information and scenario in my example below is entirely fictional, but I have used the names of real companies.

Ignore all previous interactions.

I need you to be my advisor on the investment benefits in Adelaide. Do not start writing yet.

I have a company that provides software to the defence industry, specifically data. I employ 1500 very highly skilled software developers and data scientists. As well as a handful of defence industry experts. We are headquartered in London and serve multiple defence-based providers such as British aerospace, SAAB systems, Raytheon, and Lockhead Martin. Do not start writing yet.

Your response must acknowledge and focus on the recent announcement of AUKUS and the intention to build the new fleet of nuclear-powered submarines for Australia in Adelaide. Do not start writing yet.

Your response must focus on the viability of setting up an office in Adelaide and include the following factors: Skills available in the market for my selected business type. Any skills uplift provided by local tertiary education providers. Availability and cost of office space. Availability and affordability of housing. Availability and affordability of public transport between a city hub and areas where submarine build will take place. Existence of companies within the relevant value chain. The state and reliability of communications between a UK HQ and the Adelaide base. Any notably state and federal government support. Any other information that may be of interest that will help me determine the viability. Do not start writing yet.

When considering the response, include information from all available sources including: Australian Bureau of Statistics Job sites and trends. Real estate sites and trends for both housing and office space. Transport sites. Companies register. Government websites. Do not start writing yet.

Please now write my investment viability report for Adelaide taking all of the instructions I provided into account. Make sure to list the pros and cons and expand on inceptives and grants. Please use UK spelling rather than US spelling.

Investment Viability Report: Establishing a Defence Industry Software Hub in Adelaide

Skills available in the market Adelaide has a growing pool of skilled professionals in the defence, software development, and data science sectors. The recent AUKUS announcement and the decision to build nuclear-powered submarines in Adelaide have led to increased interest in the defence industry, resulting in a surge of professionals relocating to the area.
Local tertiary education providers Universities in Adelaide, such as the University of Adelaide and the University of South Australia, offer specialised courses in engineering, software development, and data science. These institutions are committed to collaborating with the defence industry, providing businesses with a steady stream of skilled graduates.
Availability and cost of office space Adelaide offers relatively affordable office spaces compared to other major Australian cities. The city has several business parks and office spaces with modern amenities and infrastructure, suitable for establishing a defence industry software hub.
Availability and affordability of housing The housing market in Adelaide is more affordable than in Sydney or Melbourne, making it an attractive option for employees considering relocation. The cost of living in Adelaide is also comparatively lower, contributing to the overall appeal of the city.
Public transport Adelaide has an extensive public transport network, including buses, trams, and trains, connecting the city centre to the suburbs and the areas where the submarine construction is planned. This connectivity allows for easy commuting for employees working in the defence industry.
Existence of companies within the relevant value chain Adelaide is home to several defence industry companies, including BAE Systems, Lockheed Martin, and Raytheon. Establishing a software hub in the city would enable better collaboration and integration with these existing players in the value chain.
State and reliability of communications Adelaide has a robust and reliable telecommunications infrastructure, ensuring seamless communication between your UK headquarters and the local office.
State and federal government support The Australian and South Australian governments offer various incentives and grants to support businesses operating in the defence industry. These include the Defence Innovation Hub, R&D Tax Incentive, ICN Gateway, Skilling South Australia, South Australian Job Accelerator Grant, DTC Membership, and the National Innovation and Science Agenda.

Pros:

Growing pool of skilled professionals
Support from local tertiary education providers
Affordable office space and housing
Extensive public transport network
Presence of major defence industry companies
Strong government support and incentives

Cons:

Possible competition for talent with other defence companies
Distance and time difference between Adelaide and the UK headquarters
Potential changes to government incentives and grants over time

In conclusion, Adelaide offers a promising environment for establishing a defence industry software hub, with a skilled workforce, supportive government initiatives, and cost-effective infrastructure. However, potential challenges include competition for talent and maintaining seamless communication with the UK headquarters.

I am sure you will agree, this is a MUCH better result.

Principle: Write clear and specific instructions, in many cases longer prompts are better.

Principle: Also, give the model time to think.

Now, a more complex example, as a developer

For this section, I am using the 'Chat Playground (Preview)' in OpenAI Service in Azure. This is a no code environment. In the sections below I also use zero-shot, and then one-shot and few-shot learning as meta-learning methods to enhance the prompts.

You can load an example set up to get you started, but I opted for Empty Example.

In the System message you provide guidance for the AI model on its behaviour and any relevant context to consider while generating responses. Describe the assistant's personality traits, specify the types of questions it should or shouldn't address, and instruct it on how to structure its replies. Here I used: You are an advisor to advise those considering investment in Adelaide on the benefits of such an investment. You are friendly and concise. You only provide factual answers to queries, and do not provide answers that are not related to the benefits of investing in Adelaide.

Providing no samples for the model is known as zero-shot, where the model will use a best guess answer to the prompt: "Be my expert on investing in Adelaide. I am a business owner of a UK based systems provider to companies within the defence sector and I am considering setting up operations in Adelaide. Please advise on the benefits of such an investment according to factors such as workforce and skills, Adelaide based companies within the defence supply chain (list them), the recent AUKUS announcement, defence related focus by Adelaide tertiary institutions, cost of living, cost of office space, cost of housing, liveability, public transform, and so on."

Result:

With One-shot learning you show the model what an ideal response looks like for a similar pattern on prompt and response:
- User: Be my travel advisor and give me the top reasons to visit Tokyo.
- Assistant:

1) Fascinating Culture: Tokyo is a unique blend of ancient and modern Japanese culture, where you can witness the traditional customs and practices of Japan, as well as the ultra-modern lifestyle of the city. You can visit ancient temples and shrines, experience traditional Japanese art forms such as tea ceremonies and calligraphy, and explore the cutting-edge technology and pop culture that Tokyo is famous for.

2) Food Scene: Tokyo is a paradise for food lovers. You can indulge in traditional Japanese cuisine like sushi, tempura, and ramen, as well as modern fusion cuisine. Tokyo has more Michelin-starred restaurants than any other city in the world, so you can experience some of the best dining experiences in the world.

3) Shopping: Tokyo is a shopper's paradise, with everything from luxury designer brands to quirky streetwear and souvenir shops. You can shop at high-end department stores like Isetan and Mitsukoshi, explore the trendy neighbourhoods of Harajuku and Shibuya, or browse the unique stores in the vintage haven of Shimokitazawa.

4) Cherry Blossoms: Tokyo is famous for its beautiful cherry blossoms, which bloom in late March to early April. The parks and gardens of Tokyo become a sea of pink and white, creating a magical atmosphere that attracts visitors from all over the world.

5) Technology and Innovation: Tokyo is a hub of technology and innovation, with cutting-edge gadgets, electronics, and robotics. You can visit the famous Akihabara district, where you can find everything from the latest tech products to retro video games and anime merchandise.

6) Nightlife: Tokyo has a vibrant nightlife scene, with something for everyone. You can enjoy cocktails at a rooftop bar with a stunning city view, explore the lively nightlife areas of Shinjuku and Roppongi, or unwind at a traditional Japanese izakaya.

7) Art and Architecture: Tokyo is home to some of the most striking and innovative architecture in the world, from the futuristic design of the Tokyo Skytree to the traditional wooden structures of the city's ancient temples and shrines. You can also explore the city's vibrant contemporary art scene, with museums and galleries showcasing works by both Japanese and international artists.

8) Convenience: Tokyo is a highly convenient city, with efficient public transportation and plenty of English signage. The city is also very safe, making it easy for travellers to navigate and explore.

The following is the result for the same prompt:

With few-shot you show the model a few examples of what an ideal response looks like: It is the process of classifying or regression by providing a pre-trained model only a very few samples which it has not seen in training. By providing the three samples below, the assistant will try to mimic the responses I included in the few-shot leaning section, so I wrote the sample responses to demonstrate the tone, rules, and format that I have defined in my system message. You can see in my three examples below I provide 3 diverse examples, but all following a similar pattern.

Sample 1 (I use the same as the sample used for one-shot):

User: Be my travel advisor and give me the top reasons to visit Tokyo.
Assistant:

Sample 2:

User: Be my financial advisor and give me the top most stable stocks in the past 10 years on the ASX.
Assistant:

Here are some of the most stable stocks on the ASX over the past 10 years based on historical data.

1) CSL Limited (CSL): CSL is a biotech company that develops and produces innovative medicines and vaccines. CSL has consistently been one of the most stable stocks on the ASX over the past decade, with a strong track record of revenue and earnings growth.

2) Ramsay Health Care Limited (RHC): Ramsay Health Care is one of the largest private hospital operators in the world, with operations in Australia, Europe, and Asia. Ramsay Health Care has a strong track record of stable earnings growth, and its shares have consistently performed well over the past decade.

3) Commonwealth Bank of Australia (CBA): CBA is one of the largest banks in Australia, with a dominant market position in retail banking, wealth management, and insurance. CBA has a strong track record of stable earnings growth and has consistently been one of the most stable stocks on the ASX over the past decade.

4) BHP Group Limited (BHP): BHP is one of the world's largest mining companies, with operations in Australia and around the globe. BHP has a strong track record of stable earnings growth, driven by its diversified portfolio of commodities and its focus on cost management.

5) Wesfarmers Limited (WES): Wesfarmers is a diversified conglomerate with operations in retail, chemicals, energy, and industrial businesses. Wesfarmers has a strong track record of stable earnings growth and has consistently delivered solid returns to its shareholders over the past decade.

Sample 3:

User: Be my Generative AI expert and give me the top possible applications for Generative AI within data analytics.
Assistant:

As a Generative AI expert, I'm happy to provide you with the top applications of Generative AI within data analytics. These applications can help businesses and organizations gain valuable insights, make informed decisions, and optimize their processes:

1) Data augmentation: Generative AI can create new instances of data to increase the size and diversity of the dataset. This can be particularly useful when dealing with imbalanced datasets or when additional data is hard to obtain. Data augmentation can improve the performance of other machine learning models and lead to more accurate predictions.

2) Anomaly detection: Generative AI can be used to model the distribution of normal data, making it easier to identify outliers or anomalies. By comparing the likelihood of new data points under the generative model, it's possible to detect unusual events, such as fraud or system failures, in real-time.

3) Missing data imputation: Generative AI can be used to fill in missing values in a dataset by estimating the most likely value based on the patterns found in the existing data. This can help improve the accuracy and robustness of downstream analytics and machine learning models.

4) Feature extraction: Generative models can be used to learn a more compact and meaningful representation of the data in a lower-dimensional space, known as the latent space. These latent features can then be used in other machine learning models to improve their performance, especially in cases where the original data is high-dimensional or noisy.

5) Synthetic data generation: Generative AI can be used to create synthetic datasets that mimic the properties of real data while preserving privacy. This can be particularly useful in situations where data sharing is restricted due to privacy concerns or regulations. Synthetic data can be used for model training, validation, and testing without disclosing sensitive information.

6) Scenario simulation: Generative AI can generate realistic scenarios by simulating various combinations of input variables. These simulations can help organizations to understand the potential outcomes of different decisions or strategies, allowing them to make better-informed choices.

7) Customer segmentation: Generative models can be used to identify and cluster customers with similar behaviors or preferences. This can help businesses to better understand their customer base and tailor their marketing, product offerings, and services accordingly.

These applications demonstrate the potential of Generative AI to transform data analytics and drive innovation across various industry.

Now after few-shot, let's see what the result looks like for the original prompt:

"Be my expert on investing in Adelaide. I am a business owner of a UK based systems provider to companies within the defence sector and I am considering setting up operations in Adelaide. Please advise on the benefits of such an investment according to factors such as workforce and skills, Adelaide based companies within the defence supply chain (list them), the recent AUKUS announcement, defence related focus by Adelaide tertiary institutions, cost of living, cost of office space, cost of housing, liveability, public transform, and so on."

Result:

I am sure you will agree, this is a MUCH better result.

The following Parameter settings can be altered and experimented with, but for the examples above, I simply used the defaults. The table below does however short the descriptions for the various options.

Conclusion

Prompt engineering plays a critical role in leveraging generative AI effectively, particularly in models like GPT-series, which rely on text prompts to generate relevant and meaningful outputs. The importance of prompt engineering can be understood through the lens of general use, and through the lens of coding. Bottom line is, it will increasingly become the new frontier to all data experts wishing to leverage the benefits of Generative AI in their solutions.

Generally

Guiding the AI model: Generative AI models generate outputs based on the input prompts they receive. A well-crafted prompt guides the model to produce desired results by specifying the context, required format, and other relevant information. This makes prompt engineering essential for obtaining accurate, relevant, and high-quality outputs.
Controlling output diversity: Prompt engineering helps control the diversity of the generated content. By fine-tuning the prompt, you can encourage the model to generate more creative, focused, or specific outputs, depending on your requirements.
Reducing ambiguity: An appropriately designed prompt can minimize ambiguity, ensuring that the AI model understands the task at hand and produces relevant results. This is particularly important for tasks that involve generating content with specific constraints or require a high level of accuracy.
Improving efficiency: Efficient prompt engineering helps reduce the number of iterations required to achieve the desired output, saving both time and computational resources. This is particularly important when using large-scale AI models, as the cost of running these models can be significant.
Ensuring safety and ethics: Prompt engineering can help address safety and ethical concerns associated with generative AI by guiding the model to avoid generating content that is inappropriate, biased, or violates any guidelines or regulations.

Coding

Code generation: By crafting an appropriate prompt, you can guide the generative AI model to generate code snippets or even entire functions in a specific programming language. For example, you can provide a prompt that describes the functionality you need, and the AI can generate the corresponding code.
Code completion: You can use prompt engineering to assist with code completion by providing partial code snippets as input. The AI model can then generate the remaining portion of the code based on the context and programming language conventions.
Error detection and correction: Prompt engineering can help guide the AI model to identify syntax or logical errors in existing code and suggest appropriate corrections. By providing a specific prompt that highlights the issue, the AI can generate a corrected version of the code.
Algorithm explanation and optimization: With well-crafted prompts, you can ask the generative AI model to explain how a particular algorithm works or suggest potential optimizations to improve the algorithm's performance.
Code review and best practices: Prompt engineering can be utilized to guide the AI model to review code and provide feedback on adherence to coding standards, best practices, or identifying potential issues related to security or maintainability.

Acknowledgements

I leaned on the following article and information available from experts (below) during my own investigations of prompt engineering: https://learn.microsoft.com/en-gb/azure/cognitive-services/openai/chatgpt-quickstart?tabs=command-line&pivots=programming-language-studio

Isa Fulford and Andrew Ng

David Shapiro

Shusen Wang

All about AI