Prompt engineering refers to the process of crafting messages to successfully steer an LLM toward executing a particular task. Since prompts are user-supplied input, such input may include malicious instructions that cause unintended behaviors in the model. For instance, an attacker can guide a model to output malicious instructions such as participating in illegal activities, crafting and distributing malicious software, creating websites that promote racist activities, etc.