Artificial Intelligence and Large Language Models (LLMs) like GPT-4, Claude, and Gemini are incredible at generating text and writing code. But as any prompt engineer knows, LLMs are fundamentally non-deterministic. If you are building an AI agent or a data pipeline, you need a deterministic way to extract structured information from those unpredictable text outputs.

This is where regular expressions (regex) come in.

While AI can write the text, regex is the glue that safely extracts JSON, code blocks, and specific variables from LLM responses into your application logic. In this guide, we will cover the best regex patterns for AI prompt engineering and data extraction.

1. Extracting JSON from LLM Outputs

A very common prompt engineering pattern is asking the LLM to "output your response in JSON format." However, the LLM will often wrap the JSON in markdown formatting (like ```json) or include conversational filler text before and after the object.

To reliably extract the JSON object from the string, you can use the following regex pattern:

/\{(?:[^{}]|(?R))*\}/s

Note: The (?R) syntax represents a recursive regex pattern, which is supported in PCRE engines (like PHP) but not natively in JavaScript. For a more standard JS-compatible approach to grab the first JSON-like block, developers often use:

const text = "Here is your JSON: {\"key\": \"value\"} hope it helps!";
const match = text.match(/\{[\s\S]*\}/);
if (match) {
  const jsonString = match[0];
  // Parse the JSON securely...
}

If you're using Python for your AI backends, check out our Python Regex Cheat Sheet & Pattern Matching guide for more specific re module implementations.

2. Extracting Markdown Code Blocks

If you are building a coding copilot or an AI agent that generates code, you need to extract the raw code from the markdown code blocks.

Here is the perfect regex pattern to match a markdown code block and capture the language and the code separately:

/```(\w+)?\n([\s\S]*?)\n```/g

How it works:

``` matches the literal backticks.
(\w+)? is a capturing group that matches the optional programming language identifier (e.g., python, javascript).
\n matches the newline.
([\s\S]*?) is a non-greedy capturing group that matches the actual code content.
\n``` matches the closing backticks.

You can instantly test this pattern and see the captured groups in our Interactive Regex Tester.

3. Extracting Thoughts and Reasoning (`<think>` tags)

Modern reasoning models (like DeepSeek or customized agents) often output their chain-of-thought inside <think> or <reasoning> XML tags before providing the final answer.

To separate the AI's internal monologue from the final output, use:

/<think>([\s\S]*?)<\/think>/i

This pattern uses the i flag for case-insensitivity. If you only want to extract the final answer that comes after the think tags, you might want to leverage lookbehinds. If you are unfamiliar with lookbehinds, read our tutorial on Mastering Lookarounds in Regex.

4. Stripping Conversational Filler

If you prompt an AI to return a specific ID or an email address, it might stubbornly reply with: "Sure! Here is the email address you requested: test@example.com."

Instead of trying to parse the sentence, use specific data extraction regex patterns:

Extracting Emails: \b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b
Extracting URLs: https?:\/\/(www\.)?[-a-zA-Z0-9@:%._\+~#=]{1,256}\.[a-zA-Z0-9()]{1,6}\b([-a-zA-Z0-9()@:%_\+.~#?&//=]*)

Conclusion

Prompt engineering isn't just about writing good instructions; it's about building robust pipelines that handle unpredictable outputs. By integrating these regular expression patterns into your AI workflows, you can guarantee that your application parses data safely and efficiently.

Want to test your own extraction patterns? Head over to our Regex Sandbox and start debugging your LLM parsers today.

Best Regex Patterns for AI Prompt Engineering & Data