Prompts for the Data Engineer
AI prompts for data engineers: SQL, schema design, pipeline docs, and data quality checks.
Data engineering is full of work that's necessary, fiddly, and easy to get subtly wrong: SQL that has to be both correct and performant, schema designs that will haunt you if they're not normalized properly, regexes for parsing messy inputs, and the data-quality checks nobody enjoys writing. ChatGPT and Claude are strong here because they handle the boilerplate and the syntax recall, freeing you to focus on correctness and architecture.
This collection targets that daily grind. There are prompts for turning a business question into clean, commented SQL, designing a fully normalized schema with DDL and indexes, writing advanced window-function queries with performance notes, and building and explaining a regex with passing and failing test cases. On the pipeline side, there's a Pandas data-cleaning script prompt with before/after summaries and a production-ready Dockerfile prompt explained line by line.
Prompting well matters because in data work, a plausible-looking answer can be quietly wrong. Giving the model your real schema and constraints, and asking it to explain its reasoning, is how you catch problems before they hit production.
What makes a good prompt for a data engineer
The best data-engineering prompts hand the model your actual schema, column types, data volumes, and the engine you're on. A SQL prompt that knows your table structure and that you're on Postgres versus BigQuery produces query syntax and performance advice that's actually correct, not generic. The window-function prompt's performance notes are far more useful when the model knows your data size and indexing.
Always ask for explanation and verification artifacts. The regex prompt that returns passing and failing test cases lets you confirm the pattern actually works, and the cleaning-script prompt's before/after summaries let you validate the transformation. In a domain where wrong answers look convincing, those checks are the whole point.
Get sharper results
- 01Paste your real schema, column types, and target engine (Postgres, BigQuery, Snowflake) into SQL prompts, since dialect and structure dramatically change both correctness and the performance advice you get back.
- 02For regex work, use the prompt's passing and failing test cases to actually verify the pattern against your edge cases instead of trusting that it looks right.
- 03When generating window-function queries, give the model your approximate data volume and existing indexes so its performance notes reflect your reality, not a toy table.
- 04Run the data-cleaning script prompt's before/after summaries against a sample before applying transformations at scale, so you catch a misread column or dropped row early.
Common questions
Can I trust AI-generated SQL to be correct and performant?
Trust but verify, and feed it context. With your real schema and engine specified, the model produces solid, commented queries, but you should run them against sample data and check the query plan before shipping anything that touches production. A query that returns plausible-looking rows can still be subtly wrong, so validate the output, don't just eyeball the syntax.
How do I make sure a generated regex actually handles my edge cases?
Use the test cases the prompt produces and add your own. The regex prompt returns passing and failing examples and a line-by-line breakdown, so you can confirm it matches what you want and rejects what you don't. Then throw your real messiest inputs at it before wiring it into a pipeline.
Is it worth using AI for schema design, or is that too high-stakes?
It's a good thinking partner for it precisely because the stakes are high. The normalized-schema prompt gives you DDL, indexes, and design rationale you can critique, which surfaces tradeoffs early. Treat it as a strong first draft to review against your access patterns and scale, not a final design to apply blindly.
Write a SQL Query From a Business Question
Translate a business question into a clean, commented SQL query against your schema.
Build and Explain a Regular Expression
Get a working regex with a plain-English breakdown and a set of passing/failing test cases.
Design a Normalized Database Schema
Design a fully normalized relational database schema with DDL, indexes, and design rationale.
Analyze a Dataset With Pandas Step-by-Step
Generate step-by-step Pandas EDA code covering nulls, outliers, and a business question.
Write a Data Cleaning Script for Messy Data
Generate a step-by-step Pandas data cleaning script with issue detection and before/after summaries.
Use SQL Window Functions for Advanced Analytics
Generate advanced SQL window function queries with explanations and performance notes.
Generate a Production-Ready Dockerfile and Explain Each Line
Generate a hardened, multi-stage Dockerfile with line-by-line comments and matching .dockerignore.
Port a code snippet to another language idiomatically
Translates a snippet into idiomatic code in a target language and flags cross-language correctness pitfalls.
Design a normalized database schema from plain requirements
Turns plain-language requirements into a normalized relational schema with DDL, relationships, and trade-offs.
Diagnose and rewrite a slow SQL query using its plan
Reads a query plan to explain why a SQL statement is slow, then rewrites it and recommends indexes.
Turn a Plain-English Question Into a SQL Query
Converts a natural-language business question into a correct, commented SQL query grounded in your real schema.
Write a Pandas Snippet for a Specific Data Transformation
Generates a vectorized, commented pandas snippet for a precise transformation on your actual DataFrame.
Write a Precise, Unambiguous Metric Definition
Produces a precise metric definition with formula, grain, inclusion rules, and resolved edge cases.
Build a Data-Cleaning Checklist for a New Dataset
Generates a prioritized, column-specific data-cleaning checklist tailored to your dataset and intended analysis.