Skip to content
León Palafox

Vibe coding · data science 規則

Eight rules for vibe coding in data science

An assistant will write a whole notebook from one sentence. Whether the number at the bottom means anything depends on what that sentence left out. These are the rules I teach for closing those gaps before any code exists.

One vague prompt, many notebooks

Send the prompt below to three assistants, or to the same one three times. You get three notebooks back, and all of them run.

The prompt 
Build a classifier for this CSV:
https://…/data.csv

None of it is wrong code. Each notebook answers a different question, and none of them is the question you had in mind. The assistant filled every gap in the prompt with a guess. The rules below take the guesses away, one gap at a time.

Answer A

“Best model wins”

  • Gradient boosting, with scaling
  • SMOTE oversampling
  • Accuracy as the only metric
  • No way to explain a prediction

Answer B

“Quick and simple”

  • Logistic regression
  • Categories label-encoded as 0, 1, 2…
  • Codes treated as numbers
  • One split, one number

Answer C

“Looks rigorous”

  • Random forest with a long parameter list
  • Tuned by looking at the test score
  • Threshold picked on test
  • A score that will not hold up

Rule 01

Name the environment

The same request comes back as notebook cells in Colab and as a script on a laptop. Say where the code will run and what the answer has to look like: cells or files, how packages get installed, where plots go.

Vague 
Write the code to train a model on this data.

Notebook code pasted into a script, or the reverse, is the usual reason a first answer does not run.

In the prompt 
ENVIRONMENT
- Google Colab notebook, Python 3. pandas, scikit-learn and matplotlib are already installed.
- Give the code as separate cells, with a short markdown cell before each one.
- Install extra packages only if essential, with !pip install in the first cell.
- Read the data from its URL. No Google Drive, no local paths.
- Show plots inline.

Try it

Change where the code runs. Only this block of the prompt changes.

Browser notebook with an assistant built in

Output
Notebook cells, each with a markdown cell above it
Installs
!pip, and only if the package is missing
Files
A folder under /content
Watch for
A runtime reset loses every variable and file

Rule 02

Give a data contract

Say what the data is: where it lives, how many rows and columns, which column is the target and the type each column really has. Anything you leave out gets guessed, and the guesses are worst on columns that look like numbers and are not.

Vague 
Here is a CSV with customer data.
In the prompt 
DATA
- Source: <URL or path>. <rows> rows, <columns> columns.
- Target: "<column>" (<its values, and which one is the positive class>).
- "<column>" is a code, not a quantity: treat it as categorical.

Try it

Switch on what your table contains. Each one adds a line to the contract.

Left to a guess · Codes that look like numbers

Zip codes, area codes and store numbers get read as quantities, so 90210 counts as more than 10001.

Rule 03

Fix the model and the search space

Without limits the assistant reaches for its favorite model, adds preprocessing nobody asked for and tunes whatever it likes. Name the model family, write the grid yourself and list what is off the table. Strict limits also keep results comparable when you run the same prompt in two tools.

Vague 
Use the best model.

You decide the search space. The assistant writes the code that searches it.

In the prompt 
MODEL SETUP (hard constraints)
- Use ONLY scikit-learn DecisionTreeClassifier.
- Search exactly this grid with GridSearchCV:
  - max_depth: [2, 3, 4, 5, 6, 8, 10, 12]
  - min_samples_leaf: [1, 5, 10, 20]
  - criterion: ["gini", "entropy"]
  - class_weight: [None, "balanced"]
- No other models, no feature scaling, no resampling, no PCA.
- random_state=42 everywhere.

Try it

Switch values on and off. The grid in the prompt follows, and so does the bill.

max_depth

min_samples_leaf

criterion

class_weight

cross-validation folds

8 × 4 × 2 × 2 = 128 combinations, each fitted on 5 folds: 640 model fits.

Rule 04

Split first, fit on train

Leakage starts as soon as anything learns from test rows. The split comes before encoding, scaling, exploring or tuning, and every transformation is fitted on train only.

Vague 
Preprocess the data and train a model.
In the prompt 
VALIDATION RULES
- Split first: stratified 80/20 train/test with random_state=42, before any preprocessing.
- Fit every transformation (imputers, encoders, scalers) on train only.

Try it

Which rows each step gets to see.

Fit imputers, encoders, scalers Fitted on every row, then split
used used
Explore and select features Correlations computed on the full table
used used
Tune hyperparameters Fitted on train, compared by test score
used used
Pick the decision threshold A few cutoffs tried on test
used
Report the final number The same rows, for the fifth time
used

The test rows took part in four decisions and then graded the result.

Rule 05

Fix how to tune

“Tune it” is an invitation to peek at the test set. Say which grid, which cross-validation and which score, and say where the decision threshold comes from. The test set gets opened once.

Vague 
Optimize the hyperparameters.

If a number helped you choose, it cannot also be the number you report.

In the prompt 
VALIDATION RULES
- Tune with GridSearchCV, 5-fold StratifiedKFold on train, scoring="roc_auc".
- Choose the decision threshold on out-of-fold predictions, never on test.
- Use the test set exactly once, at the end.

Try it

What choosing by test score does to a model with no skill at all.

Every configuration here guesses at random on balanced classes, so its true accuracy is 50%. Score them all on the same test set and report the best one.

One configuration The one you would report True accuracy, 50%
Reported
–
True accuracy
50.0%
Typical best
–

Typical best: mean over 1,500 simulated runs, normal approximation to the binomial.

Rule 06

Define success

On data where one row in twenty is positive, a model that always answers “no” is 95% accurate. State the goal in business terms, the metrics that reflect it and the baseline any model has to beat.

Vague 
Make it accurate.
In the prompt 
SUCCESS
- Goal: catch the positive cases, even at the cost of some false alarms.
- Report recall and precision for the positive class, ROC AUC and the confusion matrix.
- Always show the majority-class baseline next to accuracy.

Try it

A model that predicts “negative” for every row, on 1,000 rows.

Accuracy
95%
Positives caught
0 of 50

On this data, any model below 95% accuracy is worse than doing nothing, and one at 95% may have caught no positives at all.

Rule 07

Shape the output

Output you can read and rerun is part of the spec. Ask for a short explanation before each block of code, results that were actually printed, and no new libraries without asking.

Vague 
(nothing said)

A number in the prose that no cell printed is a claim. Ask for the cell.

In the prompt 
WORKING STYLE
- Before each code block, 2–3 lines on what you will do and why.
- Print every key number. Never invent results.
- Ask before adding a library.
- In every answer, list the files you created or changed.

Rule 08

One step at a time

Small steps can be reviewed. Ask for one step per answer and check it before you type “next”. When a rule is broken, name it and ask for the smallest fix. Do not start over.

Vague 
Do everything end to end.
In the prompt 
WORKING STYLE
- Do ONE step per answer and wait for me to say "next".
- If I say a rule is broken, redo only the affected code and keep everything else.

Try it

Say which rule broke and what you saw. Send back the result.

Correction prompt 
That answer breaks my rules under MODEL SETUP: you added StandardScaler.
Keep everything else exactly as it is.
Redo only the affected code, explain the change in 2 lines, and show the new output.

The review checklist writes this for you from the boxes you mark as broken.

Takeaway

Constrain the what. Let the model write the how. Check everything in between.

Review catches errors after the code is written. Rules keep them from being written. You need both, so the guide has a tool for each.

Before the code

Prompt builder

Fill in your environment, data, model and metric. It assembles the setup prompt with all eight rules in it, ready to copy or save as a file.

Open

After every answer

Review checklist

Thirteen checks to run before you type “next”. Mark what broke and it writes the correction prompt.

Open