Data Before Opinion: How a Data Collection Plan Keeps You From Measuring the Wrong Thing
🕓  

6

  MIN READ

Data Before Opinion: How a Data Collection Plan Keeps You From Measuring the Wrong Thing


🎯 Quick Answer

What: A Data Collection Plan is a one-page agreement, written before any data is gathered, that defines what you measure, how, who collects it, how often, and how you will know it can be trusted.

Why it matters: Data collected without a plan is inconsistent, and inconsistent data cannot be defended later. Most “data arguments” are really disagreements about a definition nobody wrote down.

How to apply: Answer the five questions for your most important metric, and write a one-sentence operational definition for each measure before you collect anything.

The payoff: Your conclusions become proof instead of opinion, and one hour of planning saves weeks of re-collecting and re-arguing.

Most improvement projects do not fail in the analysis. They fail earlier, in a quiet moment nobody flags: the moment the team starts collecting data before anyone has agreed what good data even looks like.

It feels productive. Numbers are going into a spreadsheet, a chart is taking shape, the project is “moving.” But if the metric is fuzzy, the method is inconsistent, and two people are measuring the same thing in two different ways, you are not building evidence. You are building an argument you will lose the moment someone asks a hard question.

The fix is unglamorous, almost boring. It is a one-page Data Collection Plan, filled out before a single number is gathered. I have watched it quietly save more projects than any clever statistical method.

What a Data Collection Plan actually is

A Data Collection Plan is a simple agreement, written down before collection starts, that answers a handful of questions about every number you intend to gather. That is it. It is not software. It is not complicated. It is a page that turns “let us go get some data” into “here is exactly what we will measure, how, and how we will know we can trust it.”

The reason it matters is that data collected without a plan is almost always inconsistent, and inconsistent data cannot be argued with later. By the time you notice, you have weeks of numbers nobody believes.

The five questions it forces

A good plan answers five questions before anyone touches a measurement.

  • What are you measuring? The specific metric, tied directly to the project goal. Not “quality” or “speed,” but the exact number you will track.

  • How will you measure it? The method, and the operational definition behind it. More on that in a moment, because it is the part most teams skip.

  • Who collects it? One named owner per measure. A measure without an owner does not get collected, it gets assumed.

  • How often, and how much? Frequency and sample size, decided up front rather than left to whoever remembers.

  • How will you know you can trust it? The data-quality check that keeps bad numbers out before they ever reach your analysis.

None of these are difficult. The discipline is in answering them first, in writing, where the whole team can see and agree.

The column everyone skips: the operational definition

If I could get every team to fix one thing, it would be this. An operational definition is a precise, shared description of what a measurement means, specific enough that two different people, measuring the same thing, get the same answer.

“On time” is the classic trap. On time to the scheduled date, or the promised date? Measured when the work leaves your hands, or when the customer receives it? Counting business days or calendar days? Until you define it, “on time” is not a metric. It is a feeling, and everyone in the room is quietly using a different version of it.

Most of the data arguments I have sat through were not really about the data. They were about two people who never agreed on the definition. The Data Collection Plan forces that agreement into the open, before it can cost you a month.

Why this is cheaper than it looks

Spending an hour on a Data Collection Plan feels like a delay when the team is eager to act. It is the opposite. The expensive version is collecting first and discovering later that half your numbers are not comparable, then re-collecting, then re-arguing. The plan trades one boring hour now for weeks you do not lose later. It is the cheapest hour in the whole project.

It also changes the conversation at the end. When you can say exactly what you measured, how, and how you checked it, your conclusion stops being an opinion someone can wave away. It becomes proof.

Where to start

You do not need a tool to begin. Open a page and answer the five questions for the single most important metric in your current project. If you cannot answer all five cleanly, you have just found the reason your last set of numbers started an argument instead of ending one.

If you want the format already built, with a worked example so you can see what a good one looks like, I keep a free starter version with the rest of my field tools in my newsletter. Measure the spread before you touch the mean, and define the measure before you touch the data.

Frequently asked questions

What is a Data Collection Plan in Lean Six Sigma?

It is a one-page document, completed before data collection begins, that specifies what you will measure, the method, the owner, the frequency and sample size, and how the data will be checked for trustworthiness. It turns a vague “let us gather some data” into a precise, agreed plan.

What are the five questions a Data Collection Plan answers?

What are you measuring, how will you measure it, who collects it, how often and how much, and how will you know you can trust it. Answering all five in writing, before collection starts, is what separates evidence from guesswork.

What is an operational definition, and why does it matter?

It is a precise, shared description of what a measurement means, specific enough that two people measuring the same thing get the same answer. Without one, terms like “on time” or “defect” mean different things to different people, and the numbers cannot be compared.

Is a Data Collection Plan only for Six Sigma projects?

No. Any team making a decision based on data benefits from one. The plan is methodology-agnostic. It simply makes sure the numbers you act on are consistent, defined, and defensible.

Data without a plan produces numbers. A plan produces proof. The discipline is not in collecting more, but in defining what counts before you count.


More Insights

Subscribe to the Newsletter

Monthly Insights from the Field

Get practical process improvement insights, real implementation stories, and lessons learned delivered to your inbox every month.

Join 2,000+ process improvement professionals. Unsubscribe anytime.

Form Submitted. Please check your email to confirm your sign-up. Thank you.

Oops! Some Error Occurred. Please try again.

Maria Milo

35+ years of worldwide operational excellence experience across oil & gas, healthcare, and manufacturing. Focuses on practical implementation that delivers sustainable results, rather than just theoretical models.

Insights from the Field

Master's Toolkit

Let's Connect

Follow for insights from three decades of tranformation work. Connect if you value practical wisdom over theoretical frameworks..

maria@mariamilo.com

Copyright ©️ 2025 Maria Milo | Real Insights From the Field

CEO at Variance Reduction International (VRI) | Serving Oil & Gas, Healthcare, and Manufacturing Globally

www.VarianceReduction.com | Houston, Texas | USA