How we decide if something actually worked.

Three labels, never mixed

Every piece carries one label at the top. An experiment has a hypothesis, a control or baseline, a sample and a verdict. An observation is a pattern in real data without a control. An opinion is what I think, and why.

Three formats

Experiments test one claim. Benchmarks publish reference numbers from aggregated data, with the sample and period. Field notes are shorter observations and opinions from the work.

How an experiment runs

We write the hypothesis and the success metric before we look at results. We state the sample, the period and what we held constant. If the result is not significant, we say so — "we don't know yet" is a valid verdict.

Where the data comes from

Our own tests and, in some cases, anonymized and aggregated data measured with Sealmetrics. No personal data is published. The source of every dataset is named on the piece.

What's free and what needs an email

The question, hypothesis, method and dataset of every piece are public. Some pieces put the detailed results behind a free subscription. Search engines and people see exactly the same thing: we never show crawlers content that readers can't see.

Independence

Unmodeled is written by Rafa, founder of Sealmetrics. Sealmetrics may provide measurement infrastructure; it does not choose topics or verdicts. When a piece was measured with Sealmetrics, or touches a category Sealmetrics competes in, it says so.

Corrections

When we get something wrong, we fix it, update the "last updated" date and log the change at the bottom of the piece.