Paste raw timings for several implementations and get mean, median, standard deviation, min, max and relative speed in one comparison table, plus a Markdown version to paste anywhere.
Turn a pile of timings into a comparison table. One suite per line, a label, a colon, then your raw measurements. It computes the usual statistics, ranks everything against the fastest, and gives you a Markdown table for the pull request.
One suite per line as "name: value, value, value". Commas, semicolons or spaces all work as separators.
Suite
Runs
Mean
Median
Std dev
Min
Max
Relative
Add at least one line in the form "name: 1, 2, 3" to see the comparison.
Reading a benchmark table without being misled
Mean and median disagree for a reason
The table shows both, and the gap between them is the first thing worth looking at. If the mean sits well above the median, a small number of slow runs are dragging the average up - a garbage collection pause, a cache miss storm, the operating system deciding to schedule something else on your core. The median ignores those; the mean does not.
Neither is automatically the right one to quote. If you are sizing a server for steady-state throughput, the mean is what matters, because those slow runs really do consume time. If you are comparing two implementations to decide which algorithm is better, the median is usually the fairer number, because the outliers are measuring your machine rather than your code. Quote whichever you are actually reasoning about, and say which one it is.
Standard deviation tells you whether the comparison is real
A difference in means only means something if it is large relative to the spread. Two suites at 12.0 and 12.4 milliseconds with a standard deviation of 0.05 are genuinely different. The same two means with a standard deviation of 3.0 are the same number wearing different hats, and any conclusion drawn from them is noise.
This tool uses the sample standard deviation, dividing by n-1 rather than n. That is the right choice here: your runs are a sample drawn from the population of all possible runs, not the entire population, and dividing by n would systematically underestimate the spread. With a single run there is no spread to compute, so the column reads zero - which is a reminder that one measurement is not a benchmark.
Relative speed is the number people remember
The last column expresses every suite as a multiple of the fastest one, because that is the form the result usually gets repeated in. "The new parser is 2.4x faster" travels; "the new parser averages 8.2 milliseconds" does not, unless the listener already knows the baseline.
Two cautions. First, the ratio inherits all the uncertainty of the two means behind it, so a 1.05x difference on noisy data is not a difference at all. Second, ratios of means are not the same as means of ratios, and if the runs vary a lot in size the two can point in different directions. When the numbers are close, report the spread alongside the ratio and let the reader judge.
Frequently asked questions
What format do the measurements need to be in?
One line per suite: a label, a colon, then the numbers. Commas, semicolons, spaces and tabs all count as separators, so most copy-pasted output works without editing.
Does the unit selector convert anything?
No. It only labels the output. The statistics are computed on the numbers exactly as you typed them, so all suites on the page need to be in the same unit already.
Why is the standard deviation zero for a single run?
Sample standard deviation divides by n-1, which is zero when n is one. A single measurement has no spread to report, and the zero is a hint that you should collect more runs.
How is "relative" calculated?
Every suite's mean is divided by the smallest mean on the page. The fastest suite is labelled as such rather than showing 1.00x, since that reads better in the exported table.
Can I paste output straight from a benchmarking tool?
Usually. As long as each line has a name, a colon and some numbers, the extra punctuation is treated as a separator. Lines without a colon are skipped rather than causing an error.
Is the Markdown table safe to paste into a pull request?
Yes - it is plain GitHub-flavoured Markdown with a header row and a separator row. The column headers follow the page language, so switch language first if you want them in English.