A timing needs a statistical meaning
A benchmark number is not self-explanatory. The same set of runs can produce a minimum, median, mean or percentile, and each answers a different question. Before comparing two values, ask how observations were collected and which statistic was reported. A very fast run demonstrates that an operation can complete quickly under some observed conditions. It does not establish what a typical user should expect.
The supplied CompareGrids methodology uses a relatively fast observation: for short actions it describes ten to twenty runs and selects the third-smallest result; for long actions it describes roughly three runs and selects the smallest. Values are then rounded. This is an important interpretation boundary. The published number should not be described as a median, an average or a prediction of production latency.
Minimums answer a narrow question
A minimum can help estimate a best observed execution path with limited interference. It is sensitive to the number of attempts: collecting more runs gives more opportunities to observe an unusually fast one. Comparing the minimum of three attempts with the minimum of twenty is therefore not a neutral choice. The sampling plan belongs with the result.
The third-smallest value also changes meaning with sample count. In a set of ten it occupies a different relative position than in a set of twenty. It is an order statistic, but it is not a fixed percentile estimate across varying counts. Without the raw observations, readers cannot reconstruct the distribution or quantify how far that value sits from the center.
This does not make the source useless. It makes the supported claim narrower. The results can help identify large performance differences and operations that failed in the supplied setup. They are less suitable for declaring a small timing difference decisive. Present the statistic honestly and avoid adding a confidence interval that the available data cannot support.
Define the interval before collecting samples
A timer around an initialization call may stop before the browser paints. A timer that starts after data arrives excludes transfer and parsing. Both can be legitimate measurements if labelled correctly, but neither is automatically the user's complete wait. Decide what event starts the clock and what observable condition ends it.
For a sort action, a useful end condition might require the correct visible order and an updated indicator. For editing, it might require a confirmed save rather than a local visual change. For an export, it could be file generation or completed download. Keep component timing and end-to-end timing separate so differences can be attributed to the right stage.
The supplied initialization measurements exclude script, page and dataset loading. Quoting them as total page-load time would change their meaning. Carry this limitation into articles, charts and procurement documents. A number loses value when its definition disappears during reuse, even if the digits remain accurate.
Control ordinary sources of variation
Record the device, operating system, browser, viewport and build configuration. Keep background workload reasonably controlled and document relevant power settings. Use a consistent fixture and action sequence. These controls make comparisons more interpretable; they do not eliminate all variation or turn one machine into a representative sample of every user's environment.
Distinguish cold and warm scenarios. Cached resources, previously created objects and earlier interactions can affect later work. Choose the scenario that matches the question and label it. If both first use and repeated use matter, report both rather than averaging them into a value with no clear real-world counterpart.
Alternate or randomize candidate order where practical to reduce systematic effects from warming, background activity or device conditions. Preserve the run order in the record. When an unexpected result appears, investigate the cause without silently removing it. An exclusion needs a documented rule, such as a known unrelated interruption, applied consistently to every candidate.
Report completion as well as duration
Failed attempts are part of the result. A grid that completes two of three attempts cannot be summarized fairly by the fastest successful attempt alone. Record the number of attempts, successful completions and failure categories. Keep unsupported operations separate from crashes, and keep unmeasured cases separate from both.
Do not convert status codes into zero milliseconds for a timing chart. Zero suggests an instantaneous successful operation. Likewise, dropping failures from an average without explanation creates survivorship bias. A completion table alongside successful timing summaries makes the trade-off visible without inventing a numerical penalty that looks like measured time.
Investigate whether failures are reproducible and implementation-specific. A configuration mistake may be repairable; a resource limit may require a different architecture. Retain the original observation and document any corrected rerun separately. This preserves the history of what changed and prevents a revision from becoming an unexplained improvement.
Use distributions when the evidence supports them
For a new evaluation, retain individual samples. Report a central statistic and a measure of spread appropriate to the sample size and purpose. A median is often useful for a typical observed run, while a high percentile can describe slower experiences when enough observations exist. Avoid presenting a stable-looking tail percentile from a handful of runs.
Averages can be useful for some aggregate cost questions, but they can hide distinct modes such as first-use and cached behavior. Plot or inspect the samples before reducing them to one value. If two clusters correspond to different scenarios, separate those scenarios. A single summary should not conceal a known architectural distinction.
Choose sample counts and stopping rules before looking for a preferred winner. Repeating only the candidate that looks bad creates an uneven comparison. If exploratory work reveals a problem, fix the protocol and rerun comparable cases. Separate exploratory diagnosis from the final evidence used for a recommendation.
Preserve enough evidence for a rerun
Store the run sequence, individual observations and configuration alongside the summarized result. Include the rule used to identify an invalid run and any observation excluded under that rule. A future reviewer should be able to recompute the published statistic without reverse-engineering a chart or relying on a recollection of the testing session.
When a new version is evaluated, retain the old fixture and protocol long enough to distinguish product changes from methodology changes. If the protocol improves, publish that fact and avoid presenting an incompatible comparison as a pure speedup. Better methodology is valuable even when it breaks a continuous trend line.
Match precision to uncertainty
The source rounds values, sometimes to only one significant digit. Small differences between rounded values should not be treated as precise ratios. A displayed value of one second does not mean the underlying operation consistently takes exactly one thousand milliseconds. Explain precision limits where they affect the conclusion.
Score transformations introduce another layer. A score may interpolate between thresholds, clamp extremes and combine categories. It is a decision aid built from measurements, not another unit of time. Two equal scores can conceal different raw results, and a change in weights can alter ranking without any measured improvement. Link back to raw values when interpreting a score.
A strong benchmark report states the workload, timing boundary, sampling rule, completion profile and uncertainty. It then connects those observations to a user task. The goal is not to produce the most impressive smallest number. It is to support a decision that remains understandable when another engineer repeats the experiment or a user encounters a slower run.