Ask what an error bar means

An error bar can show standard deviation — open definition, a standard error — open definition, a confidence interval or another stated range. These represent different things. A chart must define the bars before the viewer can interpret their size or compare them across studies. The same-looking marks need not have the same statistical meaning.

A set of repeated measurements can vary substantially while its estimated mean is more precise. Conversely, highly repeatable measurements can share a calibration offset. Small scatter therefore does not automatically establish accuracy relative to the intended physical quantity.

Under an ideal model of independent observations with a fixed standard deviation of 4 units, the standard deviation of their sample mean is 4 divided by the square root of the sample count. For 16 observations it is 1 unit; for 64 it is 0.5 unit. Four times the observations halves this component of uncertainty.

These fictional values illustrate a statistical model, not a universal rule for all data. Correlated samples, changing conditions and systematic effects can invalidate a simple application. Repeating the same biased procedure does not guarantee convergence to the right physical answer.

More readings help under stated assumptions

A frequentist 95% confidence procedure is designed to cover the fixed parameter in about 95% of repeated uses under its assumptions. It does not mean that 95% of individual observations lie inside a particular interval for the mean. It also does not provide a blanket probability that every part of a scientific explanation is correct.

For a fictional estimate reported as 10 with an interval from 8 to 12, identify the parameter, confidence level, procedure and assumptions. Comparing two sets of bars by eye is not a universal significance test; the correct comparison depends on their definitions and any dependence between the estimates.