In the age of the 3Rs (reduction, refinement and replacement of animals in research) journals and policy makers are beginning to recognize (at last!) that well designed, properly analyzed, experiments are less likely to go wrong, and to produce unrepeatable results.
Nature's
checklist of statistical
adequacy. Reviewers for Nature Group journals have to check through this, and answer yes/no to
all of the questions!
For graphs
- Reported n at start of study and for each analysis
- Provided sample size calculation or justification
- Described randomization procedures or other ways to eliminate bias in sampling (in particular for experiments involving animals)
- Identified all statistical methods unambiguously
- If statistical methods were described adequately, were any of them clearly inappropriate?
- Provided alpha for all statistical tests
- Specified whether tests were one-sided or two-sided
- Stated whether the data met the assumptions of the test
- Reported actual P values for primary analyses
- Were the statistical measures (mean, standard error, standard deviation, etc.) reported, and were they clearly labeled?
- Was the unit of analysis clearly stated in all comparisons?
- Are mean and standard deviation used to describe data sets that may be non-normally distributed or when the sample size is very small?
- Explanation of unusual or complex statistical methods
- Explanation of data exclusions, if any
- Explained reasons for any discrepancy between initial n and n for each analysis
- Explained method of treatment assignment (randomization, if any)
- Explained any data transformation
- Discussed adjustments for multiple testing
For graphs
- Were effect sizes distorted? (by truncation of y axis, etc.)
- Were error bars unlabeled?
- Were error bars absent?
Right, if you're ready to start planning your experiment, please choose your subjects!
