Power analysis: How many subjects do I need?
YOU WILL NOT BE ABLE
TO DETECT A DIFFERENCE IF THE SAMPLE SIZE IS SMALL RELATIVE TO THE
MAGNITUDE OF THE DIFFERENCE!
Power analysis allows us to do two things:
- To
determine
the sample size required to uncover a significant result for a given
effect size with a given degree of confidence.
- To work out, if our sample size is limited, the probability of acheiving a significant result with a given effect size.
IT IS YOUR RESPONSIBILITY AS A RESEARCHER TO ENSURE THAT YOU HAVE SUFFICIENT STATISTICAL POWER TO CARRY OUT YOUR EXPERIMENT.
So, we have the sample size (how many subjects in each treatment), the effect size (the magnitude of the treatment on the response), the α-level (P(Type-I error), which means falsely rejecting the null hypothesis), and the power (1 - P(Type-II error), which means falsely accepting the null hypothesis). If we have any three of these bits of information, we can calculate the fourth. To give you a heads-up, the α-level is typically p < 0.05, and the power should typcially be no less than 0.8.