Experimental design, power and statistics for in vivo behavioural scientists
  • Downloding and installing R
  • Designing Experiments
  • Power Analysis
  • Planning Data Analysis
  • Data analysis
As we talked about previously, usually the 'sig.level' = 0.05 and 'power' = 0.8. In an ideal world, the power will be 0.8, which means that you have an 80% chance of detecting a significant result if one exists.

Calculate effect size:

For chi-squared tests, the effect size, w, is calculated by the following equation:
Picture
where p0i is the probability in the ith cell under the null hypothesis and p1i is the probability in the ith cell under the alternative hypothesis. In R, there are functions in the pwr library to calculate this effect size (ES.w1), and the w statistic for 2-way contingency tables (ES.w2).

If you have four probabilities under the alternative hypothesis (0.3, 0.4, 0.2, 0.1) and four under the null hypothesis (0.25, 0.25, 0.25, 0.25), type into R:
Picture
and then calculate the effect size w1:
Picture
If you have a two-way contingency table e.g.,

A
B
C
D
a
0.225
0.125
0.125
0.125
b
0.160
0.160
0.040
0.040

this can be entered into R as a matrix:
Picture
Then, the effect size can be calculated as:
Picture
Cohen suggests that w values of 0.1, 0.3, and 0.5 represent small, medium, and large effect sizes respectively.

Calculate sample size:

Calculating the required sample size is the same, regardless of which effect size you have. The basic command in R for calculating the required sample size for a chi-squared test is:
Picture
where 'w' is the effect size calculated above, 'N' is the total sample size (number of observations), 'df' is the degrees of freedom (i.e., the number of cells minus one, and the number of columns minus 1), 'sig.level' is the required significance level (i.e., p < 0.05) and 'power' is the observed or required power of the design, given the effect size w.

For example, say you were planning to run an experiment looking at the proportion of a population of animals showing four distinct phenotypes. You are expecting a medium effect size, based on pilot data and you want to be 80% confident that you will detect this effect at the p = 0.05 cut-off. You would therefore type the following into R (note that I omit the 'N = ...' command, as this is what I want R to calculate for me):
Picture
This means that in order to be 80% confident of detecting the effect, I will need 121 subjects. What if I only have 90 subjects? How confident can I be of detecting a significant result? Let's ask R. Note that this time, I leave the 'power = ...' command out, and enter the number of subjects I have in the 'N = ...' space.
Picture
According to this, I can only be 66% confident that I will detect the effect given the sample size I have at my disposal. Time to re-design the experiment I think!
Return to previous page
Powered by Create your own unique website with customizable templates.