Fisher’s exact test is used to compare counts and proportions between groups when small samples of nominal variables are available. It assumes that the individual observations are independent, and that the row and column totals are fixed, or “conditioned.” An example would be putting 12 female hermit crabs and 9 male hermit crabs in an aquarium with 7 red snail shells and 14 blue snail shells, then counting how many crabs of each sex chose each color (you know that each hermit crab will pick one shell to live in). The total number of female crabs is fixed at 12, the total number of male crabs is fixed at 9, the total number of red shells is fixed at 7, and the total number of blue shells is fixed at 14. You know, before doing the experiment, what these totals will be; the only thing you don’t know is how many of each sex-color combination there are.
Use Fisher’s exact test when the total sample size is less than 1000, and use the chi-square or G–test for larger sample sizes.
The lady and the tea example.
The null hypothesis in this famous experiment is that there is no association between the true order of pouring and the woman’s guess. By the way, the lady in question was Dr Muriel Briston, another statistician at the research lab where Fisher worked. The alternative hypothesis is that there is a positive association (that the odds ratio is greater than 1).
TeaTasting <- matrix (c(3, 1, 1, 3),
nrow = 2,
dimnames = list(Guess = c("Milk", "Tea"),
Truth = c("Milk", "Tea")))
TeaTasting## Truth
## Guess Milk Tea
## Milk 3 1
## Tea 1 3f <- fisher.test(TeaTasting, alternative = "greater"); f##
## Fisher's Exact Test for Count Data
##
## data: TeaTasting
## p-value = 0.2429
## alternative hypothesis: true odds ratio is greater than 1
## 95 percent confidence interval:
## 0.3135693 Inf
## sample estimates:
## odds ratio
## 6.408309Cannot reject the null hypothesis, meaning that an association between the true order of pouring and the woman’s guess could not be established. This may have been an example of an underpowered experiment, because according to Fisher’s daughter, Dr Bristol did convince him that she could indeed separate the pour sequences.
The alternative for a one-sided test is based on the odds ratio, so alternative = “greater” is a test of the odds ratio being bigger than or.
May decide to extract just one value
f$conf.int; #f$estimate; #f$p.value## [1] 0.3135693 Inf
## attr(,"conf.level")
## [1] 0.95One-tailed or two-tailed tests?
You should almost always use a two-tailed test, unless you can articulate a very good reason (Handbook of Biological Statistics, by John H. McDonald); should probably use a level of <0.025 for a one sided test to guard against bias.
diet <- matrix (c(1,11,9,3),
nrow=2,
dimnames = list(Diet=c("Dieting", "Not dieting"),
Sex=c("Men", "Women")))
diet## Sex
## Diet Men Women
## Dieting 1 9
## Not dieting 11 3fisher.test(diet, alternative="two.sided") #is there a difference?##
## Fisher's Exact Test for Count Data
##
## data: diet
## p-value = 0.002759
## alternative hypothesis: true odds ratio is not equal to 1
## 95 percent confidence interval:
## 0.0006438284 0.4258840381
## sample estimates:
## odds ratio
## 0.03723312fisher.test(diet, alternative="less") #do men (1st column) diet less?##
## Fisher's Exact Test for Count Data
##
## data: diet
## p-value = 0.00138
## alternative hypothesis: true odds ratio is less than 1
## 95 percent confidence interval:
## 0.0000000 0.3260026
## sample estimates:
## odds ratio
## 0.03723312fisher.test(diet, alternative="greater") # do men diet more?##
## Fisher's Exact Test for Count Data
##
## data: diet
## p-value = 1
## alternative hypothesis: true odds ratio is greater than 1
## 95 percent confidence interval:
## 0.001297208 Inf
## sample estimates:
## odds ratio
## 0.03723312barfs <- matrix (c(2,12,7,7),
nrow=2,
dimnames = list(Barf =c("Wet", "Dry"),
Beast=c("Patches", "Pepper")))
barfs## Beast
## Barf Patches Pepper
## Wet 2 7
## Dry 12 7fisher.test(barfs, or)##
## Fisher's Exact Test for Count Data
##
## data: barfs
## p-value = 0.1032
## alternative hypothesis: true odds ratio is not equal to 1
## 95 percent confidence interval:
## 0.01422997 1.29320684
## sample estimates:
## odds ratio
## 0.1784072Larger tables
In tables larger than 2×2 the p-values are computed using a Monte Carlo simulation
Agresti (2002, p. 57) Job Satisfaction
Job <- matrix(c(1,2,1,0, 3,3,6,1, 10,10,14,9, 6,7,12,11),
4, 4,
dimnames = list(income = c("< 15k", "15-25k", "25-40k", "> 40k"),
satisfaction = c("VeryD", "LittleD", "ModerateS", "VeryS")))
Job## satisfaction
## income VeryD LittleD ModerateS VeryS
## < 15k 1 3 10 6
## 15-25k 2 3 10 7
## 25-40k 1 6 14 12
## > 40k 0 1 9 11fisher.test(Job)##
## Fisher's Exact Test for Count Data
##
## data: Job
## p-value = 0.7827
## alternative hypothesis: two.sidedfisher.test(Job, simulate.p.value = TRUE, B = 1e5) #Monte Carlo sim##
## Fisher's Exact Test for Count Data with simulated p-value (based on
## 1e+05 replicates)
##
## data: Job
## p-value = 0.7833
## alternative hypothesis: two.sidedThere are very few biological experiments where both the row and column totals are conditioned. In the much more common design, one or two of the row or column totals are free to vary, or “unconditioned.” (JH Macdonald).