Niche Overlap


Based in part on Gause’s (1934) classic laboratory experiments, the competitive exclusion principle (Hardin 1960) is that coexisting species must differ in some aspect of their resource utilization. The idea was expanded into mathematical theories of limiting similarity (MacArthur and Levins 1967) and more recent treatments of the ecological niche (Chase and Leibold 2003). Ecologists have spent a lot of time measuring niche axes of potential resource competition, especially those related to the utilization of food, space, or time (Schoener 1974). The ?ghost of competition past? predicts that species currently coexisting should exhibit relatively low overlap in resource utilization (Connell 1980). Alternatively, if competition for limited resources is current, there should be high overlap in the niches of two species.

Whether niche overlap is high or low has to first be measured relative to some null expectation. This module of EcoSimR tests for overlap in resource use among a set of coexisting species using a set of unordered, discrete resource categories. The analysis reveals whether the average niche overlap, calculated among all unique pairs of species, is more or less than would be expected if species used resource categories independently of one another. If the resources are measured on an ordered scale (such as prey size) or even a circular scale (such as time of day), then other kinds of null models are more appropriate (Castro-Arellano et al. 2010). Different algorithms are also needed if the utilization data are collected on a continuous scale (prey body size measured in g) versus a set of discrete categories (prey classified as invertebrates, small fish, or large fish).


The data for a niche overlap analysis consists of a data frame in which each row is a species and each column is a discrete, unordered resource category (such as host plant species, diet categories, or microhabitats). The first column of the data frame optionally contains the species names. Each entry is the utilization (measured in appropriate units) of a particular resource category by a particular species. Entries must be non-negative real numbers. Zero values are allowed (some species may not use certain resource categories), but missing NA values are not allowed. All entries must be in the same units, but they do not have to be proportions or percentages. All of the niche overlap metrics first scale the data to proportions by dividing each entry by its row sum.

The supplied data set macwarb for niche overlap analysis is taken from Table 2 of MacArthur’s (1958) classic study on niche overlap of coexisting warblers. Each row represents one of 5 species of warbler that coexist in northern New England. Each column represents one of 16 subregions of an idealized coniferous tree. The entries are the percentage of time that each species was observed foraging in a different subregion of a (hypothetical) tree. MacArthur (1958) proposed that the species specialized on different subregions, and that this specialization was the basis for resource partitioning and coexistence. As we will see, the null model analysis provides a different perspective on this classic data set.

MacArthur’s (1958) warbler data.
Species X1T X1M X1B X2T X2M X2B X3T X3M X3B X4T X4M X4B X5T X5M X5B X6
Cape May warbler 49.9 13.2 0.0 20.6 8.3 0.0 4.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.5
Myrtle warbler 6.6 4.1 0.3 7.8 4.9 1.3 9.3 9.8 3.6 1.7 1.3 5.1 0.0 0.6 15.9 27.7
Black-throated green warbler 12.1 5.7 0.0 17.3 8.8 0.7 21.8 14.1 1.5 6.2 4.7 4.5 1.4 0.3 0.9 0.0
Blackburnian warbler 34.8 10.5 3.2 15.1 8.3 2.7 13.1 11.0 0.7 0.3 0.3 0.0 0.0 0.0 0.0 0.0
Bay-breasted warbler 3.5 1.9 1.4 6.5 8.0 5.9 11.4 19.1 13.1 7.7 8.8 10.1 0.0 0.1 2.5 0.0


The basic metrics for niche overlap are either the pianka or the czekanowski index. The pianka index is similar to the competition coefficient \(\alpha\) in the Lotka-Volterra equations, and the czekanowski index is the area of intersection of two resource utilization histograms. These indices are calculated for an individual species pair, and are then averaged over all possible pairs of species in the assemblage. Both indices range from 0.0 (no overlap) to 1.0 (complete overlap). They are highly correlated with one another, and it makes little difference which one you use.

The basic pianka and czekanowski indices give an index that is the mean overlap of all possible species pairs. You can also analyze the variance of pairwise overlap with pianka_var and czekanowski_var, and the skewness of pairwise overlap with pianka_skew and czekanowski_skew. A large value of the variance indicates that some species pairs have very high overlap and others have very overlap, relative to the null model. A large positive value of skewness indicates asymmetry with some strong high-overlap species pairs in the data matrix. A large negative value of skewness indicates asymmetry with some strong low-overlap species pairs in the matrix. As with all higher moments of a distribution, the variance and skewness can be very sensitive to outliers in the data, although the randomization test controls for this to some extent. The variance and skew functions have not been well-tested, but they may provide some additional insight into the niche overlap patterns.


Following the terminology of Lawlor (1980a) and the benchmark testing of Winemiller and Pianka (1990), EcoSimR provides four randomization algorithm. We recommend the default ra3 or the slightly more conservative ra4.

In brief, ra3 reshuffles the row values, and ra4 reshuffles the non-zero row values. Both of these algorithms retain the observed ?niche breadth? of each species- that is the relative degree of specialization, but they randomly alter which particular resource categories are used.

ra2 preserves the non-zero elements, but replaces the non-zero elements by a (0,1) random uniform value. ra1 replaces all matrix entries by a random (0,1) uniform value, so the null hypothesis is that each species is an isotropic super-generalist.

ra1 and ra2 are too liberal and do not give good results with artificial test matrices (Winemiller and Pianka 1990). For the dataMacWarb, try running these variations. Compared to ra1, the observed (Pianka) niche overlap is significantly greater than expected. Compared to ra2, the observed (Pianka) niche overlap is not different from random, and compared to ra3 and ra4, the observed (Pianka) niche overlap is significantly greater than expected. Run these options yourself in EcoSimR to confirm these patterns.

The statistical properties of ra3 and ra4 are good, although it is an open question whether resource categories that were never utilized by a species might have been utilized in a null assemblage. All four of the algorithms assume that all resource categories are equally abundant, which is often not the case.

If independent data on resource abundance are available, the observed utilization can be divided by resource abundance to create an “electivity” index (Lawlor 1980b). This index effectively provides more weight for resource categories that are rare. Such a weighting is useful because we expect high overlap in abundant resource categories. However, the weightings may be sensitive to sampling error for very scarce resource states.

Although it might seem reasonable to do so, you should not sum the resource utilization across species in order to estimate resource abundance. This estimate is not independent of observed utilization and can lead to Type I errors (incorrect rejection of a true null hypothesis).

Some people are disturbed and suspicious of the fact that the null model results may depend on which algorithm is used and how the data are processed. But this is exactly as it should be: the results depend on the assumptions of the model. In the more familiar world of a simple ANOVA, the results will depend on whether the factors are treated as fixed or random and whether the data are transformed or not. And of course the results will depend on the sample size, the spatial scale of the study, and whether the data were collected as random, haphazard, systematic, or representative samples. Although there is often a preferred set of options for a null model analysis (which we have used in our defaults), we think it is always worthwhile to explore different variations of the model so that you can understand the result. See Gotelli and Ulrich (2012) for a discussion of other general issues in null model analysis.


The niche overlap module offers two graphic outputs. `plot(type=“hist”) generates a standard histogram of simulated metric values (blue bars) based on the selected algorithm and metric. The vertical red line indicates the observed metric for the original data, the pair of vertical long-dash lines indicate the 95% one-tailed cutpoints, and the short-dash lines indicate the 95% two-tailed cutpoints. The latter would constitute a proper 95% confidence interval for the null distribution. Note that these intervals are not based on a normal or other parametric distribution. Instead, they are always estimated directly from the histogram, which means they could be asymmetric, and will be sensitive to the number of replicates used.

plot(type="niche") generates a rendering of the species \(\times\) utilization matrix for the original data matrix (red) and one simulated data matrix (blue). The area of each circle depicted is proportional to the utilization of a resource category by a species. If no circle is shown, the utilization was 0.0.



speciesData          # user must supply a data frame; speciesData=dataMacWarb for default run
algo = "ra3"           # reshuffle elements within each row of the matrix
metric = "pianka"      # pianka niche overlap index
nReps = 1000         # number of null assemblage created
rowNames=TRUE        # reads speciesData as a data frame wtih row labels in the first column
saveSeed=FALSE       # if TRUE, saves random number seed
algoOpts=list()      # list of other specific options for the algorithm
metricOpts=list()    # list of other specific options for the metric
suppressProg= FALSE  # suppress printing of progress bar (for creating markdown files)


str(dataMacWarb)  # structure of MacArthur's warbler data set
summary(myModel)  # output summary of null model analysis

#create a random data set with uniform (0,1) values
myRandomData <- matrix(runif(300), nrow=30)

# run null model with czekanowski index and ra1, 5000 replications
myRandomModel <- niche_null_model(speciesData=myRandomData, rowNames=FALSE, 
                            algo="ra1", metric="czekanowski", 

# print summary of model and plot histogram


This null model is appropriate only for unordered niche categories, such as host plant species or dietary type. Different kinds of null models are appropriate for continuous data (such as prey body size) or circular data (such as temporal niches measured over diurnal or annual time scales). The niche overlap model used here also assumes all resource categories are equally available to all species. However, ra2 and ra4 fix the zero states, which means that resource categories that were never used by a species are assumed to be unavailable to that species even in the absence of competition. None of the algorithm explicitly account for sampling variation in the number of observations per category. However, ra3 and ra4 do account somewhat for sampling variation because the percent utilization values are themselves randomized among resource categories.


Castro-Arellano, I., T.E. Lacher, Jr., M.R. Willig, and T.F. Rangel. 2010. Assessment of assemblage-wide temporal niche segregation using null models. Methods in Ecology & Evolution 1: 311-318.

Chase, J.M. and M.A. Leibold. 2003. Ecological Niches: Linking Classical And Contemporary Approaches. University of Chicago Press, Chicago.

Connell, J.H. 1980. Diversity and the coevolution of competitors, or the ghost of competition past. Oikos 35: 131-138.

Gause, G.F. 1934. The Struggle For Existence. Williams & Wilkins, Baltimore.

Hardin, G. 1960. The competitive exclusion principle. Science 131: 1292-1297.

Gotelli, N.J. and W. Ulrich. 2012. Statistical challenges in null model analysis. Oikos 121: 171-180.

Lawlor, L.R. 1980a. Structure and stability in natural and randomly constructed model ecosystems. American Naturalist 116: 394-408.

Lawlor, L.R. 1980b. Overlap, similarity, and competition coefficients. Ecology 61: 245-251.

MacArthur, R.H. 1958. Population ecology of some warblers of northeastern coniferous forests. Ecology 39: 599-699.

MacArthur, R.H. And R. Levins. 1967. The limiting similarity, convergence, and divergence of coexisting species. American Naturalist 101: 377-385.

Schoener, T.S. 1974. Resource partitioning in ecological communities. Science 185: 27-39.

Winemiller, K.O. and E.R. Pianka. 1990. Organization in natural assemblages of desert lizards and tropical fishes. Ecological Monographs 60: 27-55.