Inferring Diversity: Life After Shannon
Univerisity at Albany
Last modified: December 20, 2007
Diversity is a concept that is used in many fields to describe the variability of different entities in a group. In ecology, the Shannon entropy and Simpson's index are the predominate measures of diversity. In this paper we focus on the Shannon entropy for two reasons: First, it has been shown that Simpson's index is an approximation of Shannon's. Second, Shannon's entropy is closely tied to many other areas of research, such as information theory and physics.
It is often the case that the species in a community cannot be fully counted. In this case, when one has incomplete information, one must rely on methods of inference. The two preeminent inference methods are the MaxEnt method, which has evolved to a more general method, the method of Maximum (relative) Entropy (ME) and Bayes' rule. Choosing between the two methods has been dictated by the nature of the information being processed (either constraints or observed data). However, it has been shown that one can accommodate both types of information in one method, ME (Giffin 2007). The purpose of this paper is to demonstrate how the ME method can be used as a measure of diversity that is able to include more information that Shannon's measure allows.
It is critical to note that, as is shown in the examples, any work done with Bayesian techniques can be implemented into the ME method directly and as is. There is no need to 'reinterpret' the distributions obtained by Bayesian methods for use in ME.