image: Sound is transmitted to the inner ear (purple) by the bony ossicles (light blue), both in mammals (left) and birds (right). Among the three ossicles of mammals, the stapes (S) corresponds to the columella (Co) of birds, whereas the incus (I) and the malleus (M) evolved from bones of the primary jaw joint.
The vertebrate ear is a remarkable structure. Tightly encapsulated within the densest bone of the skeleton, it comprises the smallest elements of the vertebrate skeleton (auditory ossicles) and gives rise to several different senses: balance, posture control, gaze stabilization, and hearing. Nowhere else in the vertebrate skeleton are different functional units packed so close together and jointly embedded in its skeletal environment, which also hampers the independent evolution of the ear components.
Even the growth pattern of the ear deviates from that of the remaining skeleton: In humans and other mammals, the inner and middle ears achieve their final size already before or early after birth, which further challenges evolutionary change because postnatal development substantially contributes to anatomical differences between many mammals otherwise.
All this makes it puzzling how mammals, as a predominantly nocturnal group reliant on hearing, were able to occupy such a vast diversity of environments in the aquatic, terrestrial, subterranean, and aerial realms that require an amazing disparity not only in hearing abilities, but also in locomotion and posture. How could the different, tightly connected parts of the ear adapt independently to these diverse functional and environmental regimes?
A group of researchers around Philipp Mitteroecker from the University of Vienna proposed a new explanation for this evolutionary puzzle. Despite its similar function, the ear is composed of different bones in mammals, birds, and reptiles. In birds and reptiles, the lower jaw and its joint are composed of multiple bones, and they have a single auditory ossicle that transmits the sound. Extant mammals, by contrast, have three ossicles (malleus, incus, stapes) and one ectotympanic bone, supporting the tympanic membrane, all of which are separate from the jaw. This evolutionary transformation of the primary jaw joint into the mammalian ear ossicles is one of the most iconic transitions in vertebrate evolution, but it is not clear why this complex transition has happened.
The Austrian research team proposed that this substantial evolutionary change of mammalian ear anatomy has - in addition to any direct enhancements of mastication and hearing - also increased the "evolvability" (capacity for adaptive evolution) of the ear and its associated sensory functions. The incorporation of the bones of the primary jaw joint into the ear has considerably increased the genetic, regulatory, and developmental complexity of the mammalian ear. This increase in the number of genetic and developmental factors, in turn, has increased the evolutionary degrees of freedom for an independent adaptation of the different functional units of the ear: the number of genetic and developmental "knobs" for natural selection to turn.
They suggest that despite the tight spatial entanglement of functional ear components, the increased evolvability of the mammalian ear may have contributed to the evolutionary success and adaptive diversification of mammals in the vast diversity of ecological and behavioral niches observable today. In their article, they show that mammals, as compared to birds, were indeed able to evolve a much wider morphological and functional diversity, including numerous evolutionary "novelties", even though birds are more diverse in species number than mammals.