It is widely thought that reduced olfactory capacity in apes is linked to the development of acute vision, especially trichromacy.

Whilst the loss of any sense can be a grave burden, the extent to which we rely on olfaction is sometimes underestimated, as sufferers from anosmia will confirm. However, in comparison with many animals, our sense of smell is comparatively slight.

In most mammals, olfaction is essential for survival, but different groups depend on smell to different degrees. Macrosmatic animals, such as rodents and dogs, possess a highly acute and behaviourally relevant sense of smell, whereas humans and other microsmats have less developed olfactory capacities. Dogs are up to 100 times more sensitive to certain odour compounds than humans. These differences are reflected not only in the anatomy of the olfactory apparatus (e.g. size of the olfactory epithelium and the olfactory bulb, a brain region involved in odour detection), but also in the genome. Micro- and macrosmats differ in size and diversity of the olfactory receptor (OR) gene family, which is the largest gene superfamily in mammalian genomes. OR genes are organised in clusters across multiple chromosomes and, as the name suggests, code for the olfactory receptors. These short proteins with seven-transmembrane domains detect specific sets of odour molecules, thus constituting the molecular basis for the sense of smell.

OR genes are thought to evolve rapidly and during mammalian evolution, their numbers have undergone extensive changes. It is generally assumed that a reduced sense of smell could be due to a loss of functional OR genes. In different animal groups, the OR genes show varying degrees of pseudogenisation, where genes lose their protein-coding ability due to deleterious mutations (e.g. frameshifts). While house mouse (Mus musculus) and dog (Canis familiaris) both possess approximately 1300 OR genes, of which about 20% and 27%, respectively, are pseudogenes, the lineage leading to humans has experienced a vast increase in the proportion of OR pseudogenes relatively recently in evolutionary history.

Loss of olfactory capacity in primates

In many primate species, the proportion of OR pseudogenes is comparatively large, with the highest levels observed in chimpanzees (Pan troglodytes; about 48%), gorillas (Gorilla gorilla; about 50%) and humans. In humans, only about 400 OR genes are functional, while about 600 are pseudogenes (interestingly, there seem to be differences between populations in the size of the intact olfactory repertoire). However, New World monkeys are an exception to this – they have retained a high proportion of functional OR genes.

How can this difference between New World monkeys and other primates be explained? It is widely thought that the reduction in olfactory capacity in Catarrhini (Old World monkeys and apes) is linked to the development of acute vision, especially trichromacy. Trichromatic colour vision is very effective for perceiving long-distance environmental signals, which reduces the importance of olfaction and accordingly the selective advantage of having a large functional OR gene repertoire. Interestingly, the New World howler monkeys (genus Alouatta) do not show a corresponding diminution of olfactory capacities and still employ pheromone signalling, despite having independently evolved full trichromatic vision. One argument is that they require both sensory modalities because of a jungle habitat.

Loss of olfactory capacity in cetaceans

In the cetaceans (whales and dolphins) there has been an independent loss of functional OR genes, very much in line with humans, but in this case linked to the return to the oceans and the loss of the terrestrial olfactory register (which is thought to have little function in a marine habitat, especially for air breathers).

Generally, olfaction is more important in a terrestrial than in an aquatic environment, which is probably why the repertoire of OR genes expanded widely at the fish-tetrapod transition (although chance elements, such as random gene duplication, need to be considered here as well). Furthermore, the set of OR genes required is different – while airborne odours are important for terrestrial animals, marine animals need to detect water-soluble compounds.

When the wholly aquatic cetaceans evolved from a terrestrial ancestor quite similar to cattle (as recently demonstrated by molecular biology), their sensory abilities underwent major changes. The extent to which the olfactory apparatus was reduced differs between the two major clades. In baleen whales (Mysticeti), olfaction is significantly impaired (e.g. about 58% OR pseudogenes in minke whales Balaenoptera acutorostrata), while the toothed whales (Odontoceti) seem to have lost their olfactory sense completely. Adult toothed whales lack olfactory structures and show extremely high pseudogene levels (e.g. about 77% in dwarf sperm whales Kogia sima). This is reminiscent of the blind Southern marsupial mole (Notoryctes typhlops), where an anatomical degeneration of the eyes is accompanied by a mutation in an inter-photoreceptor protein gene.

Interestingly, the proportion of OR pseudogenes is comparatively low in two other marine vertebrates, the Steller sea lion (Eumetopias jubatus; about 37%) and the loggerhead turtle (Caretta caretta; about 31%). It is likely that these species still have maintained their functional OR genes as they are only semi-adapted to an aquatic habitat and still rely on a land habitat for several important roles (e.g. for breeding). In contrast, the cetaceans are perfectly adapted to a marine life and do not depend on a terrestrial environment at all. Therefore, as a consequence of relaxed selective pressure, they have probably lost a large fraction of their OR genes (and presumably this is one reason why they evolved echolocation instead).

One study also found evidence that also platypus OR genes exhibit a high degree of pseudogenisation, which could be related to the semi-aquatic lifestyle of this species and their well-developed mechano- and electroreceptive capacities.