PASADENA, Calif.-- Individuals with synesthesia perceive the world ina different way from the rest of us. Because their senses arecross-activated, some synesthetes perceive numbers or letters ashaving colors or days of the week as possessing personalities, evenas they function normally in the world. Now, researchers at theCalifornia Institute of Technology have discovered a type ofsynesthesia in which individuals hear sounds, such as tapping,beeping, or whirring, when they see things move or flash.Surprisingly, the scientists say, auditory synesthesia may not beunusual--and may simply represent an enhanced form of how the brainnormally processes visual information.
Psychologists previously reported visual, tactile, and tastesynesthesias, but auditory synesthesia had never been identified.Caltech lecturer in computation and neural systems Melissa Saenzdiscovered the phenomenon quite by accident.
"While I was running an experiment at the Caltech Brain ImagingCenter, a group of students happened to pass by on a tour, and Ivolunteered to explain what I was doing," explains Saenz, who, alongwith Christof Koch, the Lois and Victor Troendle Professor ofCognitive and Behavioral Biology at Caltech and professor ofcomputation and neural systems, reports the finding in the August 5issue of the journal Current Biology.
"As part of the experiment, a moving display was running on mycomputer screen with dots rapidly expanding out, somewhat like theopening scene of Star Wars. Out of the blue, one of the studentsasked, "Does anyone else hear something when you look at that?" Aftertalking to him further, I realized that his experience had all thecharacteristics of a synesthesia: an automatic sensorycross-activation that he had experienced all of his life," says Saenz.
A search of the synesthesia literature revealed that auditorysynesthesia--of any kind--had never been reported. Intrigued, Saenzbegan to look for other individuals with the same ability, using theoriginal movie seen by the student as a test. "I queried a fewhundred people and three more individuals turned up," she says.Having that specific example made it easy to find more people. Thatmovie just happens to be quite "noisy" to the synesthetes and was agreat screening tool. When asked if it made a sound, one of theindividuals responded, "how could it not?" I would have been lesssuccessful had I just generally asked, "Do you hear sounds when yousee things move or flash?" because in the real environment, thingsthat move often really do make a sound," for example, a buzzing bee.
This may be why auditory synesthesia hadn't been detected byneurobiologists. "People with auditory synesthesia may be even lesslikely than people with other synesthetic associations to fullyrealize that their experience is unusual. These individuals have anenhanced soundtrack in life, rather than a dramatically differentexperience, compared to others," says Saenz. However, when asked, allof the synesthetes could name examples of daily visual events thatcaused sounds that they logically knew to be only in their minds,such as seeing a fluttering butterfly or watching television with thesound turned off.
Saenz and Koch found that the four synesthetes outperformed a groupof nonsynesthetes on a simple test involving rhythmic patterns offlashes similar to visual Morse code. Normally, such patterns areeasier to identify with sound (beeps) than with vision (flashes), sothe researchers predicted that synesthetes would have an advantagewith visual patterns because they actually heard a sound every timethey saw a flash.
In the test, the subjects saw a series of flashes and had to guess ifa second sequence, played afterward, represented the same temporalpattern or not. As a baseline measurement, a similar test was givenusing sequences of beeps. Both the synesthetes and the control groupperformed equally well when given beeps. However, with visual flashessynesthetes were much more accurate, responding correctly more than75 percent of the time, compared to around 50 percent--the levelpredicted by chance--in the control group. "Synesthetes had anadvantage because they not only saw but also heard the visualpatterns," Saenz says.
Saenz and Koch suspect that as much as 1 percent of the populationmay experience auditory synesthesia. In fact, she and Koch think thatthe brain may normally transfer visual sensory information over tothe auditory cortex, to create a prediction of the associated sound."This translation might result in actual sound perception insynesthetes, perhaps due to stronger than normal connections, saysSaenz, who has begun brain imaging experiments to study thisconnectivity in synesthetes and nonsynesthetes.
"We might find that motion processing centers of the visual cortexare more interconnected with auditory brain regions than previouslythought, even in the 'normal' brain," Saenz says. "At this point,very little is known about how the auditory and visual processingsystems of the brain work together. Understanding this interaction isimportant because in normal experience, our senses work together allthe time."