video: University of Toronto Department of Psychology researchers have developed a new tool called a "scene wheel" to help understand how we perceive and remember visual experiences such as witnessing crimes or accidents.
TORONTO, ON - Researchers at the University of Toronto have developed an innovative tool to aid in the investigation of how we perceive and remember visual experiences.
The new tool, referred to as a "scene wheel," will help researchers study how accurately we construct mental representations of visual experiences for later retrieval -- for example, how well an eyewitness recalls details of a crime or accident.
"We know that eyewitness testimony is not reliable," says Gaeun Son. "With the new scene wheel, we can start to characterize the specific nature of those memory failures."
Son is a PhD student in the Faculty of Arts & Science's Department of Psychology and lead author of a paper published in Behavior Research Methods that describes the scene wheel methodology.
"Studying how people perceive and remember the world requires careful control of the physical stimuli presented in experiments," says Michael Mack. "This kind of control isn't difficult in experiments using simple stimuli like colour. But it's very challenging for more complex, realistic scenes."
Mack and Dirk Bernhardt-Walther are both professors of psychology in the department and co-authors of the study.
Traditional experiments in this field involve test subjects performing tasks such as identifying which colour or which arrangement of graphic symbols most resembles a previously viewed colour or graphic. While these methods provide some insight, their simplicity imposes a fundamental limit to what they can reveal.
The scene wheel moves into a whole new experimental realm by using highly realistic images that more closely simulate our day-to-day visual experiences -- while still providing the rigorous control needed.
The wheel is a continuous, looping series of gradually changing images depicting typical domestic spaces: dining rooms, living rooms and bedrooms. The images are detailed and realistic, and vary continuously in subtle ways: tables subtly transform into desks, mirrors become framed pictures, walls become windows, etc.
The collaborators used deep-learning methods in computer vision -- specifically, generative adversarial networks (GAN) -- to create the images and arrange them in a continuous "spectrum" analogous to a 360-degree colour wheel.
"The success of this project is all thanks to the recent revolution in deep-learning fields," says Son. "Especially in GANs which is the same sort of approach used in creating so-called 'deep fake' videos in which one person's face is very realistically replaced with someone else's."
To test whether their approach worked, the researchers had subjects view a still image of a scene from the wheel for one second, followed by a blank screen. Next, the subjects were presented with a scene similar to the one they just viewed.
The subjects then altered the second image by moving their cursor in a circle around it. As they moved their cursor, the scene changed. Subjects were asked to stop their cursor when the image matched their memory of the original image.
"With the scene wheel, we've provided a new experimental bridge that brings more of the richness of everyday experience into a controlled experimental setting," says Son. "We anticipate that our method will allow researchers to test the validity of classic findings in the field that are based on experiments using simple stimuli."
What's more, the approach could lead to different applications. For example, it could potentially lead to a wheel that uses faces instead of rooms. Such a "face wheel" could take the place of police lineups which are not particularly reliable in identifying individuals.
Says Mack, "Our method will allow for a better understanding of how precise that identification of individuals actually is."