University researchers have carried out the largest systematic review and meta-analysis to date of how people's perceptions of their screen time compare with what they do in practice, finding estimates of usage were only accurate in about five per cent of studies.
The international team say this casts doubt on the validity of research on the impact of screen time on mental health, and its influences on government policy, as the vast majority rely on participants to estimate (self-report) how long they spend on digital devices, rather than logs of actual usage, or tracked time.
"For decades, researchers have relied on estimates of how we use various technologies to study how people use digital media and the potential outcomes this behaviour can lead to. Our findings suggest that much of this work may be on unstable footing," said lead researcher Dr Doug Parry at Stellenbosch University.
"The screen time discrepancies highlight that we simply do not know enough yet about the actual effects (both positive and negative) of our media use. Researchers, journalists, members of the public, and crucially policy makers need to question the quality of evidence when they consider research on media uses and effects. We can no longer simply take claims of harmful effects at face value."
The researchers also investigated whether questionnaires and scales addressing 'problematic' media use, such as excessive or so-called 'addictive' media use, were suitable substitutes for logged usage. They found an even smaller association with usage logs for these measures.
Published in Nature Human Behaviour, the research identified every existing study that compares logged or tracked media use measures with equivalent self-reports. They screened over 12,000 articles for inclusion and found 47 studies that included both types of measures. From here they were able to identify and extract 106 comparisons, based on 50,000 individuals, to address the question of how closely self-report estimates relate to logs of actual usage.
"These highly flawed studies are over-inflating the relationships between digital media use and typically negative outcomes, such as mental health symptoms and cognitive impairments, which of course explains the pervading view that smartphones among other technologies are bad for us," said Dr Brit Davidson from the University of Bath's School of Management.
"Media and technology use takes the blame for everything from increases in teenage depression and suicide to higher incidence of Attention Deficit Hyperactivity Disorder (ADHD) and violence. If we want to properly investigate harms, we must first tackle assumptions about screen time and disentangle how people are actually using their phones or other technologies of interest.
"Importantly, these questionable studies are also being used to influence policy. The UK and Canada both have forms of screen time guidelines based on poorly conducted research, which is clearly worrying and hard to reverse."
The research team also included Dr Craig Sewall at The University of Pittsburgh; Dr Jacob Fisher at the University of Illinois Urbana-Champaign; Hannah Mieczkowski at Stanford University; and Dr Daniel Quintana at the University of Oslo.
The researchers hope that their study will lead to a shift in measurement practices regarding technology, alongside starting to correct the narrative regarding technology and society. They say that only by better understanding what people actually do with their technologies, can we start to genuinely understand the impact of them on people and society.
A systematic review and meta-analysis of discrepancies between logged and self-reported digital media use is published by Nature Human Behaviour: