Is Online Research Losing Its Edge? The Case for In-Person Research in the Age of AI
January 26, 2024
The landscape of research has been profoundly transformed by the COVID-19 pandemic, propelling a mass transition to online research methodologies. However, the advent of sophisticated AI technologies capable of mimicking human responses has introduced new complexities, potentially undermining the authenticity of data collected through online surveys. As the Director of the Interdisciplinary Behavioral Research Center (IBRC) at Duke University, I am both a witness to the allure of online research and an advocate for the indispensable value of in-person research, especially as we face the threat of AI contamination.
The whispers of concern are growing louder in the academic community that relies on online survey data to draw scientific conclusions. The concern with AI’s interference in online surveys is multifaceted. AI like ChatGPT can be used to answer surveys, generating full responses for open-ended questions, selecting multiple-choice answers, and even watching videos in place of the participant to provide answers to content questions or even opinions. Why do participants even bother to use AI to complete surveys? There are probably many reasons. For one, this saves time and effort for people who are often paid very little for their work (researchers often pay the minimum rate, i.e., $6 for CloudResearch or $8 for Prolific platforms for example) or who have limited resources.
Whatever the reason, this new phenomenon begs the question:
How can we learn about people’s opinions, emotions, beliefs, behaviors, if the data we get is made up by ChatGPT?
This phenomenon isn’t just a hiccup; it’s a siren call to reevaluate the quality and authenticity of online research data. Could this signal a renaissance for in-person research?
The controlled environment of in-person research has always been its hallmark, ensuring that participant responses are genuine and not influenced by the distractions or multitasking common with online surveys. But the inherent value of in-person research extends beyond control and precision. In-person research allows researchers to employ nuanced designs and gather ecologically valid data that digital methods simply cannot capture. As such, it is still the preferred method for collecting behavioral measures, physiology data, video and audio from participants’ interactions, to cite a few examples. Moreover, nothing can substitute the hands-on experience and direct human interaction offered by in-person studies. It’s a vital space for refining research methods and gaining candid participant feedback, particularly valuable in early-stage research. It also provides an indispensable learning ground for students and emerging researchers to engage intimately with the nuances of human behavior—a dimension of education that virtual environments cannot match.
However, the truth is, online research isn’t going anywhere. Its convenience and expansive reach remain unmatched, especially for engaging hard-to-reach demographics or conducting cross-cultural studies. Given this landscape, we must both fortify our methodologies with safeguards against AI manipulation and facilitate access and reach for in-person research.
At the IBRC, we support researchers engaged in social and behavioral science on both counts. I highlight two particularly relevant resources. We offer a large and versatile research space, with individual and group testing rooms equipped with computers. The IBRC is conveniently located on 9th street and has free parking to facilitate in-person research. We also manage a large community participant pool, which can be leveraged for both in-person and online studies. For in-person studies, the pool is meant to increase ease of recruitment without sacrificing representativity and diversity, a common critique against in-person research that has often relied on convenience student samples. For online studies, the pool offers additional safeguards against low engagement by paying participants a living wage. We further provide advice to add data quality checks for online studies.
In sum, with the rise of AI contaminating online research, in-person research stands to regain prominence, not just for its data integrity but for the invaluable insights gleaned from direct human interaction. I extend an invitation to my colleagues in the field of social and behavioral research to rediscover the merits of in-person studies and explore the resources and support offered by the IBRC. Let’s work together to preserve the integrity of our scientific inquiries and ensure the training of future researchers includes the irreplaceable component of human experience.
Patty Van Cappellen, Ph.D.
Director, Interdisciplinary Behavioral Research Center
Assistant Research Professor, Social Science Research Institute