Making social media research more reliable and reproducible

For researchers interested in studying human behavior, the explosion of social media data provides incredible opportunities. The result has been an explosion of research using this data, which was only brought to the attention of many users by Facebook’s infamous study on emotional words, in which researchers manipulated the emotions of unsuspecting users. “Powerful computational resources combined with the availability of massive social media data sets has given rise to a growing body of work that [measures] population structure and human behavior at unprecedented scale,” write researchers Juergen Pfeffer and Derek Ruths in a recent Science paper that discusses the shortcomings of social media research methods. However, despite the potential of these data sets to provide insights into human behavior, “mounting evidence suggests that many of the forecasts and analyses being produced misrepresent the real world,” Pfeffer and Ruths caution. The use of this data is rife with problems, many of which have already been addressed by other disciplines and are not insurmountable, but which must be dealt with to improve accuracy in the field.

Problems include:

• Data being obtained from skewed populations
• The lack of unrestricted access to data
• Privacy constraints which prevent researchers from retaining their data sets
• User data being effectively self-reported
• Publication bias