Popular Science’s Action Highlights Gap at Social Science/Media/Democracy Interface

graphic

The reining-in of a well-known science magazine’s comments section is aimed at improving science understanding and ‘literacy.’ The move highlights critical communications puzzles in the quickly evolving digital media world.

Popular Science editors recently shut down the site’s comment section for most articles because they feel uncivil comments reduce the credibility of the science presented in the articles themselves. It’s a step, the editors hope, in helping roll back the “war on expertise.”
Popular Science cover
But some immediately questioned if it was also a rollback of the open ethos of the Web and a rejection of the 21st century media commitment to two-way conversation. The science community has been abuzz, and the wider media world is also chiming-in.

Simultaneously, a group of scholars convening for the Sackler Colloquium at the National Academy of Sciences contemplated these very questions about distortion of science in a digital media era. Few definitive policy-oriented conclusions there, but lots of plans for future research agendas.

The two events highlight a gnawing problem unsettled at any level of the information and communications system.

The core issues for the media are not new. How much should news outlets let the public “in”? How prominent should those voices be on a website? What editorial strategies and technologies can promote moderation and civility? Jeff Sonderman and Andrew Beaujon at the Poynter Institute have made this perpetual debate something of a beat, in and of itself.

Internet “trolls” have long known that posting early and often can tilt discourse toward their viewpoint and scare away more reasonable commenters. But well-intentioned efforts to moderate user comments in some cases have resulted in dubious or unintended outcomes.

Questions for News Editors … and Thomas Jefferson

Those on the digital side of a newsroom are used to struggling with questions of trolls and flaming, “sock-puppeting,” and legal questions about hosting hate speech. Many have looked to solutions such as comment registration through Facebook — which requires real names, thus limiting anonymous messages — or deploying tools like Disqus, which allows votes up or down on individual comments, relying on, and hoping for, the wisdom of the “crowd.”

Every business-side news manager must ponder, at some level, a ruthlessly practical cost-benefit question and calculus: Does all this “engagement” entice sufficient Web traffic that can be monetized through “CPMs” — advertising rates per 1,000 views? Or does messy hoi polloi engagement instead hurt the product brand, diminishing subscription rates? Are comment forums — civil and uncivil alike — a service that readers appreciate and that make a news product more valuable, with some journalists now seeing their role as that of conversation “convener”? Meanwhile, journalists love to see their work light up with hundreds of comments, but the tradeoffs can involve tolerating incivility and enduring personal attacks — as has long been the case when reporting on climate change.

Dominique Brossard addresses ‘the nasty effect’ of online incivility.

There are loftier issues at stake, too, such as those involving cultivating an informed public and serving democracy. Popular Science raises exactly these questions in its editorial policy announcement; its rationale leans heavily on research publicized earlier this year in a New York Times opinion piece by the University of Wisconsin science communications scholars Dominique Brossard and Dietram Scheufele. That article was based on a study they co-authored in the Journal of Computer-Mediated Communication, “The ‘Nasty Effect’: Online Incivility and Risk Perceptions of Emerging Technologies.” They found in their study that uncivil comments can filter and distort reader understanding; but this was mostly true for those with strong prior beliefs or higher levels of religiosity. The news subject tested was nanotechnology, a kind of “blank slate” where the public is not particularly “primed” ideologically in one direction or another.

Popular Science online content director Suzanne LaBarre wrote: “If you carry out those results to their logical end — commenters shape public opinion; public opinion shapes public policy; public policy shapes how and whether and what research gets funded — you start to see why we feel compelled to hit the ‘off’ switch.” LaBarre then faced pushback from, among others, Columbia Journalism Review’s Alexis Sobel Fitts, who argued that the merely suggestive and circumscribed research findings don’t justify the Pop Sci editorial policy conclusion so tightly restricting comments.

Social Science Under-delivers for News Insight

So what do the authors of the study in question make of all this?

“I think Popular Science really should be commended for turning to research,” Scheufele said in an interview. “We are making so many decisions based on intuition, based on hunches — on what we think will work. We have increasingly slim margins in the news industry. The more we can rely on empirical research to make those decisions to guide us to a) commercially viable solutions and b) solutions that produce the best outcomes in terms of democratic citizenship, the better.”

Still, he maintains that “there’s not a one-size-fits-all solution based on our study.” And he also highlights a major research gap:

I would very much argue that social science has under-delivered on this topic. A lot of editors have asked us, “What is the other research that is out there? Is there other research?” And there really is not that much yet. I think this is social science not living up to its ability to do research on how we are going to navigate this new and rapidly changing information environment. The burden is really on us, and we really do need to respond to that challenge, and respond to that challenge in a timely fashion to be relevant to news organizations that are struggling.

Dietram Scheufele says social sciences research needed to meet pressing newsroom needs. 

Scheufele — whose 2013 essay “Communicating Science in Social Settings” in Proceedings of the National Academy of Sciences (PNAS) highlights the more general problems for explaining science in any context — points to a rather striking fact about the communications research community. On a central issue of how the 21st century public square should be constructed and governed — how the Fourth Estate should moderate and encourage digital speech about issues before society — he says America’s sociologists, mass communications and journalism scholars, and political scientists have largely been absent.

News editors have simply eyeballed trends and guessed or randomly fiddled around with policy tweaks to improve comment threads. Sure, their Google Analytics can tell them something about unique page views and bounce rates; and these might be correlated with comment thread policies. But there remains much to learn about citizen knowledge and the wider civic good.

Brossard’s new literature review article in PNAS, “New Media Landscapes and the Science Information Consumer,” concludes: “In short, a well-written, balanced news story about a scientific topic may be interpreted differently because of the comments that follow it. Obviously, these results have important implications for science communication online and point to the importance of carefully evaluating how scientific information is presented, under which format, and with what type of contextualization.” She also notes that research in this area remains “scant.”

(As an aside, many news outlet audiences represent their own distinct, representative sample of the American public — skewing by income, race, geography, education, cultural values, etc. Although it makes general research findings all the more difficult, it may speak to the need for closer, embedded collaboration between social scientists and their regional news outlets.)

Research on Anonymous Comments

A few other studies also stand out.

In July 2013, Arthur Santana, a former newspaper reporter now on the communications faculty at the University of Houston, published a study in Journalism Practice that found requiring users to register under their real names reduces incivility in comment forums. He coded and analyzed content relating to immigration issues from a sample of news outlets that allowed anonymous postings in 2010 — Los Angeles Times, Arizona Republic and Houston Chronicle. Santana then compared the results with an analysis of a group of newspapers not allowing anonymity: Buffalo Daily News, Statesman Journal (Salem, OR), Burlington Free Press (Burlington, VT), USA Today, Detroit Free Press, Indianapolis Star, San Jose Mercury News, El Paso Times, Hartford Courant, Wall Street Journal, and Los Angeles Times blogs.

Santana concluded that anonymous comments are more likely to be uncivil: “Alarming is the finding that more than half of the comments in the anonymous forums following news on immigration were uncivil, meaning they contained some sentiment of hate, xenophobia or bigotry, thus representing a flagrant violation of the guidelines that online newspapers set for themselves.” By contrast, only about 29 percent of comments on sites with registered users were uncivil.

Other recent studies complement these findings: “Opinion Expression during Social Conflict: Comparing Online Reader Comments and Letters to the Editor,” published in Journalism by Jay Hmielowski and Michael McCluskey of Ohio State University; “Discussions in the Comments Section: Factors Influencing Participation and Interactivity in Online Newspapers’ Reader Comments,” published in New Media & Society by Patrick Weber of the University of Zurich; and “That’s Not the Way It Is: How User-Generated Comments on the News Affect Perceived Media Bias,” published in The Journal of Computer-Mediated Communication by Eun-Ju Lee of Seoul National University.

Still, there are those who argue anonymity has a strong value in the digital age — that enforcing “real names” policies is to be lamented, on any Web platform — particularly for those who may lack institutional support and may be vulnerable to online attacks for expressing their views.

Reporter/Editor Engagement — The ‘Gold Standard’?

The New York Times chooses 17 articles each day that will feature an open reader comment section. An editorial team is dedicated to moderating each comment, a luxury that few other outlets can afford. But many smaller outlets do have “community engagement” managers and editors who do some of this kind of work.

Natalie Jomini Stroud finds that reporter involvement in moderating content can lead to less incivility.

Research does suggest that a policy of editorial staff moderation may help achieve some balance between gatekeeper iron curtains and a verbal free-for-all.

A report from the Engaging News Project, from the Annette Strauss Institute for Civic Life at the University of Texas at Austin, finds preliminary evidence that comment moderation by reporters can pay off. That conclusion fits with the experiences of many newsroom reporters and editors. It is an approach practiced by many prominent bloggers in the climate change and science journalism communities.

“We’re only beginning to understand best practices, but our research with one local news station found that reporter involvement in the comment section resulted in lower levels of incivility,” Natalie Jomini Stroud, a scholar who oversees the Engaging News Project, said in an e-mail interview. “The reporter didn’t cure incivility, but the study does suggest that there are ways of reducing incivility. Perhaps other practices, such as badges, eliminating anonymity, or highlighting strong comments, also reduce uncivil commenting and can help to create a productive discussion space where citizens learn from each other.”

The Commenting Future: Rollback, Refinement — or Retrograde?

As news organizations have entered the digital age, two fundamental news audience-related decisions have colored much of media culture.

The first was the decision, originating in the late 1990s, to give away content for free, conditioning expectations among consumers. It’s an issue still being researched and debated — and some call it an “original sin.” As this decision continues to be revisited, more news paywalls have been erected as media outlets rethink digital age economics.

The other big decision, less obviously consequential from a business standpoint, was the opening up of news Web platforms to readers. Between 2007 and 2008, the percentage of the top 100 newspapers that allowed online comments went from 33 to 75 percent. It may be no coincidence that this transformation took place at precisely the time that Twitter first began to attract notice and Facebook was taking off. In other words, open comment forums were of a piece with the historical ethos of “Web 2.0″ — a cultural shift toward an ethic of digital openness and “platform” thinking.

Further movement toward news outlet paywalls could reshape audience engagement, as many visitors would already be logged-in, registered, paying consumers.

Meanwhile, as research and media editorial policy struggle to catch up, the Web is changing toward an inchoate “3.0″ stage, and social media engagement with news content has taken off. Many articles are boiled down and remixed into 140 character nuggets and “tweets,” with context, facts and quotes lifted out, and content is often being unbundled from the static page and the traditional news article format.

Smarter algorithms that can do more sophisticated “sentiment analysis” and other innovations around what’s being called the “semantic Web” might be enlisted to surface and feature certain kinds of audience comments. One can even imagine these algorithmically gathered comments as an active, endlessly flowing part of any news story, in the way reporters have traditionally gathered quotations from people through “shoe-leather” interviewing and newsgathering. Google has begun pioneering the use of machine learning and algorithmic curation of social media to help better order and surface relevant comments on YouTube posts. It suggests a future where all comments are “personalized” for individual visitors.

Even if social science catches up with the relatively “old” problem of comment thread trolls — and as journalists over time settle on best practices — the speed of the digital world may already be complicating things further for issues such as improved public understanding of climate science and science generally.

John Wihbey

A regular contributor to The Yale Forum, John Wihbey is an editor and researcher at the John F. Kennedy School of Government at Harvard. (E-mail: johnwihbey@gmail.com, Twitter: @wihbey)
Bookmark the permalink.

5 Responses to Popular Science’s Action Highlights Gap at Social Science/Media/Democracy Interface

  1. The dubious quality of the vast majority of comments in public comment forums generally adds absolutely nothing to any serious discussion. Even the best seldom contribute to increased understanding. They almost always are simply a place for readers to vent their emotions, devoid of fact. I completely agree with the editors: the bile most comments express takes away from — rather than add to — the articles that are the subject of the comment. The anonymous nature of comments makes these sections little more than graffitti painted on a wall. If a person has something serious, well-based and worthwhile to say, they should put it in a letter to the editor, complete with signature. This sill not be the last publication to take this step (I hope!)

  2. Miner49er says:

    What, we’re supposed to just accept uncritically the opaque conclusions of a claque of brahmins whose own qualifications are far from proven? Conclusions arrived at in secret and vetted through a “peer-review” process that seems most like a junior high school cafeteria?

    Computer models are highly suspect in guiding public policy. They can be useful when the modelers are risking their own capital. (But even then they can lead things awry. Remember the Long Term Capital Management and mortgage derivatives scandals?) Can “climate scientists” (an ambiguous term, indeed) prove their models are better? So, no–models are simply not ready for prime time in setting public policy that compels expending $65 trillion.

    When a political administration is as vindictive as this one, anonymous comments are essential.

  3. Nullius in Verba says:

    “I cannot praise a fugitive and cloistered virtue, unexercised and unbreathed, that never sallies out and sees her adversary but slinks out of the race, where that immortal garland is to be run for, not without dust and heat. Assuredly we bring not innocence into the world, we bring impurity much rather; that which purifies us is trial, and trial is by what is contrary.”

    Milton, Areopagitica.

    That which purifies science is trial, too. The reason we can trust science – the reason it works – is that every claim can be challenged. Our confidence is founded on it being endlessly checked and rechecked, and having survived every challenge thrown at it. Conversely, there can be no confidence in claims for which challenges have been silenced or hidden – for we cannot ever know whether the reason we know of no counterarguments is that there are none, or merely that publicly speaking them is not permitted. To censor opposition is to sabotage our reasons for trust in science at their root, to cripple it, to leave it defenseless. For criticism and scepticism are science’s immune system, scouring its body for errors and misunderstandings, and mercilessly casting them out.

    Everyone knows that the most common reason for refusing to debate is knowing that one’s argument cannot withstand it, and that its weaknesses would be revealed. Whether true or not, it is what many will assume. There is a certain glamour in censored ideas, and hearing that there is a counterargument without knowing what it is may prove more persuasive than if it was shown out in the open – its every flaw unveiled, and visible for all to see.

    This move will be painted by its critics as both fundamentally unscientific, and evidence of a craven lack of confidence in their own arguments – and with full justification. They’re shutting down the debate because they keep losing. And so they can still pretend to themselves it is not lost.

    And finally, the incivility is generally in response to incivility. It is a response to intolerance for different beliefs, anger – even hatred – for those who disagree, contempt for those who do not ‘know’ as we do, who do not hold the same faith. The vast majority on *both* sides, including those who speak for the scientific mainstream, do not understand the science themselves. They only know what they have been told “science says”, and hold it to be true (or not) out of trust in their authority.

    “The first is, to allege the opinions of men, whose parts, learning, eminency, power, or some other cause has gained a name, and settled their reputation in the common esteem with some kind of authority. When men are established in any kind of dignity, it is thought a breach of modesty for others to derogate any way from it, and question the authority of men who are in possession of it. This is apt to be censured, as carrying with it too much pride, when a man does not readily yield to the determination of approved authors, which is wont to be received with respect and submission by others: and it is looked upon as insolence, for a man to set up and adhere to his own opinion against the current stream of antiquity; or to put it in the balance against that of some learned doctor, or otherwise approved writer. Whoever backs his tenets with such authorities, thinks he ought thereby to carry the cause, and is ready to style it impudence in any one who shall stand out against them. This I think may be called Argumentum ad Verecundiam.”

    Locke, An Essay Concerning Human Understanding.

    You can reduce the incivility by participating in the debate (too many authors write their article but then abandon it, avoiding the debate that follows) and showing a respect for the other person and their point of view. Make a space for them, where they can have their say without being disruptive. Ask questions, clarify, explore implications – don’t dismiss disagreement without thought. At the least, remaining polite in the face of incivility claims the moral high ground, and leaves the other looking the worse for it. It helps to admit to uncertainty and fallibility, to seek understanding rather than victory, to try for a more complete truth rather than to persuade. Constructive debate is a cooperative activity in which people with different opinions aid one another in each finding the strongest arguments against their own ideas, as a check on possible errors. Lead by example, because people judge what behaviour is expected of them by what they see. Treat others as you would have them treat you, and drown out the incivility with civil and intelligent commentary. And remember, it’s not a perfect world, but we should do the best we can.

  4. John Wihbey says:

    Steve, thanks for the comment. What is also interesting is how luridly attractive comment forums are to many (most) people — even people who detest their content! Of course, they fall at the END of most articles, so it actually takes extra effort to read to them. So why do you read them, then? Why do they commonly frustrate? Do many just wish the “public” were better educated and informed — and comment threads are the perpetual evidence that the public apparently disappoints? But isn’t that a flawed conclusion? Why can we not remind ourselves cognitively that these forums are not a representative sample of the public? I throw out those questions to you and others, as I’m curious to hear more opinions.

  5. John Garrett says:

    The deliberate attempt to induce a panic over catastrophic anthropogenic global warming in order to cause the imposition of ill-advised and ill-considered actions by unscrupulous activists has failed. The effort to cloak political machinations beneath a patina of “science” was only foiled because cooler and wiser heads prevailed.