{"id":699,"date":"2021-01-29T00:03:59","date_gmt":"2021-01-29T07:03:59","guid":{"rendered":"http:\/\/ctlj.colorado.edu\/?p=699"},"modified":"2021-03-16T15:15:20","modified_gmt":"2021-03-16T21:15:20","slug":"heller-1-20-21-final-draft","status":"publish","type":"post","link":"https:\/\/ctlj.colorado.edu\/?p=699","title":{"rendered":"Enlisting Useful Idiots: The Ties between Online Harassment and Disinformation"},"content":{"rendered":"

<\/a> Enlisting Useful Idiots: The Ties between Online Harassment and Disinformation<\/p>\n

Brittan Heller[1]<\/a><\/sup>*<\/p>\n

Online harassment has been a blind spot for major platforms for many years. The problem became mainstream with Gamergate in 2014, the first public reckoning with intimidation of women in the online gaming community. This problem is still plaguing social media, with progress being made in fits and starts after publicized incidents of bullying or silencing of minority voices.<\/em><\/a>[2]<\/a><\/sup> However, the problem has grown beyond these applications to new harms. Online harassment has created serious policy, technical, and structural vulnerabilities that have been exploited by malign actors and gone largely unnoticed\u2014or unprioritized\u2014by defenders. Trolling has become the vocabulary and testing ground of digital authoritarians. Understanding how online harassment works is integral to combatting State-based disinformation and efforts to undermine faith in both democracy and the internet.<\/em><\/a>[3]<\/a><\/sup><\/em><\/p>\n

This Article describes the threat posed by online harassment and then outlines how lessons learned from combatting online harassment can be used to counter a wide range of disinformation actors. To do this, the Article will define \u201cdigital authoritarians\u201d to show how disinformation and online harassment are connected. It will then use two examples of online harassment to demonstrate how the phenomenon has been a continual problem for online platforms, using post-Gamergate examples. These stories show how harassment has grown more complex\u2014allowing malign actors to go beyond intimidating individuals to targeting entire communities via systemic platform vulnerabilities. The weaknesses are created by missing or underenforced content moderation policies. While many articles and essays discuss the harm of online harassment, few scholars and advocates discuss the influence of harassment on elections or democracy in general.<\/em><\/p>\n

This Article concludes with three concrete suggestions of how industry can counter harassment-based disinformation and the targeting of vulnerable persons and groups. First, companies should commit to a risk-based allocation of resources. This is especially important when it comes to addressing harassment and disinformation. Second, companies should also follow best practices in other industries by conducting human rights impact assessments. Harassment should be included in the topics surveyed. And third, companies should take a systemic, and not a piecemeal, approach to online harassment by looking at behaviors instead of actors and content. If these are accomplished, online harassment can be taken out of the digital authoritarians\u2019 toolbox as a means to influence elections, undermine democracy, and eliminate political critics\u2014which means it is more critical than ever to take online harassment seriously.<\/em><\/p>\n

<\/a> Introduction<\/p>\n

Maria Ressa is a world-renowned journalist with a problem.<\/a>[4]<\/a><\/sup> She is the founder and CEO of Rappler, an independent investigative news outlet founded in the Philippines in 2012.[5]<\/a><\/sup> Her background consists of over thirty years of experience, including running CNN\u2019s Manila and winning the 2018 Person of the Year Award from Time Magazine.[6]<\/a><\/sup><\/p>\n

Despite her renown and expertise, in recent years Ressa has found both Rappler\u2014and herself\u2014the targets of online smear campaigns. Ressa\u2019s prominence grew with her coverage of the populist and authoritarian government in the Philippines. After the 2016 election ushered in Rodrigo Duterte\u2019s administration, a series of widespread disinformation campaigns targeted the president’s critics, his political opponents, and independent media.[7]<\/a><\/sup><\/p>\n

Ressa documented these systematic campaigns of online harassment and mapped them to government sources. The attacks included networks of pro-Duterte bloggers and bot-driven social media accounts designed to malign the independent press and increase support for the president.[8]<\/a><\/sup> Ressa described the ferocity of the harassment and its origin in her own words:<\/p>\n

Rappler and I became a target after we did a series on the \u201cpropaganda wars.\u201d We released stories that showed how this hate was being used to create doubt in institutions and in journalists\u2026 That triggered a wave of attacks against me and against Rappler that reached as many as ninety hate messages per hour\u2026 ninety messages a week, you can handle, but an hour? That becomes a whole different ball game, and our response to it was to do what we do as journalists: to shine a light and tell people that these attacks were happening, that journalists were being targeted.\u00a0.\u00a0.<\/p>\n

After journalists were targeted, opposition politicians were next, and the one who I think really bore the brunt of the propaganda machine\u2019s attacks was Senator Leila de Lima, the former Commission on Human Rights chief and justice secretary who had been investigating Duterte and then became a Senator; President Duterte began targeting her and within a few months, she was jailed.\u00a0.\u00a0.\u00a0. The harbinger of the attacks against her in the real world was a social media campaign. What we saw with these attacks is not just an attempt to tear down the credibility of anyone questioning or perceived to be a critic of government, but also to seed doubt in truth, and this is where you can see the disinformation campaign that continues today.[9]<\/a><\/sup><\/p>\n

Ressa\u2019s story, unfortunately, is not unique. Online harassment, including the targeting of journalists, is an old problem.<\/a>[10]<\/a><\/sup> However, a recent development is the harnessing of online harassment as a weapon by governments around the world to silence critics and bend the truth to their will.[11]<\/a><\/sup> As this Article will show, in countries around the world\u2014from the Philippines to China, Russia and elsewhere\u2014governments have adopted techniques from trolls, exploited platform vulnerabilities, and used harassment to spread disinformation via unwitting accomplices.<\/p>\n

<\/a> Online harassment is a continual problem for online platforms.<\/p>\n

The story of Maria Ressa exemplifies how governments have invested in the online harassment game. The term \u201cdigital authoritarians\u201d describes primarily State actors who use the internet to enable mass manipulation and intimidation to meet their political ends and silence dissent.[12]<\/a><\/sup> As this Article will describe, malign actors often use online harassment to fuel their own agendas.<\/a>[13]<\/a><\/sup><\/p>\n

We can best understand how this threat has evolved, and why it is so effective, by looking at the experiences of those who have been targeted by online violence and harassment, whether or not a government agent was responsible for the harassment. As two of these examples show, harassment is an effective hammer to silence voices because the experiences of those targeted go beyond disruptive to being frightening, or even dangerous\u2014and they cannot be solved by simply turning off a computer.<\/p>\n

Noor is a young woman who lives in the Washington D.C. metro area.<\/a>[14]<\/a><\/sup> Around January 2017, at the time of President Trump\u2019s inauguration celebration, Noor was targeted by online accusations.[15]<\/a><\/sup> She was accused of setting a Trump supporter\u2019s hair on fire, and supposed video evidence was posted online and viewed by 1.5 million people.[16]<\/a><\/sup> However, the truth was that the woman in the video was not Noor.[17]<\/a><\/sup> In fact, it was a woman with only a passing resemblance to her.[18]<\/a><\/sup> When the incident happened, Noor was actually visiting her gravely ill father.[19]<\/a><\/sup><\/p>\n

The targeting of Noor was racist, sexist, and anti-Muslim, and paired offline threats with online harassment to terrorize Noor and her family. Noor was misidentified\u2014most likely intentionally by the original poster\u2014by crowdsourcing on forums like 4Chan, WeSearchr, and right-wing political forums.[20]<\/a><\/sup> The cybermob combined its fixation on her with calls to make her pay for her supposed transgressions.[21]<\/a><\/sup> Anonymous calls were placed to Noor\u2019s work.[22]<\/a><\/sup> Her email inbox and Facebook page began to fill with angry and violent communications from strangers.[23]<\/a><\/sup> After Noor\u2019s family noticed cars circling her house, they moved into a hotel for a week.[24]<\/a><\/sup><\/p>\n

Noor sought to get the misidentifying posts taken down, but at least one website that played host to the amateur sleuths demanded a statement from police indicating that Noor was not a subject or target in the investigation of the alleged assault on the protestor.[25]<\/a><\/sup> The people behind the website saw themselves as vigilante online investigators who were unmasking the perpetrator of an assault, and were suspicious of efforts to remove what they considered to be their good detective work.[26]<\/a><\/sup> This Article posits that disinformation is often a component of online harassment\u2014threats are often founded on a mixture of true and false statements, weaving a tapestry around the victim that leaves them struggling to separate truth from fiction.[27]<\/a><\/sup><\/p>\n

Initially, local law enforcement was unsympathetic to Noor\u2019s plight. Her experience was common to many harassment victims[28]<\/a><\/sup>\u2014local police told her that all she could do was to disengage from the internet and turn off her devices. Law enforcement protocol prevented the department from publicly clearing her before the investigation was positively concluded.[29]<\/a><\/sup> However, once they saw the extent of the ongoing harassment, after being presented with a dossier proving that Noor was with her father and his hospice nurse at the time of the assault, the police issued a statement exonerating her, which helped dampen the cybermob.[30]<\/a><\/sup><\/p>\n

Julie is a journalist, who must engage with the public on Twitter in order to do her job.[31]<\/a><\/sup> During coverage of the 2016 U.S. presidential campaign, Julie was targeted by masses of anti-Semitic trolls.[32]<\/a><\/sup> For example, for nineteen hours straight, Julie received a series of Tweets: herself superimposed into a concentration camp wearing a yellow Jewish star; her face pasted on a string of victims\u2019 bodies; images of herself in a gas chamber; and images of the gates of Auschwitz altered to read \u201cMachen Amerika Great.\u201d[33]<\/a><\/sup> After receiving thousands of messages like these, Julie bought herself a firearm.[34]<\/a><\/sup> Julie\u2019s fellow political journalist Elizabeth also received similar negative attention, going beyond social media to include a phone call from \u201cOvernight Caskets.\u201d[35]<\/a><\/sup><\/p>\n

This carefully seeded falsehood was enough to make both Elizabeth and the business owner part of this effort\u2014one as a victim, the other as an unwitting tool. It is telling that there is a term from Russian disinformation operations\u2014a \u201cuseful idiot\u201d\u2014that describes exactly this technique, where an unwitting bystander is used to perpetuate a disinformation campaign.<\/a>[36]<\/a><\/sup><\/p>\n

Unfortunately, these stories are not unusual. Each could be taken from the headlines in the mid-2010s, when Gamergate and other controversies like the harassment of female online personalities like Kathy Sierra introduced online harassment to the public, and would not be out of place today.[37]<\/a><\/sup> While most cases of online harassment are not as extreme as the examples listed above, what begins online often moves to offline consequences.<\/a>[38]<\/a><\/sup> Outside authorities\u2014be it platform content reviewers, school officials, or police officers\u2014are enlisted as useful idiots, squandering their resources while further victimizing the targets.[39]<\/a><\/sup> Furthermore, in many cases, the perpetrators are intimately familiar with the contours and weaknesses of the law.[40]<\/a><\/sup> They use this knowledge to their advantage to weaken enforcement or enable their own ends. For example, it is common to see coded language in calls to harass people, or reminders to harassers to let targets \u201cknow your opinion\u201d to make it more difficult for online platforms to take action against it by wrapping themselves in the mantle of freedom of expression and the First Amendment.[41]<\/a><\/sup><\/p>\n

However, online harassment has not been static for the past decade\u2014in fact, the techniques behind it have evolved and the actors using it have evolved. As people\u2019s offline and online selves grow closer together, advocates have seen more aggressive blends of online and offline harassment. And in addition to angry individuals, politically-motivated and state-backed actors are now leveraging these techniques.<\/p>\n

<\/a> Modern online harassment has evolved in the last decade, now targeting entire communities, rather than only individuals.<\/p>\n

Since 2016, researchers and advocates have learned more about how online harassment functions as a tool, sometimes for ends other than solely individual intimidation. Increasingly, online harassment is being used to target entire communities\u2014everyone who looks like or identifies with a particular target\u2014and to exacerbate social divisions and societal fissures.<\/p>\n

Today\u2019s harassment is often multivariable and intersectional. A study that the author conducted with U.C. Berkeley\u2019s D-Lab[42]<\/a><\/sup> and the Center for Technology and Society used machine learning to study xenophobic language on Reddit in the build-up to the 2016 election.[43]<\/a><\/sup> Going into the study, the researchers expected to see primarily anti-immigrant slurs and targeting of particular communities. The team\u2019s expectation was that harassers would be provoked by specific animus, like the way hate crimes target specific people based on their protected class, like race, religion, or ethnicity. But what came back was far more diverse. Top phrases ranged as broadly as \u201cJew,\u201d \u201cWoman,\u201d \u201cBlack,\u201d \u201cWhite,\u201d and \u201cHate,\u201d and the campaigns seemed to be disturbingly equal-opportunity.[44]<\/a><\/sup> Harassers targeted many communities, making these efforts look less like targeted hate crimes and more like efforts to systematically exacerbate social fissures in American communities.[45]<\/a><\/sup><\/p>\n

Harassment remains constant, but its application and nuances are not fixed; other assumptions about online harassment may not play out as one may presume based on past experiences. Advocates\u2014the author included\u2014sounded alarms about rising hate speech in the lead-up to the 2016 election, but new studies show that online hate speech was not more prevalent during the 2016 election than it is normally.[46]<\/a><\/sup> Does this mean that online harassment is normally much worse than we think, but has been historically underreported? How deeply is hate speech now embedded in American political expression because of increased polarization and what does this look like? More study is needed to understand this phenomenon and how online harassment evolves.<\/p>\n

<\/a> Harassment is a vital tactic used in the spread of disinformation and electoral interference.<\/p>\n

What we do know, thanks to the Senate Intelligence Report about Russia\u2019s actions in the 2016 election, is that online harassment has been tied into other vectors, like disinformation and electoral interference.<\/a>[47]<\/a><\/sup> The report described how Russian actors, using bots and accounts impersonating Americans, sought to magnify divisions in the United States, particularly in flashpoint areas like race relations.[48]<\/a><\/sup> This targeting of minority groups is most effective when it is used to provoke domestic actors, fusing expressions of aggression and hatred into legitimate political expression by these unwitting proxies. An example of this occurred in 2016 in Houston Texas, where Russian-linked Facebook groups organized two offline protests\u2014one anti-Muslim and one pro-Muslim\u2014to occur at the same time, in front of the same mosque, on the same day.[49]<\/a><\/sup> The event appeared to be protest and counter-protest to American observers and resulted in confrontations and exchanges of verbal abuse.[50]<\/a><\/sup> The atmosphere was tense and the potential for violence was high.[51]<\/a><\/sup> Notably, the instigators, safe in Russia, never had to set foot in the United States.[52]<\/a><\/sup><\/p>\n

Harassment is also leveraged by state actors and politically-motivated actors for reasons that go beyond the personal targeting of their immediate victims. This has been intricately tied to politics. For example, in Brazil, the Supreme Court is investigating the so-called \u201coffice of hate,\u201d a group of online influencers, bots, and digital marketing firms allegedly hired by President Bolsonaro and his allies to disparage critics of the regime and create manufactured waves of pro-government public opinion.[53]<\/a><\/sup> Observers have noticed the professional nature and narrow scope of the attacks, like individual harassment of Bolsonaro\u2019s political opponents and multiple daily pro-Bolsonaro issue-based campaigns.[54]<\/a><\/sup> Facebook did an investigation and took down networks of fake accounts tied to employees in the office of Bolsonaro, his sons, and other conservative allies of the president.[55]<\/a><\/sup> These accounts commented on \u201cdomestic politics and elections\u201d and posted \u201ccriticism of the political opposition, media organizations and journalists.\u201d[56]<\/a><\/sup><\/p>\n

Sometimes masses of people will organize to harass political opponents and abuse the platform architecture designed to protect people from harassment. For example, in Vietnam, cybermobs reportedly coordinated in 2014 to participate in a tactic known as brigading: coordinated groups targeting critics online; in this case, the brigade hit the \u201creport\u201d button en masse on their target\u2019s content, resulting in Facebook kicking political activists or independent journalists off the platform.[57]<\/a><\/sup> The mechanisms intended to mitigate abuse resulted in at least forty-four independent Vietnamese journalists being removed from Facebook:<\/p>\n

For journalist Pham Doan Trang, it\u2019s meant that the free space of Facebook has become effectively controlled by her political opponents. \u201cThey have been escalating since mid-June (2014),\u201d Trang says, \u201cby now, it\u2019s hundreds of pages that have been knocked down,\u201d both from individuals and larger publications. Generally, the pages were taken offline through Facebook raids like the one that targeted Trang\u2014a large group of people all pressing the Report Abuse button at once. Anything they were trying to say is effectively silenced, whether it\u2019s breaking news or reports from a protest.[58]<\/a><\/sup><\/p>\n

While attribution is difficult, the pattern is clear: pro-government forces report critics and manipulate platform mechanisms to silence critiques of the Vietnamese government. Furthermore, tactics for mass media manipulation adapt according to digital authoritarians\u2019 evaluation of the political environment and the best way to spread their intended message. The groups comprising useful idiots are not always the usual suspects\u2014and are increasingly volunteer conscripts. For example, in China, it was commonly known that pro-government forces would pay individuals to post positive content or to disparage opponents.<\/a>[59]<\/a><\/sup> Starting in 2016, an organic movement emerged, starting on an online soccer forum called Diba that became the center of nationalistic online attacks:<\/p>\n

[Diba is] known for its highly organised nationalist \u201cbattle missions\u201d. Its troops are divided into groups and assigned different tasks for its actions, which are always advertised in advance on their social networks.\u00a0.\u00a0.\u00a0. During the attacks, some Diba members translate poems and pro-China slogans into various languages or create memes \u2013 the \u201cammunition\u201d as it is known \u2013 while others are administrators, directing the \u201cbattle\u201d from headquarters, sending out links and new materials for the troops to copy, paste and spam.[60]<\/a><\/sup><\/p>\n

On Diba, a forum like Reddit, users publicly brag of their successes targeting Taiwanese pro-independence celebrities and politicians, including Taiwanese President Tsai Ing-wen; hundreds of thousands of Diba users inundated the comments of their targets\u2019 Facebook pages, leading to Diba accounts being banned by Twitter, Facebook, and YouTube.[61]<\/a><\/sup><\/p>\n

Current Diba efforts target other non-Chinese interests, like pro-Hong Kong democracy activists, and misuse online architecture to stifle debate.<\/a>[62]<\/a><\/sup> Reports state that fans of \u201cA-zhong\u201d (a fictional persona standing in metaphorically for the Chinese state, who is often depicted like a pop star) \u201care encouraged to refrain from engagement with pro-Hong Kong commenters, who may try to \u2018brainwash\u2019 them; instead, team leaders suggest, it\u2019s better to immediately report those Instagram users who appear to be spreading propaganda.\u201d[63]<\/a><\/sup><\/p>\n

The prevalence of these digital brigades has led to an online expression, wumao\u00a0or \u201c50 cent trolls,\u201d a term alluding to the fact that previous online armies were known to be paid by the Chinese government; the term denigrates a poster, assuming they are trolling-for-hire and insulting their expression as cheap.[64]<\/a><\/sup> China\u2019s online citizen armies are a newer technique that enlist patriotic youth who volunteer to troll and spread Chinese nationalistic memes and pro-government content out of a sense of civic pride.[65]<\/a><\/sup> This organic brand of troll is diverse and has included Chinese overseas expatriates, rappers, and teenage fangirls\u2014all of whom \u201crefuse to be labelled as nationalists, saying they simply want to offer counterpoints to \u2018misinformation\u2019 about China, or to protect China from being smeared. Like Diba, they find strength in numbers and work in a similar, coordinated fashion.\u201d[66]<\/a><\/sup><\/p>\n

The relationship between these trolls and the Chinese government officials is very close, suggesting it may be sanctioning the efforts.[67]<\/a><\/sup> Politicians have been supportive of some of these groups, and the Chinese Communist Youth League has endorsed the movement.[68]<\/a><\/sup> These newer efforts contrast with paid astroturfing by government or private parties, and instead weaponize internet subcultures to harass political opponents.[69]<\/a><\/sup> China may have learned that the best useful idiots may not be idiots at all\u2014but willing authentic parties eager to harass others.<\/p>\n

<\/a> Because online harassment is hard to define, and rules prohibiting it are underenforced, it is systematically exploited by malign actors.<\/p>\n

Advocates have been complaining about online harassment for almost as long as the internet has existed.[70]<\/a><\/sup> While the problem itself is undeniable, it is notoriously hard to get a handle around the scope, scale, and shifting content of online harassment.<\/a>[71]<\/a><\/sup> Because of this volatility, it should be no surprise that digital authoritarians and malign online actors with ulterior motives would seek to exploit it.<\/p>\n

Harassment is extremely hard to quantify for any single actor because it often spans multiple platforms.<\/a>[72]<\/a><\/sup> Perpetrators exploit this weakness, often transferring abuse from platform to platform, so no single company can fully grasp, or mitigate, the entirety of the abuse that a target is facing.[73]<\/a><\/sup> Restrictions on companies sharing information about their users are designed to protect privacy, and while this is an important goal, one unfortunate consequence is to limit platform coordination and increase the burden on targets themselves to combat multi-platform harassment.[74]<\/a><\/sup><\/p>\n

Different platforms have their own unique terms of service, and their users have distinct understandings of what constitutes harassing behavior on different platforms.<\/a>[75]<\/a><\/sup> This means that even though online harassers use multiple platforms, the platforms themselves are not created equal. Different types of social media and policy approaches to online harassment lend themselves to campaigns of online harassment in different ways.<\/p>\n

For this analysis, it is useful to divide up social media platforms in two dimensions: (1) based on the type of social media they host (images, video, message boards); and (2) based on the approach their content policies take to addressing online harassment (permissive and mostly unmoderated; mixed tolerance of harassing content; or restrictive against harassment). This second distinction is particularly important. Some platform policies clearly condemn online harassment, while others take a far more permissive approach, often rooted in theories of free speech or libertarianism.[76]<\/a><\/sup> As will later be discussed, whether a platform has taken an aggressive stance against online harassment\u2014demonstrated through its policies, product features, and enforcement record\u2014should be a significant factor for legislators, regulators, and civil society groups assessing how to treat that platform.<\/p>\n

 <\/p>\n\n\n\n\n\n\n\n\n
Table: Platform Approaches to Online Harassment<\/td>\n<\/tr>\n
<\/td>\nType of Social Media Platform<\/strong><\/td>\n<\/tr>\n
Image Sharing<\/strong><\/td>\nVideo<\/strong><\/td>\nMultiple Formats<\/strong><\/td>\nDiscussion Board<\/strong><\/td>\nMessaging<\/strong><\/td>\n<\/tr>\n
Approach to online harassment<\/strong><\/td>\nRestrictive<\/strong><\/td>\nInstagram, Snapchat<\/td>\nTikTok<\/td>\nFacebook, Twitter, LinkedIn<\/td>\nReddit<\/td>\n<\/td>\n<\/tr>\n
Mixed<\/strong><\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\nWhatsApp, Telegram<\/td>\n<\/tr>\n
Permissive<\/strong><\/td>\n<\/td>\n<\/td>\nParler, VKontakte<\/td>\nGab, 4Chan, 8Chan<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

2021] ENLISTING USEFUL IDIOTS<\/em> 33<\/p>\n

Obtaining inside information to evaluate the prevalence of online harassment is not easy. Scholars have a difficult time getting back-end information from platforms for researching online activity, although this is starting to change.[77]<\/a><\/sup> For example, Facebook now issues requests for proposals from academics to partner with them to study online phenomena, like hate speech and disinformation.[78]<\/a><\/sup> Twitter has also worked with researchers and issued requests for proposals to study how to increase the health of online conversations.[79]<\/a><\/sup> But currently, the most detailed research comes from internal teams at platforms or market research firms. Microsoft has conducted an annual global survey of digital civility since 2016.[80]<\/a><\/sup> Pew Research has also done two landmark surveys of abusive behaviors throughout the internet ecosystem.[81]<\/a><\/sup><\/p>\n

This combined body of work has shown interesting trends. Notably, different demographics have distinct thresholds for acceptable versus harassing behaviors. While there is more online harassment reported than ever, younger generations have grown up expecting abuse or \u201cflaming\u201d to be part of their online experience, and have lower baseline expectations about user safety, privacy, and security.[82]<\/a><\/sup> This can also be seen manifesting in China\u2019s Diba community, who see their trolling as an expression of national pride, and not as a demonstration of harmful online behavior.[83]<\/a><\/sup><\/p>\n

Perhaps due in part to this realignment of user expectations, combined with the lack of transparency into platform enforcement and challenges with information sharing, online harassment presents a strategic vulnerability for many platforms. This is substantial, just like a cybersecurity breach, even though it is a failure of company policy and enforcement. It is no surprise, given the consistent complaints of inaction by victims, that harassment has become a tool used by bad actors who want to use the internet to influence elections, undermine democracy, and eliminate critics.[84]<\/a><\/sup> Oftentimes, as seen with Russia, China, Vietnam, Brazil, and the Philippines, harassment is integrated into influence campaigns, meaning \u201cany actions taken.\u00a0.\u00a0. to distort political sentiment\u2026 to achieve a strategic gain and\/or geopolitical outcome.\u201d[85]<\/a><\/sup><\/p>\n

<\/a> Because online harassment is a frequent part of influence campaigns, we need to understand how it works to grapple with emerging online threats to elections and information integrity.<\/p>\n

Examples from around the world show that trolling is the template used by digital authoritarians\u2014particularly related to the targeting of minority groups.[86]<\/a><\/sup> While journalists and political activists are particularly at-risk, it is also religious, ethnic, and racial minorities in a society that can bear the brunt of harassment from useful idiots.[87]<\/a><\/sup><\/p>\n

As previously mentioned, within studies of disinformation and foreign influence operations, we see other types of harmful online activity, like hate speech and online harassment, being used to target minority groups.[88]<\/a><\/sup> This transition is how offline phenomenon, like racial tensions, become translated into online environments. In other words, disinformation can leverage social pressures and prejudice that explode into online harassment; oftentimes the harassers are useful idiots, especially when influence operations provide them with an impetus for action. According to the Institute for the Future, eight different minority groups experienced targeted disinformation before the 2016 U.S. Presidential election.[89]<\/a><\/sup> The Oxford Internet Institute has similarly studied the gender-based facets of disinformation.[90]<\/a><\/sup><\/p>\n

Examples of disinformation leading to harassment include situations with potentially dangerous outcomes. As previously mentioned, Russian trolls organized a protest against the \u201cIslamization\u201d of a Houston community along with a counter protest, across the street, supporting Muslim neighbors in 2016.[91]<\/a><\/sup> The foreign actors never had to set foot in Texas, yet they created a situation with a real potential for violence.<\/p>\n

Those who broker disinformation know that harassment can be a bridging mechanism to offline violence and apply it accordingly. How does this work? According to Ben Nimmo, one of the foremost experts on information operations in the world, the most common tactic used by digital authoritarians is to dismiss the speaker.[92]<\/a><\/sup> In a talk given at RSA Conference 2020, the author and Nimmo describe how Morgan Freeman, who had previously espoused anti-Russia sentiments for its interference in American political processes, was targeted in Russian troll tweets for marijuana usage.[93]<\/a><\/sup> This has nothing to do with Freeman\u2019s credibility about politics, but is akin to an ad hominem<\/em> attack, insulting him to attempt to stop people from listening to his message. The dismissal tactic is designed to prevent audiences from examining evidence or taking critique seriously.[94]<\/a><\/sup> If you combine dismissal with the aim of disinformation (creating chaos by amplifying preexisting social fissures in a society), it is easy to understand how online harassment can be deployed against a speaker. Since harassers often attack using social biases against the target, this can easily be deployed in the service of information operations to amplify existing prejudices and undercut pluralistic societies.[95]<\/a><\/sup><\/p>\n

Many platforms have begun taking action to prepare for the 2020 U.S. elections, but this varies from company-to-company and on each platform\u2019s perceived level of risk.[96]<\/a><\/sup> Furthermore, while disinformation campaigns often begin with elite hackers and carefully seeded dumps of stolen documents, the end game is to engage average citizens to spread the message organically.[97]<\/a><\/sup> As previously mentioned, a successful operation will engage brigades of useful idiots, who drive home the message via attacking the speakers. In the United States, this can create a dilemma, as platforms do not want to squelch the legitimate (even if odious, hateful, or aggressive) expression of their users, within their terms of service. This is especially true when companies are under increasing scrutiny from political constituencies, who allege their political viewpoints have been censored.[98]<\/a><\/sup> It can create the perfect storm: harassment that looks like typical online hate and vitriol that some audiences have become accustomed to online, but it is more than meets the eye. This content moves the Overton window, primes audiences with disinformation, and\u2014as Russia\u2019s actions demonstrate\u2014can potentially impact democratic processes.<\/p>\n

<\/a> Potential solutions for online harassment that promotes disinformation<\/p>\n

What can be done about the new application of online harassment in service of disinformation? First, companies should apply a risk-based allocation of resources.<\/a>[99]<\/a><\/sup> When previously confronted with harassment, some companies did not have the staff, the resources, or the expertise to handle such a problem.[100]<\/a><\/sup> However, given the intersection with other platform integrity issues, this Article recommends that companies treat harassment with the same gravity that applies to other security-focused threats. As part of this analysis, companies should expand their focus to other regional markets and categories of vulnerable users, like journalists and political candidates, during especially critical times. Specifically, elections and periods of civil unrest are indicators of times when platform manipulation is more likely to occur.[101]<\/a><\/sup> For example, companies should look to users or customer bases in Africa and southeast Asia around their elections as the testing grounds for new types of media manipulations. This is where the most sophisticated threat actors are testing out new theories, and where harassment and hate speech have been gaining ground.[102]<\/a><\/sup><\/p>\n

Companies can also learn from other industries\u2019 best practices and commit to independent human rights impact assessments (HRIAs), which should include examining the prevalence, means, and likelihood of online harassment.[103]<\/a><\/sup> HRIAs are a mainstay of corporate social responsibility, and have been conducted by other volatile industries like oil, mining, and gas for years.[104]<\/a><\/sup> Internal assessments are part of some companies\u2019 commitment to multi-stakeholder processes, like the Global Network Initiative, and are part of the policies, standards, and procedures they undertake to adhere to the U.N. Guiding Principles on Business and Human Rights; however, these efforts do not incorporate online harassment as a serious user safety threat.[105]<\/a><\/sup> The benefit of looking at harassment as part of a HRIA would be that it would provide a feedback loop for companies to hear from impacted populations and allow them to anticipate risks before they hit the breaking point. As situations around the world erupt into ethnic or political violence\u2014like the targeting of the Rohingya in Myanmar\u2014online harassment of minorities can serve as a litmus test showing that tensions may be reaching the level of mass violence.[106]<\/a><\/sup><\/p>\n

Companies should understand that regulatory sands may be shifting, and legislators and the courts may pay increased attention to how platforms confront online harassment. Section 230 of the Communications Decency Act of 1996 (CDA) provides online platforms and websites broad immunity from liability for content posted by their users.[107]<\/a><\/sup> This broad protection has come under increasing fire in recent years, with lawmakers, activists, and advocates calling for it to be repealed or revised.<\/a>[108]<\/a><\/sup><\/p>\n

The intermediary liability protections afforded by Section 230 are an important enough component of the internet\u2019s legal framework that limiting its protections should be taken carefully and with due consideration for secondary consequences. One of the most promising methods to draw a line like this was proposed by Danielle Citron and Benjamin Wittes, who recommend conditioning the protections of Section 230 to apply only to sites that \u201ctake[] reasonable steps to prevent or address unlawful uses of [their] services.\u201d[109]<\/a><\/sup><\/p>\n

This Article has argued that online harassment is both harmful in its own right, and is increasingly a tool of digital authoritarians, who use it to silence critics and fuel aggressive disinformation campaigns. It has noted further that social media platforms have taken different approaches to combatting online harassment. While the scope and scale of social media means that gaps (some significant) will always exist, platforms can be categorized based on how seriously they have taken their responsibility to confront online harassment among their users.\u00a0Given this interplay it would be worth legislators or the courts considering whether they should apply Citron and Wittes\u2019 \u201cGood Samaritan\u201d proposal to combat online harassment. Perhaps platforms that knowingly encourage, or \u201cdo not take reasonable steps to prevent\u201d systemic online harassment should incur liability for the individual and societal consequences of that endemic harassment.<\/p>\n

Assessing the boundaries of this definition will be particularly delicate. An overbroad imposition of liability could burden innovation and even become a tool of the very same governments that leverage online harassment today. However, as Citron and Wittes argue, a carefully construed limitation on the broad immunity currently provided to social media platforms can avoid given \u201cplatforms a free pass to ignore destructive activities\u201d while still defending free speech online.[110]<\/a><\/sup> Given how online harassment is being wielded by repressive governments, this is an essential balance to strike not only for individual rights, but for society.<\/p>\n

Finally, companies should move to systemic approaches to combat harassment.[111]<\/a><\/sup> Patchwork efforts tend to focus only on harassing content, after it has already been posted, and thus are already too late. Best practices, derived from studies of disinformation, would include looking for harassing behaviors, instead of just content or actors.[112]<\/a><\/sup> This is a better indicator due to the rapidly shifting context behind much harassment, and the prevalence and volatility of the useful idiot problem.<\/p>\n

Companies can also build awareness of harassment into their product development pipeline, along with mechanisms that minimize the burden on targets.[113]<\/a><\/sup> If this were to occur, it is likely we would see companies as more than triage teams, and a decrease in the impact of harassment on individuals, communities, and nations\u2014and a lessening of the impact of any useful idiots or related disinformation.<\/p>\n

<\/a> Conclusion<\/p>\n

With the onset of disinformation as a tool of statecraft, it behooves companies, regulators, and citizens to view online harassment in a new light. It is no longer just an issue of personal online safety. The examples in this paper have shown how governments across the globe and their affiliated actors have used online harassment to terrorize journalists, activists, and political opponents; to widen social fissures in societies around the world based on race, religion, and ethnicity; and to disrupt democratic processes. Because of this, platforms should view online harassment as a strategic vulnerability\u2014and take immediate steps to close the policy gap, lest their products and services become playgrounds for the useful idiot problem.<\/p>\n

    \n
  1. * Brittan Heller is Counsel in the Global Business and Human Rights practice at Foley Hoag LLP. She was an inaugural 2019-2020 Technology and Human Rights Fellow at Harvard Kennedy School. Heller was the founding director of the Center for Technology and Society and prosecuted human rights and technology-related crime at the U.S. Department of Justice. She received her J.D. from Yale Law School and her M.A.\/B.A. from Stanford University.This Article is based on an address, \u201cTech Pessimism,\u201d given at Silicon Flatirons Conference on February 9, 2020. Special thanks to Amie Stepanovich, Ben Nimmo, Camille Francois, Nathaniel Gleicher, Danielle Citron, Ben Wittes, Mark Lemley, and Gare Smith for their input and support. \u2191<\/a><\/li>\n
  2. . Caitlin Dewey, The Only Guide to Gamergate You Will Ever Need to Read<\/em>, Wash. Post (Oct. 14, 2014), https:\/\/www.washingtonpost.com\/news\/the-intersect\/wp\/2014\/10\/14\/the-only-guide-to-gamergate-you-will-ever-need-to-read\/ [https:\/\/perma.cc\/7VNQ-AYJW]. \u2191<\/a><\/li>\n
  3. . See<\/em>, e.g.<\/em>, Adrian Shahbaz, The Rise of Digital Authoritarianism<\/em>, Freedom House, https:\/\/freedomhouse.org\/report\/freedom-net\/2018\/rise-digital-authoritarianism [https:\/\/perma.cc\/MWU3-XRM8] (last visited Oct. 12, 2020) (discussing digital authoritarianism). \u2191<\/a><\/li>\n
  4. . See<\/em> Maria Ressa on Digital Disinformation and Philippine Democracy in the Balance<\/em>, Nat\u2019l Endowment for Democracy, https:\/\/www.ned.org\/wp-content\/uploads\/2018\/02\/Maria-Ressa-on-Digital-Disinformation-and-Philippine-Democracy-in-the-Balance.pdf [https:\/\/perma.cc\/P8W8-GHRC] (last visited Oct. 12, 2020). \u2191<\/a><\/li>\n
  5. . The People Behind Rappler<\/em>, Rappler (June 16, 2012, 4:19 PM), https:\/\/www.rappler.com\/about\/the-people-behind-rappler [https:\/\/perma.cc\/UH6M-APWG]; About Rappler<\/em>, Rappler (Dec. 14, 2011, 8:00 AM), https:\/\/www.rappler.com\/about\/about-rappler [https:\/\/perma.cc\/2ASK-5XD2]. \u2191<\/a><\/li>\n
  6. . Maria Ressa on Digital Disinformation and Philippine Democracy in the Balance<\/em>, supra<\/em> note 3; Karl Vick, 2018: The Guardians: Maria Ressa<\/em>, Time Mag., (Mar. 5, 2020, 6:50 AM), https:\/\/time.com\/5793800\/maria-ressa-the-guardians-100-women-of-the-year\/ [https:\/\/perma.cc\/FQ56-G4MF]. \u2191<\/a><\/li>\n
  7. . Maria Ressa on Digital Disinformation and Philippine Democracy in the Balance<\/em>, supra<\/em> note 3. \u2191<\/a><\/li>\n
  8. . Id.<\/em> \u2191<\/a><\/li>\n
  9. . Id.<\/em> \u2191<\/a><\/li>\n
  10. . See<\/em>, e.g.<\/em>, Anti-Semitic Targeting of Journalists During the 2016 Presidential Campaign<\/em>, Anti-Defamation League (Oct. 19, 2016), https:\/\/www.adl.org\/sites\/default\/files\/documents\/assets\/pdf\/press-center\/CR_4862_Journalism-Task-Force_v2.pdf [https:\/\/perma.cc\/PJ8G-KJYS]; Caroline Sinders, An Incomplete (but growing) History of Harassment Campaigns since 2003<\/em>, Medium (Nov. 25, 2018), https:\/\/medium.com\/digitalhks\/an-incomplete-but-growing-history-of-harassment-campaigns-since-2003-db0649522fa8 [https:\/\/perma.cc\/SQT5-BGPK]. \u2191<\/a><\/li>\n
  11. . See Maria Ressa on Digital Disinformation and Philippine Democracy in the Balance<\/em>, supra<\/em> note 3 (discussing Chinese and Russian state-sponsored social media accounts). \u2191<\/a><\/li>\n
  12. . Shahbaz, supra <\/em>note 2. \u2191<\/a><\/li>\n
  13. . Lisa Reppell & Erica Shein, Disinformation Campaigns and Hate Speech: Exploring the Relationship and Programming Interventions<\/em>, Int\u2019l Found. for Electoral Sys. (Apr. 2019), https:\/\/www.ifes.org\/sites\/default\/files\/2019_ifes_disinformation_campaigns_and_hate_speech_briefing_paper.pdf [https:\/\/perma.cc\/6SQ7-K5ZY]. \u2191<\/a><\/li>\n
  14. . Terry Collins, Here\u2019s the Brutal Reality of Online Hate: Attack of the Trolls<\/em>, CNET (Nov. 27, 2017, 3:20 PM), https:\/\/www.cnet.com\/news\/the-brutal-reality-of-online-hate-neo-nazis\/ [https:\/\/perma.cc\/ZR3L-X8XF] (name changed for privacy considerations). \u2191<\/a><\/li>\n
  15. . Id.<\/em> \u2191<\/a><\/li>\n
  16. . Id.<\/em> \u2191<\/a><\/li>\n
  17. . Id.<\/em> \u2191<\/a><\/li>\n
  18. . Id.<\/em> \u2191<\/a><\/li>\n
  19. . Id.<\/em> \u2191<\/a><\/li>\n
  20. . Id.<\/em> \u2191<\/a><\/li>\n
  21. . Id.<\/em> \u2191<\/a><\/li>\n
  22. . Id.<\/em> \u2191<\/a><\/li>\n
  23. . Id.<\/em> \u2191<\/a><\/li>\n
  24. . Id.<\/em> \u2191<\/a><\/li>\n
  25. . The author personally knows this from her background in law enforcement, particularly from working as a prosecutor in the District of Columbia where this incident occurred. \u2191<\/a><\/li>\n
  26. . The author personally knows this from her background in law enforcement, particularly from working as a prosecutor in the District of Columbia where this incident occurred. \u2191<\/a><\/li>\n
  27. . See<\/em>, e.g.<\/em>, Reppell & Shein, supra<\/em> note 12. \u2191<\/a><\/li>\n
  28. . See<\/em> Julie Zeilinger, An interview with Cynthia Lowen, director of online harassment documentary \u2018Netizens\u2019<\/em>, Women\u2019s Media Ctr. (May 30, 2018), https:\/\/womensmediacenter.com\/fbomb\/an-interview-with-cynthia-lowen-director-of-online-harassment-documentary-netizens [https:\/\/perma.cc\/PQM4-UNLY] (explaining that disengaging from the internet is common advice given to victims of cyber abuse). \u2191<\/a><\/li>\n
  29. . The author personally knows this from her background in law enforcement, particularly from working as a prosecutor in the District of Columbia where this incident occurred. \u2191<\/a><\/li>\n
  30. . Collins, supra<\/em> note 13. \u2191<\/a><\/li>\n
  31. . Name has been changed for privacy considerations. \u2191<\/a><\/li>\n
  32. . Anti-Semitic Targeting of Journalists During the 2016 Presidential Campaign<\/em>, supra <\/em>note 9, at 3, 8. \u2191<\/a><\/li>\n
  33. . Id. <\/em>at 11\u201314. Note that while the original Tweets were deleted by Twitter, copies of the tweets using this imagery are available within the report. \u2191<\/a><\/li>\n
  34. . Matt Katz, Trump-Inspired Anti-Semitism Prompts Fear, Police Reports\u2026and a Gun Purchase<\/em>, WNYC (June 28, 2016), https:\/\/www.wnyc.org\/story\/trump-inspired-anti-semitism-spikes-prompting-conservative-writer-protect-herself-gun\/ [https:\/\/perma.cc\/X5DY-TJYJ]. \u2191<\/a><\/li>\n
  35. . Lauren Gambino, Journalist who profiled Melania Trump hit with barrage of antisemitic abuse<\/em>, The Guardian (Apr. 28, 2016), https:\/\/www.theguardian.com\/us-news\/2016\/apr\/28\/julia-ioffe-journalist-melania-trump-antisemitic-abuse [https:\/\/perma.cc\/DR5N-5MTC] (name changed for privacy considerations). \u2191<\/a><\/li>\n
  36. . RSA Conference, Your Democracy Needs You: Taking On Digital Dictatorships<\/em>, YouTube (Feb. 28, 2020), https:\/\/www.youtube.com\/watch?v=5jy4saeW5Ho#action=share&ab_channel=RSAConference [https:\/\/perma.cc\/6S62-7UDB]. \u2191<\/a><\/li>\n
  37. . Dewey, supra<\/em> note 1; Kathy Sierra, Why the Trolls Will Always Win<\/em>, Wired, (Oct. 8, 2014, 4:49 PM), https:\/\/www.wired.com\/2014\/10\/trolls-will-always-win\/ [https:\/\/perma.cc\/AYH4-56SU]. \u2191<\/a><\/li>\n
  38. . See<\/em>, e.g.<\/em>, Jacqueline Beauchere, Digital civility at lowest level in 4 Years, new Microsoft research shows<\/em>, Microsoft on the Issues (Feb. 10, 2020), https:\/\/blogs.microsoft.com\/on-the-issues\/2020\/02\/10\/digital-civility-lowest\/ [https:\/\/perma.cc\/D6SC-UZX2]. \u2191<\/a><\/li>\n
  39. . See<\/em>, e.g<\/em>., Robert Salonga, Facebook exec targeted by hoax call<\/em>, Mercury News (Jan. 9. 2019), https:\/\/www.mercurynews.com\/2019\/01\/09\/facebook-exec-targeted-by-hoax-call-drawing-heavy-police-response\/ [https:\/\/perma.cc\/S4SF-FA9L]. \u2191<\/a><\/li>\n
  40. . Luke O\u2019Brien, The Making of an American Nazi<\/em>, The Atlantic (Dec. 2017), https:\/\/www.theatlantic.com\/magazine\/archive\/2017\/12\/the-making-of-an-american-nazi\/544119\/ [https:\/\/perma.cc\/WT4V-FWLG]. But cf. <\/em>Alexia Fern\u00e1ndez Campbell, The limits of free speech for white supremacists marching at Unite the Right 2<\/em>, Vox (Aug. 12, 2018), https:\/\/www.vox.com\/policy-and-politics\/2018\/8\/10\/17670554\/unite-the-right-dc-free-speech-first-amendment [https:\/\/perma.cc\/FZ7W-WZ4T]. \u2191<\/a><\/li>\n
  41. . Through Conspiracies and Coded Language, White Supremacists Use Social Media Networks to Aid and Abet Terror, New Study Finds<\/em>, Anti-Defamation League, https:\/\/www.adl.org\/news\/press-releases\/through-conspiracies-and-coded-language-white-supremacists-use-social-media [https:\/\/perma.cc\/2KJ5-4624] (last visited Oct. 12, 2020). \u2191<\/a><\/li>\n
  42. <\/a>. Online Hate Index: Scalable Detection of Online Hate Speech<\/em>, D-Lab, https:\/\/dlab.berkeley.edu\/landing-page\/online-hate-index-scalable-detection-online-hate-speech [https:\/\/perma.cc\/7P24-BT6F] (last visited Oct. 12, 2020). \u2191<\/a><\/li>\n
  43. . The Online Hate Index<\/em>, Anti-Defamation League, https:\/\/www.adl.org\/resources\/reports\/the-online-hate-index#implications [https:\/\/perma.cc\/HFZ9-5VXN] (last visited Oct. 12, 2020). \u2191<\/a><\/li>\n
  44. . Id.<\/em> \u2191<\/a><\/li>\n
  45. . See<\/em> id.<\/em> \u2191<\/a><\/li>\n
  46. . Alexandra A. Siegel, Evgenii Nikitin, Pablo Barber\u00e1, Joanna Sterling, Bethany Pullen, Richard Bonneau, Jonathan Nagler & Joshua A. Tucker, Trumping Hate on Twitter? Online Hate Speech in the 2016 US Election Campaign and its Aftermath<\/em> (Mar. 6, 2019) at 1, https:\/\/alexandra-siegel.com\/wp-content\/uploads\/2019\/05\/Siegel_et_al_election_hatespeech_qjps.pdf [https:\/\/perma.cc\/GU4U-3556]; cf. Anti-Semitic Targeting of Journalists During the 2016 Presidential Campaign<\/em>, supra<\/em> note 9. \u2191<\/a><\/li>\n
  47. . See<\/em> Philip Ewing, Report: Russian Election Trolling Becoming Subtler, Tougher To Detect<\/em>, NPR, (Mar. 5, 2020, 3:05 PM), https:\/\/www.npr.org\/2020\/03\/05\/812497423\/report-russian-election-trolling-becoming-subtler-tougher-to-detect [https:\/\/perma.cc\/295Q-KAVK]. \u2191<\/a><\/li>\n
  48. . See<\/em> Tim Mak, Senate Report: Russians Used Social Media Mostly to Target Race in 2016<\/em>,NPR, (Oct. 8, 2019, 2:50 PM), https:\/\/www.npr.org\/2019\/10\/08\/768319934\/senate-report-russians-used-used-social-media-mostly-to-target-race-in-2016 [https:\/\/perma.cc\/V3HW-PVTR]. \u2191<\/a><\/li>\n
  49. . Claire Allbright, A Russian Facebook page organized a protest in Texas and the counterprotest.<\/em> Texas Tribune, (Nov. 1, 2017, 4:00 PM), https:\/\/www.texastribune.org\/2017\/11\/01\/russian-facebook-page-organized-protest-texas-different-russian-page-l\/ [https:\/\/perma.cc\/L357-46XA]. \u2191<\/a><\/li>\n
  50. . Id.<\/em> \u2191<\/a><\/li>\n
  51. . See<\/em> id.<\/em> \u2191<\/a><\/li>\n
  52. . See<\/em> id.<\/em> \u2191<\/a><\/li>\n
  53. . Andrew Rosati & Mario Sergio Lima, In Hunt for \u201cOffice of Hate,\u201d Brazil\u2019s Supreme Court Closes In<\/em>, Bloomberg, (June 22, 2020, 2:00 AM), https:\/\/www.bloomberg.com\/news\/articles\/2020-06-22\/in-hunt-for-office-of-hate-brazil-s-supreme-court-closes-in [https:\/\/perma.cc\/PM7U-D3RT]. \u2191<\/a><\/li>\n
  54. . Id.<\/em> \u2191<\/a><\/li>\n
  55. . Chandler Thornton & Rodrigo Pedroso, Facebook Shuts Down Network of Fake Accounts Tied to Employees of Brazil\u2019s Bolsonaro and Sons<\/em>, CNN, (July 9, 2020, 2:55 PM), https:\/\/www.cnn.com\/2020\/07\/09\/americas\/brazil-bolsonaro-facebook-fake-accounts-intl\/index.html [https:\/\/perma.cc\/6A48-72E7]. \u2191<\/a><\/li>\n
  56. . Id.<\/em> \u2191<\/a><\/li>\n
  57. . Russel Brandom, Facebook\u2019s Report Abuse Button has become a Tool of Global Oppression<\/em>, The Verge (Sept. 2, 2014, 10:16 AM), https:\/\/www.theverge.com\/2014\/9\/2\/6083647\/facebook-s-report-abuse-button-has-become-a-tool-of-global-oppression [https:\/\/perma.cc\/3F4Q-V5KG]. \u2191<\/a><\/li>\n
  58. . Id.<\/em> \u2191<\/a><\/li>\n
  59. . Phoebe Zhang & Laurie Chen, The Emergence and Evolution of China\u2019s Internet Warriors Going to Battle over Hong Kong Protests<\/em>, S. China Morning Post (Sept. 4, 2019, 7:15 AM), https:\/\/www.scmp.com\/news\/china\/society\/article\/3024223\/emergence-and-evolution-chinas-internet-warriors [https:\/\/perma.cc\/A47Q-RSET]. \u2191<\/a><\/li>\n
  60. . Id.<\/em> \u2191<\/a><\/li>\n
  61. . Id.<\/em> \u2191<\/a><\/li>\n
  62. . See <\/em>Lauren Teixeira, China is Sending Keyboard Warriors Over the Firewall<\/em>, Foreign Pol\u2019y (Aug. 26, 2019, 9:25 AM), https:\/\/foreignpolicy.com\/2019\/08\/26\/china-is-sending-keyboard-warriors-over-the-firewall\/ [https:\/\/perma.cc\/66DV-BT7C]. \u2191<\/a><\/li>\n
  63. . Id.<\/em> \u2191<\/a><\/li>\n
  64. . See <\/em>Zhang & Chen, supra<\/em> note 58. \u2191<\/a><\/li>\n
  65. . See<\/em> id.<\/em> \u2191<\/a><\/li>\n
  66. . Id.<\/em> \u2191<\/a><\/li>\n
  67. . Teixeira, supra<\/em> note 61. \u2191<\/a><\/li>\n
  68. . See<\/em> id.<\/em> \u2191<\/a><\/li>\n
  69. . See <\/em>Zheng & Chen, supra<\/em> note 58. \u2191<\/a><\/li>\n
  70. . See<\/em>, e.g.<\/em>, About ADL\u2019s Work Combating Cyberhate and Countering Violent Extremists Online<\/em>, Anti-Defamation League (Feb. 29, 2016), https:\/\/www.adl.org\/news\/article\/about-adls-work-combating-cyberhate-and-countering-violent-extremists-online [https:\/\/perma.cc\/5R9M-7PPV] (ADL\u2019s first report on online abuse was in 1985). \u2191<\/a><\/li>\n
  71. . See generally<\/em>, Maeve Duggan, Online Harassment 2017<\/em>, Pew Res. Ctr. (July 11, 2017), https:\/\/www.pewresearch.org\/internet\/2017\/07\/11\/online-harassment-2017\/ [https:\/\/perma.cc\/7CY7-NNPW]. \u2191<\/a><\/li>\n
  72. . See Online Abuse 101<\/em>, Women\u2019s Media Ctr., https:\/\/www.womensmediacenter.com\/speech-project\/online-abuse-101\/#crossPlatformHarassment [https:\/\/perma.cc\/N2RU-VUHK] (last visited Oct. 12, 2020). \u2191<\/a><\/li>\n
  73. . See <\/em>Jordan Kraemer & Danya Glabau, The Trolls are Teaming up\u2014and Tech Platforms Aren\u2019t Doing Enough to Stop Them<\/em>, Fast Company (Dec. 10, 2019), https:\/\/www.fastcompany.com\/90440915\/the-trolls-are-teaming-up-and-tech-platforms-arent-doing-enough-to-stop-them [https:\/\/perma.cc\/6F8L-YLN8]. \u2191<\/a><\/li>\n
  74. . See generally id.<\/em> \u2191<\/a><\/li>\n
  75. . See<\/em> Jessica A. Pater, Moon K. Kim, Elizabeth D. Mynatt & Casey Fiesler, Characterizations of Online Harassment: Comparing Policies across Social Media<\/em>, Proc. of the 19th Int\u2019l Conf. on Supporting Grp. Work 373 (Nov. 2016), https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2957276.2957297 [https:\/\/perma.cc\/9W68-FAZ7] (describing that platforms, too, have distinct terms of service or community standards that characterize content as \u201charassing\u201d in different ways; there is no one standard definition of harassment). \u2191<\/a><\/li>\n
  76. . See id. <\/em>at 370.<\/em> \u2191<\/a><\/li>\n
  77. . See<\/em>, e.g.<\/em>, Content Policy Research on Social Media Platforms Request for Proposals<\/em>, Facebook Res. (Jan. 30, 2019), https:\/\/research.fb.com\/programs\/research-awards\/proposals\/content-policy-research-on-social-media-platforms-request-for-proposals\/ [https:\/\/perma.cc\/37N6-JFR8]; Vijaya Gadde & Yoel Roth, Enabling Further Research of Information Operations on Twitter<\/em>, Twitter (Oct. 17, 2018), https:\/\/blog.twitter.com\/en_us\/topics\/company\/2018\/enabling-further-research-of-information-operations-on-twitter.html [https:\/\/perma.cc\/24SD-5UR4]; see also <\/em>Jeffrey Mervis, Researchers finally get access to data on Facebook\u2019s role in political discourse<\/em>, Science (Feb. 13, 2020, 5:10 PM) https:\/\/www.sciencemag.org\/news\/2020\/02\/researchers-finally-get-access-data-facebook-s-role-political-discourse [https:\/\/perma.cc\/X9GZ-4J3K]. \u2191<\/a><\/li>\n
  78. . Pater et al., supra<\/em> note 74. \u2191<\/a><\/li>\n
  79. . Twitter Health Metrics Proposal Submission<\/em>, Twitter (Mar. 1, 2018), https:\/\/blog.twitter.com\/en_us\/topics\/company\/2018\/twitter-health-metrics-proposal-submission.html [https:\/\/perma.cc\/L5EG-M4LL]. \u2191<\/a><\/li>\n
  80. . See<\/em> Beauchere, supra <\/em>note 37. \u2191<\/a><\/li>\n
  81. . See <\/em>Duggan, supra <\/em>note 70 (Pew\u2019s prior study was done in 2014). \u2191<\/a><\/li>\n
  82. . See generally Online Abuse 101<\/em>, supra<\/em> note 71; see generally<\/em> Amanda Lenhart, Michelle Ybarra, Kathryn Zickuhr & Myeshia Price-Feeney, Online Harassment, Digital Abuse, and Cyberstalking in America<\/em>, Data & Soc\u2019y, (Nov. 21, 2016), https:\/\/www.datasociety.net\/pubs\/oh\/Online_Harassment_2016.pdf [https:\/\/perma.cc\/6ZPJ-TA9P]. \u2191<\/a><\/li>\n
  83. . See <\/em>Zhang & Chen, supra<\/em> note 58. \u2191<\/a><\/li>\n
  84. . See generally <\/em>Ewing, supra<\/em> note 46. \u2191<\/a><\/li>\n
  85. . See<\/em> Reppell & Shein, supra<\/em> note 12, at 10 (an earlier term is also \u201cinformation operations\u201d). \u2191<\/a><\/li>\n
  86. . See id.<\/em> \u2191<\/a><\/li>\n
  87. . RSA Conference, supra <\/em>note 35. \u2191<\/a><\/li>\n
  88. . Id.<\/em> \u2191<\/a><\/li>\n
  89. . The Human Consequences of Componential Propaganda, Eight Case Studies from the 2018 US Midterm Elections, Institute for the Future, (Katie Josef & Samuel Woolley eds., 2019), https:\/\/www.iftf.org\/fileadmin\/user_upload\/downloads\/ourwork\/IFTF_Executive_Summary_comp.prop_W_05.07.19_01.pdf [https:\/\/perma.cc\/Y7F8-6A36]. \u2191<\/a><\/li>\n
  90. 88<\/sup> AtlanticCouncil, 360\/OS London \u2013 The Digital Authoritarian\u2019s Toolbox<\/em>, YouTube (June 21, 2019), https:\/\/www.youtube.com\/watch?v=3Dj2DE9VzGo [https:\/\/perma.cc\/HW32-Q3LG]. \u2191<\/a><\/li>\n
  91. . Donie O\u2019Sullivan, Russian trolls created Facebook events seen by more than 300,000 users<\/em>, CNN (Jan. 26, 2018, 5:13 PM), https:\/\/money.cnn.com\/2018\/01\/26\/media\/russia-trolls-facebook-events\/index.html [https:\/\/perma.cc\/9PTD-47LN]. \u2191<\/a><\/li>\n
  92. . RSA Conference, supra<\/em> note 35. \u2191<\/a><\/li>\n
  93. . Id.<\/em> \u2191<\/a><\/li>\n
  94. . Id.<\/em> \u2191<\/a><\/li>\n
  95. . Id.<\/em> \u2191<\/a><\/li>\n
  96. . For a comprehensive survey of platform policies, see <\/em>Jared Newman, Tech Platforms Screwed Up the Last Election: Here\u2019s How They\u2019re Prepping for 2020<\/em>, Fast Company (Mar. 4, 2020), https:\/\/www.fastcompany.com\/90467733\/tech-platforms-screwed-up-the-last-election-heres-how-theyre-prepping-for-2020 [https:\/\/perma.cc\/6YCY-UY74]. \u2191<\/a><\/li>\n
  97. . Id.<\/em>; Exposing Russia\u2019s Effort to Sow Discord Online: The Internet Research Agency and Advertisements<\/em>, U.S. H.R. PERMANENT SELECT COMM. ON INTELLIGENCE (last visited Oct. 12, 2020), https:\/\/intelligence.house.gov\/social-media-content\/ [https:\/\/perma.cc\/8WWA-TTGF]. \u2191<\/a><\/li>\n
  98. 97. Emily A. Vogels, Andrew Perrin & Monica Anderson, Most Americans Think Social Media Sites Censor Political Viewpoints<\/em>, Pew Research Ctr. (Aug. 19, 2020), https:\/\/www.pewresearch.org\/internet\/2020\/08\/19\/most-americans-think-social-media-sites-censor-political-viewpoints\/ [https:\/\/perma.cc\/55QD-DZXC]. \u2191<\/a><\/li>\n
  99. . Brittan Heller, Is this Frog a Hate Symbol or Not?<\/em>, N.Y. Times (Dec. 24, 2019), https:\/\/www.nytimes.com\/2019\/12\/24\/opinion\/pepe-frog-hate-speech.html [https:\/\/perma.cc\/FF2T-DGDL]. \u2191<\/a><\/li>\n
  100. . See <\/em>Jessica Valenti, If Tech Companies Wanted to End Online Harassment, They Could Do It Tomorrow<\/em>, The Guardian (Dec. 1, 2014), https:\/\/www.theguardian.com\/commentisfree\/2014\/dec\/01\/tech-companies-online-harassment-courts-social-media [https:\/\/perma.cc\/6GY7-XRJJ]. \u2191<\/a><\/li>\n
  101. . RSA Conference, supra<\/em> note 35 (While major platforms are starting to realize this, the majority of focus has been U.S.-based, when the audience of companies like Facebook is primarily international). \u2191<\/a><\/li>\n
  102. . Researchers are just beginning to look into countries in the Global South and Asia. See<\/em>, e.g.<\/em>, Shelby Grossman, Daniel Bush & Ren\u00e9e DiResta, Evidence of Russia-Linked Influence Operations in Africa<\/em>, Stan. Internet Observatory (Oct. 29, 2019), https:\/\/fsi-live.s3.us-west-1.amazonaws.com\/s3fs-public\/29oct2019_sio_-_russia_linked_influence_operations_in_africa.final_.pdf [https:\/\/perma.cc\/4LYU-RYC9]; Taiwan Election: Disinformation as a Partisan Issue<\/em>, Stan. Internet Observatory (Jan. 21, 2020), https:\/\/cyber.fsi.stanford.edu\/io\/news\/taiwan-disinformation-partisan-issue [https:\/\/perma.cc\/P8NH-Y6JJ]. \u2191<\/a><\/li>\n
  103. . See<\/em> Heller, supra<\/em> note 98. \u2191<\/a><\/li>\n
  104. . See<\/em>, e.g.<\/em>, Principles for Responsible Investment, Digging Deeper: Human Rights and the Extractives Sector 9\u201310 (2018), https:\/\/www.unpri.org\/download?ac=5081 [https:\/\/perma.cc\/U62V-XPB6]. \u2191<\/a><\/li>\n
  105. . See<\/em> The GNI Principles<\/em>, Global Network Initiative, https:\/\/globalnetworkinitiative.org\/gni-principles\/ [https:\/\/perma.cc\/748A-WU7M] (last visited Oct. 12, 2020). \u2191<\/a><\/li>\n
  106. . Facebook did commission a HRIA for Myanmar, which detailed the role that social media played in inciting mass violence against religious and ethnic minorities. However, this was after the violence had already swept through the country and the company had been criticized for its inaction for several years. While Facebook has begun to do HRIAs, this is not yet widespread for analyzing the risk of emergent situations of violence instigated or exacerbated by social media platforms. See <\/em>BSR, Human Rights Impact Assessment: Facebook in Myanmar 24 (2018), https:\/\/about.fb.com\/wp-content\/uploads\/2018\/11\/bsr-facebook-myanmar-hria_final.pdf [https:\/\/perma.cc\/VL7N-2AT5]. \u2191<\/a><\/li>\n
  107. . See<\/em> 47 U.S.C. \u00a7 230 (2020). \u2191<\/a><\/li>\n
  108. . Danielle Keats Citron & Benjamin Wittes, The Internet Will Not Break: Denying Bad Samaritans Section 230 Immunity<\/em>, 86 Fordham L. Rev. 404 (2017); Lauren Feiner, Republican Bill Seeks to Limit Social Media Liability Protections Without Getting Rid of Them<\/em>, CNBC (Sept. 10, 2020, 10:07 AM), https:\/\/www.cnbc.com\/2020\/09\/10\/republican-bill-seeks-to-limit-section-230-protection-for-tech-platforms.html [https:\/\/perma.cc\/Z7WF-W97W]. \u2191<\/a><\/li>\n
  109. . Citron & Wittes, supra <\/em>note 107, at 419. \u2191<\/a><\/li>\n
  110. . Id.<\/em> at 413. \u2191<\/a><\/li>\n
  111. . See<\/em> Heller, supra<\/em> note 98 (arguing that platforms must embrace multiple methods to effectively fight online harassment, extremism, and hate). \u2191<\/a><\/li>\n
  112. . See<\/em> Camille Francois, Actors, Behaviors, Content: a Disinformation ABC<\/em>, Transatlantic Working Grp. (Sep. 20, 2019), https:\/\/science.house.gov\/imo\/media\/doc\/Francois%20Addendum%20to%20Testimony%20-%20ABC_Framework_2019_Sept_2019.pdf [https:\/\/perma.cc\/3GWB-EJ7P] (describing the ABC\u2019s of viral deception\u2014actors, behaviors, and content). \u2191<\/a><\/li>\n
  113. . See<\/em> Stine Eckert, Fighting Online Abuse Shouldn\u2019t Be Up To The Victims<\/em>, The Conversation (Nov. 26, 2017, 6:45 PM), https:\/\/theconversation.com\/fighting-online-abuse-shouldnt-be-up-to-the-victims-87426 [https:\/\/perma.cc\/ABC4-C662] (advocating that software companies should step up efforts to combat online harassment). \u2191<\/a><\/li>\n<\/ol>\n

     <\/p>\n","protected":false},"excerpt":{"rendered":"

    Enlisting Useful Idiots: The Ties between Online Harassment and Disinformation Brittan Heller[1]* Online harassment has been a blind spot for major platforms for many years. The problem became mainstream with Gamergate in 2014, the first public reckoning with intimidation of women in the online gaming community. This problem is still plaguing social media, with progress […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_uag_custom_page_level_css":"","site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[6,9,8],"tags":[],"class_list":["post-699","post","type-post","status-publish","format-standard","hentry","category-6","category-printed","category-volume19"],"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false,"portfolio_item-thumbnail":false,"portfolio_item-thumbnail@2x":false,"portfolio_item-masonry":false,"portfolio_item-masonry@2x":false,"portfolio_item-thumbnail_cinema":false,"portfolio_item-thumbnail_portrait":false,"portfolio_item-thumbnail_portrait@2x":false,"portfolio_item-thumbnail_square":false},"uagb_author_info":{"display_name":"Brittan Heller","author_link":""},"uagb_comment_info":0,"uagb_excerpt":"Enlisting Useful Idiots: The Ties between Online Harassment and Disinformation Brittan Heller[1]* Online harassment has been a blind spot for major platforms for many years. The problem became mainstream with Gamergate in 2014, the first public reckoning with intimidation of women in the online gaming community. This problem is still plaguing social media, with progress…","featured_media_urls":[],"_links":{"self":[{"href":"https:\/\/ctlj.colorado.edu\/index.php?rest_route=\/wp\/v2\/posts\/699","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ctlj.colorado.edu\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ctlj.colorado.edu\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ctlj.colorado.edu\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ctlj.colorado.edu\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=699"}],"version-history":[{"count":4,"href":"https:\/\/ctlj.colorado.edu\/index.php?rest_route=\/wp\/v2\/posts\/699\/revisions"}],"predecessor-version":[{"id":746,"href":"https:\/\/ctlj.colorado.edu\/index.php?rest_route=\/wp\/v2\/posts\/699\/revisions\/746"}],"wp:attachment":[{"href":"https:\/\/ctlj.colorado.edu\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=699"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ctlj.colorado.edu\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=699"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ctlj.colorado.edu\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=699"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}