An NYU report offers a plan for the content moderation industry

The job is a nightmare. But tech companies might be able to make it a little more humane.


June 9, 2020

Fixing one of tech’s most inhumane jobs is not an enviable task.

Moderators have to dig through some of the darkest content humanity has to offer — and the work is often traumatizing. But AI moderation isn’t quite up to snuff either, which means humans are an unfortunate necessity.

But in a new report, NYU professor Paul M. Barrett has a few ideas for revamping the industry — starting with one big one. Instead of outsourcing content moderators, he argues, companies like Facebook need to bring them on as full-time workers.

How content moderation went so wrong

Big tech companies currently “marginalize the people who do content moderation” to give themselves “plausible deniability” over content failures, according to the report.

Facebook’s content moderators are often subcontractors. Many worked for a company called Cognizant, earning salaries as low as ~$28k and receiving few health benefits. (Last October, Cognizant left the business.)

After a string of moderators received PTSD diagnoses, they sued Facebook — and last month, the company agreed to shell out $52m to 11,250 of its moderators.

But moderators say they are blocked from “voicing concerns and contributing to the public discussion.” A group of moderators on Monday expressed solidarity with virtual walkouts at Facebook, writing, “We would walk out with you — if Facebook would allow it.”

The new moderation action plan

Here are a few more of Barrett’s prescriptions:

  • Double the number of human moderators. Moderators will be able to rotate shifts more often and overall view less terrifying content.
  • Put moderation teams in every country. Moderators trained in local languages and politics are best suited to sleuth out inappropriate posts.
  • Give moderators more mental health support. Trauma counseling is urgently needed — but most have a tricky time getting it.

Join 1.5m+ professionals getting The Hustle daily news brief

Business and tech news in 5 minutes or less

100% free, no ads or spam, unsubscribe anytime

Psst

How'd Bezos build a billion dollar empire?

In 1994, Jeff Bezos discovered a shocking stat: Internet usage grew 2,300% per year.

Data shows where markets are headed.

And that’s why we built Trends — to show you up-and-coming market opportunities about to explode. Interested?

We’re shooting our shot…

Start your mornings with The Daily.

Get the freshly baked 5-minute newsletter every day except Saturdays.

It’s filled with the most interesting stories on business, tech, and the internet.

And written for innovators of every industry. Sign up for the news that slaps.

If you don’t like it, unsubscribe any time. Privacy policy.

[email-submission-form redirect-to-home="true" button-text="Join Free" include-trends-opt-in="true" id="main-signup-form" fail-url="" success-url="https://thehustle.co/signup" default-source="thehustleco" default-campaign="home"]
<script type="text/javascript"> var onloadCallback = function() { grecaptcha.render('verify-your-humanity', { 'sitekey' : '6LdddrcZAAAAALyttpvOqiwQGwq5BNhgDz4tMQGE' }); }; function getCookieValue(a) { var b = document.cookie.match('(^|[^;]+)\\s*' + a + '\\s*=\\s*([^;]+)'); return b ? atob(decodeURIComponent(b.pop())) : ''; } function setHiddenFieldValue(wrappingDiv, searchParams, className, utmName, cookieName, defaultValue) { var el = wrappingDiv.getElementsByClassName(className)[0]; var existingVal = el.getAttribute('value'); var newVal = searchParams.get(utmName) || getCookieValue(cookieName); if ((existingVal == null || existingVal == '' || existingVal == defaultValue) && (newVal != null && newVal != '')) { el.setAttribute('value', newVal); } } function setHiddenFieldValueFromUtm( wrappingDiv, searchParams, className, utmName, defaultValue ) { var el = wrappingDiv.getElementsByClassName(className)[0]; if (el != null) { var existingVal = el.getAttribute("value"); var newVal = searchParams.get(utmName); if (newVal != null && newVal != "") { el.setAttribute("value", newVal); } } } function initForm() { var wrappingDivs = document.getElementsByClassName('email-submission'); wrappingDivs.forEach(wrappingDiv => { var sp = new URLSearchParams(window.location.search); setHiddenFieldValue(wrappingDiv, sp, 'funnel-source', 'utm_source', 'funnel_source', 'thehustleco'); setHiddenFieldValue(wrappingDiv, sp, 'funnel-campaign', 'utm_campaign', 'funnel_campaign', 'home'); setHiddenFieldValue(wrappingDiv, sp, 'funnel-medium', 'utm_medium', 'funnel_medium', ''); setHiddenFieldValue(wrappingDiv, sp, 'funnel-referral', 'ref', 'funnel_referral', ''); setHiddenFieldValueFromUtm(wrappingDiv, sp, 'funnel-a', 'a', ''); setHiddenFieldValueFromUtm(wrappingDiv, sp, 'funnel-c', 'c', ''); setHiddenFieldValueFromUtm(wrappingDiv, sp, 'funnel-o', 'o', ''); setHiddenFieldValueFromUtm(wrappingDiv, sp, 'funnel-oc', 'oc', ''); setHiddenFieldValueFromUtm(wrappingDiv, sp, 'funnel-e', 'e', ''); setHiddenFieldValueFromUtm(wrappingDiv, sp, 'funnel-f', 'f', ''); setHiddenFieldValueFromUtm(wrappingDiv, sp, 'funnel-r', 'r', ''); setHiddenFieldValueFromUtm(wrappingDiv, sp, 'funnel-t', 't', ''); setHiddenFieldValueFromUtm(wrappingDiv, sp, 'funnel-s1', 's1', ''); setHiddenFieldValueFromUtm(wrappingDiv, sp, 'funnel-s2', 's2', ''); setHiddenFieldValueFromUtm(wrappingDiv, sp, 'funnel-s3', 's3', ''); setHiddenFieldValueFromUtm(wrappingDiv, sp, 'funnel-s4', 's4', ''); setHiddenFieldValueFromUtm(wrappingDiv, sp, 'funnel-s5', 's5', ''); var error_message = getCookieValue('funnel_error_message'); if (error_message && error_message.trim() != '') { var error = wrappingDiv.getElementsByClassName('funnel-error')[0]; var prev_email = getCookieValue('funnel_email'); error_message = prev_email + " is not valid. Please try again"; error.innerHTML = error_message; error.style = ''; } }); if (false) { initCaptchaFormV2(); } } function initCaptchaFormV2() { var v3RecaptchaResponseEl = document.getElementById('recaptcha-response-v3'); v3RecaptchaResponseEl.parentNode.removeChild(v3RecaptchaResponseEl); var wrappingDiv = document.getElementById('main-signup-form'); var subForm = wrappingDiv.getElementsByClassName('email-submission')[0]; var captchaVersion = document.createElement('input'); captchaVersion.class = 'g-recaptcha hidden-input'; captchaVersion.type = 'hidden'; captchaVersion.name = 'g-recaptcha-response-v2'; captchaVersion.value = 'true'; subForm.appendChild(captchaVersion); var captchaEl = document.createElement('div'); captchaEl.id = 'verify-your-humanity'; subForm.appendChild(captchaEl); var captchaApiScriptEl = document.createElement('script'); captchaApiScriptEl.src = 'https://www.google.com/recaptcha/api.js?onload=onloadCallback&render=explicit'; captchaApiScriptEl.async = true; captchaApiScriptEl.defer = true; document.head.appendChild(captchaApiScriptEl); } function appendCheckboxes() { var optInDivs = document.querySelectorAll('.trends-opt-in'); optInDivs.forEach(el => { if (el.getElementsByClassName('trends-opt-in-checkbox').length < 1) { var checkbox = document.createElement('input'); checkbox.setAttribute('class', 'trends-opt-in-checkbox'); checkbox.setAttribute('type', 'checkbox'); checkbox.setAttribute('name', 'trends_opt_in'); var label = document.createElement('label'); label.setAttribute('class', 'trends-opt-in-text'); label.textContent = "Yes, I'd like to receive updates on market opportunities before they explode from Trends by The Hustle"; el.appendChild(checkbox); el.appendChild(label); checkbox.click(); } }) } window.addEventListener('DOMContentLoaded', (event) => { var funnel_email_cookie = getCookieValue('funnel_email'); if ( (true) && (funnel_email_cookie != null && funnel_email_cookie != '') ) { window.location.replace('/home'); } initForm(); if (true) { appendCheckboxes(); } }); </script> <div class="email-signup" id=main-signup-form> <div class="funnel-error" style="display:none;"></div> <form class="email-submission" action="https://cms.thehustle.co/api/v1/contacts/wordpress_create" method="post"> <div class="email-form-wrap"> <input class="funnel-source hidden-input" type="hidden" name="source" value="thehustleco"> <input class="funnel-campaign hidden-input" type="hidden" name="campaign" value="home"> <input class="funnel-medium hidden-input" type="hidden" name="medium"> <input class="funnel-referral hidden-input" type="hidden" name="referral_code"> <input class="funnel-fail-url hidden-input" type="hidden" name="fail_url" value=""> <input class="funnel-a hidden-input" type="hidden" name="a" value=""> <input class="funnel-c hidden-input" type="hidden" name="c" value=""> <input class="funnel-o hidden-input" type="hidden" name="o" value=""> <input class="funnel-oc hidden-input" type="hidden" name="oc" value=""> <input class="funnel-e hidden-input" type="hidden" name="e" value=""> <input class="funnel-f hidden-input" type="hidden" name="f" value=""> <input class="funnel-r hidden-input" type="hidden" name="r" value=""> <input class="funnel-t hidden-input" type="hidden" name="t" value=""> <input class="funnel-s1 hidden-input" type="hidden" name="s1" value=""> <input class="funnel-s2 hidden-input" type="hidden" name="s2" value=""> <input class="funnel-s3 hidden-input" type="hidden" name="s3" value=""> <input class="funnel-s4 hidden-input" type="hidden" name="s4" value=""> <input class="funnel-s5 hidden-input" type="hidden" name="s5" value=""> <input class="funnel-success-url hidden-input" type="hidden" name="success_url" value="https://thehustle.co/signup"> <input id="recaptcha-response-v3" class="g-recaptcha hidden-input" type="hidden" name="g-recaptcha-response" value=""> <input class="signup-email" type="email" name="email" placeholder="Your email address" required> <input class="email-submit" type="submit" value="Join Free"> </div> <div class="trends-opt-in"></div> <div submit-success> <template type="amp-mustache"> <p class="c-message c-message--success">Thank you for subscribing.</p> </template> </div> <div submit-error> <template type="amp-mustache"> <p class="c-message c-message--failed">Your submission failed. Please try again!</p> </template> </div> </form> </div>