The Hidden Struggles of Social Media Moderators: A Necessary but Traumatic Role

In the shadows of vibrant social media platforms like TikTok, Facebook, and Instagram lies an unsettling reality - a network of content moderators dedicated to filtering out the most disturbing online material. Over the past months, the BBC has delved into the traumatic experiences of these individuals whose jobs involve reviewing graphic and distressing videos, including beheadings, child abuse, and hate speech. While their work is crucial for online safety, the mental toll it takes on these moderators has raised serious concerns.Although technology has advanced and organizations have invested in tools to automate content moderation, human moderators remain at the forefront. Often hired by third-party companies, these individuals sift through reports from users or content flagged by AI, many times dealing with content that is too brutal for public consumption. In a recent BBC series, "The Moderators," many former moderators from East Africa shared harrowing accounts of their experiences. One of them, known as Mojez, recounted a duality in social media; the joyous content users scroll through contrasts sharply with the horrific videos he was forced to watch and evaluate on a daily basis. He expressed a deep commitment to the role, stating, “I took it upon myself. Let my mental health take the punch so that general users can continue going about their activities..."Many moderators have spoken out about the psychological damage their work has caused, with some forming unions to advocate for support and change within the industry. This is underscored by ongoing legal actions against prominent tech firms like Meta, previously known as Facebook, which in 2020 settled a lawsuit for $52 million concerning moderators whose mental health deteriorated due to their duties.Yet, amid these challenges, former moderators revealed a compelling sense of pride in their work. They viewed themselves as guardians of online safety, likening their roles to emergency responders. One anon user, referred to as David, expressed a strong sense of duty, emphasizing that their efforts were crucial in training AI systems designed to filter harmful content.As tech companies explore the future of moderation through AI tools, concerns persist about their limitations. While those in the industry like Dave Willner, former head of trust and safety at OpenAI, acknowledge the potential of AI in moderation, experts warn about over-blocking and misunderstandings that could arise without human oversight. Social media companies, in response to the increasing scrutiny over moderator welfare, state that they are taking steps to offer support and mental health resources to their moderation teams. TikTok and OpenAI have emphasized their commitment to ensuring a caring working environment, while Meta highlights the importance of on-site support for moderators.The dialogue surrounding content moderation continues, with discussions about the necessity of human insight in combination with technological advancements highlighting the complexities of keeping online communities safe. The series "The Moderators" will air on BBC Radio 4, elucidating the profound challenges and responsibilities borne by these unseen defenders of digital spaces.