https://doi.org/10.1140/epjds/s13688-024-00505-x
Research
Consumption-based approaches in proactive detection for content moderation
1
Spotify Ltd, London, UK
2
Mathematical Institute, The University of Oxford, Oxford, UK
Received:
8
February
2024
Accepted:
23
October
2024
Published online:
15
November
2024
Implementing effective content moderation systems at scale is an unavoidable and complex challenge facing technology platforms. Developing systems that automate detection and removal of violative content is fraught with performance, safety and fairness considerations that make their implementation challenging. In particular, content-based systems require large amounts of data to train, cannot be easily transferred between contexts, and are susceptible to data drift. For these reasons, platforms employ a wide range of content classification models and rely heavily on human moderation, which can be prohibitively expensive to implement at scale. To address some of these challenges, we developed a framework that relies on consumption patterns to find high-quality leads for human reviewers to assess. This framework leverages consumption networks, and ranks candidate items for review using two techniques: Mean Percentile Ranking (MPR), which we have developed, and an adaptation of Label Propagation (LP). We demonstrate the effectiveness of this approach to find violative material in production settings using professional reviewers, and on a publicly available dataset from MovieLens. We compare our results with a popular collaborative filtering (CF) baseline, and we show that our approach outperforms CF in production settings. Then, we explore how performance can improve using Active Learning techniques. The key advantage of our approach is that it does not require any content-based data; it is able to find both low- and high-consumption items, and is easily scalable and cost effective to run.
Key words: Content moderation / Proactive detection / Consumption networks / Label propagation / Node ranking / Collaborative filtering
Supplementary Information The online version contains supplementary material available at https://doi.org/10.1140/epjds/s13688-024-00505-x.
Shahar Elisha and John N. Pougué-Biyong contributed equally to this work.
© The Author(s) 2024
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
