https://doi.org/10.1140/epjds/s13688-021-00288-5
Regular Article
Multilayer networks for text analysis with multiple data types
1
School of Mathematics and Statistics, The University of Sydney, NSW, 2006, Sydney, Australia
2
Wikimedia Foundation, San Francisco, USA
3
Department of Network and Data Science, Central European University, Quellenstraße 51, 1100, Vienna, Austria
4
Department of Mathematical Sciences, University of Bath, Claverton Down, BA2 7AY, Bath, United Kingdom
a
christopherhyland95@gmail.com
f
eduardo.altmann@sydney.edu.au
Received:
5
March
2021
Accepted:
14
June
2021
Published online:
28
June
2021
We are interested in the widespread problem of clustering documents and finding topics in large collections of written documents in the presence of metadata and hyperlinks. To tackle the challenge of accounting for these different types of datasets, we propose a novel framework based on Multilayer Networks and Stochastic Block Models. The main innovation of our approach over other techniques is that it applies the same non-parametric probabilistic framework to the different sources of datasets simultaneously. The key difference to other multilayer complex networks is the strong unbalance between the layers, with the average degree of different node types scaling differently with system size. We show that the latter observation is due to generic properties of text, such as Heaps’ law, and strongly affects the inference of communities. We present and discuss the performance of our method in different datasets (hundreds of Wikipedia documents, thousands of scientific papers, and thousands of E-mails) showing that taking into account multiple types of information provides a more nuanced view on topic- and document-clusters and increases the ability to predict missing links.
Key words: Stochastic block models / Multilayer networks / Natural language processing / Complex systems / Data science
Supplementary Information The online version contains supplementary material available at https://doi.org/10.1140/epjds/s13688-021-00288-5.
© The Author(s) 2021
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
