The research group POPNET has been analyzing the 2018 social network of the Netherlands to explore its network properties and their relationship with socio-economic indicators. Recently, data on the family, school, work, neighbour, and household connections became available annually from 2009 to 2022 via CBS’s microdata environment. The new primary goal of the group is to understand the evolution of network properties over time and their impact on upward social mobility. Changes in degree, clustering, and distance distributions are examined and we investigate the relationship between income growth, educational diversity, and the structure of ego networks.

The analysis involves 14 networks, each with 27 million nodes and approximately 2 billion edges of 33 different edge types, totalling about 2TB of data scattered over 100-150 files. Given the size, the CBS RA environment’s limited computational resources (2 virtual cores, slow 100GB disk, 32GB memory, and restricted access to 128GB desktops) are insufficient. We aim to utilize supercomputing power to create compressed files for efficient loading of the networks and outsource intensive tasks such as matrix multiplications and network measure computations. This connection allows us to run complex algorithms continuously, overcoming the limitations of CBS’s environment and enabling comprehensive longitudinal network analysis.

Photo by Ryoji Iwata on Unsplash