Horizon scanning in policy research database with a probabilistic topic model


National governments take advantage of collective intelligence when conducting foresight processes. They grasp emerging issues through expert reviews as well as public opinions. It raises national agendas and affects policy-making process. Therefore, by examining policy papers which contain societal issues, we can perceive past, current, and future environments. In this study, we exploit policy research database of Republic of Korea, which is a unique source that automatically collects all policy papers written by national research institutes, to extract latent topics and their trends over 10 years through a probabilistic topic model. Detected topics fairly correspond to expert-selected future drivers in national foresight report, implying that public discourse and policy agenda are coupled. We suggest to utilize open government data and text mining methods for building open foresight framework that various actors exchange their opinions on societal issues.

Technological Forecasting & Social Change

Link to the paper