The Dutch Ministry of Infrastructure and Water Management (I&W) is collaborating with Xomnia to better facilitate the process with which it conducts stakeholder analysis. Using data analysis technologies, the Ministry aims to better understand citizens’ needs and problems, and identify relevant stakeholders.
The final proof of concept (POC) of the solution is a web-based app that uses text mining to help the Directorate for Public Participation at the Ministry better locate and interpret the public dialogue about their services and responsibilities. Xomnia’s Lead Data Scientist Guido Faassen created the solution, from the project refinement phase all the way to deployment in Azure.
“It is important that all affected or interested stakeholders are involved in our development of policy. The tool that Xomnia helped us develop aids in searching the web to map out these stakeholders.” – Nick Boxem, Directorate Manager
Challenge
Members of the Directorate for Public Participation in the Ministry of I&W research coverage and discussion among citizens, corporations and other stakeholders about topics such as public transportation, airports, and other infrastructure. By doing so, the Directorate aims to find and thematically sort challenges that fall under the Ministry’s responsibilities, and deep dive into ways to reach solutions in collaboration with relevant stakeholders.
Members of the Directorate used to conduct this research manually on Google, Twitter and similar sources, and to come up with the best search keywords to expand their search based on the outcomes. This manual approach, however, proved to be labor intensive, time-consuming, and prone to inconsistencies, since each team member has their own search methods, personalized outcomes based on their search histories (i.e. filter bubbles), human bias, among other factors.
To tackle this issue, the Ministry approached Xomnia to utilize our expertise to enhance the research, monitoring and stakeholder detection processes using data science.
Solution
Lead Data Scientist Guido Faassen worked on the POC for the Ministry, iterating together with the Directorate for Public Participation about the solution. The final outcome is a web-based app with a front end that resembles a search engine, which runs on Azure App Service, where it is constantly updated and accessible.
The solution gathers information from services like Google, Bing and Twitter, applies text mining techniques on this information, and returns the results back to the end user:
First, users can search different keywords using the solution, which uses the APIs of Google, Twitter and Bing (and can use more APIs in the future) to obtain results. Because the search engine within the solution doesn’t have personalization or search history, all its users get the same results, with less bias and inconsistencies.
Then, the solution applies named entity recognition to extract the stakeholders from the results, which can be individuals, organizations and locations. This gives the user an impression of who the potential stakeholders are and what are the potential issues for them.
Finally, the solution applies text mining techniques to the results to extract meaning from sentences, identify the different themes that each search result fits in, and show synonyms and matches which the users can use as follow-up search queries. These outcomes appear under the “issues” section in the solution app, which researchers can use to get a better idea about what they need to search for next, a better understanding of underlying issues, and improve their research overall.
Impact
The Directorate for Public Participation in the Ministry of I&W facilitates and encourages public dialogue about its services, infrastructure, and areas of responsibility. The solution helps the Directorate save time in its research process, and better analyze the outcomes of their research.
“One of the things that the tool helped us discover is that search results of Google and Bing only overlap by 15%,”said Liset Verschoore, Lead Participation Advice. “This tool allows us to use two search engines at the same time, which we expect will also help us identify unusual or unpredictable stakeholders.”
The solution has been in production for 2 years, and Xomnia continues to provide support on it, ensuring that it keeps running safely, managing any incidents (e.g. in the case of crashes or bugs), and ensuring that its software remains up to date.