The ICT implementation service of the Dutch government (Dienst ICT Uitvoering or DICTU) is receiving support from Xomnia to refine and productionize a data platform that serves different entities within the government.
Our Machine Learning Engineer Romano Vacca joined DICTU in August 2023, strengthening two teams at the client with his technical expertise in Extract, Transform, Load (ETL) pipelines, machine learning principles, platform architecture and productionalizing existing code.
Upon its completion, the data platform will help different stakeholders in the Dutch government, from ministries to policy officers working for the Dutch parliament*, in analyzing and achieving different use cases with data. The result is a more efficient and data-driven way of working for DICTU’s customer database.
“Romano brings the well-needed expertise to the project, either from own experience or his Xomnia colleagues. We can trust on Romano to come back with a solid solutions for the challenges we face with implementing the generic data platform” - Han Adema, Technical Project Leader at DICTU
Challenge
DICTU has created a new platform in Azure that enables collecting, storing and deriving insights from data. Using this platform, different teams can get their hands on a pre-configured environment that is created per best practices to build, analyze and visualize their work.
Prior to building this platform, DICTU’s clients - representing different entities within the Dutch government - developed on their own machines, which lacked the needed computational power and speed to deliver data tools. Their ability to collaborate with each other and with other teams was also limited due to data that resided in isolated silos.
Moreover, the client’s data teams had limited capacity to access certain data that included sensitive information, because they lacked the means to handle it in compliance with standards. This prevented them from seizing it and generating value from it.
To strengthen their teams with the necessary technical assistance to move the platform from a most viable product (MVP) stage to being fully in production, the client approached Xomnia.
Solution
Romano currently works in two different teams at DICTU. In the first team, he assists in defining the previously mentioned platform, advising them about the best ways to set up its different parts. Romano brings his Azure platform knowledge to the team, which is essential because the common denominator among the use cases required by each ministry is Azure Machine Learning service**.
In the second team Romano acts as the bridge between the first team and their stakeholders, ensuring that the platform’s architecture accommodates different use cases in the best way possible. He does this by leading different use cases that act as blueprints for the capabilities that should be present in the new platform, and advising on what the best tools and resources are for the problems at hand. Afterwards, he builds the solution together with the use case team, coaching other team members in the meantime to prepare them to manage the solution themselves at a later point.
Among the use cases being developed on the platform is one based on analyzing satellite images to recognize manure silos in them, which is an important topic when it comes to detecting illegal operations surrounding these silos. Romano helped a team of data scientists to develop the model, train it, further develop it, and save the data.
Romano is also involved in a use case that aims to help the Dutch Parliament track its progress regarding its goal to make the country energy neutral by 2050. For this use case, he designed and built a Lakehouse using Azure Data Factory and Azure Databricks. This setup uses the Medallion architecture, where data is stored in different buckets, progressively increasing in quality the more it is processed. This enables the client to create dashboards to be shared with the public to show how far the current policies support their 2050 goals.
Other use cases that Romano is assisting with involve configuring geospatial data using Azure resources and building an NLP dashboard.
Impact
The collaboration resulted in finalizing the platform and preparing it to go into production. The project is currently in the migration phase, but it is estimated that the first team will fully use the new platform by the end of May 2024.
This platform enables DICTU to solidify their position as the IT provider for ministries in the Netherlands. It helps them eliminate limitations in inter-team collaboration caused by insufficient computational power, and provides them with the necessary tools to focus on their actual jobs instead of addressing challenges around them. Last but not least, the platform enables the client to safely access and generate value from data that also includes sensitive information.
* The initial stakeholders are different ministries and policy officers who create recommendations and proposals for new policies or changes to existing policies. These can then in turn be implemented/used by people in the parliament, but in the initial phase that will not be the case.
** An environment in which ministries can do analysis, develop models, and create data pipelines.
*****************