The City of Amsterdam is collaborating with Xomnia to build a blueprint for its future generic data integration platform. The blueprint will help the City provide better services to its inhabitants, enabling the exchange of relevant data across departments securely and in compliance with the GDPR and other regulations.
“This Data Integration Architecture introduced a further level of data driven innovation. It helps us in the next step to data maturity.” - Liesbeth Pot - Manager Informatievoorziening
Case: Challenges to a data-driven transformation
In 2015, the City of Amsterdam kicked off its digital transformation, adopting a data driven management orientation to improve, integrate, and streamline its business processes.
The aim of the blueprint is problem solving with a data-driven approach, to provide effective and well-tailored services to the city’s inhabitants. This especially involves the cross-disciplinary and public services, which tend to rely on data collected by multiple departments.
The City also aims to use the blueprint to better monitor its spending, making sure that its budgets are being expended in an efficient and accountable way.
One of the challenges facing this data driven policy, however, is the many laws, policies, and regulations that govern the way the that the municipality can collect and handle its large amounts of data.
Another challenge is the accessibility and integration of available data. The historically independent departments of the municipality have always collected data separately from each other. Examples of such data include the number of trash bags wrongly placed in the streets, the subsidence of quays, the percentage of households receiving a combination of various financial allowances, and others.
Those departments need a platform to efficiently exchange their data, in order to accomplish the interdisciplinary data-driven management necessary to counter modern problems across the city, such as the covid-19 pandemic, climate change, or international crime.
For this reason, The City asked Xomnia to design a blueprint for their future data integration platform, which will break through existing data silos, and serve as a platform where relevant data can be exchanged across departments.
Solution: Architecture supporting a Data-Driven Policy
First, we attempted to explore the challenges facing data-driven management within the municipality. This was done by looking for answers to underlying questions, such as the types of data processes that the future data integration platform should be able to support, and where does the current infrastructure fall short in supporting data processes.
To answer such questions, we interviewed a total of 50 stakeholders from the social departments, public works departments, financial departments, and others. From these interviews, we derived a proper sense of the problem as well as what should the solution deliver.
Based on the results obtained from the interviews, we drafted a blueprint consisting of three architectural parts:
- The conceptual architecture: This part illustrates the role and intended use of the data integration platform within the City’s organization.
- The logical architecture: It describes on a functional level, the various components and functionalities required in the platform, as well as the demarcation of the data platform (i.e. what is considered to be out-of-scope).
- The physical architecture: describes a mapping of the functional components described in the logical architecture to actual and concrete products. In this part, we compared cloud-native solutions with state-of-the-art Platform-as-a-Service (PaaS) solutions offered by market leaders across product categories following Gartner’s magic quadrants.
The blueprint was iteratively refined together with the stakeholders through focus groups, sprint reviews, and a final document review.
Impact: Problem solving with a data-driven approach
The blueprint for the data integration architecture is currently implemented in the City’s new Cloud Competence Center. It will roughly serve 80% of the future and current data processes within the City, and will be the seed for the data-driven policy within the municipality.
The City continues its collaboration with Xomnia on the implementation of generic data pipelines and data-driven management services for the various departments and stakeholders within the city.
In addition to the blueprint, the collaboration also helped give rise to important questions that will hopefully trigger a more broad and ambitious data-related strategies. Examples of those questions include:
- What does having a future-proof data platform mean to the City?
- How is it possible to use it to facilitate compliance with laws and regulations (such as GDPR) and others as much as possible?
- How can the City responsibly make use of data driven algorithms to help identify potential causes for concern in tackling the pandemics, climate change, or organized crime and cybercrime?
The municipality’s capability to deal with such problems is growing every day, as the data foundation is being built incrementally. Future projects can rely on this foundation and their success will be amplified by the functionality offered by the fast, easy, secure, and well-managed exchange of data of the new platform.