The City of Amsterdam joined hands with Xomnia and other consulting partners to embark on a transformative journey to reshape its data architecture. This collaboration marked a significant milestone in The City's pursuit of technological advancement, and sets a compelling example for cities worldwide seeking to harness the full potential of their data resources.
Commencing in Q3 2022 through June 2023, our collaborative efforts reshaped the data platform’s tooling and architecture at the client, spearheaded by data architect Ozan Dogu Tuna and a team of 5 other data architects.
The outcome was translating Data Mesh concepts into an architecture that The City can deploy on a wider basis. This forms a stepping stone in the eventual transition of The City to operate in a cloud-based infrastructure.
"Ozan brought a lot of experience by setting up an architecture on Data Mesh principles. He was able to translate the high level intentional architecture to concrete features for the teams to work on." The City of Amsterdam
Challenge: Legacy Infrastructure and Cloud Migration
Amsterdam's city-wide data platform operated within an on-premise system. This stood as a challenge in the face of The City’s ambitions to expand and achieve its full data-driven potential. To overcome this, The City wanted to shift to a cloud-based paradigm, migrate to Azure, revamp its data platform's tooling and architecture, and enable self-service analytics. However, this vision posed intricate challenges.
The City’s initial steps towards migrating to the cloud heavily relied on Infrastructure-as-a-Service (IaaS) components. Attempts to bootstrap new solutions atop this framework strained platform teams, causing delays, service quality issues, and maintenance complexities. The data teams, entrusted with making data accessible to the public and enabling self-service analytics, grappled with these impediments.
Overcoming this infrastructural puzzle required a holistic reimagining of The City's data landscape. It entailed a shift in strategy, mission and vision, as well as necessitating a profound embrace of data mesh principles. These principles focus on domain-centralized ownership, treating data as a product and data governance.
Ozan played a pivotal role in The City’s journey to achieve this. This came following Xomnia's help with lying the blueprint for a cloud-based architecture at The City from 2015 to 2020. In 2022, the City sought Xomnia's architectural expertise to translate plans into action.
Solution: Data Architect Ozan's Role
Xomnia assigned Data Architect Ozan Dogu Tuna to work alongside The City of Amsterdam's team in orchestrating a seamless transition to the cloud. His year-long assignment played a crucial role within the Security Architecture Services (SAS) team, which comprised 5 other data architects.
Starting the project demanded a comprehensive enterprise architecture overhaul, beginning with setting the mission and vision for the new cloud platform. This top-down approach traversed enterprise and contextual architecture, setting the stage for transformative changes, and transitioning to working with the Data Mesh concepts.
One of the pivotal challenges was to deliver the new platform efficiently while preserving the legacy system's functionality. This necessitated the introduction of new architectural patterns that emphasized a cloud-first approach. This was no small feat considering the vast domain size of The City of Amsterdam.
Ozan's expertise was instrumental in navigating these complex transitions. He helped in bridging gaps between the teams and 50 domain experts who used the minimum viable product versions of the platforms.
He contributed towards ensuring a deep understanding of the data teams' needs while helping to guarantee that the delivered data platforms met these requirements, were future-proof, and adhered to a well-defined architecture.
His contributions encompassed:
- Clarifying the division of responsibility and designing architectural patterns to enforce domain-centralized ownership.
- Elevating self-service quality and availability by transitioning from Infrastructure-as-a-Service to Software-as-a-Service.
- Managing data ingestion and making data accessible for analysis.
- Establishing robust data governance, treating data as a product, and ensuring data's security, privacy, sharing standards, and interoperability.
- Successfully executing the MVP stage.
“The adoption of the 'safe methodology' facilitated the development of each platform through smaller bi-weekly iterations, with quarterly milestones setting the agenda for the next quarter's goals,” says Ozan. “Architecture played a crucial role in these planning phases, and we dedicated a quarter to designing and reaching consensus on the upcoming steps."
Impact: A Robust Data Ecosystem
Ozan's assignment at The City of Amsterdam concluded in June 2023. His contributions brought The City a step closer to the establishment of a more resilient data infrastructure, both from an architectural and organizational perspective.
Following the year-long collaboration, Ozan and the team of architects successfully executed the first MVP. The platform was positioned as viable enough to replace on-premise workloads, marking a significant step towards the cloud-based platform's full adoption. The ultimate objective remains to onboard the entirety of the client's domain in the years to come.
Solution Architect Richard Kooijman joined The City starting Q3, 2023. His role focuses on supporting the platform teams and constructing use cases atop the data infrastructure. Stay tuned for further updates as the client's data transformation journey continues.