Kamer van Koophandel (KVK), the chamber of commerce in the Netherlands, is making use of Xomnia’s expertise to stream data to increase the speed and reliability of its web applications.
With the aid of technology such as Apache Kafka and Apache Cassandra, data collected and stored in the KVK’s legacy systems will become accessible in real-time. This will, in turn, increase the speed with which a response is given to inquiries placed in the KVK’s systems, and make those systems able to accommodate more state-of-the-art digital services in the future.
At the KvK we are working on a real-time data platform to provide scalable API's to get information on all commercial activities in the Netherlands. Xomnia's consultant helped us with application design and deploying Apache Kafka and Apache Cassandra on our on premise Kubernetes platform – KVK
Challenge
All businesses and legal entities in the Netherlands must register in the Dutch trade register, the KVK. In return for a fee, the KVK can give background information about any company registered within it to third parties. Usually, businesses place such inquiries to get more information for background checks, for instance ahead of forming new partnerships.
The trade registry system is a few years old and designed to do registrations very efficiently. However, the system can be relatively slow in processing other types of inquiries. This situation is not unique to the KVK, as many companies find that their legacy systems are unable to support modern services nor accommodate updates quickly, which stands in the way of creating mobile apps or accessing data in real-time. Fully replacing those legacy systems, however, can be very costly and time consuming.
Therefore, in order to make the process of placing inquiries on its platform easier and more reliable, KVK approached Xomnia to help build a new system with technology that is based on NoSQL databases and Apache Kafka. Those technologies should perform better and can stream data easily and quickly from legacy systems. This enables the agile development of new applications while keeping the legacy application running.
Solution
As mentioned earlier, business registrations in the KVK take place on a legacy system, which is hard to modify and doesn’t allow using or retrieving this data easily. To solve this issue, Xomnia’s data engineer Floris Smit is working with the developers at KVK to move the data out of the old system through a technology called Apache Kafka, a known software component that is similar to a queue through which data passes into a new application that puts it in a NoSQL database, a flexible model for the storage and retrieval of data.
Consequently, while businesses can still register using the old system, a new system can be used for badly performing queries. This will happen in 2 steps: We publish all the data and registrations to Apache Kafka. From here, multiple applications can retrieve the same data for different purposes. The new applications make use of a modern and scalable technology so that they perform without timeouts, for which we use NoSQL solutions like Apache Cassandra.
Don't miss Xomnia's latest business cases and blogs. Subscribe to our newsletter.
Impact
This project will take around 2 years to complete. Making a module in the system that lists and copies all the data to Apache Kafka needs to be done slowly and carefully to avoid any data breaks in the trade registry application.
The sought impact after the project is completed is enhancing the performance of the inquiries platform at KVK, so that users retrieve the information they require immediately without sessions timing out. This will, in turn, give visitors using KVK’s platform a more reliable and timely service.
Achieve your organization's full potential with the leading AI consultancy in the Netherlands