INFOMOTION as host of the
Databricks User Group Cologne
Databricks User Group Cologne: Insights from Flaschenpost and DAC.Digital
Bringing together the Databricks community: The Databricks user community offers a wealth of knowledge and expertise in data, analytics, and AI. We are pleased to contribute to this community by hosting an in-person Databricks User Group event as part of the series across Germany.
Join us on July 9, 2026, in Cologne at the INFOMOTION office. This user group event will feature a 30-minute talk (including Q&A) by flaschenpost and DAC.digital.
Each session will include an open discussion on key data challenges and solutions, as well as insights into the latest technologies, including the Databricks Data Intelligence Platform. In addition, you will have the opportunity to network with peers from a wide range of industries while enjoying food and beverages.
Time left until the event
Jetzt anmelden0
Tage
0
Stunden
0
Minuten
0
Sekunden
Look forward to the following highlights
flaschenpost Eventworker
Tim Wagner - Lead Developer - Cloud Data Engineering, flaschenpost & Stefan Schuler - Team Lead / PO - Cloud Data Engineering, flaschenpost
This talk covers the architecture and implementation of the flaschenpost Eventworker, which currently connects more than 500 events from different business domains to Databricks. The system consists of two clearly separated components: a C# application running in Kubernetes and a Databricks-based processing layer built with Lakeflow Declarative Pipelines. Together, these components provide the technical basis for self-service event onboarding and consumption. The talk will also cover a number of practical challenges we had to address during the rollout, including duplicates, missing data, and general stability issues. In addition, we had to migrate a large number of existing stakeholder reporting queries to the new setup. This required substantial technical and organizational effort and was supported by a customized AI tool. Overall, the session focuses on the architectural decisions, operational challenges, and migration work involved in moving from a legacy event processing framework to a scalable event platform on Databricks.
AI Agent Self-Healing System: The Future of Autonomous Data Pipelines
Dariusz Romanowski, DAC.Digital & Michal Kopczynski, DAC.Digital
Data pipelines often fail because upstream systems change unexpectedly - for example, when a column is renamed or a data type changes. Fixing these issues usually requires manual investigation and code updates. This presentation shows a Proof of Concept (POC) for an AI-based self-healing system that detects ETL failures and automatically proposes fixes. The system analyzes historical Data Lake metrics such as null values, cardinality, and schema information to identify what changed and adjust the pipeline code accordingly. The solution includes a Databricks environment for monitoring pipeline status and a Human-in-the-Loop verification step before changes are applied. The system can also reuse previous fixes to handle similar problems in the future. The presentation will include a live demo showing how the system responds to real pipeline failures and restores data flow automatically.
Our Eventpartner:
Zum Aktivieren der eingebetteten Karte bitte auf den Link klicken. Durch das Aktivieren werden Daten an den jeweiligen Anbieter übermittelt. Weitere Informationen können unserer Datenschutzerklärung entnommen werden.
Inhalt anzeigen
