Databricks offers a Rolls-Royce experience for data teams. It’s powerful, flexible, and almost limitless in possibilities. But with great power comes great responsibility: costs can spiral quickly if you don’t have the right strategy.
So, how do you optimise your spend, avoid costly mistakes, and ensure your Databricks investment delivers maximum ROI?
This was the focus of SGI’s first Databricks event in Belgium, held on 21 October in Brussels, where leading experts shared actionable insights.
Meet the Experts
We were joined by two incredible speakers:
-
Axel Vulsteke – Data Engineer, Co-Founder at Growing pAI, and organiser of the Databricks Meetup Group in Belgium. Axel has built impactful platforms for companies like Volvo Trucks, Engie, and Versele.
-
Sam Rédelé – Technology business partner with experience in European Institutions, scale-ups, and global consultancies. Co-Founder of AI Community Belgium and former leader of Deloitte’s AI Lab.
Why Data Quality Is Back in the Spotlight
A decade ago, businesses focused on BI, reporting, and analytics, with data management and quality as key priorities.
Those who ignored these fundamentals are now struggling to make sense of LLMs and AI. Once again, data quality and management are at the forefront, as they are critical for change management and any successful data strategy.
Another surprising insight? Many companies have 8, 9, or even 10 separate data strategies. It sounds crazy, but it happens often.
The solution is simple:
Pick one strategy. Do it right. Do it well. Any decision is better than indecision, especially when it comes to data. Choose a clear path, align everyone on the why and outcomes, and work toward a defined end goal.
Technical Strategies for Cost Optimisation
Axel shared practical tips based on real-world experience:
1. Increase Data Literacy
Understand your organisation’s data maturity. Axel outlined five levels:
-
Data Driven
-
Data Proactive
-
Data Active
-
Data Reactive
-
Data Blind
Belgian businesses often aim to be “data-driven”, but the journey requires awareness and education.
2. Use the Medallion Architecture
Databricks’ Medallion architecture improves data quality by structuring pipelines into three layers:
Bronze (raw data) → Silver (validated data) → Gold (enriched data)
This design pattern ensures data is progressively cleaned, transformed, and prepared for analysis, creating a single source of truth for reporting, predictive analytics, and machine learning.
3. Avoid Costly Replication
During his presentation, Axel highlighted that data source replication is usually to blame for the increase in costs. To prevent this additional cost, he suggested one of 2 ways to approach this:
-
EventHub Capture + Databricks Autoloader (cheaper, more reliable, slight delay)
-
Direct EventHub Reads (faster, but expensive and error-prone)
4. Monitoring & Alerting
-
Apply the canary in the coal mine principle
-
Avoid unnecessary replication
-
Set tiered alerts (2h, 6h, 24h, 72h)
5. Axel’s Quick Cost Checklist
-
Use cold paths for infrequent data
-
Delete raw data carefully
-
Optimise weekly and vacuum
-
Partition smartly and use liquid clustering
-
Spot VMs for DEV/QA
-
Keep RAM/CPU at 50–70% utilisation
-
Broadcast joins
-
Stop jobs during nights/weekends
Business Perspective: How to Calculate ROI of Technology
Sam Rédelé, our second speaker, delivered an insightful presentation on business strategies for data leaders, sparking lively discussion during the networking session.
He showed that technology ROI should be driven by business enablement, not only by tool analysis. To do this, your tech ecosystem must first mirror how the business really works, which means connecting systems and breaking down silos.
Sam introduced knowledge modelling on top of the existing data ecosystems. This is a business layer that connects systems that previously couldn’t talk to each other, unlocking faster interoperability and enabling systemic, end-to-end analytics rather than siloed reporting.
This knowledge-driven foundation enables an AI-First operating model with real-time operational improvements and smart decision-making.
Throughout the keynote, Sam used concrete real-life examples to show that the hardest part is not the technology, but change and adoption: shifting how people think, work and take ownership. These cases also illustrated how to gradually move IT away from a ticket-driven service model and organically grow into a Technology Business Partner that co-owns outcomes with the business.
Conclusion: Your Next Move
Databricks is a game-changer, but only if you manage cost, quality, and strategy effectively.
Whether you’re a Databricks user, a data leader, or building AI-driven solutions, the key is clear:
-
Choose one data strategy and stick to it
-
Invest in data literacy
-
Optimise continuously
At SGI, we don’t just host events; we help you hire the best Databricks talent in Belgium to make these strategies a reality.
If you’re looking to hire Databricks specialists or would like to join a future event, please reach out to Adam Jones.
