Lightdash: BI as code for the modern data team
Ideally, most development time building a data model should be spent actually building the data models that can be used to deliver insights to the business, which is where Lightdash come in.
When I first met the Lightdash team, they weren’t called Lightdash, and I tried to hire half of them.
Back then, I had just started Moonfire and was meeting Ligthdash’s now co-founder and CTO Oliver Laslett to offer him a job. He told me he was starting a company and had been accepted to YC – I offered to invest instead.
Oliver and co-founder and CEO Hamzah Chaudhary graduated from YC's S20 cohort with a tool called Hubble, which helped companies run tests on their data warehouse to identify issues with data quality. But when they realised this would be most useful inside a BI tool, they pivoted, focusing instead on empowering data analytics teams with the tools to be as productive as software engineers. Lightdash was born: an open-source business intelligence platform for modern data teams.
Having both experienced the challenges facing data analysts first-hand when working together at an insurtech startup and, later on, advising clients on their data infrastructure, Oliver and Hamzah got a clear insight into their pain points and how best to solve them. They realised that while data analysts are operating more and more like software developers, existing BI tools and workflows didn’t reflect it.
Ideally, most development time building a data model should be spent actually building the data models that can be used to deliver insights to the business. But most data analysts are stuck within a complex and manual workflow cobbled together from different tools, trying to make sure the production runs, repeatedly debugging and checking that the charts look right, before starting all over again. This means constantly switching contexts and fighting fires to keep everything in sync.
So Lightdash set out to build a data workflow as lightweight and fast as its name. It turns dbt – a data transformation tool that enables analysts to transform raw data in their warehouse using SQL – projects into a full-stack BI platform.
This allows analysts to manage BI as code, defining business logic for metrics and KPIs using developer workflows. It brings continuous integration and continuous development (CI/CD) to data, meaning changes in your dbt project automatically trigger builds. And a visual layer allows non-technical users (in the marketing and finance teams, for example) to quickly create their own charts and dashboards. SQL for the experts and an intuitive UI for everyone else.
It’s a step-change in the way data teams work: a better way to both build BI and share it with the rest of the business.
The beauty of being wholly open source is that it natively integrates with the tools that data teams already use and love, like Snowflake, Airbyte and Fivetran, making workflows simpler and giving businesses a clearer read on their data. Data teams can test drive it for free, before quickly deploying and connecting it up to the rest of the stack, all while keeping full visibility over how their data is being handled.
It also gives the product a genuine developer-friendly feel. Over the last year and a half, the team has been spearheading dbt-based BI, contributing to every major dbt- and BI-related event, and building an active community around the free, self-hosted version of its product, counting Accenture, Beauty Pie and Warner Media among its customers. They’ve also set up Lightdash University, an educational programme to help BI teams improve their craft, as well as a chance for the team to geek out about hard data problems with their customers.
And following their recent $8.4m seed round – made up in part by an earlier, unannounced $2.4m round led by us – Lightdash launched the public beta of its fully-managed cloud version in October, allowing any team to spin it up without worrying about managing their own infrastructure. This commercial version is what the team will look to build out over the next year. As for what else is coming next, you can see for yourself on their public roadmap.
We look forward to supporting Hamzah, Oliver and team as they continue to build a dbt-native, developer-friendly vision of the modern data function, and BI that’s open to all.
If you’re interested, check out their GitHub repo. For every star, they plant a tree! Better BI – and a greener planet – is but a click away.