TLDR

Mage now supports a suite of DuckDB & MotherDuck features— from reading and writing DuckDB databases to executing dbt with dbt-duckdb!

🤝 Mage & MotherDuck

Here at Mage, we're always looking for the latest and greatest ways to make data engineering simple, accessible, and fun.

That's why we've parntered with MotherDuck to introduce complete support for DuckDB and Motherduck! 🪄 🦆

🚀 New Features

In version 0.9.28 and above, you'll have access to a suite of DuckDB features like:

• Reading from/writing to DuckDB databases: Mage can read data from DuckDB databases directly into your piplines and output to DuckDB. That includes our easy-to-use templates for loading/exporting!

• Streaming data to DuckDB destinations: Since Mage supports to streaming to any of our supported destinations, DuckDB is a de-facto streaming sink!

• Executing dbt models using dbt-duckdb: With an amazing contribution by our community, Mage supports executing dbt models in DuckDB SQL!

And new in version 0.9.63 is support for:

Reading/writing to MotherDuck datasources: With DuckDB support comes out-of-the-box support for MotherDuck's serverless analytics platform.

Exporting to a database is as simple as using a data exporter, which can select directly from an upstream block.

You can also read from a database in one-line with our DuckDB data loader.

Since DuckDB is implemented in Mage as it's own IO class, that also makes DuckDB available as a streaming destination. That's right, you can stream from any source directly do a DuckDB database.

Last, but certainly not least, you can take this data and execute dbt models! Talk about a cherry on top! 🍒

Leveraging the dbt-duckdb library, Mage can seamlessly integrate your dbt project with DuckDB sources and destinations, allowing you to leverage our hybrid architecture to execute SQL, Python, and dbt models in the same pipeline. 🤯

You can read more about configuring DuckDB as a storage location here (and the MotherDuck companion here) get started building today! 🦆