Share Delta Lake and Iceberg Tables Across Databricks and Snowflake

Oct 30, 2026 316 views

Databricks and Snowflake can work with the same underlying data without copying it between platforms. The approach pairs shared Azure Data Lake Storage (ADLS) with open table formats, with each service using catalog APIs to access table metadata. The practical goal is to let different analytics engines work from a common set of files while respecting the rules that make those files a table: their schema, organization, and current state. That can simplify some data architectures, but it doesn't make the two platforms interchangeable. Storage is only one part of the system; metadata access and supported operations still shape what each service can do.

How cross-platform access works

Both platforms read a single set of Parquet files in ADLS. Databricks tables stored in Delta or Delta Uniform can be read by Snowflake through catalog integration; Databricks can access Snowflake-managed Iceberg tables through catalog federation. That shared-storage model can reduce duplication and its associated costs, while making catalog compatibility a key implementation consideration. Parquet supplies the columnar files, while the table formats and catalogs provide the structure and descriptions needed to interpret them as tables. A reader needs more than a path to a folder: it needs to discover which files belong to a table and how to interpret their fields.

The direction of access matters. In the first case, Snowflake reads Databricks table data through catalog integration. In the second, Databricks accesses Snowflake-managed Iceberg tables through catalog federation. Those descriptions don't establish that every feature or operation works in both directions. Teams need to check the capabilities of the specific table format, catalog connection, and service configuration they plan to use.

Why catalogs matter as much as storage

A data lake can contain files that look perfectly accessible to both platforms and still fail to behave like a shared table. The catalog is the coordination point: it exposes table metadata so a service can find and interpret the data. If one platform updates a table and another reads outdated or incompatible metadata, the shared files alone won't resolve the mismatch. This is why catalog APIs belong in the architecture discussion from the start, rather than being treated as a small connection detail after storage has been chosen.

Open table formats are intended to make table metadata and organization usable beyond one engine. That can reduce dependence on a single platform's private representation of a table, but “open” doesn't mean every implementation supports every feature in the same way. Delta, Delta Uniform, and Iceberg each have their own format details and compatibility boundaries. Catalog integration and federation are the mechanisms described here for connecting those systems; they don't erase the need to validate what each side can read and how it responds to changes.

That distinction is easy to miss. A successful read test proves that one service can access a table in a particular state. It doesn't, by itself, prove that writes, schema changes, or concurrent updates will behave as a team expects. (And this is the part most people overlook.) The intended workload matters: analysts querying stable datasets face different coordination demands from pipelines that repeatedly change tables or publish new data.

What shared storage can change

Keeping one set of files can reduce duplication and the costs associated with storing and maintaining multiple copies. It may also simplify the path from data creation to analysis: a team can make a dataset available to another platform without first building a separate copy and keeping it synchronized. That's a meaningful architectural benefit, especially when organizations have reasons to use both services.

But a shared copy doesn't eliminate operational work. Teams still need to manage access to ADLS, configure the catalog connections, and establish how table changes are discovered. They may also need to account for the different query behaviors and feature sets of each platform. If users see different results or delays between updates, the cause may be in metadata synchronization, permissions, or the way a particular format is supported—not in the Parquet files themselves.

There can also be a trade-off between avoiding copies and keeping workloads independent. Separate copies can consume more storage and require synchronization, but they can give teams more control over the timing and shape of data prepared for a particular workload. A shared-storage design makes the common dataset more direct; it also makes sound coordination around that dataset more important. The right choice depends on how often data changes, which services need to read it, and how much complexity the organization is prepared to manage in its catalogs.

Implementation questions teams should answer

Before treating a table as shared, teams should verify the exact access path for each direction. Which catalog API is involved? Which table format and features are supported by the receiving service? How are credentials and permissions handled in ADLS? What happens when the table's schema or metadata changes? These aren't peripheral questions. They determine whether a design that works in a demonstration remains predictable under normal workloads.

It also helps to distinguish read access from broader interoperability. A platform being able to query a table doesn't necessarily mean it can safely modify that table or manage its lifecycle. Teams should define which service owns table creation and updates, and which services consume the result. Clear ownership can reduce conflicts and make troubleshooting easier, particularly when a problem appears in one platform but originates in a shared catalog or storage configuration.

If you're working in this space, test representative tables and workflows rather than relying on a format label alone. Include the data changes your pipelines actually make, and confirm that both platforms observe the expected metadata and results. A narrow test can establish that a connection is possible; operational testing shows whether it is dependable for the intended use.

Implications and outlook

This is more significant than it looks, but the benefit is architectural rather than automatic. Shared ADLS and open table formats can give Databricks and Snowflake a route to work from common data, reducing the need for platform-specific copies. The catalog layer is what makes that sharing legible to each service, and compatibility there is a central design constraint.

For organizations using both platforms, the approach offers a way to preserve choice about where workloads run while keeping data in a shared location. That flexibility comes with a practical test: whether the formats, catalogs, permissions, and update patterns line up for the work at hand. Here's the thing: a common storage layer is a useful starting point, not a guarantee of frictionless exchange. The details of metadata access will decide how much of the promised simplicity users actually experience.

And yet, even a carefully configured shared-data setup won't remove every reason to move or transform data. A workload may still need a prepared dataset, a different schema, or processing that happens inside one service. The stronger claim is narrower: some cross-platform access can happen without copying the underlying files. For teams that can make the catalog and format requirements fit, that can mean fewer duplicated datasets and a clearer route to using both platforms. A useful capability. With conditions.

Source: Hanza Parayil Salim · dzone.com

Comments

Sign in to comment.
No comments yet. Be the first to comment.

Related Articles

Share Delta Lake Tables Between Databricks and Snowflake