What Is Dremio Open Catalog?

Dremio Open Catalog is Dremio's Apache Iceberg catalog service — the metadata store that registers Iceberg tables, manages namespaces, and coordinates access across multiple query engines. It implements the Iceberg REST Catalog specification, meaning any engine with an Iceberg REST catalog client can connect to it as a drop-in catalog backend.

Under the hood, Dremio Open Catalog is powered by Project Nessie — the Git-like transactional catalog developed by Dremio. Nessie adds branching and tagging semantics on top of the Iceberg snapshot model: data engineers can create isolated branches for experiments, merge changes atomically when validated, and tag specific table states for reproducible ML training datasets.

Dremio Open Catalog is available both as part of Dremio Cloud (managed, serverless) and as a standalone service that can be deployed independently for organizations that want a managed open catalog without using Dremio as their primary query engine.

Key Open Catalog Features

Dremio Open Catalog provides capabilities beyond basic catalog registration:

Iceberg REST API Compliance

Full implementation of the Iceberg REST Catalog specification — namespaces, table CRUD, metadata load, transaction commit, token-based auth. Any conformant Iceberg client connects without modification.

Git-Like Branching (Nessie)

Create branches for isolated data experimentation. Commit changes atomically. Merge branches when validated. Tag specific commits for reproducible ML datasets. All the power of Git workflows applied to lakehouse tables.

Credential Vending

The catalog returns short-lived, scoped cloud storage credentials alongside table metadata. Engines get exactly the permissions needed to read or write the specific table requested — no broad long-lived credentials required.

Multi-Engine Interoperability

Tables registered in Dremio Open Catalog are accessible simultaneously by Dremio, Apache Spark, Trino, Apache Flink, and any other Iceberg REST catalog client. Schema changes from one engine are immediately visible to all others.

Dremio Open Catalog Architecture diagram
Figure 1: Dremio Open Catalog as the shared Iceberg catalog for a multi-engine lakehouse.

Connecting to Dremio Open Catalog

Connecting any Iceberg-compatible engine to Dremio Open Catalog uses the standard Iceberg REST catalog configuration:

Apache Spark:

spark.sql.catalog.dremio=org.apache.iceberg.spark.SparkCatalog
spark.sql.catalog.dremio.type=rest
spark.sql.catalog.dremio.uri=https://your-catalog.dremio.cloud
spark.sql.catalog.dremio.credential=<token>

Trino:

connector.name=iceberg
iceberg.catalog.type=rest
iceberg.rest-catalog.uri=https://your-catalog.dremio.cloud
iceberg.rest-catalog.security=OAUTH2

Once connected, tables created or modified by any engine appear in all other engines immediately — through the shared catalog's consistent metadata view.

Open Catalog and Data Governance

Dremio Open Catalog enforces access control at the catalog API layer — not in the query engine. This means security policies are consistent regardless of which engine is querying. A principal without access to a namespace cannot load that namespace's table metadata, regardless of whether they're using Dremio, Spark, or Trino.

Access control in Dremio Open Catalog is managed through role-based policies on namespaces and tables: read access, write access, manage (create/drop) access, and admin. Combined with credential vending, this creates a security model where neither the query engine nor the storage credentials need to carry the full security policy — the catalog is the enforcement point.

Open Catalog Multi-Engine Governance diagram
Figure 2: Open Catalog enforces governance consistently across all connected engines.

Summary

Dremio Open Catalog is the practical implementation of the open lakehouse vision: a vendor-neutral, Iceberg REST Catalog compliant metadata store that any engine can use, governed consistently regardless of client, and extended with Git-like branching for advanced data engineering workflows. For organizations building multi-engine lakehouses on Apache Iceberg, Dremio Open Catalog provides the catalog foundation that makes true engine interoperability a reality.