What Is Catalog Interoperability?

Catalog interoperability is the technical property that allows multiple heterogeneous query engines to access, create, and modify tables in a shared catalog through a standard API — without requiring custom, engine-specific integrations for each catalog backend. In the data lakehouse context, catalog interoperability is what makes the 'multi-engine' promise real: not just theoretically possible, but operationally seamless.

Before the Iceberg REST Catalog specification, catalog integrations were point-to-point: Spark had its own HiveCatalog implementation, Trino had its own HMS connector, Presto had its own. Adding a new catalog backend required implementing a new connector for each engine — an O(engines × catalogs) engineering burden. With the REST catalog specification, each engine implements one REST client and each catalog implements one REST server — reducing the integration burden to O(engines + catalogs).

How REST Catalog Enables Interoperability

The Iceberg REST Catalog specification defines a standard HTTP API for all catalog operations. Any engine implementing the REST client can connect to any catalog implementing the REST server:

  • Table discovery: List namespaces and tables via standard GET endpoints
  • Metadata loading: Load table metadata (current metadata file location + storage credentials) via standard GET endpoint
  • Commit: Commit new snapshots via standard POST endpoint
  • Auth: OAuth2 token-based authentication — any engine can authenticate with any catalog using the same flow

When Dremio and Apache Spark both connect to the same Apache Polaris catalog, they share one namespace. A table Spark creates is immediately queryable by Dremio, with the same schema and the same access control policies applied to both engines' requests.

Catalog Interoperability REST API Architecture diagram
Figure 1: REST Catalog interoperability — any engine connects to any catalog via the standard HTTP API.

Interoperability Across Catalog Implementations

Catalog interoperability also means that organizations can switch catalog backends without reconfiguring engines. If a team starts with AWS Glue as their catalog and later migrates to Apache Polaris for better governance, the engine configuration change is minimal — update the REST catalog URI in the engine's Iceberg catalog config. No engine-specific plugins need to be replaced, no data needs to be copied, and no table registrations need to be redone (tables are re-registered in the new catalog pointing to the same physical data files).

This portability is fundamental to avoiding catalog lock-in — one of the most insidious forms of vendor lock-in in the pre-REST-catalog era. Organizations that standardized on a proprietary catalog (like AWS Glue before its REST API) found that migrating to a different catalog required significant re-engineering of all engine configurations and integrations.

Catalog Interoperability Multi-Engine Access diagram
Figure 2: Multi-engine catalog interoperability — Spark, Trino, Flink, and Dremio sharing one catalog.

Summary

Catalog interoperability — enabled by the Iceberg REST Catalog specification — is the technical foundation of the open, multi-engine data lakehouse. It transforms the catalog from a proprietary integration point into a commodity service: organizations can choose their preferred catalog implementation (Apache Polaris, Project Nessie, AWS Glue, Dremio Open Catalog) based on features and governance needs, confident that all their engines will connect to it through the same standard API. This is the vendor-neutral, open-ecosystem architecture that the data lakehouse community has been building toward.