Overview
Stigg’s integration with Apache Iceberg allows you to export your product catalog, customer, and subscription data as Iceberg tables, backed by a catalog of your choice, for use with Spark, Athena, Snowflake, and other query engines.Stigg’s native data export integrations are included in the Scale plan, and are also available as an optional add-on to the Growth plan. See Stigg’s pricing for more details.
Apache Iceberg is currently a beta destination with an upper limit of 1 billion rows per month. Contact Stigg support if you expect to exceed that volume.
View the full entity schema
See every table and column exported to your destination, organized by entity group.
Setting up the integration
Iceberg requires a catalog to manage table metadata and provide atomic transactions. Stigg supports connecting via an AWS Glue Catalog, an AWS S3 Tables Catalog, an Iceberg REST Catalog (such as Cloudflare R2 Data Catalog or Tabular), or a Google Lakehouse Catalog (BigLake).Prerequisites
- An S3 bucket for the underlying data files
- An AWS Glue Data Catalog database
This guide covers AWS Glue Catalog as the primary flow, since it’s the most common Iceberg catalog. If you use a different catalog, see Other catalogs below instead.
1
Create the destination S3 bucket
In the AWS S3 console, click Create bucket. Enter a Bucket name. AWS recommends setting Object Ownership to ACLs disabled and Block Public Access settings for this bucket to Block all public access.Click Create bucket, and note the bucket name and AWS Region.
2
Create an IAM policy and role
In the AWS IAM console, create a policy that grants the following permissions, replacing Create a role with a Custom trust policy, pasting in the trust policy shown in the Stigg connect form to allow
BUCKET_NAME, ACCOUNT_ID, and DATABASE with your values:If your bucket uses KMS encryption (CMK), also grant
kms:GenerateDataKey and kms:Decrypt on the key.AssumeRole access. Attach the policy above, then note the role’s ARN.3
Connect Stigg with Iceberg
In Stigg, navigate to Integrations > Apps > Iceberg.Enter the following information in the connection form:
Click Test & connect.
4
Select entities to export
After entering your connection details, expand the Entities to export section to choose which entity groups to include in the sync. All groups are selected by default.See Exported entities for a description of each group.
Other catalogs
Once connected via any of these catalogs, finish by clicking Test & connect, then expand Entities to export on the connect form — see Exported entities for a description of each group.AWS S3 Tables Catalog
This catalog requires two S3 buckets: an S3 Tables bucket for the catalog itself, and a standard staging S3 bucket in the same region. The IAM role’s trust policy must allow the role to assume itself, in addition to the usual service principal — the role has to be created first, then its trust policy updated afterward, since it can’t be self-assuming at creation time. Connection fields:Iceberg REST Catalog
Cloudflare R2 Data Catalog — benefits from zero egress fees. Create an R2 bucket and an API token with Admin Read & Write permissions in the Cloudflare dashboard, then provide:
Tabular — create a credential with table-creation and write-data permissions in your Tabular organization dashboard, then provide:
Google Lakehouse Catalog (BigLake)
Requires a GCP service account with GCS read/write and Lakehouse catalog management permissions, granted either via impersonation (recommended — grant Service Account Token Creator) or a service account key. Your Lakehouse catalog must be configured to use end-user credentials, not credential vending. Grant the service account Storage Object Admin on the destination bucket and BigLake Editor on the target project/catalog (or the equivalent fine-grainedstorage.* and biglake.* permissions).
BigLake tables are queryable directly in BigQuery without needing an external table definition.
Configuration options
Data is staged under a_write_ahead_staging prefix before being atomically committed into the final table. We recommend configuring an object storage lifecycle policy to delete objects under that prefix after 30 days.
retention_window_days sets how long historical snapshots are retained — configure it according to your organization’s data retention policy.
Treat destination tables as read-only. Executing write or delete operations against them manually may corrupt the table state. Snapshot expiry and compaction are handled automatically — no manual maintenance is required, though delete permissions are still needed for this background maintenance to run.