KnowForge/Apache Iceberg 1.11.0/ 返回书籍
Tables

Configuration

qianmoQqianmoQ· 更新于 2026-09-21· 阅读 28 分钟· 0 次阅读

登录后可跨设备保存划线和私人笔记登录

Configuration🔗

Table properties🔗

Iceberg tables support table properties to configure table behavior, like the default split size for readers.

Read properties🔗

PropertyDefaultDescription
read.split.target-size134217728 (128 MB)Target size when combining data input splits
read.split.metadata-target-size33554432 (32 MB)Target size when combining metadata input splits
read.split.planning-lookback10Number of bins to consider when combining input splits
read.split.open-file-cost4194304 (4 MB)The estimated cost to open a file, used as a minimum weight when combining splits.
read.parquet.vectorization.enabledtrueControls whether Parquet vectorized reads are used
read.parquet.vectorization.batch-size5000The batch size for parquet vectorized reads
read.orc.vectorization.enabledfalseControls whether orc vectorized reads are used
read.orc.vectorization.batch-size5000The batch size for orc vectorized reads

Write properties🔗

PropertyDefaultDescription
write.format.defaultparquetDefault file format for the table; parquet, avro, or orc
write.delete.format.defaultdata file formatDefault delete file format for the table; parquet, avro, or orc
write.parquet.row-group-size-bytes134217728 (128 MB)Parquet row group size
write.parquet.page-size-bytes1048576 (1 MB)Parquet page size
write.parquet.page-versionv1Parquet data page version: v1 (DataPage V1) or v2 (DataPage V2)
write.parquet.page-row-limit20000Parquet page row limit
write.parquet.dict-size-bytes2097152 (2 MB)Parquet dictionary page size
write.parquet.compression-codeczstdParquet compression codec: zstd, brotli, lz4, gzip, snappy, uncompressed
write.parquet.compression-levelnullParquet compression level
write.parquet.shred-variantsfalseWhen true, variant columns are written with shredded Parquet encoding for improved query performance
write.parquet.variant-inference-buffer-size100Number of rows to buffer for schema inference when variant shredding is enabled
write.parquet.bloom-filter-enabled.column.col1(not set)Hint to parquet to write a bloom filter for the column: 'col1'
write.parquet.bloom-filter-max-bytes1048576 (1 MB)The maximum number of bytes for a bloom filter bitset
write.parquet.bloom-filter-fpp.column.col10.01The false positive probability for a bloom filter applied to 'col1' (must > 0.0 and < 1.0)
write.parquet.bloom-filter-ndv.column.col1(not set)The expected number of distinct values for a bloom filter applied to 'col1' (must > 0)
write.parquet.stats-enabled.column.col1(not set)Controls whether to collect parquet column statistics for column 'col1'
write.avro.compression-codecgzipAvro compression codec: gzip(deflate with 9 level), zstd, snappy, uncompressed
write.avro.compression-levelnullAvro compression level
write.orc.stripe-size-bytes67108864 (64 MB)Define the default ORC stripe size, in bytes
write.orc.block-size-bytes268435456 (256 MB)Define the default file system block size for ORC files
write.orc.compression-codeczlibORC compression codec: zstd, lz4, lzo, zlib, snappy, none
write.orc.compression-strategyspeedORC compression strategy: speed, compression
write.orc.bloom.filter.columns(not set)Comma separated list of column names for which a Bloom filter must be created
write.orc.bloom.filter.fpp0.05False positive probability for Bloom filter (must > 0.0 and < 1.0)
write.location-provider.implnullOptional custom implementation for LocationProvider
write.metadata.compression-codecnoneMetadata compression codec; none or gzip
write.metadata.metrics.max-inferred-column-defaults100Defines the maximum number of columns for which metrics are collected. Columns are included with a pre-order traversal of the schema: top level fields first; then all elements of the first nested struct; then the next nested struct and so on.
write.metadata.metrics.defaulttruncate(16)Default metrics mode for all columns in the table; none, counts, truncate(length), or full
write.metadata.metrics.column.col1(not set)Metrics mode for column 'col1' to allow per-column tuning; none, counts, truncate(length), or full
write.target-file-size-bytes536870912 (512 MB)Controls the size of files generated to target about this many bytes
write.delete.target-file-size-bytes67108864 (64 MB)Controls the size of delete files generated to target about this many bytes
write.distribution-modenot set, see engines for specific defaults, for example Spark WritesDefines distribution of write data: none: don't shuffle rows; hash: hash distribute by partition key ; range: range distribute by partition key or sort key if table has an SortOrder
write.delete.distribution-mode(not set)Defines distribution of write delete data
write.update.distribution-mode(not set)Defines distribution of write update data
write.merge.distribution-mode(not set)Defines distribution of write merge data
write.wap.enabledfalseEnables write-audit-publish writes
write.summary.partition-limit0Includes partition-level summary stats in snapshot summaries if the changed partition count is less than this limit
write.metadata.delete-after-commit.enabledfalseControls whether to delete the oldest tracked version metadata files after each table commit. See the Remove old metadata files section for additional details
write.metadata.previous-versions-max100The max number of previous version metadata files to track
write.spark.fanout.enabledfalseEnables the fanout writer in Spark that does not require data to be clustered; uses more memory
write.object-storage.enabledfalseEnables the object storage location provider that adds a hash component to file paths
write.object-storage.partitioned-pathstrueIncludes the partition values in the file path
write.data.pathtable location + /dataBase location for data files
write.metadata.pathtable location + /metadataBase location for metadata files
write.delete.modecopy-on-writeMode used for delete commands: copy-on-write or merge-on-read (v2 and above)
write.delete.isolation-levelserializableIsolation level for delete commands: serializable or snapshot
write.update.modecopy-on-writeMode used for update commands: copy-on-write or merge-on-read (v2 and above)
write.update.isolation-levelserializableIsolation level for update commands: serializable or snapshot
write.merge.modecopy-on-writeMode used for merge commands: copy-on-write or merge-on-read (v2 and above)
write.merge.isolation-levelserializableIsolation level for merge commands: serializable or snapshot
write.delete.granularitypartitionControls the granularity of generated delete files: partition or file

Encryption properties🔗

PropertyDefaultDescription
encryption.key-id(not set)ID of the master key of the table
encryption.data-key-length16 (bytes)Length of keys used for encryption of table files. Valid values are 16, 24, 32 bytes

See the Encryption document for additional details.

Table behavior properties🔗

PropertyDefaultDescription
commit.retry.num-retries4Number of times to retry a commit before failing
commit.retry.min-wait-ms100Minimum time in milliseconds to wait before retrying a commit
commit.retry.max-wait-ms60000 (1 min)Maximum time in milliseconds to wait before retrying a commit
commit.retry.total-timeout-ms1800000 (30 min)Total retry timeout period in milliseconds for a commit
commit.status-check.num-retries3Number of times to check whether a commit succeeded after a connection is lost before failing due to an unknown commit state
commit.status-check.min-wait-ms1000 (1s)Minimum time in milliseconds to wait before retrying a status-check
commit.status-check.max-wait-ms60000 (1 min)Maximum time in milliseconds to wait before retrying a status-check
commit.status-check.total-timeout-ms1800000 (30 min)Total timeout period in which the commit status-check must succeed, in milliseconds
commit.manifest.target-size-bytes8388608 (8 MB)Target size when merging manifest files
commit.manifest.min-count-to-merge100Minimum number of manifests to accumulate before merging
commit.manifest-merge.enabledtrueControls whether to automatically merge manifests on writes
history.expire.max-snapshot-age-ms432000000 (5 days)Default max age of snapshots to keep on the table and all of its branches while expiring snapshots
history.expire.min-snapshots-to-keep1Default min number of snapshots to keep on the table and all of its branches while expiring snapshots
history.expire.max-ref-age-msLong.MAX_VALUE (forever)For snapshot references except the main branch, default max age of snapshot references to keep while expiring snapshots. The main branch never expires.
gc.enabledtrueAllows garbage collection operations such as expiring snapshots and removing orphan files

Reserved table properties🔗

Reserved table properties are only used to control behaviors when creating or updating a table. The value of these properties are not persisted as a part of the table metadata.

PropertyDefaultDescription
format-version2Table's format version as defined in the Spec. Defaults to 2 since version 1.4.0.

Informational properties🔗

Informational properties can be set to provide additional context about a table. They can be useful for documentation, discovery, and integration with external tools. They do not affect read/write behavior or query semantics.

PropertyDefaultDescription
comment(not set)A table-level description that documents the business meaning and usage context.

Compatibility flags🔗

PropertyDefaultDescription
compatibility.snapshot-id-inheritance.enabledfalseEnables committing snapshots without explicit snapshot IDs (always true if the format version is > 1)

评论

登录后参与评论

正在加载评论…