KnowForge/Apache Ozone 简体中文
Ozone Debug

LDB 工具

师成师成· 更新于 2026-09-28· 阅读 22 分钟· 0 次阅读
登录后可跨设备保存划线和私人笔记登录

Ozone 大量使用 RocksDB 来存储元数据。此工具用于解析属于各 Ozone 角色的 RocksDB 中的内容。支持的数据库：Ozone Manager（om.db）、StorageContainerManager（scm.db）、Datanode/Container（container.db）。使用方法如下：

Usage: ozone debug ldb --db=<dbPath> [COMMAND]
Parse rocksdb file content
      --db=<dbPath>   Database File Path
Commands:
  scan                      Parse specified metadataTable
  list_column_families, ls  list all column families in db.
  value-schema              Schema of value in metadataTable
  checkpoint                Create checkpoint for specified db

list_column_families 命令

list_column_families 命令列出所提供数据库中的所有列族。

$ ozone debug ldb --db=/path/to/scm.db ls
default
sequenceId
revokedCertsV2
pipelines
crls
crlSequenceId
meta
containers
validCerts
validSCMCerts
scmTransactionInfos
deletedBlocks
statefulServiceConfig
revokedCerts
move

scan 命令

scan 命令会解析所提供的 RocksDB 中的特定列族，并打印其中的记录。

Usage: ozone debug ldb scan [--compact] [--count] [--with-keys]
                            [--batch-size=<batchSize>] --cf=<tableName>
                            [--cid=<containerId>] [-d=<dnDBSchemaVersion>]
                            [-e=<endKey>] [--fields=<fieldsFilter>]
                            [--filter=<filter>] [-l=<limit>] [-o=<fileName>]
                            [-s=<startKey>] [--thread-count=<threadCount>]
Parse specified metadataTable
      --batch-size=<batchSize>
                          Batch size for processing DB data.
      --cf, --column_family, --column-family=<tableName>
                          Table name
      --cid, --container-id=<containerId>
                          Container ID. Applicable if datanode DB Schema is V3
      --compact           disable the pretty print the output
      --count, --show-count
                          Get estimated key count for the given DB column family
                            Default: false
  -d, --dnSchema, --dn-schema=<dnDBSchemaVersion>
                          Datanode DB Schema Version: V1/V2/V3
  -e, --ek, --endkey=<endKey>
                          Key at which iteration of the DB ends
      --fields=<fieldsFilter>
                          Comma-separated list of fields needed for each value.
                            eg.) "name,acls.type" for showing name and type
                            under acls.
      --filter=<filter>   Comma-separated list of "<field>:<operator>:<value>"
                            where <field> is any valid field of the record,
                            <operator> is [EQUALS,LESSER, GREATER or REGEX].
                            (EQUALS compares the exact string, REGEX compares
                            with a valid regular expression passed, and
                            LESSER/GREATER works with numeric values), <value>
                            is the value of the field.
                          eg.) "dataSize:equals:1000" for showing records
                            having the value 1000 for dataSize,
                               "keyName:regex:^key.*$" for showing records
                            having keyName that matches the given regex.
  -l, --limit, --length=<limit>
                          Maximum number of items to list.
  -o, --out=<fileName>    File to dump table scan data
  -s, --sk, --startkey=<startKey>
                          Key from which to iterate the DB
      --thread-count=<threadCount>
                          Thread count for concurrent processing.
      --with-keys         Print a JSON object of key->value pairs (default)
                            instead of a JSON array of only values.

默认情况下，内容会打印到控制台，但可以使用 --out 选项将其重定向到文件。

  • --length 可用于限制打印的记录数量。
  • --count 不会打印记录，而是显示记录的大致数量。该数值并不精确。
  • ozone debug ldb scan 命令提供了许多过滤选项，以便更轻松地进行调试，详见下文：

tip

在单个命令中可以同时使用多个过滤选项。

--startkey 和 --endkey

顾名思义，这两个选项用于指定迭代的起始键和结束键。

  • --startkey 指定从哪个键开始迭代，包含该键。
  • --endkey 指定迭代到哪个键结束，不包含该键。
ozone debug ldb --db=/path/to/om.db scan --cf=volumeTable --startkey=vol3 --endkey=vol5
{
  "/vol3": {
    "metadata": {},
    "objectID": -9999,
    "updateID": 4000,
    "adminName": "om",
    "ownerName": "om",
    "volume": "vol3",
    "creationTime": 1707192335309,
    "modificationTime": 1714057412205,
    "quotaInBytes": 22854448694951936,
    "quotaInNamespace": 100000000,
    "usedNamespace": 1,
    "acls": [
      {
        "type": "USER",
        "name": "om",
        "aclScope": "ACCESS"
      }
    ],
    "refCount": 0
  },
  "/vol4": {
    "metadata": {},
    "objectID": -888,
    "updateID": 5000,
    "adminName": "om",
    "ownerName": "om",
    "volume": "vol4",
    "creationTime": 1696280979907,
    "modificationTime": 1696280979907,
    "quotaInBytes": 2251799813685250,
    "quotaInNamespace": 100000000,
    "usedNamespace": 2,
    "acls": [
      {
        "type": "USER",
        "name": "om",
        "aclScope": "ACCESS"
      }
    ],
    "refCount": 0
  }
}

--fields

每条记录中包含多个字段。--fields 选项允许我们选择需要显示的特定字段。

ozone debug ldb --db=/path/to/om.db scan --cf=keyTable -l=1 --fields="volumeName,bucketName,keyName,keyLocationVersions.version,acls.name"
{
  "/vol1/ozone-legacy-bucket/10T-1-terasort-input/": {
    "keyLocationVersions": [
      {
        "version": 0
      }
    ],
    "keyName": "10T-1-terasort-input/",
    "bucketName": "ozone-legacy-bucket",
    "acls": [
      {
        "name": "om"
      },
      {
        "name": "scm"
      },
      {
        "name": "testuser"
      }
    ],
    "volumeName": "vol1"
  }
}

--filter

--filter 可用于筛选其值符合给定条件的记录。过滤器采用如下格式：<field>:<operator>:<value>，其中 <field> 是记录值中的任意有效字段，<operator> 是 4 种受支持的操作之一 [equals, regex, lesser, greater]，<value> 是用于比较的值。

  • equals 和 regex 适用于字符串、布尔和数值字段，lesser 和 greater 仅适用于数值。
  • 也可以在一条命令中给出多个过滤器，它们需要用逗号分隔。

使用 equals 运算符：

ozone debug ldb --db=/path/to/om.db scan --cf=volumeTable --filter="usedNamespace:equals:2"
{
  "/vol4": {
    "metadata": {},
    "objectID": -888,
    "updateID": 5000,
    "adminName": "om",
    "ownerName": "om",
    "volume": "vol4",
    "creationTime": 1696280979907,
    "modificationTime": 1696280979907,
    "quotaInBytes": 2251799813685250,
    "quotaInNamespace": 100000000,
    "usedNamespace": 2,
    "acls": [
      {
        "type": "USER",
        "name": "om",
        "aclScope": "ACCESS"
      }
    ],
    "refCount": 0
  },
  "/vol5": {
    "metadata": {},
    "objectID": -956599,
    "updateID": 45600,
    "adminName": "om",
    "ownerName": "om",
    "volume": "vol5",
    "creationTime": 1807192332309,
    "modificationTime": 1914057410005,
    "quotaInBytes": 7785494951936,
    "quotaInNamespace": 100000000,
    "usedNamespace": 2,
    "acls": [
      {
        "type": "USER",
        "name": "om",
        "aclScope": "ACCESS"
      }
    ],
    "refCount": 0
  }
}

使用 lesser 运算符（greater 运算符的用法完全相同）：

ozone debug ldb --db=/path/to/om.db scan --cf=volumeTable --filter="usedNamespace:lesser:2"
{
  "/vol2": {
    "metadata": {},
    "objectID": -73548,
    "updateID": 2384,
    "adminName": "om",
    "ownerName": "om",
    "volume": "vol2",
    "creationTime": 11980979907,
    "modificationTime": 1296280979900,
    "quotaInBytes": 417913685250,
    "quotaInNamespace": 100000000,
    "usedNamespace": 1,
    "acls": [
      {
        "type": "USER",
        "name": "om",
        "aclScope": "ACCESS"
      }
    ],
    "refCount": 0
  }
}

使用 regex 运算符：

ozone debug ldb --db=/path/to/om.db scan --cf=volumeTable --filter="volume:regex:^v.*2$"
{
  "/vol2": {
    "metadata": {},
    "objectID": -73548,
    "updateID": 2384,
    "adminName": "om",
    "ownerName": "om",
    "volume": "vol2",
    "creationTime": 11980979907,
    "modificationTime": 1296280979900,
    "quotaInBytes": 417913685250,
    "quotaInNamespace": 100000000,
    "usedNamespace": 1,
    "acls": [
      {
        "type": "USER",
        "name": "om",
        "aclScope": "ACCESS"
      }
    ],
    "refCount": 0
  }
}

使用多个过滤器：

ozone debug ldb --db=/path/to/om.db scan --cf=volumeTable --filter="usedNamespace:equals:2,volume:regex:^.*4$"
{
  "/vol4": {
    "metadata": {},
    "objectID": -888,
    "updateID": 5000,
    "adminName": "om",
    "ownerName": "om",
    "volume": "vol4",
    "creationTime": 1696280979907,
    "modificationTime": 1696280979907,
    "quotaInBytes": 2251799813685250,
    "quotaInNamespace": 100000000,
    "usedNamespace": 2,
    "acls": [
      {
        "type": "USER",
        "name": "om",
        "aclScope": "ACCESS"
      }
    ],
    "refCount": 0
  }
}

value-schema 命令

value-schema 命令用于显示 RocksDB 中某个列族（column-family）所存储值的 schema，即显示值中存储的字段及其数据类型。可以使用可选参数 --depth 来限制获取字段的层级深度。

ozone debug ldb --db=/data/metadata/om.db value-schema --cf=keyTable --depth=1
{
  "OmKeyInfo": {
    "bucketName": "String",
    "metadata": "struct",
    "fileName": "String",
    "creationTime": "long",
    "isFile": "boolean",
    "acls": "struct",
    "keyName": "String",
    "replicationConfig": "struct",
    "encInfo": "struct",
    "dataSize": "long",
    "tags": "struct",
    "keyLocationVersions": "struct",
    "updateID": "long",
    "ownerName": "String",
    "modificationTime": "long",
    "parentObjectID": "long",
    "volumeName": "String",
    "fileChecksum": "struct",
    "objectID": "long"
  }
}
ozone debug ldb --db=/data/metadata/om.db value-schema --cf=keyTable
{
  "OmKeyInfo": {
    "bucketName": "String",
    "metadata": {},
    "fileName": "String",
    "creationTime": "long",
    "isFile": "boolean",
    "acls": {
      "toStringMethod": {},
      "hashCodeMethod": {},
      "name": "String",
      "type": {
        "name": "String",
        "value": "String",
        "ordinal": "int"
      },
      "aclScope": {
        "name": "String",
        "ordinal": "int"
      },
      "aclBits": "int"
    },
    "keyName": "String",
    "replicationConfig": {},
    "encInfo": {
      "ezKeyVersionName": "String",
      "keyName": "String",
      "edek": {},
      "cipherSuite": {
        "unknownValue": {
          "value": "int"
        },
        "name": "String",
        "algoBlockSize": "int",
        "ordinal": "int"
      },
      "version": {
        "unknownValue": {
          "value": "int"
        },
        "name": "String",
        "description": "String",
        "version": "int",
        "ordinal": "int"
      },
      "iv": {}
    },
    "dataSize": "long",
    "tags": {},
    "keyLocationVersions": {
      "isMultipartKey": "boolean",
      "locationVersionMap": {},
      "version": "long"
    },
    "updateID": "long",
    "ownerName": "String",
    "modificationTime": "long",
    "parentObjectID": "long",
    "volumeName": "String",
    "fileChecksum": {},
    "objectID": "long"
  }
}

checkpoint 命令

checkpoint 命令用于在指定路径上为 RocksDB 创建检查点。

ozone debug ldb --db=/data/metadata/om.db checkpoint --output=/tmp/om-checkpoint

下一个 >>

评论

登录后参与评论

正在加载评论…