Ozone is a scalable, redundant, and distributed object store for Hadoop and Cloud-native environments. Apart from scaling to billions of objects of varying sizes, Ozone can function effectively in containerized environments such as Kubernetes and YARN.

Apache Ozone

This section documents the internal workings of the Datanode.

[children]


Datanode

KnowForge 新版本 2026.0.6 正式发布！！！

产品

资源

其他

Overview

Docker

Kubernetes

Bare Metal

Installation

Reading and Writing Data in Ozone

Quick Start

Ozone Manager

Storage Container Manager

Ozone S3 Gateway

Datanodes

Recon

Architecture

Storage Containers

Write Pipelines

Ratis

Erasure Coding

Replication

Owners

Quotas

Volumes

Layouts

Default Replication Type

Encryption

Links

Snapshots

Buckets

Keys

Namespace

Kerberos

Native ACLs

Ranger authorization policies

ACLs

Security

SCM High Availability

Ozone Manager High Availability

High Availability

Comparison with Other Storage Technologies

Core Concepts

Ozone Shell

Securing S3

S3 API

HttpFS Gateway

Java Client API

Accessing Apache Ozone from Python

CSI Protocol

Native C/C++ Client Access to Ozone

Client Interfaces

Ozone

HDFS

Clients

Hive

Iceberg

Impala

Oozie

Spark

Trino

DistCp

Flink

HBase

Integrations

User Guide

Deployment Architecture

Reference Architecture

Hardware and Sizing

Installing the Ozone Binaries

Initializing Cluster

Configuration Files

Environment Variables

Network

Directories

Basic

Application Logs

Audit Logs

Logging

Administrators

HTTPS

Apache Knox

Apache Ranger

Securing S3 Secrets

Production Deployment

Topology

Multi-Raft

RocksDB In Apache Ozone

Streaming Write Pipeline

Calculating EC Pipeline Limits

Calculating Ratis Pipeline Limits

S3 Gateway Load Balancing

Fairness

HttpFS load balancer (Kerberos)

Performance

SCM HA Configuration

OM HA Configuration

Client Failover

Appendix

Configuration

Start and Stop

Upgrade and Downgrade

Decommissioning and Maintenance

Disk Replacement

Container Replication Report

Balancing Data Among Datanodes

Data Balancing

Certificate Rotation

Setup

Tenant Commands

Access Control

S3 Multi-Tenancy

Snapshots Overview

Snapshot Configuration Properties

Web UIs

Prometheus

Grafana

Distributed tracing

Observability

Leader Transfer

Quota in Ozone

Ozone Repair

Ozone Admin

Ozone Debug

Tools

Trash

Dynamic Property Reload

Operations

Administrator Guide

Startup

Safemode

Read and Write

Delete

PLACEHOLDER

Certificates

Authentication

Authorization

Upgrades and Downgrades

RocksDB

OM HA snapshot installation

Troubleshooting

Disk Layout

RocksDB Schema

In-Memory Storage

Startup Sequence

Roles

Write Request Flow

Read Request Flow

Admin Write Request Flow

Admin Read Request Flow

Datanode Container Schema v3

RocksDB Layout

SQL Schema

Syncing Data From Ozone Manager

Syncing Data From Storage Container Manager

Syncing Data From Datanodes

Request Flow

S3 Gateway

Jars

Client

Components

Write

Read

Data Operations

Checksums

Datanode Container Scanner

Datanode Volume Scanner

Data Integrity

Metadata

Containers

SCM Replication Manager

SCM Container Balancer

Data

Symmetric Encryption

Tokens

SASL

S3 Credentials

Server

Network Protocols

Filesystem Optimization (FSO)

Snapshot Deletion Lifecycle

Bucket Snapshot

Snapshot Defragmentation

OM Bootstrapping with Snapshots

Features

System Internals

Maven

Intellij

Docker Images

Build

Docker Compose

Unit Tests

Integration Tests

Acceptance Tests

Continuous Integration

Static Analysis

Test

Developer Guide

Datanode

评论