1. Project Overview

JuiceFS is a high-performance POSIX file system specially optimized for cloud-native environments and released under the Apache 2.0 open-source license. When using JuiceFS to store data, the data itself is persisted in object storage (such as Amazon S3), while the corresponding metadata can be persisted in various database engines, including Redis, MySQL, and TiKV, depending on specific requirements.

JuiceFS makes it easy to integrate massive cloud storage directly into big data, machine learning, artificial intelligence, and various application platforms already running in production. No code modifications are required, allowing you to use massive amounts of cloud storage as efficiently as local storage.

2. Open Source License

Licensed under Apache-2.0

3. Screenshots

4. Feature Overview

JuiceFS is an open-source POSIX file system built on Redis and object storage (such as Amazon S3), designed and optimized for cloud-native environments.

Core Features

1. POSIX Compatible: Works like a local file system, seamlessly integrates with existing applications, and requires no changes to business logic;
2. HDFS Compatible: Fully compatible with the HDFS API and provides higher metadata performance;
3. S3 Compatible: Provides an S3 Gateway for access through an S3-compatible interface;
4. cloud-native: Easily use JuiceFS in Kubernetes through the Kubernetes CSI driver;
5. Multi-Client Sharing: The same file system can be mounted simultaneously on thousands of servers, supporting high-performance concurrent reads and writes and shared data access;
6. Strong Consistency: Confirmed changes become immediately visible on all servers mounting the same file system, ensuring strong consistency;
7. Powerful Performance: Millisecond-level latency and virtually unlimited throughput (depending on the scale of the object storage). View the performance test results;
8. Data Security: Supports encryption in transit and encryption at rest. View details;
9. File Locking: Supports BSD locks (flock) and POSIX locks (fcntl);
10. Data Compression: Supports LZ4 or Zstandard compression to reduce storage space usage.

Supported Object Storage Services

• Amazon S3
• Google Cloud Storage
• Microsoft Azure Storage
• Alibaba Cloud OSS
• Tencent Cloud COS
• QingCloud QingStor Object Storage
• Ceph RGW
• MinIO
• Local Directory
• Redis
• ……

JuiceFS supports almost all mainstream object storage services. View details.

Who Is Using It

JuiceFS is ready for production use, with thousands of nodes already running it in production environments. We have compiled a list of users, which is available here. JuiceFS also has several integration projects with other open-source projects, which we have documented here. If you are also using JuiceFS, please feel free to let us know. We also welcome you to share your specific usage experiences with the community.

The JuiceFS storage format is stable and will be supported by all subsequent releases.

Product Roadmap

• Support FoundationDB as a metadata engine
• Directory-based quotas
• User- and group-based quotas
• Snapshots
• Write Once, Read Many (WORM)

5. Tech Stack

Architecture

JuiceFS consists of three components:

1. JuiceFS Client: Coordinates object storage and metadata storage engines, and implements file system interfaces such as POSIX, Hadoop, Kubernetes, and S3 Gateway;
2. Data storage: Stores the actual data and supports local disks and object storage;
3. Metadata Engine: Stores metadata corresponding to the data and supports multiple engines, including Redis, MySQL, and SQLite;

JuiceFS relies on Redis to store file metadata. Redis is a high-performance, memory-based key-value data store that is ideal for storing metadata. Meanwhile, all data is stored in object storage through the JuiceFS client. Learn more

Every file stored in JuiceFS is split into fixed-size "Chunk", with a default maximum capacity of 64 MiB. Each Chunk consists of one or more "Slice" . The length of a Slice is variable and depends on how the file is written. Each Slice is further divided into fixed-size "Block", which default to 4 MiB. Finally, these Blocks are stored in object storage. Meanwhile, JuiceFS stores metadata for each file, including its Chunks, Slices, and Blocks, in the metadata engine. Learn more

When using JuiceFS, files are ultimately split into Chunks, Slices, and Blocks and stored in object storage. Therefore, you will notice that the original files stored in JuiceFS cannot be found in the file browser of your object storage platform. The storage bucket contains only a chunks directory and a collection of directories and files with numeric names. Don't panic! This is exactly the secret behind JuiceFS's high-performance operation!

Get Started

To create a JuiceFS file system, you need to prepare the following three things:

• Prepare a Redis database
• Prepare object storage
• Download and install the JuiceFS client

Follow the Quick Start Guide to start using JuiceFS right away!

Command Index

Click here to view all subcommands and command-line arguments.

Container

JuiceFS provides persistent storage for container technologies such as Docker and Podman. See the documentation for details.

Kubernetes

Using JuiceFS in Kubernetes is very convenient. See this documentation for more information.

Hadoop Java SDK

JuiceFS integrates with the Hadoop ecosystem using the Hadoop Java SDK.

Advanced Topics

Detailed content, please refer to README.md document.