1. Project Overview

Apache Dolphinscheduler

A distributed, easily extensible visual DAG workflow task scheduling system. It aims to solve the complex dependencies in data processing workflows, making the scheduling system ready to use out of the box for data processing tasks.

2. Open Source License

Licensed under Apache-2.0

3. Screenshots

System Part Screenshots

• Home: Provides an overview of projects and workflows, including status statistics for the latest workflow instances and task instances. 

• Workflow Definitions: Create and manage workflows using drag-and-drop operations, making it easy to build and maintain complex workflows. 

• Workflow Tree Diagram: The abstract tree structure makes it easier to understand the relationships between tasks. 

• Data Sources: Supports managing multiple external data sources, including MySQL, PostgreSQL, Hive, and Trino, and provides unified data access capabilities. 

• Monitoring: View the real-time status of the master, workers, and databases, including server resource utilization and load conditions. Perform health checks quickly without logging into the servers. 

4. Feature Overview

A next-generation cloud-native workflow task scheduling platform dedicated to "resolving complex dependencies between tasks and making the entire data processing workflow ready to use out of the box."

The main features of DolphinScheduler are as follows:

• Easy to deploy, offering four deployment methods: Standalone, Cluster, Docker, and Kubernetes.

• Easy to use, supporting four ways to create and manage workflows: Web UI, Python SDK, YAML files, and Open API.

• High Reliability and High Availability: A decentralized multi-master, multi-worker architecture with native horizontal scalability.

• High Performance: N times faster than other orchestration platforms, capable of handling tens of millions of tasks per day.

• Cloud Native: DolphinScheduler supports orchestrating workflows across multiple clouds and data centers, as well as custom task types.

• Version Control: Supports version control for workflows and workflow instances, including tasks.

• Flexible State Management: Supports multiple states for workflows and tasks, allowing them to be paused, stopped, or resumed at any time.

• Multi-tenant Support

• Other features include backfill support (native Web UI), as well as permission management for projects, resources, and data sources.

5. Tech Stack

Prerequisites

• JDK: Download JDK (1.8+), install it, and configure the JAVA_HOME environment variable. Add its bin directory to the PATH environment variable. If JDK is already installed and configured in your environment, you can skip this step.

• Binary Package: Download the DolphinScheduler binary package from the download page.

Start DolphinScheduler Standalone Server

Extract and Start DolphinScheduler

The binary archive includes a startup script for Standalone mode. You can quickly start it after extraction. Switch to a user with sudo privileges and run the script.

# Extract and Run Standalone Server
tar -xvzf apache-dolphinscheduler-*-bin.tar.gz
cd apache-dolphinscheduler-*-bin
bash ./bin/dolphinscheduler-daemon.sh start standalone-server

Log in to DolphinScheduler

Visit [http://localhost:12345/dolphinscheduler/ui](http://localhost:12345/dolphinscheduler/ui) in your browser to access the system UI. The default username and password are admin/dolphinscheduler123

Start and Stop Services

The ./bin/dolphinscheduler-daemon.sh script can not only quickly start Standalone but also stop the service. All available commands are listed below:

# Start Standalone Server
bash ./bin/dolphinscheduler-daemon.sh start standalone-server
# Stop Standalone Server
bash ./bin/dolphinscheduler-daemon.sh stop standalone-server

Source: DolphinScheduler Official Documentation