1. Project Overview

A distributed task scheduling framework that provides not only standard task scheduling capabilities but also features such as pausing or canceling running tasks, resuming paused tasks, task splitting, retrying failed tasks, broadcast tasks, task dependencies, workflow tasks (DAG), separate deployment of managers and executors, and a web-based management console.

Lightweight and easy to use, particularly suitable for executing long-running tasks. It offers excellent scalability, extensibility, and stability, and has been proven in production environments.

2. Open Source License

Licensed under Apache-2.0

3. Screenshots

4. Feature Overview

Features

• It has two roles: Manager (Supervisor) and Executor (Worker). Supervisor and Worker can be deployed separately.

• Supervisor and Worker discover each other through a registry. Supported registries include: Database, Redis, Consul, Nacos, Zookeeper, and Etcd.

• Supervisor is responsible for creating tasks and dispatching them to Workers for execution. Supported task dispatch methods include: Redis and Http.

• A Job group (job-group) must be specified. Jobs are dispatched only to Workers belonging to the designated group.

• Provides task sharding capabilities. By overriding the JobHandler#split method, a task can be split into multiple subtasks to enable distributed task processing and parallel execution.

• Supports pausing and canceling running tasks. Paused tasks can be resumed, and failed tasks can be retried.

• Supports saving task execution states through savepoint, allowing manually paused or unexpectedly interrupted tasks to resume from their previous execution state.

• If a task throws PauseTaskException during execution, all tasks under the corresponding instance will be paused, including tasks running on different Worker machines.

• Supports broadcast tasks, which are dispatched to all Workers under the specified job-group for execution.

• Supports dependencies between Jobs. Once dependencies are configured for multiple Jobs, they will execute sequentially according to the predefined dependency order.

• Supports DAG workflows. The jobHandler can be configured with complex DAG expressions, such as: A->B,C,(D->E)->D,F->G

• Provides a Web-based administration console for job configuration, task monitoring, and more.

5. Tech Stack

Architecture

• Overall Workflow

• Code Structure

disjob                                        # Main Project ①
├── disjob-admin                              # Admin Console Project ② (Further Developed Based on the Ruoyi Framework)
├── disjob-bom                                # Maven Project BOM Module
├── disjob-common                             # Common Utility Module
├── disjob-core                               # Core Classes Related to Task Scheduling (e.g., Data Models, Enums, Abstract Interface Layers)
├── disjob-dispatch                           # Task Dispatch Module
│   ├── disjob-dispatch-api                   # Abstract Interface Layer for Task Dispatch
│   ├── disjob-dispatch-http                  # HTTP Implementation for Task Dispatch
│   └── disjob-dispatch-redis                 # Redis Implementation for Task Dispatch
├── disjob-id                                 # Distributed ID Generation Module
├── disjob-registry                           # Server (Supervisor & Worker) Registration Module
│   ├── disjob-registry-api                   # Abstract Interface Layer for the Server Registry
│   ├── disjob-registry-consul                # Server Registry: Consul Implementation
│   ├── disjob-registry-database              # Server Registry: Database Implementation
│   ├── disjob-registry-etcd                  # Server Registry: Etcd Implementation
│   ├── disjob-registry-nacos                 # Server Registry: Nacos Implementation
│   ├── disjob-registry-redis                 # Server Registry: Redis Implementation
│   └── disjob-registry-zookeeper             # Server Registry: Zookeeper Implementation
├── disjob-reports                            # Aggregates Test Coverage Reports from All Modules
├── disjob-samples                            # Samples Project ③
│   ├── disjob-samples-frameless-worker       # Example of Standalone Worker Deployment (Standard Java Main Application)
│   ├── disjob-samples-springboot-common      # Common Module for Samples Spring Boot
│   ├── disjob-samples-springboot-merged      # Example of Combined Supervisor and Worker Deployment (Spring Boot Application)
│   ├── disjob-samples-springboot-supervisor  # Example of Standalone Supervisor Deployment (Spring Boot Application)
│   └── disjob-samples-springboot-worker      # Example of Standalone Worker Deployment (Spring Boot Application)
├── disjob-supervisor                         # Supervisor Code
├── disjob-test                               # For Testing Assistance
└── disjob-worker                             # Worker Code

Comparison

Maven Dependencies

<dependency>
  <groupId>cn.ponfee</groupId>
  <artifactId>disjob-{xxx}</artifactId>
  <version>2.0.4</version>
</dependency>

Build from source code

./mvnw clean install -DskipTests -Dcheckstyle.skip=true -Dmaven.javadoc.skip=true -U

1. Import each project separately into your development environment (divided into three independent projects sharing one Git repository).

• Main Project — disjob

• Admin Console — disjob-admin

• Example Project — disjob-samples

Different ports have been configured, so all applications can run simultaneously. You can run Java classes directly from your development environment or use the java -jar command to run the packaged JAR files.

• Start the embedded local MySQL & Redis. If you encounter a MariaDB "Library not loaded" error on macOS, refer to the relevant instructions. When connecting to MySQL using a client tool, use root as the username; no password is required. The initialization SQL scripts are executed automatically during startup.

• Start the standalone Worker Spring Boot application.

• Start the standalone Worker standard Java main application.

• Start the combined Supervisor + Worker Spring Boot application.

• Start the standalone Supervisor Spring Boot application.

• Start the Admin Console Spring Boot application.

• Visit 【 [http://127.0.0.1:80/](http://127.0.0.1:80/) 】 in your browser to log in to the Admin Console. Username/Password: admin/admin123

• After logging in, find the Scheduling Management menu in the left sidebar to access the scheduling management features.

• Scheduling Configuration: Manage job configurations, including viewing, adding, editing, deleting, triggering, and disabling jobs.

• Scheduling Instances: A specific execution instance (instance) triggered at a particular time for a job. Each instance is divided into multiple tasks (task). The first paginated table queries root instances and supports drill-down. A root instance refers to the initially triggered instance, including: ordinary non-retry instances, lead instances in workflows (DAG), and the first instance in a dependency chain. Scroll down the page to find the second paginated table, which queries all instances.

User Guide

1. The project includes two SQL scripts.

• SQL script for the core framework

• SQL script for the Admin Console

• Samples Project

• Admin Project

• By default, disjob-registry-redis is used as the registry center, and disjob-dispatch-http is used for task dispatch.

• MySQL Configuration for the Supervisor Role

• Core Configuration for the Supervisor Role

• Core Configuration for the Worker Role (Spring Boot Application)

• Redis Configuration (Shared by Worker and Supervisor; required when using Redis as the registry center or task dispatcher)

• Spring Boot Web Configuration (Shared by Worker and Supervisor)

• Configuration for the Standard Java Main Application in the Worker Role

• MySQL Configuration for the Supervisor Role

• Redis Configuration (Shared by Worker and Supervisor; required when using Redis as the registry center or task dispatcher)

• The @EnableWorker annotation can be added to enable the Worker role (the Admin Console must enable the Supervisor role).

• MySQL Configuration for Admin Console Features

• RuoYi Framework Configuration Reference (Only the ruoyi-disjob module has been added to disjob-admin; the other original RuoYi modules have remained largely unchanged.)

• Consul

• Database

• Zookeeper

• Nacos

• Etcd

For example, when using Consul as the configuration center, you can add the following configuration.

disjob.registry.consul:
  namespace: consul_namespace
  host: localhost
  port: 8500
  token:

6. Some Embedded Local Servers

• Mysql Server

• Redis Server

• Zookeeper Server

• Consul Server

• Nacos Server (requires a local Docker environment)

• Etcd Server (requires a local Docker environment)

For more information, please refer to the README.md document