1. Project Overview

PowerJob (formerly OhMyScheduler) is a new generation of distributed scheduling and computing framework, which can easily complete job scheduling and complex tasks distributed computing.

A new generation of distributed task scheduling and computing framework, supporting CRON, API, fixed frequency, fixed delay, etc. scheduling strategies, providing workflow to orchestrate tasks and solve dependency relationships, simple to use, powerful, and fully documented, welcome to use!

2. Open Source License

Licensed under Apache-2.0

3. Screenshots

4. Feature Overview

Main Features

• Simple to use: provides a front-end Web interface, allowing developers to visually complete task management (add, delete, modify, query), task execution status monitoring, and execution log viewing, etc. functions.
• Perfect timing strategy: supports CRON expressions, fixed frequency, fixed delay, and API four types of scheduling strategies.
• Rich execution modes: supports single machine, broadcast, Map, MapReduce four execution modes, among which Map/MapReduce processor can make developers obtain cluster distributed computing capabilities with a few lines of code.
• DAG workflow support: supports online configuration of task dependency relationships, visualized task arrangement, and also supports data transmission between upstream and downstream tasks.
• Executor supports a wide range: supports Spring Bean, built-in/external Java classes, Shell, Python, etc. processors, with a wide range of applications.
• Convenient operation and maintenance: supports online log function, the logs produced by the executor can be displayed in real-time on the front-end console page, reducing debugging costs and greatly improving development efficiency.
• Dependency minimization: only depends on relational databases (MySQL/Oracle/MS SQLServer...), and extends to MongoDB (used for storing large online logs).
• High availability & high performance: the scheduling server is carefully designed, and the traditional scheduling framework based on database locking strategy is replaced, achieving lock-free scheduling. Multiple scheduling servers can be deployed to achieve high availability and performance improvement (supporting unlimited horizontal expansion).
• Fault tolerance and recovery: after task execution fails, the task can be retried according to the configured retry strategy, as long as the executor cluster has enough computing nodes, the task can be completed smoothly.

Applicable scenarios

• Scenarios with scheduled execution requirements: such as full data synchronization every morning, generating business reports, etc.
• Scenarios that require all machines to execute together: such as using broadcast execution mode to clean up cluster logs.
• Scenarios that require distributed processing: for example, updating a large batch of data, which takes a very long time to execute on a single machine, can use Map/MapReduce processor to complete task distribution and mobilize the entire cluster to accelerate calculation.
• Scenarios that requireScenarios with delayed executionScenarios for certain tasks with delays: such as handling expired orders, etc.

Design Goal

The design goal of PowerJob is to be an enterprise-level distributed task scheduling platform, namely a company's internalTask Scheduling Middleware。The entire company uniformly deploys the scheduling center powerjob-server, and all business lines under it only need to depend on powerjob-worker to access the scheduling center to get task scheduling and distributed computing capabilities.

5. Tech Stack

Architecture Diagram (Module View)

Server Startup Process Analysis

server Function Panorama Diagram, thanks Community Friend @Dahai contribution

Worker Startup Process Analysis

worker Function Overview, thanks to friend @Dahai 's contribution

MapReduce Principle

Similar Product Comparison

QuartZ

xxl-job

SchedulerX 2.0

PowerJob

Scheduled Type

CRON

CRON

CRON, Fixed Frequency, Fixed Delay, OpenAPI

CRON, Fixed Frequency, Fixed Delay, OpenAPI

Task Type

Embedded Java

Embedded Java, GLUE Java, Shell, Python, etc.

Embedded Java, External Java (FatJar), Shell, Python, etc.

Embedded Java, External Java (Container), Shell, Python, etc.

Distributed Computing

No

Static Sharding

MapReduce Dynamic Sharding

MapReduce Dynamic Sharding

Online Task Governance

Not supported

Support

Support

Support

Log White Screen

Not supported

Support

Not supported

Support

Scheduling Method and Performance

Based on Database Lock, Performance Bottleneck

Based on Database Lock, Performance Bottleneck

Unknown

No Lock Design, High Performance No Limit

Alarm Monitoring

No

Email

SMS

WebHook, Email, DingTalk, and Custom Extension

System Dependency

JDBC Supported Relational Database (MySQL, Oracle, ...)

MySQL

RMB

Any Spring Data Jpa Supported Relational Database (MySQL, Oracle, ...)

DAG Workflow

Not supported

Not supported

Support

Support

Project Structure Explanation

This project consists of the main project (PowerJob) and the front-end project (PowerJob-Console). The PowerJob modules are as follows:

├── LICENSE
├── powerjob-client // powerjob-client, ordinary Jar package, provides OpenAPI
├── powerjob-common // common dependencies for all components, developers do not need to be aware of
├── powerjob-server // powerjob-server, scheduling server implemented based on SpringBoot
├── powerjob-worker // powerjob-worker, ordinary Jar package, applications that need to access powerjob-server need to depend on this Jar package
├── powerjob-worker-agent // powerjob-agent, executable Jar file, can directly access powerjob-server's proxy application
├── powerjob-worker-samples // tutorial project, contains various Java processor writing samples
├── powerjob-worker-spring-boot-starter // powerjob-worker's spring-boot-starter, spring boot applications can uniformly introduce this dependency to access powerjob-server
├── powerjob-official-processors // official processors, contains a series of commonly used Processor, depending on this jar package can be used
├── others
└── pom.xml