1. Project Overview

Zhiqingyun

Ultra-Lightweight Spark Computing Platform

Zhiqingyun is an ultra-lightweight, enterprise-grade big data computing product and a data computing platform built around the Spark ecosystem. One-click deployment and ready to use out of the box. Quickly implement Spark computing, offline big data ETL, real-time computing, scheduled job orchestration, and other functions.

2. Open Source License

Licensed under Apache-2.0

3. Screenshots

Can You Write Code? It's Not Just SQL

Supports a wide variety of job types, including SparkSql, SparkJar, data synchronization, JdbcSql, Prql, Excel imports, Python scripts, Bash scripts, Curl scripts, API calls, and more.

Drag-and-Drop Job Scheduling in Every Direction

Supports the complete set of job lifecycle operations, including run, stop, interrupt, take offline, publish, rerun, rerun downstream jobs, rerun the current job, and external invocation.

Real-Time Computing for Greater Data Efficiency

Supports real-time computing and real-time database synchronization. Uses Kafka and Debezium to efficiently process data streams, ensuring rapid data transmission and updates.

Multi-Platform Deployment, Built for Flexibility

Supports various resource scheduling platforms, including Kubernetes, Yarn, CDH, Apache Hadoop, Spark Standalone, Spark Cluster, and more.

Data Visualization Dashboard for Big Data

Powerful data processing and visualization technologies transform complex data into intuitive charts. Supports real-time monitoring and analysis, helps enterprises make informed decisions, and flexibly adapts to industry requirements.

Source: Zhiqingyun Official Documentation

4. Feature Overview

Choose Zhiqingyun

Zhiqingyun is an ultra-lightweight, enterprise-grade big data computing platform. One-click deployment and ready to use out of the box. Quickly implement big data ETL, real-time computing, visual scheduling, custom APIs, data dashboards, shared forms, and other use cases. Helps enterprises process massive amounts of data and unlock greater business value.

Product Features

▪ Private Deployment

One-click Docker deployment without installing additional big data components, allowing you to quickly gain big data computing capabilities.

▪ Native Spark Integration

Fully integrates Spark into the project with built-in Spark features, making better use of Spark's capabilities.

▪ Rich Job Types

Supports more than 10 job types, including SparkSql, sparkJar, Bash, Python, data synchronization, and real-time computing.

▪ Visual Scheduling

Supports individual job scheduling, workflow-wide scheduling, cron expression scheduling, and more.

▪ Multi-Tenant Support

Supports multi-tenant architecture with data isolation between tenants.

▪ Excellent Ecosystem

Zhiqingyun will work together with the Zhiyao product series to build an enterprise-grade, open-source all-in-one solution.

5. Tech Stack

Build Environment

▪ java-1.8
▪ nodejs-18.14.2

Source code download

Visit Yifei Open Source: https://code.exmay.com/ Download

Dependency Download

cd spark-yun
./gradlew install

Code Compilation

cd spark-yun
./gradlew clean package

Package Output Path

spark-yun/spark-yun-dist/build/distributions/zhiqingyun.tar.gz

Extract and Deploy

tar -vzxf zhiqingyun.tar.gz
bash zhiqingyun/bin/start.sh

Access the Project

▪ Access URL: [http://localhost:8080](http://localhost:8080)
▪ Administrator Account: admin
▪ Administrator Password: admin123