1. Project Overview

Frostmourne is the open-source version of the monitoring system developed by the Technical Department of Autohome's Dealer Division. It is designed to help monitor data from almost all databases and data sources, including Elasticsearch, Prometheus, SkyWalking, MySql, and others. If you have already established a logging system and metrics infrastructure but are struggling with the lack of a supporting monitoring system, Frostmourne might be able to help. When used properly, this system can ensure that whenever an issue occurs in a production application, the relevant developers are immediately informed of the error and know where the problem lies, allowing them to respond and resolve it quickly. You can stay informed about production issues ahead of almost everyone else and handle various inquiries with confidence because you know exactly what is happening. Your team deserves it.

2. Open Source License

Licensed under MIT

3. Screenshots

Feature Screenshots

• Alert Messages

• 
• Data Source Management

• 
• 
  
• Data Name Management

For Elasticsearch data sources, the data name is equivalent to the concept of an index.

• Elasticsearch Data Query

After configuring the data source and data name, you can use the query page to verify whether the data configuration is correct.

• Add or Edit Monitoring

• 
• 
  
• HTTP Data Monitoring

Taking Elasticsearch cluster health monitoring as an example.

The detection conditions are: the cluster status field status is not green, or the number of cluster nodes is not equal to 11.

• Monitoring List

After a monitoring configuration is saved successfully, it will appear in the monitoring list.

4. Feature Overview

Project Background

After establishing our logging system with ELK, we discovered that there were no other solutions for application log monitoring besides ElastAlert. Initially, we used ElastAlert to address our log monitoring needs. However, as the number of configurations increased, both management and usage costs became high, and its stability did not meet our expectations. Therefore, to improve usability and stability, we decided to develop our own simple monitoring system to solve the problem of log monitoring.

The project is not limited to Elasticsearch data. It also supports HTTP, PING, and Telnet monitoring, as well as data monitoring for Prometheus, SkyWalking, InfluxDB, MySQL, ClickHouse, IoTDB, and SqlServer. More commonly used data sources will be added in the future, such as Loki, MongoDB, Redis, and Oracle. There is still a lot of work to be done, and we need more developers to join us. Feel free to contact us and help make this project bigger and stronger together.

✨ Key Features

• Easily set up monitoring with just a single data query.
• Multiple data sources supported: Elasticsearch, HTTP, SkyWalking, Prometheus, InfluxDB, MySQL/TiDb, ClickHouse, SqlServer, PING, IotDB, Telnet
• Numerical calculation monitoring: count, min, max, avg, sum, unique count, percentiles, standard deviation; Elasticsearch data supports bucketing.
• Multiple alert notification channels: DingTalk (bot), WeCom (bot), Feishu bot, Email, SMS, HTTP
• Multiple message formats: text, markdown
• Flexible customization of alert message Freemarker templates with variable placeholders; message template management.
• Distributed scheduling implementation, with each monitoring task scheduled independently without affecting others.
• Alert messages include short links to log queries, taking you directly to the cause of the alert.
• Year-over-year and period-over-period numerical monitoring
• HTTP data monitoring, JavaScript expression-based alert evaluation; PING connectivity monitoring, Telnet port connectivity monitoring
• Simple and user-friendly frontend: monitoring management, testing, saving as, execution logs, and historical messages
• Elasticsearch data querying, sharing, and downloading
• Alert message suppression to prevent message flooding; also supports alert escalation to ensure that relevant parties are not left uninformed about incidents for extended periods.
• Built-in account, team, and department information management modules, with support for implementing custom internal integrations
• Integrated LDAP login authentication
• Permission control and data isolation, ensuring that teams do not interfere with one another

5. Tech Stack

Quick Startup

Provides a docker-compose deployment option to help you get started faster and better understand the project's purpose. For details, see the documentation: Quick-Start

Deployment

Prerequisites: MySQL database table creation

The SQL statements for creating all Frostmourne tables are located in the frostmourne.sql file.

Database passwords are stored in plain text by default, without an encryption strategy. If you need to encrypt passwords, please refer to the official Druid documentation: Druid Database Password Encryption

1. k8s Deployment

For k8s deployment, refer to the following three configuration files.

• frostmourne-monitor-namespace.yaml
• frostmourne-monitor-deployment.yaml
• frostmourne-monitor-service.yaml

The relevant parameters are configured in

the frostmourne-monitor-deployment.yaml file. Note that the externally mapped port must be specified in frostmourne-monitor-service.yaml. The default nodePort is 30054.

kubectl applt -f frostmourne-monitor-namespace.yaml
kubectl applt -f frostmourne-monitor-deployment.yaml
kubectl apply -f frostmourne-monitor-service.yaml

2. ZIP Package Deployment

Dependency Environment

• JDK 1.8
• MySQL 5.7.8+

For the latest Release ZIP package, download it from releases. After extracting it, modify the application configuration file application.properties and the environment variable configuration file env according to your environment, then execute the following command to start the application:

./scripts/startup.sh

Execute the following command to stop the application:

./scripts/shutdown.sh

3. Self-Built Deployment

Dependency Environment

• JDK 1.8
• Maven 3.2.x+
• MySQL 5.7.8+

Run the Maven build command in the project's main frostmourne directory:

mvn -U clean package -DskipTests=true

The frontend project frostmourne-vue automatically builds its resources into frostmourne-monitor/resources/dist, so you only need to deploy frostmourne-monitor.

frostmourne-monitor is already configured with assembly packaging. A ZIP package will be generated in the target directory. You only need to extract the ZIP package, modify the application configuration file application.properties and the environment variable configuration file env according to your environment, then execute the following command to start the application:

./scripts/startup.sh

Execute the following command to stop the application:

./scripts/shutdown.sh

Feature Usage Guide

Detailed content, please refer to README.md document.

Elasticsearch Data Monitoring Guide

HTTP Monitoring Instructions

Prometheus Data Monitoring Guide

SkyWalking Data Monitoring Guide

MySQL/TiDB Data Monitoring Guide

ClickHouse Data Monitoring Guide

InfluxDB Data Monitoring Guide

PING Monitoring Guide

Year-over-Year Numerical Monitoring Guide

Message Template Configuration

Alert Delivery

Alert Suppression

User Management and Login Authentication

Note

Other Features

Period-over-Period Numerical Monitoring Guide

IoTDB Data Monitoring Guide

Telnet Port Connectivity Monitoring Guide

Development and Debugging

Debugging Environment Requirements

• JDK 8 ~ 14
• Node 16.14.2 (Recommended)
• Yarn 1.22.10 (Recommended) or Npm 8.7.0
• MySQL 5.7.8+
• Elasticsearch 6.3.2+

Start the frostmourne-monitor project and add the following startup parameter:

-Dmysql.host=localhost -Dmysql.user=root -Dmysql.password=example -Dlog.console.level=INFO

Modify the MySQL-related parameters to match your environment and set the active profile to local. Test URL: [http://localhost:10054](http://localhost:10054) Open the frostmourne-vue directory using VS Code for frontend debugging. Execute the following commands:

# install dependency
yarn install

# It is recommended not to use cnpm directly to install dependencies, as it may cause various unexpected bugs. You can resolve slow npm download speeds using the following approach
yarn install --registry=[https://registry.npm.taobao.org

#](https://registry.npm.taobao.org

#) develop
yarn dev

Will open automatically: [http://localhost:9528](http://localhost:9528)

If you encounter any difficulties setting up a local development and debugging environment or performing secondary development, feel free to join the discussion group. Developers from all backgrounds are welcome to contribute PRs.

Technical Notes: The project is implemented using Java 8. For details, see: Technical Notes

⚙ Future Plans

• Release 1.0-RELEASE
• Add application log collection from this project to MySQL and provide a query page to facilitate troubleshooting and monitoring
• Add Kibana link configuration to Elasticsearch data names, and add a link to the Kibana address on the data query page, making it easy to switch data queries to Kibana.
• Implement SMS alert notifications, using Alibaba Cloud SMS by default.
• Add Alibaba Cloud log data monitoring.
• Add Tencent Cloud log data monitoring.
• Add Huawei Cloud log data monitoring.
• Complete English internationalization.
• Add Loki data monitoring and alerts.
• Add an online configuration page for email settings.
• Add an online configuration page for WeCom settings.
• Display short-link IDs in hexadecimal format to avoid excessively long IDs when the numeric values are large.
• Optimize pom.
• When transferring an employee to another team, add an option to determine whether to migrate their monitoring configurations to the new team. If selected, the monitoring configurations created by that employee will also be transferred to the new team.
• Add support for alert groups.
• Add traceid field configuration to Elasticsearch data names, allowing redirect links to be configured. For example, configuring a SkyWalking link will redirect to the corresponding call chain in SkyWalking.
• Add historical data comparison rules for time-series data.
• Add a setting for the permitted alert message delivery time period. Messages generated outside the permitted period will only be recorded and not sent. Their sending status will be FORBID.
• Add a default implementation for sending messages through enterprise DingTalk (I don't have a local environment for testing. Help is welcome through PRs or sample code. Thanks in advance!).
• Additional alert notification methods (PRs are welcome).
• Add support for SkyWalking Database Layer data monitoring and alerts.
• Add a toggle option to the monitoring list to display only my monitoring configurations.
• Automatically suggest index names when configuring Elasticsearch data names.
• Introduce an object iterator, which represents a list of data obtained from a data source or custom input. It is used to dynamically generate monitoring query statements, for example: Level: ERROR AND Project: ${ITEM_VALUE}, enabling monitoring through data iteration.
• Release 1.0-RELEASE
• 3-sigma outlier detection alert rule.
• Add time-series anomaly detection algorithm rules (feasibility testing required; time-series-based anomaly detection).
• Summarize the technical concepts and knowledge used in the project.