1. Project Overview

A log center implemented in Go, called glogcenter (GLC for short), designed to replace the traditional ELK stack.

Background

Log centers have traditionally relied on the classic three-component stackELK. Despite our gratitude, the overall experience was still unsatisfactory (customized installation was not straightforward, page initialization was too slow, the interface was unfamiliar, and most importantly, resource consumption was excessive, with occasional crashes). A suitable alternative was nowhere to be found. Finally, I decided to usegoto write a prototypelogcenter. The various outstanding performance results genuinely amazed me, so I named itglogcenter, abbreviated asGLC, and open-sourced the project.

Goal: Gradually replace the existing onlineELK。

2. Open Source License

Licensed under LGPL-3.0 open source protocol

3. Screenshots

4. Feature Overview

Features

• Usagegolangimplementation, providinggowith all its features and advantages, while focusing on low resource consumption and high performance.
• Usinggoleveldbfor data storage, with some optimizations based on the write-heavy, read-light characteristics of logs, the result is remarkably fast.
• Full-text keyword search with Chinese word segmentation (usingjiebagoTokenization is performed with millisecond-level response times, delivering a natural and smooth experience
• Handles nearly 10,000 log entries per second, with idle-time indexing speeds of several thousand entries per second, meeting the needs of most projects
• Supports customizable environment variable switches and automated log repository maintenance, offering flexibility and peace of mind
• ProvidesdockerProvides images with containerized deployment support for maximum convenience
• ProvidesjavaProject log collection package,javaEnd-to-end project support
• ProvidesgolangProject log collection package,golangEnd-to-end project support
• Supports importing fromRabbitMQCollects log information to meet more end-to-end workflow requirements
• Comes with a built-in, lightweightVUELog query and management interface implementation
• Supports multi-service cluster deployment to ensure service and data redundancy

5. Tech Stack

Simple example of standalone Docker deployment

# Simple Example
docker run -d -p 8080:8080 gotoeasy/glc

# Mount External Data Directory
docker run -d -p 8080:8080 -v /glc:/glogcenter gotoeasy/glc

Simple Example of Docker Cluster Deployment

# Start the following 3 instances in cluster mode. Simply configure the address of this node and the addresses of the associated nodes
# Uses an "optimistic cluster" approach: simplified leader election (basic sorting) + log broadcasting (failures ignored) + data compensation (historical data from previous days)

# Service 1
docker run -d -p 8080:8080 -e GLC_CLUSTER_MODE=true -e GLC_SERVER_URL=[http://172.27.59.51:8080](http://172.27.59.51:8080) \\
       -e GLC_CLUSTER_URLS=http\://172.27.59.51:8080;http\://172.27.59.52:8080;http\://172.27.59.53:8080 \\
       gotoeasy/glc
# Service 2
docker run -d -p 8080:8080 -e GLC_CLUSTER_MODE=true -e GLC_SERVER_URL=[http://172.27.59.52:8080](http://172.27.59.52:8080) \\
       -e GLC_CLUSTER_URLS=http\://172.27.59.51:8080;http\://172.27.59.52:8080;http\://172.27.59.53:8080 \\
       gotoeasy/glc
# Service 3
docker run -d -p 8080:8080 -e GLC_CLUSTER_MODE=true -e GLC_SERVER_URL=[http://172.27.59.53:8080](http://172.27.59.53:8080) \\
       -e GLC_CLUSTER_URLS=http\://172.27.59.51:8080;http\://172.27.59.52:8080;http\://172.27.59.53:8080 \\
       gotoeasy/glc

Docker Startup Environment Variables

• GLC_STORE_NAME_AUTO_ADD_DATEWhether the log repository automatically stores logs by day. Default:true
• GLC_SAVE_DAYSNumber of retention days when automatic daily log repository maintenance is enabled (0~180)，0Disables automatic deletion. Default:180days
• GLC_ENABLE_LOGINWhether to enable user password authentication. Default:false
• GLC_USERNAMEUsername for logging into the query interface. Default:glc
• GLC_PASSWORDPassword for logging into the query interface. Default:GLogCenter100%666
• GLC_ENABLE_SECURITY_KEYWhether to enable API key verification for the log submission API. Default:false
• GLC_HEADER_SECURITY_KEYAPI keyheaderKey name. Default:X-GLC-AUTH
• GLC_SECURITY_KEYAPI key. Default:glogcenter
• GLC_ENABLE_AMQP_CONSUMEWhether to enablerabbitMqConsumer for receiving logs. Default:false
• GLC_AMQP_ADDRMessage queuerabbitMqConnection address, e.g.:amqp://user:password@ip:port/Empty by default
• GLC_AMQP_JSON_FORMATMessage queuerabbitMqWhether message text is injsonformat. Default:true
• GLC_CLUSTER_MODEWhether to start in cluster mode. Default:false
• GLC_SERVER_URLService address of the current node in cluster mode. Empty by default
• GLC_CLUSTER_URLSService addresses of associated nodes in cluster mode. When multiple addresses are specified, separate them with;. Empty by default
• GLC_LOG_LEVELLog level. Allowed values:debug/info/warn/error. Default:info
• GLC_GOMAXPROCSUses the maximum number of CPUs. If the specified value is outside the actual range, the maximum value is used. Defaults to the maximum value, usually non-dockerUses the mode to start

Command-line startup parameters (supported in version 0.6.* and later)

• Supports command-line parameters-vDisplay version
• Supports command-line parameters on Linux systems-dStart in the background
• Supports command-line parameters on Linux systemsstopStop the program
• Supports command-line parameters on Linux systemsrestartRestart the program
• The server port can be configured through the environment variableGLC_SERVER_PORTSet to the default:8080
• Other environment variables are the same as abovedockerStartup Environment Variables

API

• /glc/v1/log/addAdd logs,POST，application/json
• Fieldssystem: String, corresponding to the page'sCategory
• Fieldsdate: String, corresponding to the page'sDate and time
• Fieldstext: String, corresponding to the page'sLog content

For Java projects using Logback, supports log collection. Ensure that the major and minor versions match the GLC version

<!-- POM coordinates -->
<dependency><groupId>top.gotoeasy</groupId><artifactId>glc-logback-appender</artifactId><version>0.8.0</version>
</dependency>
<!-- Logback configuration example 1: Send to glogcenter -->
<appender name=\"GLC\" class=\"top.gotoeasy.framework.glc.logback.appender.GlcHttpJsonAppender\"><glcApiUrl>[http://127.0.0.1:8080/glc/v1/log/add</glcApiUrl>](http://127.0.0.1:8080/glc/v1/log/add\</glcApiUrl\>); <!-- Can be configured via the GLC_API_URL environment variable --><glcApiKey>X-GLC-AUTH:glogcenter</glcApiKey>                <!-- Can be configured via the GLC_API_KEY environment variable --><system>Demo</system>                                       <!-- Can be configured via the GLC_SYSTEM environment variable --><layout><pattern><![CDATA[%p %m %n]]></pattern></layout>
</appender>

<!-- Logback configuration example 2: Send to RabbitMQ -->
<appender name=\"GLC\" class=\"top.gotoeasy.framework.glc.logback.appender.GlcAmqpAppender\"><amqpHost>127.0.0.1</amqpHost>                <!-- Can be configured via the GLC_AMQP_HOST environment variable --><amqpPort>5672</amqpPort>                     <!-- Can be configured via the GLC_AMQP_PORT environment variable --><amqpUser>rabbitmqUsername</amqpUser>         <!-- Can be configured via the GLC_AMQP_USER environment variable --><amqpPassword>rabbitmqPassword</amqpPassword> <!-- Can be configured via the GLC_AMQP_PASSWORD environment variable --><system>Demo</system>                         <!-- Can be configured via the GLC_SYSTEM environment variable --><layout><pattern><![CDATA[%p %m %n]]></pattern></layout>
</appender>

For projects written in Go, a ready-to-use toolkit is provided.

# Import the toolkit
go get github.com/gotoeasy/glang

# Configure environment variables as needed
export GLC_API_URL='[http://127.0.0.1:8080/glc/v1/log/add](http://127.0.0.1:8080/glc/v1/log/add)'
export GLC_API_KEY='X-GLC-AUTH:glogcenter'
export GLC_SYSTEM=demo
export GLC_ENABLE=true
export GLC_LOG_LEVEL=debug # Log levels (trace/debug/info/warn/error/fatal)
// Method 1: Use cmn.Debug(...), cmn.Info(...), etc. to print logs and send them to the log center simultaneously
// Method 2: Create a client object using cmn.NewGLogCenterClient() and then use it
//        For more details, see the documentation [https://pkg.go.dev/github.com/gotoeasy/glang

import](https://pkg.go.dev/github.com/gotoeasy/glang

import) \"github.com/gotoeasy/glang/cmn\"

func main() {
	cmn.Info(\"Starting WEB service\")
	err := cmn.NewFasthttpServer().Start()
	if err != nil {
		cmn.Fatalln(\"Startup failed\", err)
	}
}