1. Project Overview

Moonbox: Moonlight Treasure Box

Moonbox (Moonlight Treasure Box) is a traffic replay platform product developed under the JVM-Sandbox ecosystem, rebuilt based on jvm-sandbox-repeater. It provides a richer set of features than jvm-sandbox-repeater while making online deployment and usage more convenient.

2. Open Source License

Licensed under Apache-2.0

3. Screenshots

4. Feature Overview

Usage scenario

Have you encountered any of the following problems?

A user request in the production environment keeps failing. I want to debug it in the test environment. Can you help me reproduce the issue? I don't know how to generate load-testing traffic. The data structures are too complex, and the load-testing model is difficult to evaluate. Is there a good solution? I don't want to write API testing scripts anymore. I want to build a traffic recording system that can perform business regression testing based on real production user scenarios. It may need to integrate with many service systems, and I don't want to modify every individual system. Are there any good framework recommendations? I want to build a business monitoring system that samples core production APIs and performs business validations to monitor business correctness in real time.

Moonbox (Moonlight Treasure Box) is anon-intrusiveproductiontraffic recording andtraffic replayplatform. It follows the SPI design of jvm-sandbox-repeater and provides numerous commonly used plugins, along with data statistics and storage capabilities. Moonbox enables automated testing, production issue tracking, business monitoring, and other capabilities.

Terminology

• Recording: The process of serializing and storing the input parameters, output parameters,downstream RPC, DB, and cache callsof a request.
• Replay: The process of restoring recorded data and reinitiating a request once or N times, while MOCKing specific downstream nodes.
• Entry Call: An entry call is generally the traffic source of an application,Traffic Source, such as HTTP/Dubbo. During the call, the input parameters and return values are recorded. During replay, they serve as the basis for initiating traffic and comparing execution results.
• Sub-call: Unlike an entry call, a sub-call is a method invocation that occurs during the execution of another call. During recording, the sub-call's input parameters and return values are captured. During replay, the recorded return value is used for MOCK.
• MOCK: During replay, intercepted sub-callsdo not execute real calls, using dynamic bytecode instrumentation to directly return the value recorded during recording.
• Noise Reduction: During replay, some sub-call input parameters or response fields may differ from the original traffic. Noise reduction excludes these non-essential fields from the comparison process.

Function introduction

• Traffic Recording: Traffic recording template management, recording task dispatch, and recorded traffic viewing.
• Traffic Replay: Traffic replay task management, replay data viewing, and success rate statistics.
• Replay MOCK: Special MOCK configurations for traffic recording and replay (used as sub-call MOCKs).
• Comparison Configuration: Configuration for ignoring specific fields during traffic replay comparison.

Key Differences Between MoonBox and Jvm-Sandbox-Repeater

MoonBox (Moonlight Treasure Box) is a traffic replay platform built on the JVM-Sandbox ecosystem and developed based on jvm-sandbox-repeater. The differences between MoonBox and jvm-sandbox-repeater are as follows:

Comparison

Function Point

jvm-sandbox-repeater

MoonBox (Moonlight Treasure Box)

Deployment Method

Frontend and backend are not separated; supports local deployment

Frontend and backend are separated; supports both local and remote deployment

Number of Plugins

Relatively few

Numerous, covering most common middleware

Agent Stability

Some bugs remain to be fixed

Proven stable through extensive use in vivo's internal production systems

Data Storage

Uses local storage and MySQL

Uses Elasticsearch (ES) for storage and supports large-scale data storage

Platform Usability

Limited functionality

Manages recording and replay data by task and interface, provides various configuration options, and offers good usability

Feature Completeness

More features are yet to be supported

Supports various capabilities, including agent file management, task management, Mock management, and replay comparison configuration

5. Tech Stack

MoonBox Platform Deployment

Resource Preparation

Resource Type

Resource Version

Description

jdk

1.8

JDK Version Used by the Project

mysql

5.x

Reference SQL Statements for Creating Storage Configuration Tables

elasticsearch

7.x (7.16.2 recommended)

Stores JSON data for recording and replay (refer to the ES installation documentation)

Virtual Machine/Container

No

Deploy moonbox-server and the VUE frontend resources on two separate machines (the server and Vue frontend can also be deployed on the same machine).

Technical principle

System architecture diagram

The Moonbox platform consists of two components: moonbox-agent and moonbox-server.

moonbox-agent

• Uses java-attach technology (actual dynamic bytecode enhancement is implemented by JVM-Sandbox) to dynamically attach to the target process, providing traffic recording and replay enhancements.

moonbox-server

• The agent exposes APIs that provide services such as configuration queries, recorded traffic storage, traffic queries, and replay result storage.
• Configuration of recording/replay tasks, remote management of agent tasks, and the administration interface (frontend and backend deployed separately).

System Flowchart

traffic recording

Traffic recording is accessed via [Admin Console - Traffic Recording]. This diagram illustrates the general workflow. The left side shows execution by the Moonbox backend, while the right side shows execution on the target JVM machine.

traffic replay

There are two paths for traffic replay: [Admin Console - Traffic Recording - Recording Tasks - Traffic Replay] and [Traffic Replay - Failed Replay/Replay Again].

The download process for sandbox-agent and moonbo-agent is the same as the previous section traffic recording section.

Heartbeat Request

When traffic recording and replay tasks start, heartbeat threads are started simultaneously to manage the agents loaded on the target JVM. When recording and replay tasks finish (or enter an abnormal state), the agent's heartbeat thread calls the sandbox-jetty container API to unload the Sandbox and Moonbox agents. This process does not affect the normal operation of the target JVM.

Agent Loading

This diagram describes the startup process of the Sandbox agent and Moonbox agent, including attaching to the target Java process, starting the Jetty service, and loading the traffic replay plugin.

Core Principles

Recording and Replay Principles

The recording and replay workflow follows the design of jvm-sandbox-repeater, with some non-core features omitted.

traffic recording The core recording logic uses the BEFORE, RETRUN, and THROW event mechanisms of JVM-Sandbox to control the recording process. See DefaultEventListener for details.

traffic replay Traffic replay is dispatched through FlowDispather. Each replay plugin type implements RepeaterSPI to initiate replay requests. Each replay request can determine whether local replay should use mocking. Plugins can also freely implement their own mocking logic. See the mock workflow code.

Mock Replay: Sub-calls within replay traffic (e.g., MyBatis/Dubbo) are not actually executed. Instead, matching sub-calls are searched from recorded sub-calls according to MockStrategy. JVM-Sandbox's execution interception capabilities are used to return the matched result immediately via throwReturnImmediately. If no matching result is found, an exception is thrown to interrupt the process and prevent duplicate calls from polluting the data.

Illustration