1. Project Overview

OpenMiniDojo Open-source Embodied Robot Platform

A lightweight open-source embodied intelligent full-stack robot platform project, including Robohome home training robot platform, OpenMiniDojo low-cost open-source robot platform, and RoboImitate skill imitation learning software package. The project provides a digital robot model that can collect demonstration data without physical samples, currently supporting keyboard and gesture control methods, and further based on Stanford's open-source ACT diffusion model to complete skill imitation and training. RoboGrasp provides a self-capture software package based on point cloud segmentation and neural network GPD.

2. Open Source License

Licensed under Apache-2.0

3. Screenshots

4. Feature Overview

A lightweight open-source embodied intelligent full-stack robot platform project, including Robohome home training robot platform, OpenMiniDojo low-cost open-source robot platform, and RoboImitate skill imitation learning software package. The project provides a digital robot model that can collect demonstration data without physical samples, currently supporting keyboard and gesture control methods, and further based on Stanford's open-source ACT diffusion model to complete skill imitation and training. RoboGrasp provides a self-capture software package based on point cloud segmentation and neural network GPD.

5. Tech Stack

Software package	Description	Dependency
Robohome	Embodied intelligent home training platform	Webots2021a，Ubuntu20.04，pinocchio
RoboImitate	Skill imitation learning software package optimized by ACT diffusion model	pytorch，cuda
RoboButler	Open-source low-cost embodied intelligent capture robot control software package	QT，RBDL
RoboGrasp	Based on GPD point cloud end-to-end grasp pose estimation algorithm	ROS，PCL

1. Robohome operation manual

The home embodied intelligent training platform is a virtual data training environment that can load RoboButler robots to collect imitation learning data and can be used for autonomous navigation, capture control, and algorithm development. The digital platform uses shared memory to communicate with ROS, achieving hardware and software decoupling and isolation. Currently, Webots2021a is used as the main physical simulation engine.
Webots2021a
ROS noetic
Pinocchio
RBLD
The Robohome project source code is located in the Robohome folder of this project, including Webots necessary controllers, models, and worlds:

1.1 Installation and compilation

(1) Complete the installation and installation of Webots2021a and ROS noetic on Ubuntu20.04 systems.
(2) Copy the pinocchio dynamics library in Third party to the local directory, complete cmake .. and make, make install for installation.
(3) Copy the Robohome software package to the local directory, extract the world folder materials and textures, and create a build directory in the controllers\tinyscretch folder.
(4) Open kin_math_pino.cpp in the math_src file, modify the called URDF path, and perform cmake .. and make in the build directory.
(5) Copy the tinystech executable file generated in the build directory to \controllers\tinyscretch and replace the original file. (You can also install QT creator and load cmakelist for compilation).
(6) Open Webots and load the apartment_simple_big1 file. If there are no errors in the console, the robot controller is loaded normally.
(7) Click the Webots simulation window and use the following keyboard to operate.
The number keys 1~5 can make the robot enter autonomous mode and realize autonomous movement to the WAY* route point position in the simulation:
handcontrol2.py is for gesture control, and you need to install Python and the Google open-source recognition framework by yourself. When starting, perform gesture control in front of the adopted camera at a distance of 30~70cm.