Getting Started:

Advantages of Dataset

The RS-ManiRW dataset, with its top-tier hardware platform and software technology stack, provides embodied intelligence data products that far exceed industry standards. Its core advantages are reflected in the following six aspects:

1. Multi-modal Sensor Spatio-temporal Hard Synchronization Technology

• Advantage: Provides strictly aligned multi-view vision for the model, fundamentally ensuring the consistency of cross-modal data.
• Technical Details: The data streams of all visual sensors (such as RGB cameras on the head, left arm, and right arm) are accurately time-hard synchronized and timestamped through hardware-level methods. This means that each frame of image captured by different cameras has a unified and precise timestamp, providing a solid data foundation for training models that require multi-view fusion.

2. Ultra-low Frame Loss Rate and High-fidelity Data Collection

• Advantage: Ensures the integrity and continuity of data streams, avoiding the model learning wrong or broken action sequences due to data loss.
• Technical Details: The data collection system follows the standard of an ultra-low frame loss rate of less than 0.5%. Through an optimized data transmission, caching, and processing pipeline, it can still maximize the integrity of each frame of sensor data in scenarios of high frame rate and multi-channel data parallel collection, thus recording high-quality and high-fidelity task demonstration data.

3. High-precision Robotic Arm Motion Control and State Feedback

• Advantage: Provides accurate, smooth, and real robotic motion trajectory data, which is an ideal data source for algorithms such as imitation learning and inverse reinforcement learning.
• Technical Details: The robot's body state data is sampled at a high frequency, accurately recording joint angles, velocities, accelerations, end-effector poses, and torques. At the same time, the system supports real-time high-precision joint velocity control, making the collected actions both smooth and accurate, perfectly reproducing the operating skills of human experts.

4. Factory-level High-precision Calibration and Completeness of Calibration Parameters

• Advantage: Provides ready-to-use accurate internal and external sensor parameters. Users do not need additional calibration and can directly use them for image correction, hand-eye calibration, and 3D reconstruction, which greatly reduces the threshold for data use.
• Technical Details: Each data collection robot and sensor undergoes strict factory calibration. The dataset not only provides original image data but also synchronously provides complete camera calibration parameters (internal and external parameters).

5. Deep Generalization Data Collection for the Same Task

• Advantage: For a single task instruction (such as "pick up a cup"), a large number of demonstration data executed under diverse variables are collected, which greatly enhances the model's robustness to object appearance, environmental interference, and action execution, and effectively prevents overfitting.
• Technical Details: This is the core of the data strategy. When performing each basic task, we will systematically introduce multiple dimensions of variation:
  • Object Attribute Generalization: The same task is performed using objects of different shapes, sizes, materials, colors, and weights (for example, using goblets, mugs, disposable paper cups, and metal cups to perform "pick up a cup").
  • Environmental Context Generalization: Change the initial position and orientation of objects in the same scene, introduce different background interferences and lighting conditions, and simulate the uncontrollable factors of the real world.
  • Action Trajectory Generalization: The same task is completed by different operators or in different ways, and diverse but equally successful action paths and grasping postures are recorded.
  • View Generalization: Hard-synchronized cameras capture the first-person view (head), third-person view (over-the-shoulder), and hand-eye view of the same task at the same time, providing the model with an innate multi-view understanding ability.
    This collection concept of "one task, thousands of changes" ensures that the trained model can understand the essence of the task instead of rote memorizing specific visual patterns or motion trajectories, thereby obtaining strong generalization ability.

6. Advantage of Replicating Real Human Decision Trajectories Based on Exoskeleton Teleoperation

• Advantage: The exoskeleton-type high-precision teleoperation equipment is used for data collection, which can highly restore the whole-body motion intention and decision-making process of the human operator (both arms) at a 1:1 ratio. Different from abstract input devices such as data gloves and VR controllers, the exoskeleton directly captures the real biomechanical movements of human joints such as shoulders, elbows, and wrists. The recorded data is the most natural and smooth expert-level operation trajectory optimized in real-time by the human brain, avoiding the distortion and unnaturalness caused by instruction conversion.
• Technical Details:
  • Real Biological Motion Mapping: The exoskeleton device completely records the joint kinematics and dynamics data of the operator, realizing lossless transmission from human intelligence to machine execution.
  • Dual-arm Coordination Strategy: Directly capture the natural dual-arm coordination and cooperation strategy of humans when completing complex tasks, providing extremely valuable cooperative operation data for the model.
  • Essential Intention Learning: The data collected thereby enables the model to learn the essential "intention" and advanced strategies of the task, rather than just imitating rigid mechanical actions, thereby obtaining human-like adaptability and robustness.