VR Teleoperation

Usage Notes and Disclaimer

The performance and functionality of the teleoperation features released on this site are subject to the technical specifications and functional descriptions officially published by our company. During actual use, the company assumes no liability for any faults, anomalies, or losses (whether direct or indirect) caused by the following circumstances:

1. Failure to correctly deploy the runtime environment, network configuration, or use the equipment in accordance with the operation manual or tutorials;
2. Unauthorized modification of code, integration of incompatible components, or use of the teleoperation features in a manner beyond their designed scope;
3. Factors beyond the company's control, such as network interruptions, power failures, or loss of control of third-party hardware;
4. Latency, jitter, packet loss, and similar phenomena that cannot yet be fully avoided due to the technical characteristics of the industry.

By downloading the teleoperation SDK and using this feature, you are deemed to have read and agreed to the above terms.

1. Preparation Before Startup

1.1 Hardware Preparation

Item	Quantity	Notes
Meta Quest 3 VR device	1	Includes 1 headset, 2 handheld controllers, and a Type-C cable.
R1 Pro Base	1	Robot body
R1 Pro remote controller	1	Controls the R1 Pro robot
R1 Pro host computer (dual-boot, not a virtual machine)	1	Laptop OS: Ubuntu 22.04
Local area network	1	Used to connect the VR device and the R1 Pro Base

1.2 Software Preparation

Check the R1 Pro Software Version Changelog to obtain the latest SDK version. The full-robot ATC SDK version includes all software resources for the product.

At the same time, download the VR device configuration SDK:

• Baidu Cloud: https://pan.baidu.com/s/1SCpvwFIjvAV7EtchW-cssA?pwd=vrvr

• Meta-related installation package .zip: used to activate new devices.

• platform-tools-latest-windows.zip: the adb files, used to install the data collection APP inside the VR headset.

• vr_teleop-0.1.7-20250818.apk: the data collection APP inside the VR headset.

2. VR Device Configuration

If your VR device is provided by Galaxea, you can skip Sections 2.1 through 2.5 and go directly to Section 2.6.

2.1 Activating Developer Mode on the VR Device

Please refer to the Meta Quest 3 Developer Mode Instructions to complete the activation.

2.2 Installing the VR Device SDK (requires a Windows computer)

1. Extract the ADB files: download and extract platform-tools-latest-windows.zip.
2. Connect the VR device: connect the VR device to the computer using a Type-C cable.
3. If an older version of the apk is already installed, remove it first.
4. Authorize the USB connection: on the VR device, confirm and allow the USB connection (as shown).
5. Enter the ADB extraction path: open File Explorer and navigate to the folder where the ADB tools were extracted.
6. Copy the APK file: copy vr_teleop-0.1.7-20250818.apk into that folder.

7. Install the APK: open a Command Prompt (CMD) in that folder and run the following command to install the app:

   .\adb.exe install vr_teleop-0.1.7-20250818.apk
   

   If the command displays Success after execution, the installation was successful.

2.3 VR Device Configuration

On the initial screen of the Meta Quest 3, connect to the same WiFi network as the R1 Pro.

Note: A "network restricted" prompt is normal, because this network cannot access the internet.

2.4 Disabling the Quest Boundary System

1. Open Settings > Advanced > Developer.
2. Turn off the Physical Space feature.

Notes:

• After disabling it, the system no longer prompts you to set a boundary, and the collection process will not be interrupted even if you move or leave your original position.
• Passthrough view cannot be enabled on the main menu page.
• After disabling the boundary, be sure to keep the operating environment safe to avoid collision risks.

2.5 Preventing System Sleep

1. Open Menu → Settings
2. Click General > Power > Display Off
3. Set the screen-off time to 4 hours

⚠️ Note: Disabling system sleep is intended to enable the "removable wearing" approach. After use, manually turn off the Meta Quest 3S to avoid draining the battery. Try to keep the VR headset and controllers sufficiently charged; a low battery affects the frame rate at which the VR controllers send pose data, causing teleoperation lag.

2.6 Getting the VR Device's IP Address

On the VR device, click the connected WiFi, open the network page, and scroll down to find and record the IP address, e.g. 192.168.5.24. In the following text this is referred to as ${VR_IP}.

2.7 How to Wear the Device During Data Collection

To avoid discomfort from wearing the headset for long periods, we provide a "removable wearing" approach. Before use, first complete the "Preventing System Sleep" and "Disabling the Boundary System" settings.

First, detach the top strap from its connection to the top of the headset, and remove the facial interface, to make it easier to hang the headset around your neck and adjust it flexibly. Then hang the headset around your neck and adjust it so it faces forward, with the display surface perpendicular to the ground (as shown).

This mounting method ensures the controllers always remain within the visible range of the headset cameras (about a 100° horizontal field of view), preventing them from moving out of the tracking range during operation. This wearing method not only keeps controller tracking stable but also makes long data collection sessions more comfortable and effortless.

⚠️ Note: Use in a safe, open environment and make sure no collisions can occur during operation.

3. Robot Configuration

If you do not need to upgrade the robot's software version, skip this chapter.

1. Download and copy the SDK file (for example: {r1pro_vr_teleop_sdk_name}.tar.gz), and run the following command to copy the SDK to the R1 Pro.

   scp {r1pro_vr_teleop_sdk_name}.tar.gz nvidia@${R1_Pro_IP}:/home/nvidia/Downloads
   

2. Log in to the R1 Pro

   ssh nvidia@${R1_Pro_IP}
   

3. Extract the SDK on the R1 Pro

   mkdir /home/nvidia/galaxea
   tar -zxvf /home/nvidia/Downloads/{r1pro_vr_teleop_sdk_name}.tar.gz -C /home/nvidia/galaxea
   

4. Install additional dependencies

   sudo apt install ros-humble-rosbag2-storage-mcap
   pip3 install websockets pyquaternion tos 
   

5. Configure the device serial number and device name

   # Get the R1Pro serial number (the SN sticker at the connection between the robot torso and the chassis; if you cannot find it, contact Galaxea customer service)
   # e.g.: S2RL100P25110
   sudo mkdir -p /opt/galaxea
   sudo chown -R nvidia:nvidia /opt/galaxea
   mkdir -p /opt/galaxea/body
   echo "S2RL100P25110" | tee /opt/galaxea/body/RSN > /dev/null
   echo "S2RL100P25110" | tee /opt/galaxea/body/RSN_REAL > /dev/null
   echo "R1PRO" | tee /opt/galaxea/body/ROBOT_NAME > /dev/null
   # Verify that the values were written successfully
   cat /opt/galaxea/body/RSN
   cat /opt/galaxea/body/RSN_REAL
   cat /opt/galaxea/body/ROBOT_NAME
   

6. Configure the data collection module

   Data recording and visualization platform, EDP platform free trial - after completion, please refer to the First Data - EDP Getting Started manual.

   After the upgrade is complete, power off the R1 Pro and restart it. After the restart, the package configuration is complete and the VR teleoperation feature is ready to use.

   After the upgrade is complete, power down the robot and restart it. After the restart, the package configuration is complete and the VR teleoperation feature is ready to use.

4. Starting Teleoperation

Note: You must complete and confirm all the operations in this chapter every time you start up.

4.1 Starting the R1 Pro Onboard Program

1. Log in to the R1 Pro

   ssh nvidia@${R1_Pro_IP}
   

2. Enter the package startup directory

   cd /home/nvidia/galaxea/install/startup_config/share/startup_config/script/
   

3. Start the program

   ./robot_startup.sh boot ../sessions.d/ATCStandard/R1PROVRTeleop.d/
   

Note: After starting the onboard program, the forearms of both of the R1Pro's arms will raise 90 degrees, ending up perpendicular to the body. Be aware of the safety of the operating space in front.

4.2 Starting the VR Device Program

Note: Please put on the VR device and hold the two controllers before beginning the following steps.

4.2.1. Connect to WiFi

Confirm that the VR device has successfully connected to the same WiFi network as the R1 Pro.

4.2.2. Launch the GalaxeaVR APP

1. Open the GalaxeaVR app:

   Click the cube icon in the lower-right corner to launch the GalaxeaVR app.

2. Enter the R1 Pro's IP address:

   After entering the GalaxeaVR app, aim the ray emitted by the VR controller at the green IP input box.

   Wait for the green input box to change color slightly, then use the T key (index-finger trigger) on the right controller to click the input box (the cursor position needs to be a bit lower).

   Once the keyboard appears, enter the robot's IP address (i.e. the R1 Pro's IP address).

   After setting the IP, click the Start button to begin (or press and hold the B key on the right controller). If the volume on your VR headset is turned on, you will hear a "connecting" prompt tone followed by a "connection successful" tone. If you do not hear the "connection successful" tone, please check whether the previous steps were performed incorrectly.

3. Operate the VR device:

   At this point the VR default operating mode is reset mode. Refer to Section 5.3 to switch to bimanual control mode, then briefly press the G key on the left and right controllers to release the pause and begin bimanual teleoperation.

   Note: At this point the robot will follow your movements. Be careful for safety: make small movements first and ensure there are no obstacles around. ​

Use the following steps to perform basic controller operations:

• Stop operation: Press and hold the B key on the right controller for more than 2 seconds to stop VR teleoperation. (Note: To enter teleoperation again after stopping, you need to exit the GalaxeaVR app and re-enter it.)
• Control the grippers: Moving your hands controls the movement of the robot arms. The T key on the left controller and the T key on the right controller control the closing of the left and right grippers respectively.
• Pause the right arm: Briefly press the G key on the right controller once, and the right arm will stop at its current position; press the G key again to release the pause.
• Pause the left arm: Briefly press the G key on the left controller once, and the left arm will stop at its current position; press the G key again to release the pause.

For detailed instructions, refer to Chapter 6: Teleoperation Control Instructions.

5. Teleoperation Control Instructions

Please practice using this product in an open area while ensuring the safety of personnel and objects.

We recommend familiarizing yourself with the product before beginning formal data collection.

5.1 Controller Instructions

5.1.1 R1 Pro Aviation-style Remote Controller

Instructions for using the remote controller are as follows:

Note: Before performing any operation, confirm that all toggle switches (SWA/SWB/SWC/SWD) are set to the topmost position. This keeps the R1 Pro in a stopped state and prevents it from running unexpectedly.

Instructions for switching each toggle switch to different positions under different functions are as follows:

• Host computer controls the chassis (down, down, up, up): In this mode, the host computer must start the chassis motion control program first before control can be enabled.
• Remote controller controls the chassis (down, down, middle, up): After the robot is powered on, you can directly use the remote controller to control the chassis movement in this mode, with no operation required from the host computer.
• Enable emergency stop (D down): In any mode, you can toggle SWD to the down position to enable emergency stop; however, after enabling emergency stop, the robot must be restarted.

5.1.2 VR Handheld Controllers

5.2 VR Device APP Display Buttons

5.3 Mode Switching

After teleoperation starts, the default mode is reset mode. Users can switch between different operating modes using the VR device controllers.

Note: If you find that the teleoperation posture of the arms is slightly off after repeatedly using the controller G key to pause and unpause, do not worry. This is caused by the incremental control mechanism of teleoperation. You can reset the arms using reset mode to control the robot more effectively.

Mode	How to Switch	Notes
Reset mode （Reset）	Press and hold the left joystick down for 1 second	Default mode
Bimanual control mode （BIMANUAL）	Press and hold both the left and right joysticks down at the same time for 1 second	To control the chassis, you need to turn on the R1 Pro remote controller and set SWA, SWB, SWC, and SWD all to the up position.
Torso mode （Torso）	Press and hold the right joystick down for 1 second	This mode is disabled when using whole-body teleoperation

5.3.1 Reset Mode

Function	Description	Notes
Reset both arms to the initial position	VR left controller T + G keys VR right controller T + G keys Pressing all four keys together will reset the robot's arm positions to the initial startup position	-

5.3.2 Bimanual Control Mode

Function	Description	Notes
Hands follow	The R1 Pro arm end-effector positions follow the movement of the VR handheld controllers.	-
Gripper grasping	The T key on the VR right controller controls the opening and closing of the right gripper The T key on the VR left controller controls the opening and closing of the left gripper	-
Arm pause	The G key on the VR right controller controls pausing the right arm The G key on the VR left controller controls pausing the left arm Press once to pause, press again to unpause	-
Chassis forward/backward	VR left controller joystick forward/backward	-
Chassis lateral movement	VR left controller joystick left/right	-
Chassis rotation	VR right controller joystick left/right	-

5.3.3 Torso Mode

Function	Description	Notes
Waist left/right rotation (Joint 1)	VR left controller joystick left/right
Waist forward/backward pitch (Joint 2)	VR left controller joystick forward/backward
Torso up/down lift (Joint 3)	VR right controller joystick forward/backward
Torso forward/backward pitch (Joint 4)	VR right controller T/G keys

# To modify the initial startup position, edit the joint positions in the launch file
vim /home/nvidia/galaxea/install/teleoperation_ros2/share/teleoperation_ros2/launch/vr_teleoperation.launch.py
# Modify the following initial startup joint positions
{'vr_initializer_left_target_joint_states_r1pro': [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]},
{'vr_initializer_right_target_joint_states_r1pro': [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]}

# Modify the following to change the gripper range
'gripper_openning_threshold', default value='80.0' #Maximum value: 100.0
'gripper_closing_threshold', default value='0.0'  #Minimum value: 0.0

6. Data Collection Workflow

Data recording and visualization platform, EDP platform free trial - after completion, please refer to the First Data - EDP Getting Started manual.

6.1 Data Format Introduction

The file format for data collection is ros2 bag, with the file suffix *.mcap.

6.2 Data Retrieval

Default storage path: /home/nvidia/GalaxeaDataset/{data}/

date is the current date, in the following format: 20250307

6.3 Data Recording Configuration File Introduction

The default configuration file used for data recording is located at:

/opt/galaxea/data_collection/data_task_config.json

Configuration file description: The configuration file describes the information of this collection task. Users can modify it as needed.

{
    "project_info": {
        "project_name": "sop_test"
    },
    "task_info": {
        "task_name": "sop_test_data_collection",
        "task_owner": "hengyi.fei"  
    },
    "operation_info": {
        "teleoperation_type": "VR",
        "location": "suzhou",
        "operator_name": "jiahao.wu"
    }
}

6.4 Introduction to Saved Data Files

Data is saved in a folder + json format, with files in one-to-one correspondence. For example:

 # For example, the following folder and json file represent one data package
 # The folder contains the .mcap data file and meta information

 S2R12000P18245_20240213173320125_RAW
 S2R12000P18245_20240213173320125_RAW.json

 # The format is robot_serial_number+timestamp+RAW
 # robot_serial_number: the robot serial number, located at /opt/galaxea/body/RSN
 # timestamp: the data collection timestamp, accurate to the millisecond.
 # RAW: represents the raw saved data of the collection

6.5 Start Recording

Perform data recording operations using the VR left controller:

Function	Operation	Description
Start recording	Click the X key on the left controller once	Start data recording
Stop recording	Click the Y key on the left controller once	Stop data recording
Delete the current recording	Click the X key on the left controller once	If a recording is already in progress, clicking the X key again will end and delete the current recording without saving it to disk.

6.6 Recording Prompt Tones

You can enable the feature where the VR headset plays status prompt tones for the start and end of data recording by configuring \${VR_IP} in the ~/.bashrc file.

# Open ~/.bashrc
vim ~/.bashrc
# Add the following to ~/.bashrc, replacing ${VR_IP} with your VR device's address on the local network (Section 2.6)
export VR_IP=${VR_IP}
# Restart the program
tmux kill-server

7. Whole-Body Teleoperation Mode

Note: To use whole-body teleoperation, update the SDK to V2.1.4 or later, and update the VR headset configuration file to V0.1.7. In whole-body teleoperation mode, the torso also follows the movement of your body. Please read the following carefully and familiarize yourself with the workflow before using it.

7.1 Key Configuration

Please first read the Usage Notes, then contact the Galaxea after-sales team to obtain the key for enabling whole-body teleoperation, and configure it according to the following workflow.

sudo chown -R nvidia:nvidia /opt/galaxea
mkdir /opt/galaxea/galaxeaKey
cp keys.env /opt/galaxea/galaxeaKey/

7.2 Usage Workflow

7.2.1 Starting Whole-Body Teleoperation

1. Similar to Section 4.1, log in to the R1 Pro

   ssh nvidia@${R1_Pro_IP}
   

2. Enter the package startup directory

   cd /home/nvidia/galaxea/install/startup_config/share/startup_config/script/
   

3. Start the software

   # Clear any already-started programs
   tmux kill-server
   
   # Start whole-body teleoperation
   ./robot_startup.sh boot ../sessions.d/ATCStandard/R1PROVRTeleop_whole_body.d/
   

Note: After starting the onboard program, the forearms of both of the R1Pro's arms will raise 90 degrees, ending up perpendicular to the body. Be aware of the safety of the operating space in front.

7.2.2 Starting the VR Device Program

Please refer to Section 4.2 to complete the startup of the VR headset program and perform whole-body teleoperation.

Note: Since in whole-body teleoperation mode the R1Pro's torso posture follows changes in the VR headset's posture, make sure the VR headset is properly worn and the straps are securely fastened, as shown below. For safety, always ensure the headset is stably fixed to your neck. If there is any anomaly, press and hold the B key on the right controller to exit.

For safety, always ensure the headset is stably fixed to your neck. If there is any anomaly, press and hold the B key on the right controller to exit.

If you have any questions during installation and startup, please contact us promptly at support@galaxea-dynamics.com or call 4008-780-980 for technical support!