XR Teleoperation Setup#

Follow these steps to install and run a full XR-to-robot teleoperation sample on a PICO 4 Ultra headset and a Linux x86 PC.

Linux x86 PC: Ubuntu 22.04
PICO 4 Ultra: User OS >5.12. Currently supports PICO 4 Ultra and PICO 4 Ultra Enterprise.

Install Roboverse
- Install Roboverse with your preferred simulators following this instruction, the XR teleoperation is tested with Isaac v1.4
- Install cuRobo

Install xrobotoolkit-sdk

Use the following command

git clone https://github.com/XR-Robotics/XRoboToolkit-PC-Service-Pybind.git
cd XRoboToolkit-PC-Service-Pybind
./setup_ubuntu.sh

Install XRoboToolkit-PC-Service
- Download deb package for ubuntu 22.04, or build from the repo source.
- To install, use command
```
 sudo dpkg -i XRoboToolkit-PC-Service_1.0.0_ubuntu_22.04_amd64.deb
```
Install the XR App on Headset
- Turn on developer mode on Pico 4 Ultra headset first (Enable developer mode on Pico 4 Ultra), and make sure that adb is installed properly.
- Download XRoboToolkit-PICO.apk on a PC with adb installed.
- To install apk on the headset, use command
```
 adb install -g XRoboToolkit-PICO.apk
```
Run XR teleoperation in Roboverse
- Connect robot PC and Pico 4 Ultra under the same network
- On robot PC, double click app icon of XRoboToolkit-PC-Service or run service /opt/apps/roboticsservice/runService.sh
- Open app XRoboToolkit on the Pico headset. Details of the Unity app can be found in the Unity source repo.
- Run the XR teleoperation demo
```
python metasim/scripts/teleop_xr.py --task=PickCube
```