The "Foculus Rift" part 2: Reverse engineering the motion tracker

The function of the motion tracker is to estimate the current roll, pitch and yaw angles of the head mounted display (HMD), which allows the 3D game to know what you are looking at and render the scene accordingly.

The original Rift uses the following configuration (as nicely shown by ifixIt):

• Invensense MPU-6000, gyroscope and accelerometer sensor
• Honeywell HMC5983, magnetic field sensor / compass
• ST  32F103C8, ARM 32 bit microcontroller (with USB connection)

The Oculus Rift Tracker board (Picture from oculusvr.com)

The ARM reads out the three sensors over an SPI and I2C interface and sends the data to the host PC over USB. The host runs the Rift SDK, which performs the sensor fusion and estimates the head position. New data is sent at a rate of ~ 1 kHz to minimize latency.

Now the interesting bit: the company ST is aggressively marketing their STM32 microcontrollers. You can get the STM32F3DISCOVERY board for as little as 15 Euro. This is an evaluation board, containing the three types of sensors mentioned above and comes with a programmable USB connection. It is an excellent hardware platform to emulate the Rift tracker.

The STM32F3 discovery kit. (Picture from ST.com)

The masterplan is the following:

1. Reverse engineer the Rift USB communication protocol. Understand which messages are exchanged and how the raw sensor values are encoded.
2. Write a firmware for the STM board to appear just like a Rift to the Host PC. This includes setting the USB descriptors (including Vendor ID, Product ID and the String descriptors) to identical values.
3. Write a firmware which periodically reads out the sensors and sends the data to the host PC in Rift compatible format.

Reverse Engineering

Much information can be found on the web. For example the Vendor and Product ID can be found in many logfiles posted by Rift users in various forums. Google is your friend.

Furthermore an USB packet sniffer (such as Wireshark) can be used to get an idea of the USB traffic and what kind of binary data is sent. This is an important debugging tool.

Perhaps one of the most insightful options is to have a close look at the source code of the Rift SDK. A lot of interesting stuff can be found in the file: OVR_SensorImpl.cpp
I found it extremely useful to modify the SDK code slightly to print me debug messages, showing what is sent over USB. Specifically I modified OVR_Linux_HIDDevice.cpp on line 630:

bool HIDDevice::SetFeatureReport(UByte* data, UInt32 length){
    if (DeviceHandle < 0)
        return false;
    UByte reportID = data[0];
    if (reportID == 0) {
        // Not using reports so remove from data packet.
        data++;
        length--;
    }
    LogText("HACK::SetFeature, length = %d, data[0-end] = ", length);
    int temp;
    for (temp=0; temp < length; temp++){
        LogText("%02x ", data[temp]);
    }
    LogText("\n");
    int r = ioctl(DeviceHandle, HIDIOCSFEATURE(length), data);
 return (r >= 0);
}

bool HIDDevice::GetFeatureReport(UByte* data, UInt32 length) {
    if (DeviceHandle < 0)
        return false;
    int r = ioctl(DeviceHandle, HIDIOCGFEATURE(length), data);
    LogText("HACK::GetFeature, length = %d, data[0-end] = ", length);
    int temp;
    for (temp=0; temp < length; temp++){
        LogText("%02x ", data[temp]);
    }
    LogText("\n");
    return (r >= 0);
}

With this change I got the following output, running the OculusWorld demo (with the fully functioning tracker):

Configuration data exchange between Oculus Rift SDK and Tracker

At the start of a game, the SDK will read several configuration parameters from the tracker, which contain information about the screen, the lenses and the tracker. This is done by exchanging HID "feature reports" over the USB endpoint 0. Here is a snippet showing some of the parameters describing the screen:

Snippet from the Rift SDK, which decodes the 56 byte long configuration packet, containing information about the display.

To get an idea what kind of configuration packets are exchanged and what is encoded by their binary values, have a look at the foculusTypes.h file.

Next, the host PC will send periodically (every 3 s) a "keep alive" packet to the tracker, telling it to keep sending sensor data. After the first received keep alive packet, the tracker will start to transmit the sensor data to the PC, typically every 1 ms. This is done in an "interrupt IN transfer" on USB endpoint 1.

The sensor data is arranged in a 62 byte long USB frame in the following way:

uint8_t messageType;    // = 1 indicates a sensor data frame
uint8_t sampleCount;    // How many sensorData slots are used (1, 2 or 3)
uint16_t timeStamp;     // Incremented for each packet
uint16_t commandID;     // Not used yet
int16_t temp;           // Temperature
uint8_t sensorData1[16];// Sensor data packet, 16 bytes each.
uint8_t sensorData2[16];// Sensor data packet, 16 bytes each.
uint8_t sensorData3[16];// Sensor data packet, 16 bytes each.
int16_t mX;             // 16 bit Magnetometer data
int16_t mY;
int16_t mZ;

The magnetometer data is directly encoded in three 16 bit signed integers (mX, mY and mZ).
The data from the gyroscope and accelerometer is formated as 21 bit signed integers and stored in the sensorData blocks.

The three 16 byte arrays contain up to three readings of the gyroscope and accelerometer. 

Let's show an example of how the data is packed in these blocks.

We start with the data, read out from the gyroscope, formated as 21 bit signed integers:

x - axis:  x20 x19 x18 ... x3  x2  x1  x0   (21 bits)

y - axis:  y20 y19 y18 ... y3  y2  y1  y0

z - axis:  z20 z19 z18 ... z3  z2  z1  z0

This is aranged in the first 8 bytes of the sensorData block as such:

sensorData[0] = x20 x19 x18 x17 x16 x15 x14 x13

sensorData[1] = x12 x11 x10 x9  x8  x7  x6  x5

sensorData[2] = x4  x3  x2  x1  x0  y20 y19 y18

sensorData[3] = y17 y16 y15 y14 y13 y12 y11 y10

sensorData[4] = y9  y8  y7  y6  y5  y4  y3  y2

sensorData[5] = y1  y0  z20 z19 z18 z17 z16 z15

sensorData[6] = z14 z13 z12 z11 z10 z9  z8  z7

sensorData[7] = z6  z5  z4  z3  z2  z1  z0  0

The last 8 byte in the sensordata block contain the X, Y and Z values of the accelerometer in the same format.

Developing a Rift compatible tracker

After reverse engineering the communication protocol, it was possible to fully emulate the Rift tracker. I might post more details about the development process soon.

Until then, you can have a look at how the "Foculus" tracker has been implemented in the attached source code, which is heavily annotated with comments. I have uploaded the entire eclipse project folder:

Sourcecode on GitHub:

Foculus Tracker for STM32F3DISCOVERY V0.3
(3.4.14)

If you have a STM32F3DISCOVERY board, you might want to try out the code.

Some hints on how to set up a full development environment can be found here.

If you just want to flash the binary (.elf) file, have a look at how_to_flash.txt in the zip file.

This is pretty much a work in progress, so feedback is much appreciated. 

Let me know if you find bugs, if it works or doesn't work for you or if you have decided to continue development and came up with some new features to the firmware.

Known Issues, open questions and things to do

• The SDK can set the following flags: Flag_CalibrationTest, Flag_UseCalibration, Flag_AutoCalibration. I have no idea how the SDK expects the Tracker to behave after it set these flags. Currently they are not evaluated at all.
• Another thing is that I can not use the Oculus Config Utility to calibrate the Magnetometer, as it checks for a connected HMD. Without the calibration, there is no Yaw drift correction in most of the games. I hope to fix this problem with a EDID hack.

In part 3 I want to have a look on how to spoof the EDID, to make the LCD display panel look like an actual Oculus Rift monitor to the SDK.

Update 20.3.14

I wrote a small script in python, to receive the data from the fake rift and plot it. It turned out there were glitches in the Magnetometer data. I found this while recording data for 60 s while rotating the HMD randomly in all directions. The data as read from the magnetometer has been plotted on a 3D plane (left) and over time (right)

I think that these glitches actually come from the LSM303 magnetometer sensor. First I thought it is a synchronization issue. But it turned out, the only way to get rid of them was to reduce the output data rate of the sensor from 220 Hz to 75 Hz:

By the way, the left plot illustrates nicely that each one of the magnetometer readings should theoretically be a point on an ellipsoid. The Oculus Rift calibration procedure determines the parameters (its diameters and center point) of this ellipsoid.

Update 29.3.14

Several bugs have been fixed and new features were added.

Scaling of sensor values

In the previous releases, I was assuming the tracker does not actually do any rescaling and just transmits the raw numbers as they are read from the sensors. But as it turns out, this assumption is wrong. The Oculus SDK expects the sensor values to be scaled to 10^-4 [rad/s], 10^-4 [m/s^2] and 10^-4 [Gauss], independent on which measurement range the sensor is set to. In the current version of the Foculus Tracker I have accounted for that.

Gyroscope zero level error correction

Place the tracker on a table and make sure it absolutely does not move. Then press the blue "User button" on the STM32F3 DISCOVERY board. The LED pattern will blink a few times, to show that the calibration will now take place. Then 3 LEDs will turn on in sequence, showing which sensor range is calibrated at the moment. Then the IDLE LED pattern will be displayed again and the Tracker is calibrated and ready to use. Please note that the calibration values will be lost as soon as the Tracker is unplugged from USB.

The calibration significantly reduces gyroscope zero-level error and hence gyro drift, as shown by the following plots:

Before calibration. A bias error is clearly visible. This will be constantly integrated and lead to significant drift of tracker orientation.

After calibration. The zero level error has been reduced.

Further updates (Changelog)

// Changelog
//--------------------------------
// 09.03.2014: Fixed bug in handleConfigPacketEvent() by adding break; statements (data rate was always 1 ms before)
// 10.03.2014:  Changed wMaxPacketSize from 62 to 64
// 19.03.2014:  Changed I2C Bus speed to 400 kHz, which allows to read all 3 sensors in 0.65 ms  (before it was > 2 ms)
// 20.03.2014:  Now evaluating the "new data ready" pins of all 3 sensors (improves timing a lot, reduces jitter)
//              Enabled FIFO in Streaming mode of Accelerometer and Gyro (no samples will be lost!)
//              Fixed Glitches in Magnetometer output by setting it to 75 Hz measurement rate (was 220 Hz before)
// 23.03.2014:  Fixed a problem with the USB interrupt and atomic access, not allowing the tracker to change sensor scale
//    Changed sensor scaling to floating point numbers and scaled to values as expected from the SDK
// 29.03.2014:  Added gyroscope "set to zero" calibration routine (Press the user button on the STM board and keep it very still)
//    Added temperature readout from gyro
//    Added some nice LED animations for IDLE mode, Tracker running mode and Calibration mode