FPV Oculus VR and Raspberry Pi

GOAL


The goal of the project is to realise FPV flight with 2 raspberry PI’s, PI cam and an Oculus Rift

Quadcopter


My quadcopter is based on:

– KK 2.1.5 flight controller
– Turnigy Multistar 20 Amp Multi-rotor Brushless ESC 2-4S
– Hobby King Quadcopter Power Distribution Board
– NTM Prop Drive 28-26 1000KV / 235W
– HK Receiver
– Turnigy H.A.L. (Heavy Aerial Lift) Quadcopter Frame 585mm

FPV


For FPV we need a sender and receiver.
The sender will setup a WiFi network so receivers could hook on. The sender will transmit the video as a motion jpeg over UDP. The reason being, I’m concerned of speed, so a frame drop here and there isn’t really an issue.

Sender wireless setup


I went trough the same steps as for my suprise (http://flitjes.nl/anchor/index.php/suprise). This was for a beaglebone, but both are arm and using the same wifi chipset.

– Compile hostapd
– Install hostapd
– Configure hostapd
– Install dhcp
– Configure dhcp
– Configure /etc/network/interfaces

Receiver wireless setup


To connect to the network setup the sender I will use wpa_supplicant

– Install wpa_supplicant
– Setup wpa_supplicant
– Add starting of wpa_supplicant to rc.local

wpa_supplicant.conf:

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
network={
ssid="YOUR_NETOWRK_ID"
psk="PASSWORD"
}

/etc/rc.local

echo "Starting Wifi"
wpa_supplicant -B -i wlan0 -c /home/pi/wpa_supplicant/wpa_supplicant.conf
sleep 1
dhcpcd wlan0

exit 0

Video

To stream I’m using the gstreamer framework. For a brief explanation checkout this site: http://www.z25.org/static/_rd_/videostreaming_intro_plab/

With the use of 2 simple shell script’s I was able to stream the video stream from the PI cam to my desktop with the Oculus attached.
Due to the fact there is no VR player for Embedded Linux I’ve opened up the video stream twice and placed them on the Oculus by hand. I’ve tried to crosscompile Bino, it’s already part of the Debian armhf repo so I thought it should work. It doesn’t because GLEW, the OpenGL “version wrapper” doesn’t support OpenGL 2.0 ES, which is a embedded system version of OpenGL. I’ve considered to extend GLEW to support OpenGL 2.0 ES but that seemed a project on it’s own.

Video stream of the pi:
Some how I can only make it work on my local network.
http://wiki.oz9aec.net/index.php/Raspberry_Pi_Camera

raspivid -w 800 -h 480 -fps 20 -n -pf baseline -ex auto -o - -t 0 | \
gst-launch-0.10 -v fdsrc !  h264parse ! rtph264pay config-interval=10 pt=96 ! udpsink host=192.168.1.255 port=9000

Receive stream on PC:

gst-launch-1.0 -v udpsrc port=9000 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000,     encoding-name=(string)H264' ! \
rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false

Receiving stream on PI:

gst-launch-1.0 -v udpsrc port=9000 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! rtph264depay ! h264parse ! omxh264dec ! eglglessink sync=false

Got the Oculus Rift working on a raspberry pi with some artifacts remaining.
For external usb power is required!

The wifi seems to fail after heavy load.. So no wireless streaming yet.

hdmi_cvt parameters:

# Put your VGA monitor resolution below
# hdmi_cvt =
# width        width in pixels
# height       height in pixels
# framerate    framerate in Hz
# aspect       aspect ratio 1=4:3, 2=14:9, 3=16:9, 4=5:4, 5=16:10, 6=15:9
# margins      0=margins disabled, 1=margins enabled
# interlace    0=progressive, 1=interlaced
# rb           0=normal, 1=reduced blanking

With the following entries added to my /boot/config.txt

#uncomment to overclock the arm. 700 MHz is the default.
arm_freq=800
#with height, framerate, aspect ratio, margins, interlace, rb
hdmi_cvt 1080 1920 60 3 0 0 1

hdmi_group=2
#use added mode with hdmi_cvt
hdmi_mode=87
max_framebuffer_width=1080
max_framebuffer_height=1920
#rotate in the propper direction
display_rotate=3
#edid should be ignored otherwise the screen will be turning on and off
hdmi_ignore_edid=0xa5000080
#havent figured the value out yet
hdmi_pixel_freq_limit=400000000

gpu_mem=128

QT and Gstreamer


http://gstreamer.freedesktop.org/data/doc/gstreamer/head/qt-gstreamer/html/examples_2qmlplayer_2main_8cpp-example.html

http://cgit.freedesktop.org/gstreamer/qt-gstreamer/tree/examples/qmlplayer2

git clone http://cgit.freedesktop.org/gstreamer/qt-gstreamer/

mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=/home/<user_name>/gstreamer_install
make
make install

Setup your enviroment before starting QT

export PKG_CONFIG_PATH=/home/<user_name>/gstreamer_install/lib/x86_64-linux-gnu/pkgconfig/
export CMAKE_PREFIX_PATH=/home/<user_name>/gstreamer_install/

Get QT to build the examples you should add the path to the pkg-config files (.pc) which are refferd to in the .pro file

Open an example
Projects -> Add

PATH (append) /home/<user_name>/gstreamer_install/lib/x86_64-linux-gnu/

If libraries can’t be found symlink them to /usr/bin from /home/<user_name>/gstreamer_install/lib/x86_64-linux-gnu/xxxx.so.0

At te moment I’ve got it running on my desktop. This is running Ubuntu and connected to my home network. I’m streaming the PI cam video over upd to my home network on WiFi. This is then shown on my Qt application running on my desktop.

Next step: Crosscompiling.

4 Comments

  1. hey man, great stuff. I fly FPV since 2 years and was one of the first that got the oculus. At the beginning there was not much about rift fpv. I’ll definetly will give it a try the next days. Any updates from your side? the last status is past a while. cheers and thanks for your effort!

    • Hey,

      I’ve stalled at moving the application to the RPI. I lost abit of interest and started something new. That will end up on the site as well

  2. certainly like your web-site however you need to test
    the spelling on quite a few of your posts. A number of them are
    rife with spelling problems and I to find
    it very bothersome to tell the reality on the other hand I’ll surely come back again.

Leave a Reply

Your email address will not be published. Required fields are marked *