🌈
Sensor Stream Pipe
  • What is Sensor Stream Pipe?
  • Getting Started
  • Installation Linux
  • Installation Windows
  • Installation Mac
  • Installation iOS
  • Streaming a Video
  • Streaming with Azure Kinect DK
  • Streaming a Dataset
  • Streaming iOS ARKit RGB-D Data
  • Streaming an OAK-D using Xlink
  • Components Overview
    • Sensor Stream Server
    • Sensor Stream Client
    • Sensor Stream Tester
  • Sending Frames
    • IReaders
    • FrameStruct
    • Config File
    • IEncoders
  • Receiving Frames
    • Receiving Frames
  • How to Extend Sensor Stream Pipe
    • Add New Sensor Interface
Powered by GitBook
On this page
  • Installing with Pre-Built Binaries (Recommended)
  • 1.0 Install Pre-Requisites
  • 2.0 Install Azure Kinect SDK/Azure Kinect Body Tracking SDK (optional)
  • 3.0 Install OAK-D Pre-Requisites (optional, if you want to stream an OAK-D using Xlink and run inference using OpenVINO)
  • 4.0 Build/Install Sensor Stream Pipe
  • 5.0 Use it!
  • Installing Manually
  • 1.0 Install Pre-Requisites

Was this helpful?

Installation Linux

PreviousWhat is Sensor Stream Pipe?NextInstallation Windows

Last updated 2 years ago

Was this helpful?

We have 2 methods for installing on Linux. In one instance, we lean heavily on pre-built binaries being packaged with SSP (recommended). In the second method we manually install all necessary dependencies.

These steps have been tested on Ubuntu 20.04

  1. Pre-built binaries (recommended)

    • This will use the binaries we have already built for each platform and architecture

      • If you want to see how the binaries were built you can check out 3rdparty/build_linux_dep.sh

  2. Installing Manually

    • This will provide download instructions for each library

Problems?! (shocker)

Reach out on and we will get you going!

Installing with Pre-Built Binaries (Recommended)

1.0 Install Pre-Requisites

sudo apt-get update
sudo apt-get install build-essential
sudo apt-get install cmake
sudo apt-get install libgtk2.0-dev
sudo apt-get install git
sudo apt-get install ffmpeg
sudo apt-get install libusb-1.0

2.0 Install Azure Kinect SDK/Azure Kinect Body Tracking SDK (optional)

These steps are only required if you are interested in building with Azure Kinect and Body Tracking capabilities.

curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/prod
curl -sSL https://packages.microsoft.com/config/ubuntu/18.04/prod.list | sudo tee /etc/apt/sources.list.d/microsoft-prod.list
curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
sudo apt-get update
sudo apt install libk4a1.3-dev
sudo apt install libk4abt1.0-dev
sudo apt install k4a-tools=1.3.0=

Verify sensor library by launching camera viewer (require sudo for hardware access)

sudo k4aviewer
  • Run simple_3d_viewer to verify body tracker works.

3.1 Install python 3.8 if you do not have it (necessary for OAK-D)

3.2 Install OpenVINO (used to run inference on frames grabbed from OAK-D device)

4.0 Build/Install Sensor Stream Pipe

git clone https://github.com/moetsi/Sensor-Stream-Pipe.git
cd Sensor-Stream-Pipe
mkdir build && cd build
cmake ..
make -j4

NOTE: You must enable Kinect or Xlink functionality when building with the following flags:

-DK4A_ENABLED=TRUE (for Kinect)

-DXLINK_ENABLED=TRUE (for OAK-D)

Example:

cmake -DK4A_ENABLED=TRUE ..

or

cmake -DXLINK_ENABLED=TRUE ..

the flags are cached, so if you want to disable you will need to set them to false

5.0 Use it!

Installing Manually

1.0 Install Pre-Requisites

To get our Sensor Stream Pipe up and running, you will require the following:

Dependencies

To get our Sensor Stream Pipe up and running, you will require the following:

Download and install repo libraries

OpenCV 3.2.0

sudo apt install libopencv-dev libopencv-core-dev uuid-dev

Libav 3.4.6

sudo apt install libavformat-dev libavutil-dev libavcodec-dev libavfilter-dev

Download and extract "out-of-repo" libraries

First, create a folder where local libs are to be installed:

mkdir ~/libs
mkdir ~/libs/srcOriginal

Cereal 1.2.2

cd ~/libs/srcOriginal
wget https://codeload.github.com/USCiLab/cereal/tar.gz/v1.2.2
tar xf v1.2.2
cp -r cereal-1.2.2/include ~/libs

ZeroMQ

libzmq3 4.3.1

cd ~/libs/srcOriginal
wget https://github.com/zeromq/libzmq/releases/download/v4.3.1/zeromq-4.3.1.tar.gz
tar xf zeromq-4.3.1.tar.gz
cd zeromq-4.3.1
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=~/libs
make install -j4

cppzmq 4.3.0

cd ~/libs/srcOriginal
wget https://github.com/zeromq/cppzmq/archive/v4.3.0.tar.gz
tar xf v4.3.0.tar.gz
cd cppzmq-4.3.0
cp *.hpp ~/libs/include

yaml-cpp 0.6.0

cd ~/libs/srcOriginal
wget https://github.com/jbeder/yaml-cpp/archive/yaml-cpp-0.6.0.tar.gz
tar xf yaml-cpp-0.6.0.tar.gz
cd yaml-cpp-yaml-cpp-0.6.0
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=~/libs
make install

Zdepth

cd ~/libs/srcOriginal
git clone https://github.com/catid/Zdepth.git
cd Zdepth
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=~/libs
make install
cp libzdepth.a ~/libs/lib/
cp zstd/libzstd.a ~/libs/lib/

spdlog

cd ~/libs/srcOriginal
wget https://github.com/gabime/spdlog/archive/v1.4.1.tar.gz
tar xf v1.4.1.tar.gz
cd spdlog-1.4.1 && mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=~/libs
make -j
make install

NVPipe (optional, recommended for users with Nvidia GPU)

cd ~/libs/srcOriginal
git clone https://github.com/NVIDIA/NvPipe.git
cd NvPipe/
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=~/libs
make
make install

Azure Kinect SDK 1.3/4 (optional)

Note: to avoid getting a password prompt, run any command as sudo before starting this section of the tutorial

1) Add the Linux Software Repository for Microsoft Products.

curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/prod
sudo apt-get update

2) Install Azure Kinect SDK 1.3 (change 1.3 to 1.4 if you do not need body tracking)

sudo apt install libk4a1.3 libk4a1.3-dev

3) To be able to use the Kinect as non-root, please run the following:

wget https://raw.githubusercontent.com/microsoft/Azure-Kinect-Sensor-SDK/develop/scripts/99-k4a.rules
sudo cp 99-k4a.rules /etc/udev/rules.d/

4 a) If using 1.4, in the current package, the link to the canonical version of the depth lib is missing. You can create it by running the following command:

sudo ln -s /usr/lib/x86_64-linux-gnu/libk4a1.4/libdepthengine.so.2.0 /usr/lib/x86_64-linux-gnu/libdepthengine.so

4 a) If using 1.3, the depth engine is missing from the package. Microsoft is aware of the problem, but it only corrected it in 1.4.

cd ~/libs/lib
ln -s libdepthengine.so.2.0 libdepthengine.so

Azure Kinect Body Tracking SDK (optional)

Check instructions above to add the Linux Software Repository for Microsoft Products and then do:

sudo apt install libk4abt1.0-dev

Building Sensor Stream Pipe

Download and build the project (the ssp_server, ssp_client and ssp_tester):

git clone git@github.com:moetsi/Sensor-Stream-Pipe.git
cd Sensor-Stream-Pipe
mkdir build
cd build
cmake .. -DSSP_WITH_KINECT_SUPPORT=OFF -DSSP_WITH_K4A_BODYTRACK=OFF -DSSP_WITH_NVPIPE_SUPPORT=OFF
make

From this comment:

These are the steps I followed to install k4a-tools, libk4a and libk4abt on Ubuntu 20.04. The general steps are as outlined in , with a couple of hacks to make things work on 20.04. - use of 18.04 repo, even though OS is 20.04 - installed lower versions of tools and libraries (as latest versions of sensor and body tracker don't seem to be compatible on 20.04)

Clone and build .

3.0 Install Pre-Requisites (optional, if you want to stream an OAK-D using Xlink and run inference using OpenVINO)

Now checkout or to see how you can stream local files with Sensor Stream Server to Sensor Stream Client.

The following steps were tested on Ubuntu 18.04. Installing on other recent Linux distributions should be pretty similar, but please check the installation instructions for OpenCV and Kinect DK on your respective platform first. Installation instructions for Windows should be ready soon. If you encounter any problems or have any suggestions, please let us know by emailing contact@moetsi.com or post on our .

3.2.0 (tested on version available on Ubuntu 18.04 repo) is used for image processing.

3.4.6 (tested on version available on Ubuntu 18.04 repo) encodes, decodes and processes image frames.

1.2.2 (headers only) serializes data for network transmission.

and (libzmq3 4.3.1, cppzmq 4.3.0) perform network and low-level I/O operations.

1.4.1 Logging library.

0.6.0 reads server configuration files.

: compresses depth data.

(optional, but recommended if you have an NVidia GPU ) encodes and decodes frames. This is optional, but recommended for users with Nvidia GPUs.

1.3 (to support the Azure Kinect Body Tracking SDK), 1.4 otherwise (optional) accesses Kinect DK data.

1.0 (optional) SSP Body Tracking client.

If you want to take advantage of ZMQ pulling support to check for new frames, compile ZMQ and CPPZMQ with the . Also, you must set SSP_WITH_ZMQ_POLLING at SSP build time. Discussion available

You can get the libdepthengine.so.2.0 file from the package at . Open with Archive Manager (or equivalent), the file is in (data/./usr/lib/x86_64-linux-gnu/libk4a1.4/), and extract it to ~/libs/lib. You can then perform an equivalent command as above.

Now checkout or to see how you can stream local files with Sensor Stream Server to Sensor Stream Client.

our discord
https://feedback.azure.com/users/1291190890-mpdroid
https://docs.microsoft.com/en-us/azure/Kinect-dk/sensor-sdk-download
Azure Kinect Samples
OAK-D
https://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_apt.html#doxid-openvino-docs-install-guides-installing-openvino-apt
Streaming a Video
Streaming a Dataset
forum
OpenCV
libav
Cereal
ZeroMQ
cppzmq
spdlog
yaml-cpp
Zdepth
NvPipe
Azure Kinect SDK
Azure Kinect Body Tracking SDK
draft API suppport
here
https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/docs/depthengine.md
https://packages.microsoft.com/ubuntu/18.04/prod/pool/main/libk/libk4a1.4/libk4a1.4_1.4.1_amd64.deb
Streaming a Video
Streaming a Dataset