🌈
Sensor Stream Pipe
  • What is Sensor Stream Pipe?
  • Getting Started
  • Installation Linux
  • Installation Windows
  • Installation Mac
  • Installation iOS
  • Streaming a Video
  • Streaming with Azure Kinect DK
  • Streaming a Dataset
  • Streaming iOS ARKit RGB-D Data
  • Streaming an OAK-D using Xlink
  • Components Overview
    • Sensor Stream Server
    • Sensor Stream Client
    • Sensor Stream Tester
  • Sending Frames
    • IReaders
    • FrameStruct
    • Config File
    • IEncoders
  • Receiving Frames
    • Receiving Frames
  • How to Extend Sensor Stream Pipe
    • Add New Sensor Interface
Powered by GitBook
On this page
  • Installing with Pre-Built Binaries (Recommended)
  • 1.0 Install Azure Kinect Pre-Requisites (optional, if you want to use Azure Kinect, or Azure Kinect Body Tracking SDK)
  • 2.0 Install OAK-D Pre-Requisites (optional, if you want to stream an OAK-D using Xlink and run inference using OpenVINO)
  • 3.0 Build/Install Sensor Stream Pipe
  • 3.0 Use it!
  • Installing Manually
  • 1.0 Make sure you have Build Tools, Git, and CUDA
  • 2.0 Install vcpkg
  • 3.0 Install dependencies available on vcpkg
  • 4.0 Build and install remaining dependencies
  • 5.0 Building Sensor Stream Pipe
  • NvPipe Information

Was this helpful?

Installation Windows

We have 2 methods for installing on Windows. In one instance, we lean heavily on pre-built binaries being packaged with SSP (recommended). In the second method we manually install all necessary dependencies.

These steps have been tested on Windows 10 Build 19041, Visual Studio 2019 Community Edition (VS) and Visual Studio Code 1.53

  1. Pre-built binaries (recommended)

    • Dependencies are prebuilt and stored on a server and downloaded by cmake using FetchContent feature

    • To rebuild dependencies:

      cd Sensor-Stream-Pipe/3rdparty
      ./buid_win_dep.sh
    • This will download/build/install dependencies in a tmp directory. The result is a *_windep.tar.gz file.

      • download prebuilt ffmpeg 4.3.2

      • OpenCV 3.4.13 as a static library, only core, imgproc, imgcodecs and highgui modules are built

      • Cereal 1.3.0, header only

      • spdlog 1.8.2, header only but built as static library for faster compile

      • Zdepth (commit 9b333d9aec520 which includes a patch to generate zdepthConfig.cmake)

      • yaml-cpp 0.6.3 as a static library

      • libzmq 4.3.4 as a static library

      • cppzmq 4.7.1, header only

  2. Installing Manually

    • This will provide download instructions for each library

Problems?! (shocker)

Installing with Pre-Built Binaries (Recommended)

1.0 Install Azure Kinect Pre-Requisites (optional, if you want to use Azure Kinect, or Azure Kinect Body Tracking SDK)

1.1 Manually install CUDA Toolkit 10.0 (needed for Body Tracking SDK)

1.2 Manually install cudNN64_70 for CUDA 10.0 (needed for Body Tracking SDK)

You will need to accept some NVIDIA agreements

1.3 Manually Install Azure Kinect 1.4.1

1.4 Manual Install Azure Body Tracking SDK 1.1

2.1 Install python3 if you do not have it (necessary for OAK-D)

Ensure that python is added as a path variable

2.2 Remove max path limit

git config --system core.longpaths true

2.3 Install OpenVINO (used to run inference on frames grabbed from OAK-D device)

By default, the Intel® Distribution of OpenVINO™ is installed to the following directory, referred to as <INSTALL_DIR> elsewhere in the documentation: C:\Program Files (x86)\Intel\openvino_<version>. For simplicity, a shortcut to the latest installation is also created: C:\Program Files (x86)\Intel\openvino_2021

3.0 Build/Install Sensor Stream Pipe

Note: If planning on using Azure Kinect, Azure Kinect Body Tracking SDK, or Xlink and OpenVINO, they must be installed before SSP as SSP looks for the libraries when creating the Release

git clone https://github.com/moetsi/Sensor-Stream-Pipe.git
cd Sensor-Stream-Pipe
mkdir build && cd build
cmake -G "Visual Studio 15 2017 Win64" ..
cmake --build . --config Release

NOTE: You must enable Kinect or Xlink functionality when building with the following flags:

-DK4A_ENABLED=TRUE (for Kinect)

-DXLINK_ENABLED=TRUE (for OAK-D)

Example:

cmake -DK4A_ENABLED=TRUE -G "Visual Studio 15 2017 Win64" ..

or

cmake -DXLINK_ENABLED=TRUE -G "Visual Studio 15 2017 Win64" ..

and you will need to set the env variabels for OpenVINO so you will need to run "'C:/Program Files (x86)/Intel/openvino_2021/bin/setupvars.bat'" or where ever setupvars.bat is installed

the flags are cached, so if you want to disable you will need to set them to false

3.0 Use it!

3.1 Add Body Tracking Components (optional)

Move: - dnn_model_2_0.onnx (from body tracking sdk) and - cudNN64_7 (from step 1.2)

to /Release (or where you run the executable) if you want to run Body Tracking on Sensor Stream Client

3.2 Give it a go

Installing Manually

1.0 Make sure you have Build Tools, Git, and CUDA

1.2 Run "Build Tools for Visual Studio", choose "Language packs" and add "English". Tested with "Visual Studio Build Tools 2019 16.8.5.

1.4 Install CUDA 10 (if you would like to use Azure Kinect Body Tracking)

1.5 Install cuDNN 7.0 (if you would like to use Azure Kinect Body Tracking)

2.0 Install vcpkg

2.2 In Command window, change directories to the vcpkg root directory. Run vcpkg integrate install .

This should respond with something similar to:

C:\Users\adamm\dev\vcpkg>vcpkg integrate install
Applied user-wide integration for this vcpkg root.
All MSBuild C++ projects can now #include any installed libraries.
Linking will be handled automatically.
Installing new libraries will make them instantly available.
CMake projects should use: "-DCMAKE_TOOLCHAIN_FILE=C:/Users/adamm/dev/vcpkg/scripts/buildsystems/vcpkg.cmake"

3.0 Install dependencies available on vcpkg

3.1 Install dependencies using vcpkg (in vcpkg root directory).

vcpkg install azure-kinect-sensor-sdk:x64-windows cereal:x64-windows cppzmq:x64-windows ffmpeg:x64-windows opencv3:x64-windows spdlog:x64-windows yaml-cpp:x64-windows zeromq:x64-windows

4.0 Build and install remaining dependencies

Zdepth

Zdepth is an encoder for depth frames. This library can be used to compress depth data.

4.2 Clone the Zdepth repo to your home directory (or where ever you usually save repos)

git clone https://github.com/catid/Zdepth.git

4.3 Open CMakeLists.txt in Visual Studio Code

4.6 Prepare a directory in your home directory (or where ever you usually save repos) to place the remaining dependencies, /libs (referred henceforth as $LIBS). Create subfolders /include and /lib within /libs

mkdir libs
cd libs
mkdir lib
mkdir include

4.7 Copy the contents of Zdepth\include into $LIBS\include and output lib folders (e.g. ZDepth\out\*) to $LIBS\lib.

Azure Kinect Body Tracking SDK (optional)

4.10 Copy the SDK include and lib files from the SDK install list to $LIBS, or add the SDK path to SSP CMakeLists (see below)

  • Example

    • C:\Program Files\Azure Kinect Body Tracking SDK\sdk\include

      • Into $LIBS\include

    • C:\Program Files\Azure Kinect Body Tracking SDK\sdk\netstandard2.0 and C:\Program Files\Azure Kinect Body Tracking SDK\sdk\windows-desktop

      • Into $LIBS\lib

5.0 Building Sensor Stream Pipe

5.1 Clone the SSP repo to your home directory (or where ever you usually save repos)

git clone git@github.com:moetsi/Sensor-Stream-Pipe.git

Due to the differences in the build process, the Windows CMake file is named CMakeListsWindows.txt at the root of the SSP repo. Thus, you should:

  • 5.2 Delete CMakeLists.txt

  • 5.3 Rename CMakeListsWindows.txt to CMakeLists.txt.

  • 5.4 Open CMakeLists.txt in VS

  • 5.5 Update

    • the include ("C://Users//Andre//source//repos//vcpkg//installed//x64-windows//include")

    • and link paths ("C://Users//Andre//source//repos//vcpkg//installed//x64-windows//lib")

    • To follow the same path of where you cloned vcpkg in Step 1

5.6 Now we can build Sensor Stream Pipe. In /Sensor-Stream-Pipe run the following commands in terminal:

mkdir build
cd build
cmake .. -DCMAKE_TOOLCHAIN_FILE=/Users/adammac/dev/vcpkg-macos/vcpkg/scripts/buildsystems/vcpkg.cmake -DSSP_WITH_KINECT_SUPPORT=ON -DSSP_WITH_K4A_BODYTRACK=ON -DSSP_WITH_NVPIPE_SUPPORT=ON
make

NvPipe Information

NvPipe is deprecated so we will use an old version of the NVIDIA Video Codec SDK. An alternative will be to use the GPU integration in ffmpeg.

git clone
cd NvPipe
mkdir build && cd build
cmake -G "Visual Studio 15 2017 Win64" \
    -DNV_VIDEO_CODEC_SDK=c:/local/Video_Codec_SDK_9.1.23/ \
    -DCMAKE_INSTALL_PREFIX=c:/local/nvpipe \
    -DNVPIPE_BUILD_EXAMPLES=ON ..
cmake --build . --config Release
cmake --build . --config Release --target install

You will need to patch CMakeLists.txtDiff

index c0cf223..c0b85a9 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -58,6 +58,7 @@ configure_file(src/NvPipe.h.in include/NvPipe.h @ONLY)
 include_directories(${CMAKE_CURRENT_BINARY_DIR}/include)

 # NvPipe shared library
+include_directories(${NV_VIDEO_CODEC_SDK}/Interface)
 list(APPEND NVPIPE_SOURCES
     src/NvPipe.cu
     ${NV_VIDEO_CODEC_SDK}/Samples/Utils/ColorSpace.cu
@@ -66,7 +67,7 @@ list(APPEND NVPIPE_LIBRARIES
     ${CMAKE_DL_LIBS}
     ${CUDA_LIBRARIES}
     ${CUDA_LIB}
-    nvidia-encode
+    nvencodeapi
     )

 if (NVPIPE_WITH_ENCODER)

You could then compile Sensor-Stream-Pipe with NvPipe supportBash

cmake -G "Visual Studio 15 2017 Win64" \
  -DNvPipe_DIR=c:/local/nvpipe/share/NvPipe/cmake \
  ..

PreviousInstallation LinuxNextInstallation Mac

Last updated 3 years ago

Was this helpful?

Reach out on and we will get you going!

Download here:

Download here:

Follow the instructions here:

Info here:

Download link:

2.0 Install Pre-Requisites (optional, if you want to stream an OAK-D using Xlink and run inference using OpenVINO)

Now checkout or to see how you can stream local files with Sensor Stream Server to Sensor Stream Client.

1.1 Go to scroll down and download "".

1.3 Download and install .

2.1 Follow vcpkg installation instructions available .

4.1 Install

4.4 If you do not have them already installed, install for VS Code

4.5 (x64; Debug or Release) and (this will create an output folder usually called /build)

4.9 Install Azure Body Tracker SDK from the instructions available .

Now checkout or to see how you can stream local files with Sensor Stream Server to Sensor Stream Client.

Install in c:/localBash

our discord
https://developer.nvidia.com/cuda-10.0-download-archive?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exelocal
https://developer.nvidia.com/compute/machine-learning/cudnn/secure/7.6.5.32/Production/10.0_20191031/cudnn-10.0-windows10-x64-v7.6.5.32.zip
https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/docs/usage.md
https://docs.microsoft.com/en-us/azure/kinect-dk/body-sdk-download
https://www.microsoft.com/en-us/download/details.aspx?id=102901
OAK-D
https://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_windows.html
Streaming a Video
Streaming a Dataset
https://visualstudio.microsoft.com/downloads/
Build Tools for Visual Studio
Git for windows
here
cmake
CMake Tools
Configure the project using CMake Tools
build
here
Streaming a Video
Streaming a Dataset
NVIDIA Video Codec SDK 9.1.23
Python Release Python 3.8.10Python.org
Logo