Installation Windows

We have 2 methods for installing on Windows. In one instance, we lean heavily on pre-built binaries being packaged with SSP (recommended). In the second method we manually install all necessary dependencies.

These steps have been tested on Windows 10 Build 19041, Visual Studio 2019 Community Edition (VS) and Visual Studio Code 1.53

1. Pre-built binaries (recommended)

   • Dependencies are prebuilt and stored on a server and downloaded by cmake using FetchContent feature

   • To rebuild dependencies:

     cd Sensor-Stream-Pipe/3rdparty
     ./buid_win_dep.sh

   • This will download/build/install dependencies in a tmp directory. The result is a *_windep.tar.gz file.

     • download prebuilt ffmpeg 4.3.2

     • OpenCV 3.4.13 as a static library, only core, imgproc, imgcodecs and highgui modules are built

     • Cereal 1.3.0, header only

     • spdlog 1.8.2, header only but built as static library for faster compile

     • Zdepth (commit 9b333d9aec520 which includes a patch to generate zdepthConfig.cmake)

     • yaml-cpp 0.6.3 as a static library

     • libzmq 4.3.4 as a static library

     • cppzmq 4.7.1, header only

2. Installing Manually

   • This will provide download instructions for each library

Problems?! (shocker)

Reach out on our discord and we will get you going!

Installing with Pre-Built Binaries (Recommended)

1.0 Install Azure Kinect Pre-Requisites (optional, if you want to use Azure Kinect, or Azure Kinect Body Tracking SDK)

1.1 Manually install CUDA Toolkit 10.0 (needed for Body Tracking SDK)

Download here: https://developer.nvidia.com/cuda-10.0-download-archive?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exelocal

1.2 Manually install cudNN64_70 for CUDA 10.0 (needed for Body Tracking SDK)

Download here: https://developer.nvidia.com/compute/machine-learning/cudnn/secure/7.6.5.32/Production/10.0_20191031/cudnn-10.0-windows10-x64-v7.6.5.32.zip

You will need to accept some NVIDIA agreements

1.3 Manually Install Azure Kinect 1.4.1

Follow the instructions here: https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/docs/usage.md

1.4 Manual Install Azure Body Tracking SDK 1.1

Info here: https://docs.microsoft.com/en-us/azure/kinect-dk/body-sdk-download

Download link: https://www.microsoft.com/en-us/download/details.aspx?id=102901

2.0 Install OAK-D Pre-Requisites (optional, if you want to stream an OAK-D using Xlink and run inference using OpenVINO)

2.1 Install python3 if you do not have it (necessary for OAK-D)

Python Release Python 3.8.10Python.org

Ensure that python is added as a path variable

2.2 Remove max path limit

git config --system core.longpaths true

2.3 Install OpenVINO (used to run inference on frames grabbed from OAK-D device)

https://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_windows.html

By default, the Intel® Distribution of OpenVINO™ is installed to the following directory, referred to as <INSTALL_DIR> elsewhere in the documentation: C:\Program Files (x86)\Intel\openvino_<version>. For simplicity, a shortcut to the latest installation is also created: C:\Program Files (x86)\Intel\openvino_2021

3.0 Build/Install Sensor Stream Pipe

Note: If planning on using Azure Kinect, Azure Kinect Body Tracking SDK, or Xlink and OpenVINO, they must be installed before SSP as SSP looks for the libraries when creating the Release

git clone https://github.com/moetsi/Sensor-Stream-Pipe.git
cd Sensor-Stream-Pipe
mkdir build && cd build
cmake -G "Visual Studio 15 2017 Win64" ..
cmake --build . --config Release

NOTE: You must enable Kinect or Xlink functionality when building with the following flags:

-DK4A_ENABLED=TRUE (for Kinect)

-DXLINK_ENABLED=TRUE (for OAK-D)

Example:

cmake -DK4A_ENABLED=TRUE -G "Visual Studio 15 2017 Win64" ..

or

cmake -DXLINK_ENABLED=TRUE -G "Visual Studio 15 2017 Win64" ..

and you will need to set the env variabels for OpenVINO so you will need to run "'C:/Program Files (x86)/Intel/openvino_2021/bin/setupvars.bat'" or where ever setupvars.bat is installed

the flags are cached, so if you want to disable you will need to set them to false

3.0 Use it!

3.1 Add Body Tracking Components (optional)

Move: - dnn_model_2_0.onnx (from body tracking sdk) and - cudNN64_7 (from step 1.2)

to /Release (or where you run the executable) if you want to run Body Tracking on Sensor Stream Client

3.2 Give it a go

Now checkout Streaming a Video or Streaming a Dataset to see how you can stream local files with Sensor Stream Server to Sensor Stream Client.

Installing Manually

1.0 Make sure you have Build Tools, Git, and CUDA

1.1 Go to https://visualstudio.microsoft.com/downloads/ scroll down and download "Build Tools for Visual Studio".

1.2 Run "Build Tools for Visual Studio", choose "Language packs" and add "English". Tested with "Visual Studio Build Tools 2019 16.8.5.

1.3 Download and install Git for windows.

1.4 Install CUDA 10 (if you would like to use Azure Kinect Body Tracking)

1.5 Install cuDNN 7.0 (if you would like to use Azure Kinect Body Tracking)

2.0 Install vcpkg

2.1 Follow vcpkg installation instructions available here.

2.2 In Command window, change directories to the vcpkg root directory. Run vcpkg integrate install .

This should respond with something similar to:

C:\Users\adamm\dev\vcpkg>vcpkg integrate install
Applied user-wide integration for this vcpkg root.
All MSBuild C++ projects can now #include any installed libraries.
Linking will be handled automatically.
Installing new libraries will make them instantly available.
CMake projects should use: "-DCMAKE_TOOLCHAIN_FILE=C:/Users/adamm/dev/vcpkg/scripts/buildsystems/vcpkg.cmake"

3.0 Install dependencies available on vcpkg

3.1 Install dependencies using vcpkg (in vcpkg root directory).

vcpkg install azure-kinect-sensor-sdk:x64-windows cereal:x64-windows cppzmq:x64-windows ffmpeg:x64-windows opencv3:x64-windows spdlog:x64-windows yaml-cpp:x64-windows zeromq:x64-windows

4.0 Build and install remaining dependencies

4.1 Install cmake

Zdepth

Zdepth is an encoder for depth frames. This library can be used to compress depth data.

4.2 Clone the Zdepth repo to your home directory (or where ever you usually save repos)

git clone https://github.com/catid/Zdepth.git

4.3 Open CMakeLists.txt in Visual Studio Code

4.4 If you do not have them already installed, install CMake Tools for VS Code

4.5 Configure the project using CMake Tools (x64; Debug or Release) and build (this will create an output folder usually called /build)

4.6 Prepare a directory in your home directory (or where ever you usually save repos) to place the remaining dependencies, /libs (referred henceforth as $LIBS). Create subfolders /include and /lib within /libs

mkdir libs
cd libs
mkdir lib
mkdir include

4.7 Copy the contents of Zdepth\include into $LIBS\include and output lib folders (e.g. ZDepth\out\*) to $LIBS\lib.

Azure Kinect Body Tracking SDK (optional)

4.9 Install Azure Body Tracker SDK from the instructions available here.

4.10 Copy the SDK include and lib files from the SDK install list to $LIBS, or add the SDK path to SSP CMakeLists (see below)

• Example

  • C:\Program Files\Azure Kinect Body Tracking SDK\sdk\include

    • Into $LIBS\include

  • C:\Program Files\Azure Kinect Body Tracking SDK\sdk\netstandard2.0 and C:\Program Files\Azure Kinect Body Tracking SDK\sdk\windows-desktop

    • Into $LIBS\lib

5.0 Building Sensor Stream Pipe

5.1 Clone the SSP repo to your home directory (or where ever you usually save repos)

git clone [e-mail]:moetsi/Sensor-Stream-Pipe.git

Due to the differences in the build process, the Windows CMake file is named CMakeListsWindows.txt at the root of the SSP repo. Thus, you should:

• 5.2 Delete CMakeLists.txt

• 5.3 Rename CMakeListsWindows.txt to CMakeLists.txt.

• 5.4 Open CMakeLists.txt in VS

• 5.5 Update

  • the include ("C://Users//Andre//source//repos//vcpkg//installed//x64-windows//include")

  • and link paths ("C://Users//Andre//source//repos//vcpkg//installed//x64-windows//lib")

  • To follow the same path of where you cloned vcpkg in Step 1

5.6 Now we can build Sensor Stream Pipe. In /Sensor-Stream-Pipe run the following commands in terminal:

mkdir build
cd build
cmake .. -DCMAKE_TOOLCHAIN_FILE=/Users/adammac/dev/vcpkg-macos/vcpkg/scripts/buildsystems/vcpkg.cmake -DSSP_WITH_KINECT_SUPPORT=ON -DSSP_WITH_K4A_BODYTRACK=ON -DSSP_WITH_NVPIPE_SUPPORT=ON
make

Now checkout Streaming a Video or Streaming a Dataset to see how you can stream local files with Sensor Stream Server to Sensor Stream Client.

NvPipe Information

NvPipe is deprecated so we will use an old version of the NVIDIA Video Codec SDK. An alternative will be to use the GPU integration in ffmpeg.

Install NVIDIA Video Codec SDK 9.1.23 in c:/localBash

git clone
cd NvPipe
mkdir build && cd build
cmake -G "Visual Studio 15 2017 Win64" \
    -DNV_VIDEO_CODEC_SDK=c:/local/Video_Codec_SDK_9.1.23/ \
    -DCMAKE_INSTALL_PREFIX=c:/local/nvpipe \
    -DNVPIPE_BUILD_EXAMPLES=ON ..
cmake --build . --config Release
cmake --build . --config Release --target install

You will need to patch CMakeLists.txtDiff

index c0cf223..c0b85a9 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -58,6 +58,7 @@ configure_file(src/NvPipe.h.in include/NvPipe.h @ONLY)
 include_directories(${CMAKE_CURRENT_BINARY_DIR}/include)

 # NvPipe shared library
+include_directories(${NV_VIDEO_CODEC_SDK}/Interface)
 list(APPEND NVPIPE_SOURCES
     src/NvPipe.cu
     ${NV_VIDEO_CODEC_SDK}/Samples/Utils/ColorSpace.cu
@@ -66,7 +67,7 @@ list(APPEND NVPIPE_LIBRARIES
     ${CMAKE_DL_LIBS}
     ${CUDA_LIBRARIES}
     ${CUDA_LIB}
-    nvidia-encode
+    nvencodeapi
     )

 if (NVPIPE_WITH_ENCODER)

You could then compile Sensor-Stream-Pipe with NvPipe supportBash

cmake -G "Visual Studio 15 2017 Win64" \
  -DNvPipe_DIR=c:/local/nvpipe/share/NvPipe/cmake \
  ..