Streaming a Dataset

We have made available a few seminal computer vision/spatial computing datasets to be used with Sensor Stream Pipe.
If you have a spatial computing/computer vision pipeline that you would like to compare to the pipelines of these published papers, you can stream the data to both pipelines with Sensor Stream Pipe.
We have provided scripts that are able to take specific published datasets and generate a .txt file that can be ingested by Sensor Stream Server to stream those datasets to Sensor Stream Client.
Our currently supported datasets:
​BundleFusion - Real-time Globally Consistent 3D Reconstruction using Online Surface Re-integration
​Microsoft - Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images
​NYU Depth Dataset V2 - Indoor Segmentation and Support Inference from RGBD Images
Follow the instructions below to stream a dataset and ingest the output with Sensor Stream Client, which can be used as a component of your own spatial computing pipeline.
1.0 Download a dataset and create the frame list file
1.1 Check out our Sensor Stream Pipe Data Scripts Repo and follow the instructions to create a frame list file for the dataset of your choice.
The frame list files are used by Sensor Stream Server's config file as a way to point as the frame source. The example/ folder includes an example frame list files.
After extracting the paths of your created frame list files, change the paths on line 8 and 9 in configs/serve_frames_color_depth.yaml . The config file is currently pointed at example frame list files: examples/stairs-seq-01-frames-color.txt and examples/stairs-seq-01-frames-depth.txt.
2.0 Prepare Sensor Stream Pipe
2.1 Follow installation instructions for Linux/Windows/MacOS to build Sensor Stream Pipe on your development platform.
2.2 Move the downloaded .mkv file to the same folder as the Sensor Stream Server executable (not required but this way you will not need to define the path to the video file in the config yaml in step 2.2)
2.2 Update line 10 (path) in Sensor-Stream-Pipe/configs/serve_video.yaml to point at the downloaded .mkv file (or the path of any mkv file)
If you are using the downloaded .mkv file and placed it in the same folder as the server executable then line 10 would be:
      path: "1080p_ShadesDown_Darker_Closer_Right_Moving_Right.mkv"
3.0 Run Sensor Stream Client and Sensor Stream Server
3.1 Run Sensor Stream Client with OpenCV (this will use OpenCV to visualize incoming streams)
./ssp_client_opencv 9999
Run with whatever port you have set in the config yaml (default is 9999)
3.2 Run Sensor Stream Server
./ssp_server ../../configs/serve_frames_color_depth.yaml
You will need to provide an argument which has a path to the config yaml you want to use. In the example .yaml provided in the configs/ folder we assume you have created a frame list and put it in examples/.

Streaming with Azure Kinect DK

Streaming iOS ARKit RGB-D Data

Last modified 1yr ago

Copy link

Outline

1.0 Download a dataset and create the frame list file

2.0 Prepare Sensor Stream Pipe

3.0 Run Sensor Stream Client and Sensor Stream Server