3D Reconstruction meets Semantics – Reconstruction Challenge


Part of the workshop is a challenge on combining 3D and semantic information in complex scenes. To this end, a challenging outdoor dataset, captured by a robot driving through a semantically-rich garden that contains fine geometric details, will be released. A multi-camera rig is mounted on top of the robot, enabling the use of both stereo and motion stereo information. Precise ground truth for the 3D structure of the garden has been obtained with a laser scanner and accurate pose estimates for the robot are available as well. Ground truth semantic labels and ground truth depth from a laser scan will be used for benchmarking the quality of the 3D reconstructions.

Data & Submission

Given a set of images and their known camera poses, the goal of the challenge is to create a semantically annotated 3D model of the scene. To this end, it will be necessary to compute depth maps for the images and then fuse them together (potentially while incorporating information from the semantics) into a single 3D model.

We provide the following data for the challenge:

  • A set of training sequences consisting of
    • calibrated images with their camera poses,
    • ground truth semantic annotations for a subset of these images,
    • a semantically annotated 3D point cloud depicting the area of the training sequence.
  • A testing sequence consisting of calibrated images with their camera poses.

Both training and testing data are available here. Please see the git repository for details on the file formats.

In order to submit to the challenge, please create a semantically annotated 3D triangle mesh from the test sequence. The mesh should be stored in the PLY text format. The file should store for each triangle a color corresponding to the triangle’s semantic class (see the calibrations/colors.yaml file for the mapping between semantic classes and colors). Once you have created the mesh, please submit it using this link. In addition, please send an email to torsten.sattler@inf.ethz.ch that includes the filename of the file you submitted as well as contact information.

We will evaluate the quality of the 3D meshes based on the completeness of the reconstruction, i.e., how much of the ground truth is covered, the accuracy of the reconstruction, i.e., how accurately the 3D mesh models the scene, and the semantic quality of the mesh, i.e., how close the semantics of the mesh are to the ground truth.

The deadline for submitting to the challenge is September 3rd (23:59 GMT).

For questions, please contact torsten.sattler@inf.ethz.ch.

#TrimBot goes #offroad in the #garden @WUR #robotics #trimming https://t.co/U3TRyLt0kG5 days
The dataset for the #3drms challenge @ICCV2017 is available https://t.co/9vN2s5EZdA7 days
#DeMoN is the very first work that formulates joint egomotion and #depth #estimation as a pure #learning problem https://t.co/nDyV7cmuvA1 week
Download the dataset for the #3drms challenge #3dreconstruction https://t.co/6w9Fm9PTmX1 week
RT @PALRobotics: TIAGo knows how to work a crowd... https://t.co/vjOuJuVxJH #robots #robotics #robocup20171 week