SeasonDepth: Cross-Season Monocular Depth Prediction Dataset and Benchmark under Multiple Environments

Authors: Hanjiang Hu, Baoquan Yang, Zhijian Qiao, Ding Zhao, Hesheng Wang
Affiliations: Shanghai Jiao Tong University, Carnegie Mellon University


Update Log

  • Aug. 28th, 2021 : SeasonDepth training set v1.1 and the fine-tuned models released, including 3 new slices for model fine-tuning.
  • Jul. 20th, 2021 : SeasonDepth training set v1 and the fine-tuned models released.
  • Jun. 11th, 2021 : SeasonDepth website and benchmark toolkit released.
  • Jun. 5th, 2021: SeasonDepth test set released.

  • Introduction

    SeasonDepth is a monocular depth prediction dataset that contains multi-traverse outdoor images from changing environments. It is the first depth prediction dataset with multi-environment road scenes to benchmark the depth prediction performance under different environmental conditions. See our work for more details and check out the toolkit repo for SeasonDepth benchmark. The dataset is built through structure from motion based on CMU Visual Localization and CMU-Seasons dataset.





    SeasonDepth Dataset

    Dataset Download

    We have released the test splits of SeasonDepth for zero-shot evaluation, including RGB images and depth maps under twelve different environments. The released dataset is placed in long-term preserved figshare with persistent identifier Digital Object Identifier (DOI) of 10.6084/m9.figshare.14731323. The test set is available HERE. The training set of v1.1 has a DOI of 10.6084/m9.figshare.16442025 and is available HERE together with the fine-tuned models on that, feel free to follow BTS and SfMLearner repos to fine-tune or evaluate them for benchmark. The detailed format and structure of test and training set can be found HERE.

    Dataset Statistics

    The distributions of relative depth value for all the environments are shown below after mean quartile alignment.

    Dataset Details

    The detailed information about partitioning, structure layout and image format can be found HERE.


    SeasonDepth Benchmark

    Baseline Evaluation Results

    Here we present the baseline performance of monocular depth estimation on the SeasonDepth dataset. Please refer to our paper for more details.

    Benchmark Toolkit

    Please visit our official repository for SeasonDepth benchmark toolkit.


    License

    The SeasonDepth dataset, the toolkit code and fine-tuned models are under the license of BY-NC-SA 4.0.


    Citation

    If you use the SeasonDepth dataset please cite all of the three references:

    @article{SeasonDepth,
      title={SeasonDepth: Cross-Season Monocular Depth Prediction Dataset and Benchmark under Multiple Environments},
      author={Hu, Hanjiang and Yang, Baoquan and Qiao, Zhijian and Zhao, Ding and Wang, Hesheng},
      journal={arXiv preprint arXiv:2011.04408},
      year={2021}
    }
    
    @inproceedings{Sattler2018CVPR,
      author={Sattler, Torsten and Maddern, Will and Toft, Carl and Torii, Akihiko and Hammarstrand, Lars and Stenborg, Erik and Safari, Daniel and Okutomi, Masatoshi and Pollefeys, Marc and Sivic, Josef and Kahl, Fredrik and Pajdla, Tomas},
      title={{Benchmarking 6DOF Outdoor Visual Localization in Changing Conditions}},
      booktitle={Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2018},
    }
    
    @inproceedings{badino2011visual,
      title={Visual topometric localization},
      author={Badino, Hern{\'a}n and Huber, Daniel and Kanade, Takeo},
      booktitle={2011 IEEE Intelligent Vehicles Symposium (IV)},
      pages={794--799},
      year={2011},
      organization={IEEE}
    }
    
    


    Contact

    If you have any questions, please contact us through seasondepth@outlook.com.





    Last updated: Aug. 28th, 2021