Package: pandora-plugin-mccnn
Pandora plugin mccnnMC-CNN neural network plugin for.Overview•Install•Usage•Related•ReferencesOverviewPandorastereo matching framework is designed to provide some state of the art stereo algorithms and to add others one as plugins.ThisPandora pluginaims to compute the cost volume using the similarity measure produced by MC-CNN neural network[MCCNN], with theMCCNNlibrary .Installpandora_plugin_mccnnis available on Pypi and can be installed by:pipinstallpandora_plugin_mccnnThis command will installed required dependencies asPandoraandMCCNN.UsageLet's refer toPandora's readmeoronline documentationfor further information about Pandora general functionalities.More specifically, you can find :MCCNN configuration file exampledocumentation about MCCNN theory and parametersPretrained Weights for MCCNN networksDownload weights filesPretrained weights for mc-cnn fast and mc-cnn accurate neural networks are available in the weights directory :mc_cnn_fast_mb_weights.pt and mc_cnn_accurate_mb_weights.pt are the weights of the pretrained networks on the Middlebury dataset[Middlebury]mc_cnn_fast_data_fusion_contest.pt and mc_cnn_accurate_data_fusion_contest.pt are the weights of the pretrained networks on the Data Fusion Contest dataset[DFC]To download the pretrained weights:wgethttps://raw.githubusercontent.com/CNES/Pandora_MCCNN/master/mc_cnn/weights/mc_cnn_fast_mb_weights.pt
wgethttps://raw.githubusercontent.com/CNES/Pandora_MCCNN/master/mc_cnn/weights/mc_cnn_fast_data_fusion_contest.pt
wgethttps://raw.githubusercontent.com/CNES/Pandora_MCCNN/master/mc_cnn/weights/mc_cnn_accurate_mb_weights.pt
wgethttps://raw.githubusercontent.com/CNES/Pandora_MCCNN/master/mc_cnn/weights/mc_cnn_accurate_data_fusion_contest.ptAccess weights from pip packagePretrained weights are stored into the pip package and downloaded for any installation of mc_cnn pip package.
To access it, use theweightssubmodule :frommc_cnn.weightsimportget_weightsmc_cnn_fast_mb_weights_path=get_weights(arch="fast",training_dataset="middlebury")mc_cnn_fast_data_fusion_contest_path=get_weights(arch="fast",training_dataset="dfc")mc_cnn_accurate_mb_weights_path=get_weights(arch="accurate",training_dataset="middlebury")mc_cnn_accurate_data_fusion_contest=get_weights(arch="accurate",training_dataset="dfc")Output exampleThe figures below show disparity maps produced on mountain, and desert areas generated with the Census and MCCNN similarity measures :Left imageLeft disparity map using Census measureLeft disparity map using mc-cnn fast pretrained on MiddleburyLeft disparity map using mc-cnn fast pretrained DFCRelatedPandora- A stereo matching frameworkMCCNN- Pytorch/python implementation of mc-cnn neural networkReferencesPlease cite the following paper when using Pandora and pandora_plugin_mccnn:Defonte, V., Dumas, L., Cournet, M., & Sarrazin, E. (2021, July). Evaluation of MC-CNN Based Stereo Matching Pipeline for the CO3D Earth Observation Program. In 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS (pp. 7670-7673). IEEE.Cournet, M., Sarrazin, E., Dumas, L., Michel, J., Guinet, J., Youssefi, D., Defonte, V., Fardet, Q., 2020. Ground-truth generation and disparity estimation for optical satellite imagery. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.[MCCNN]Zbontar, J., & LeCun, Y. (2016). Stereo matching by training a convolutional neural network to compare image patches. J. Mach. Learn. Res., 17(1), 2287-2318.[Middlebury]Scharstein, D., Hirschmüller, H., Kitajima, Y., Krathwohl, G., Nešić, N., Wang, X., & Westling, P. (2014, September). High-resolution stereo datasets with subpixel-accurate ground truth. In German conference on pattern recognition (pp. 31-42). Springer, Cham.[DFC]Bosch, M., Foster, K., Christie, G., Wang, S., Hager, G. D., & Brown, M. (2019, January). Semantic stereo for incidental satellite images. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 1524-1532). IEEE.