Package: colorply
colorplyA multispectral photogrammetry applicationOverviewColorplyis an open source python application which add new wavelength channels to a 3D cloud of points from a set of referenced images and uses images calibration from MicMac.
This package comes with aGUI, to make tasks easier if you are not used toMicMac.Multispectral Cloud of PointsThese models are models oriented in the same relative coordinate system. However, these models
are composed of only one channel.Colorplymerges any channels from any models, as long as they are
in the same coordinate system.AboutThis project alone was made in a week, in a bigger project of photogrammetry engineering. The whole project focused in testing and evaluating a multispectral camera, theParot Sequoia. Then, a major part used this camera to generate multispectral 3D cloud of points for remote sensing purposes like vegetation classification, all in 3D. BecauseMicMacworks only for RGB (or maximum 3-channels images),Colorplywas created and handles as many channels as you want and complete an existing 3D points from multispectral images.DependenciesThis package depends on :PyQt5lxmlplyfilenumpyPyQt5 is only used fot theGUIof this application. As it can be a bit tricky to understand the file and directory names generated byMicMac, we decided to create an interface for it. It is not obligatory to use this interface, you can freely use and modify the python modules.InstallationTo install this package, clone this repository and from the command line in the root folder, use :pip install .This will install all the dependencies and addColorplyto your python environment, usually saved inpath_to_anaconda\lib\site-packages.UsageMicMacColorplyworks hands in hands withMicMacwhich is an open source photogrammetric software. You can download it from GitHub athttps://github.com/micmacIGN/micmac, or from the main page athttps://micmac.ensg.eu/index.php/Install.
If you are new toMicMac, take a look a the documentationhttps://github.com/micmacIGN/Documentation/blob/master/DocMicMac.pdfand this active forum :http://forum-micmac.forumprod.com/.ColorplyHow it looksThe interface is simple, but all options are linked !Colorply GUITo use this package, run main.py orfromcolorply.uiimportinterfaceinterface()If you prefer doing it manually, you can use the functional implementation:fromcolorplyimportadd_cloud_channel# Load the 3D modelinput_ply="test/data/result/RVB_GRE.ply"# Load MicMac calibration filescalibration_file="test/data/calibration/Ori-1bande_All_CampariGCP/AutoCal_Foc-4000_Cam-SequoiaSequoia-GRE.xml"# Load the orientation files for all images in the sceneorientation_dir="test/data/calibration/Ori-1bande_All_CampariGCP"# Load the images (corresponding to the orientation files)image_dir="test/data/images/RED"# Additional argumentsimage_ext="TIF"channel="RED"# Resulting plyoutput_ply="output.ply"# Project all points in `input_ply` to `images_dir`.# Create a new ply, the operation is not inplace.add_cloud_channel(input_ply,output_ply,calibration_file,orientation_dir,image_dir,image_ext,channel)You can also use the commandlines system$ python colorply   --inply path/to/input_ply--outply path/to/output_ply--calib path/to/calib_xml--oridir path/to/orientation_folder--imdir path/to/image_folder--imext image_extension--channel name_of_new_channel--mode mode_used_to_merge_new_radiometryUse help --argument for additional information.How it worksThe main algorithm is based on the so calledimage formula. This equation links 3D coordinates pointsMto image coordinates pointsm, and depends on the camera calibration (the rotationR, the distorsion and PPSS, focal lengthF).To visualize this formula, let's consider this short animation :The red crosses represent the dense cloud points, computed from the RED channel. These points are projected in NIR channel images. The radiometry at this location is then affected back to the points, adding an extra channel to the cloud.ApplicationAboutThis project was used for remote sensing classification from a multispectral camera. The cameraParrot Sequoiawas first calibrated and tested on sample areas. Then, it was fixed to a civil drone and we flew over high altitudes forest to estimate the evolution of vegetation species.Colorplywas used to create a multispectral cloud of points, to improve our classification by adding extra features. The clusters were made from a random forest skeleton, using all radiometries (RED, REG, NIR, GRE) and the points 3D positions.ClassificationIn this repository, you can run the classification on the provided datahere.
For this short example (for fast processing), the classification results are described as follows :The confusion matrix for this sample is :TerrainOakShrubGrassTerrain4100016Oak0260100Shrub01013716Grass23011192Global accuracy : 92.07%.LicenceThe code is under the MIT licence