TITLE: Structure-from-motion multi-view photogrammetry applied to linear-scan sediment cores images
AUTHORS: Kevin Jacq, William Rapuc, Claire Blanchet, Estelle Ployon, Cecile Pignol, Didier Coquin, Bernard Fanget

[DESCRIPTION]
<div class = "text-blocks"><div class = "text-block">Image acquisition is the first step to save primary color information for environmental samples before disturbing them with other, mainly destructive analyses. The improvement of RGB cameras and image processing algorithms allows to obtain a metrically calibrated image at high resolution called ortho-image. The way to obtain this image requires the processing of several raw images acquired along the sample. We propose a semi-automatic method that uses metrically calibrated targets to create the ortho-image with Agisoft Photoscan or Metashape software.</div><div class = "text-block">HAL Id : hal-03257581, version 1</div><div class = "text-block">DOI : 10.1007/s10933-021-00204-x</div></div>

[STEPS]
?. [General description of the acquisition bench]
?. [General description of the acquisition bench]
?. [Camera acquisition parameters and calibration]
It is important that the parameters related to the camera (shutter speed, aperture, etc.) are fixed throughout the acquisition. The autofocus must also be disabled.
?. [Camera acquisition parameters and calibration]
With the chosen acquisition parameters, the camera must be calibrated in two steps:
?. [Camera acquisition parameters and calibration]
The first is to perform a white balance, in order to obtain relevant colors according to the lighting, this is called radiometric correction.
?. [Camera acquisition parameters and calibration]
The second consists in taking pictures of a checkerboard from several angles and over the entire surface acquired by the camera. This allows to correct the images for geometric distortions induced by the lens that imply that the pixels are not square.The Agisoft Lens software uses these checkerboard images to estimate the corrections to be applied to the images.
?. [Recommendations on coded markers and their georeferencing]
The method described in this protocol relies on markers placed on rulers along the sample to create a planar spatial reference (z=0).The Agisoft Photoscan or Metashape software proposes a list of markers all different which will be automatically recognized by the software. These markers must be placed in a regular way so as to be present on three successive images to allow a good registration of the images and at a size not exceeding 30 pixels for the central circle. In our case, the markers are 10 cm apart and have a diameter of 1 cm.
Markers should be printed on a material that does not get damaged and remains clean from sediment and water. They should also not be plastified as this will reflect light, and the algorithm will be distorted. For these reasons, we decided to use a mat adhesive paper.
These targets should be cut in squares and not in circles, keeping in mind that the targets are circles on a white background.
If a marker is damaged, it will probably be necessary to replace it with a new one that must be different from the others.
?. [Recommendations on coded markers and their georeferencing]
Each marker, defined by the software, is numbered and must be georeferenced to constrain the models with this landmark created with the rules and markers.The georeferencing is performed by measuring the position of the marker's center relative to a point defined as the origin. In our case, the origin was chosen as one of the corners of the sedimentary core frame by the rulers.The marker numbers and their positions are recorded in a csv file which will be used by the software.
?. [Image acquisition]
The acquisition of the images must be carried out to have sufficient overlap between the successive images. In our case, we have chosen an overlap of 75% so that each point of the sample is present on 4 images.
?. [Creation of the orthoimage]
1) Importing imagesSelect the images with the "Add Photos" icon in the Workspace tab or in Workflow > Add Photos...
?. [Creation of the orthoimage]
2) Corrections with camera settingsNow that the images have been loaded, they need to be corrected with the camera settings. You have to import them by going to Tools > Camera Calibration...Then load the parameters with the "load" icon. Set the calibration to be applied to all images by checking the box.
?. [Creation of the orthoimage]
3) Marker detectionTwo methods can be used to detect and georeference the markers. One automatic and one semi-automatic.
?. [Creation of the orthoimage]
Automatic method:A Python script has been developed to detect the markers and retrieve their positions in the csv file previously created in step 6.It can be found on github. Load the script that will automatically detect the markers and associate them with their coordinates (x, y, z). To do this go to Tools > Run Script...
It is possible that this script must be corrected, because in the case of a change of version of the Agisoft software, the command tags may be different.
?. [Creation of the orthoimage]
Semi-automatic method:Markers can be detected in Tools > Markers > Detect markers... Then define the markers to be detected according to those that have been printed and placed on the rulers. After the markers have been detected, it is necessary to manually enter their positions in x,y, and z in meters.
?. [Creation of the orthoimage]
Now that everything has been calibrated and the markers defined, it is necessary to follow the order of the Workflow for the creation of the orthoimage.The different stages of the model are visible in the Workspace tab (bottom left) by unrolling Chunk:
?. [Creation of the orthoimage]
4) Registration and assembly
?. [Creation of the orthoimage]
The first step is the alignment of the images which estimates the analog pixels between them, the positions of the cameras and creates tie points that correspond to the connection points between the images.Go to Workflow > Align Photos... then select settings.This step is the most important. It is important to check that, following these calculations, the reconstruction is horizontal. If a curved shape appear, it is a well-known “dome or arc effect” due to the miss-estimations of pixel positions in x, y, and z typical of linear image acquisition.To do this, use the radius of the sphere to rotate the model, by staying pressed with the left click, you can rotate it. There may be some points that are not correct, they will not prevent the continuation of the steps. The main thing is that the model is horizontal.The reconstruction error of the marker positions is also estimated, allowing to estimate areas where new control points have to be added manually to optimize the model. To add control points, select locations on images with right click > Create Marker and right click > Place Marker for the following images to associate a marker to multiple images. It may be necessary to de-align the images, before aligning them with the new control points.
?. [Creation of the orthoimage]
The second step is the dense cloud that estimates a reconstruction of the depth of each photo and determines its coordinates with the multiview stereo correspondence and the previous estimate of the image geometry.To calculate the dense cloud, go to Workflow > Build Dense Cloud.
?. [Creation of the orthoimage]
5) Mesh and orthorectification:
?. [Creation of the orthoimage]
The algorithm then uses the points of the dense cloud to generate a closed surface with triangular surfaces, this step is called meshing.It can be made with Workflow > Build Mesh...Look at the result of the 3D model. In the case of a very dark image it means that the model is upside down, otherwise the model is in the right direction. This is to be taken into account in the next step.
?. [Creation of the orthoimage]
Finally, the orthorectified image is estimated from the projection of the colored mesh in a 2D space to correct perspective image and projection distortions.To perform this step, go to Workflow > Build Orthomosaic...If the model is in the right direction then select the default settings, otherwise change Top XY to Bottom XY.Finally it is possible to export the orthophoto with File > Export Orthomosaic > Export JPEG/TIFF/png...