TITLE: High Resolution &#34;DIY&#34; Photogrammetry - &#39;HRP&#39; Protocol
AUTHORS: Yu Tang, Jacopo Niccolo  Cerasoni, Emily Yuko Hallett

[DESCRIPTION]
Photogrammetry is a method of calculating the three-dimensional shape of an object from a set of images. The advantages of Photogrammetry include the ability to record the shape of an object in a short time and with high accuracy without contact. In addition, the generated model can be displayed without textures. Here, the High Resolution Photogrammetry method is presented, which describes the use of photogrammetric techniques to take pictures and generate models. This method aims to give a comprehensive and extensive description for the development of high resolution 3D models, merging the well known techniques used in academic and computer graphic fields, allowing anyone to independently produce high resolution and quantifiable models for any need.

[BEFORE_START]
Note that this protocol is not presenting a novel photogrammetric method. We aimed to create an easy to follow step-by-step protocol which aims to cover the basics for anyone to learn photogrammetry and create high-resolution 3D models independently. For more detailed discussions and guides on academic and computer graphics photogrammetric applications see the following sources:

In English - 
1. Photogrammetric Applications for Cultural Heritage
        https://historicengland.org.uk/images-books/publications/photogrammetric-applications-for-cultural-heritage/
2. Photogrammetry in Paleontology - A practical guide
        https://www.jpaleontologicaltechniques.org/pasta3/JPT%20N12/Pdf/JPT_n012_Jul.pdf
3. A comparison of methods for creating 3D models of obsidian artifacts
        https://www.youtube.com/watch?v=g0YaWDrl5qI&ab_channel=RecordingArchaeology
4. Photogrammetry Workflow - Unity
        https://unity3d.com/files/solutions/photogrammetry/Unity-Photogrammetry-Workflow_2017-07_v2.pdf
5. Full Photogrammetry Guide for 3D Artists
        https://80.lv/articles/full-photogrammetry-guide-for-3d-artists/
6. RealityCapture tutorial: How to create your first 3D model | Part 1-4 (Multilingual Subtitles)
        https://www.youtube.com/watch?v=DTJq1Dz0nxY&ab_channel=CapturingReality
7. The Unbearable Lightness of Mystic "Exposure" Triangle
       https://www.fastrawviewer.com/blog/mystic-exposure-triangle
8. New Old Approach to Dynamic Range
       https://www.fastrawviewer.com/blog/determining-practical-dynamic-range

In Japanese [日本語で] - 
9. 3D Measurement of Lithic Artifacts, Towards Publication and Sharing of Results. [石器の3D計測、成果の公開・共有    を目指して] - Written by Atsushi Noguchi
        http://54.64.145.19/dspace/bitstream/11177/7014/1/data_science_salon01_009_014.pdf
10. 3D Technology and Archaeology [3D技術と考古学] - Book edited by Satoru Nakazono
        https://www.yuzankaku.co.jp/products/detail.php?product_id=8377
        https://www.yuzankaku.co.jp/user_data/special_3d.php *link to the stone tools models presented in the book*

Both academic and computer graphics fields apply photogrammetric processes for different reasons. Each field has its own advantages and disadvantages. For example, heritage preservation specialists rarely discuss the principles of computer graphics (CG), while the CG experts rarely discusses how to optimize the accuracy of scanning. Here we presented both sides into one single overarching method.
 


[GUIDELINES]
In the following protocol, typographical emphases and brackets have been used for software and in-text references.
They are: (1) main sub-steps and softwares in bold, e.g. Add imagery and Adobe Photoshop; (2) "software functions" in "quotation marks", e.g. "Copy correction settings"; (3) <keyboard buttons> in <angle markers>, e.g. <right click> or <Del>.

[STEPS]
SECTION: Setting Photographic Environment
1. Collect and prepare equipment.

 
 

 

SECTION: Photography
12. Place the object on the turntable or holder, make sure it is centered on the turntable. The whole object should always be within the frame of the camera. 

Press the shutter by remote control and rotate the turntable. Repeat this operation until it rotates one circle. The angle of rotation can be smaller depending on the details of the subject. Ideally, a minimum of 18 times per rotation is preferred.

 
 
 

SECTION: Setting Photographic Environment
2. Set the camera on a tripod.

Choose a solid color for the background if possible (Black or white is recommended).
 

SECTION: Setting Photographic Environment
3. Set the turntable on the table and set the items you want to scan on it, placing at central as possible.
 

SECTION: Photography
11. If the object can be rotated and then reset on the turntable (as per a spheric object), rotate the object slightly at each rotation and repeat for one cycle. Repeat rolling and photographing until the object has been photographed from all 360 degrees.This usually requires an object that can rotate or supplemented by a stand that can support it.
 

SECTION: Importing Photographs
14. Open DxO Photolab (image processing software). You can also choose Adobe Photoshop, Adobe Lightroom or Capture One.
 

SECTION: Final Model
36.  
SECTION: Final Model
39. Below, you can find the downloadable files of the final model:
 
Model in obj. format ->  
Model texture in .tiff format ->   

SECTION: Final Model
38. Final 3D model with texture:
 

Final 3D model without texture:
 

SECTION: Setting Lighting Environment
4. If using a turntable, lighting for photogrammetry does not need to be completely uniform (360 degrees), but you need to keep the front lighting as uniform as possible. Diffused lighting is also recommended.

Ring lights are a good choice, but you can also choose 2 or more LED lights set on the left and right sides of the camera.
 

  
  

SECTION: Photography
11.1. If the object can't be rotated properly, change the height and position of the camera so that the camera looks up at the object and take photographs in a circular motion. Repeat this action and finally flip the object up and down to take the bottom half.
SECTION: Camera Settings
5. RAW format (CR2, CRW, NEF, PEF...) is usually preferred For photogrammetry, this will provide a wider color gamut and more correct color calibration in post-processing.
 

SECTION: Camera Settings
6. Manual mode is usually selected because we want to have no difference in brightness during photography.
 

Manual mode is always suggested to use for photogrammetric purposes.
SECTION: Camera Settings
7. Aperture.

Regular Photogrammetry usually selects an aperture between F/5.6 and F/16. Different aperture sizes depend on the focal length and the size of the subject. Too small aperture will cause diffraction and reduce the quality of the photo, while too big aperture will reduce the depth of field.


For small items, aperture of around F/16 should be selected in most cases.

 

SECTION: Camera Settings
8. Exposure. Setting exposure time and ISO

First set ISO to an optional value such as 200. Call up the histogram in the camera and try to shoot the object. Adjust the exposure time to the appropriate time so that the vertical part of the histogram representing the subject is in the middle and the highlighted part of the histogram.
 

 

SECTION: Camera Settings
9. Set white balance.

You can set an arbitrary white balance when taking pictures and adjust it in post-processing. If possible, use a gray card or color checker and adjust the white balance based on it in post-processing.
SECTION: Camera Settings
8.1. Advanced methods of measuring exposure.

As the in camera histogram shows a preview image in JPG format, exposure should be directly evaluted on the RAW format photograph.

After taking a photograph, decoduing of the RAW photo to check the photograph's exposure is required.

To do so, Rawdigger can be used. To open a RAW photo on Rawfigger, hold down SHIFT to frame the object. Select
 on selection at the top left to select the histogram.

 
The following exposure method was developed following enhancement of the methods explicated in sources 7 and 8 (before start section).
 

 

SECTION: Camera Settings
10. For more detailed explanations on aperture, depth of field, exposure and white balance see - Cerasoni & do Nascimento Rodrigues (2021). "Small Object and Artefact Photography - 'SOAP' Protocol" - dx.doi.org/10.17504/protocols.io.buz8nx9w
SECTION: Photography
11.2. When changing the height of the camera, it can also be moved closer to the obhect. This will make the distance between the camera and the object consistent.
SECTION: Photography
11.3. It is recommended to include a scale in the photographs of the selected object. Ideally, the scale should be fully visible in at least 4 or 5 of the photos so to aid in the allignment of all the photographs.
SECTION: Photography
13. General photography methods and ideas.

Because the basic way of photogrammetry's three-dimensional reconstruction is parallax, we need to take at least 2 photos of each surface of the subject. Therefore, before beginning the process, consider the following when photographing:

1. Imagine a regular polyhedron (e.g., a regular dodecahedron) and place the subject in its center. Subdivide the polyhedron so that each vertex of the subdivision will be the ideal camera position, with each camera facing the center.

2. Imagine a simplified object, place the camera vertically facing all the simplified faces, and all the edges for the shot.

  

SECTION: Importing Photographs
15. In the Photo Library, select the folder where the Raw format images are located. Double-click on any image to enter customize mode.

   

SECTION: Importing Photographs
16. Increase Microcontrast.

  
SECTION: Importing Photographs
17. Decrease Highlights, Midtones.

Increase Shadows, Blacks,

If necessary, adjust the exposure.

  
 

SECTION: Importing Photographs
18. Set a value for White Balance.You can use the Pick Color tool to take the values from the gray card or from the neutral gray part.

  
SECTION: Importing Photographs
19. Turn on "Denoising", if the ISO is not very high then the default value is usually very suitable.

  
SECTION: Importing Photographs
20. Right-click on the photo and select "Copy correction settings" (<Ctrl+Shift+C>).
SECTION: Importing Photographs
21. Press <Ctrl+A> at the same time to select all photos, right-click on the photo list below and select "Paste all correction settings" (<Crtl+Shift+V>).


SECTION: Importing Photographs
22. Select "Export to Disk" at the bottom right of the screen to export the corrected photos. Usually JPEG is enough, but select 100 for Quality to get good quality.
SECTION: Importing Photographs
23. Select a Path and export.

 
SECTION: Importing Photographs
24. In general, photos used for photogrammetry do not require any other processing, and it is important to note that, no correction to lens distortion should be applied. This is because the distortion correction needs to be calculated together with the parallax calculation in the photogrammetry software.
SECTION: Photogrammetry Software Post-Processing
25. Open RealityCapture.

RealityCapture is a photogrammetry software that can generate 3D models from photographs or laser scans without seams. It has several payment options and a free version is available for educational institutions.You can also choose between Metashape or 3DF Zephyr, each of which has its own advantages and disadvantages.

SECTION: Photogrammetry Software Post-Processing
26. Add imagery.

Drag and drop all photos or the folder to the software. Then you can see that all the pictures have been added to the software.
SECTION: Photogrammetry Software Post-Processing
26.1. If you use the same focal length, select on "Images" and then "Group button" at the bottom. Following this, all photos will be aligned with the same internal parameters.
SECTION: Photogrammetry Software Post-Processing
26.2. select the "ALIGNMENT TAB" and select "Settings" to see the alignment settings. In most cases, the default settings are fine.

SECTION: Photogrammetry Software Post-Processing
26.3. Generally, you just need to adjust the Image overlap to "Low" and select "Detector Sensitivity" depending on the surface texture of the object.

  

SECTION: Photogrammetry Software Post-Processing
27. Align Images.

Select "Align Images". If there are no problems with the photos, almost all of them will be aligned into one Component. If the alignment is not successful at once, you can select "Align Images" to align the remaining photos manually.

  

SECTION: Photogrammetry Software Post-Processing
28. Reconstruction.

Select "MESH MODEL TAB" and select "Set Reconstruction Region" to define a region you want to reconstruct, you can also modify the region manually in the "3Ds" window.

  

SECTION: Photogrammetry Software Post-Processing
28.1. Select "Normal Detail" to reconstruct the model. You can also select "High Detail" to get maximum detail if you are using a digital camera and the ISO is not too high.
SECTION: Photogrammetry Software Post-Processing
28.2.  

SECTION: Photogrammetry Software Post-Processing
29. Simplification Model.

If the model has too many faces, you may get a message when the reconstruction is completed that Warning "Not enough video memory", however, this is just a display issue. Next, simplication of the model can be applied to make it more manageable and easier to view.

  
SECTION: Photogrammetry Software Post-Processing
29.1. Select "Simplify Tool" in the "TOOL TAB", it is recommended to select type to "Relative" and set the value to 25 or 50. The Relative method will preserve the maximum details.

 
SECTION: Photogrammetry Software Post-Processing
29.2. You can also choose "Part Merging" as "Create a singleton" (if the model is not very large).

 
SECTION: Photogrammetry Software Post-Processing
30. Generate a texture.

Select "MESH MODEL TAB" and <click> on "Color and Texture Settings".

 
SECTION: Photogrammetry Software Post-Processing
30.1. Usually the default settings of 8K texture and other settings are almost enough. If you don't need such a large map, you can also choose 4196 or 2048.

  
SECTION: Photogrammetry Software Post-Processing
30.2. Generate texture.

  
SECTION: Photogrammetry Software Post-Processing
31. Define scale.

Select "ALIGNMENT TAB" and select "Control Points".

 
SECTION: Photogrammetry Software Post-Processing
31.1. Select 2 separate photos. Click on at least 2 points, Point 0 and Point 1, on each of the 2 sets of photos.

  
SECTION: Photogrammetry Software Post-Processing
31.2. After adding "Control Point", click the green plus symbol in "1Ds" to activate the point and then check the accuracy of the point. If the accuracy is less than the expected value such as 1px, the point is not misplaced.

  
SECTION: Photogrammetry Software Post-Processing
31.3. Select "Create distance" in the "1Ds" window and select distance 0 to change the names of the two points A and B to Point 0 and Point 1.
SECTION: Photogrammetry Software Post-Processing
31.4. Set the Defined distance to the real distance.
SECTION: Photogrammetry Software Post-Processing
31.5. Click "Update" in the "ALIGNMENT TAB" to update to the actual size.

  
SECTION: Exporting Model
32. Select the model you want to export in Components.

SECTION: Exporting Model
33. Select "Export - Dense Mesh Model" in the "MESH MODEL TAB".
SECTION: Exporting Model
34. Set a file name and file format and in the next window select "Export Textures" as "Yes".
 

SECTION: Exporting Model
35. Click "OK" to finish exporting the model.

  

SECTION: Photography
13.1. If the object is particularly reflective or shiny (e.g. obsidian), dusting or matting of the surface will be required. For more details, see video n. 3 in the "Before Start" section.
SECTION: Photogrammetry Software Post-Processing
27.1. To note, alignment is not always successful at the first try. When it fails, it is suggested to repeatedly attempt the allignment and practice. Experience and time spent practicing are key components to the successful allignment of complex objects.
SECTION: Final Model
37. Final render video with texture:
 

Final render without texture: