Getting started with Photogrammetry — with an Smartphone camera [2019]
By: Azad Balabanian
Tags: Virtual Reality, Photogrammetry, Photograph, Tutorial, Photogrammetry Tutorial

The Detailed Photogrammetry processing steps

Nodes based iteration system

Before we begin jumping into tweaking settings, you should know how Meshroom’s node based system is fantastic for iterating on tweaking settings, processing, and comparing the results.

For any node, if you’d like to change a setting to see how it affects its results, you can right click, and select the “Duplicate >>”. This will duplicate that node and every node after (which is useful because it’s hard to remember how to link all the nodes together and in which order).

Duplicate a set of nodes to compare results of different settings

You will see how we’ll use this iteration system for getting the best results of our mesh for our needs.

Image and camera properties

Step 1: Drag and drop the images into the Images sidebar.

Make sure the Meshroom is detecting the correct camera and lens properties by hovering over the green icon on the images. If you’re using a common phone or camera, you should be just fine.

If not, you can look up your lens and sensor parameters and manually input them [guide].

confirm the make and model of your camera

Feature matching and Camera alignment

The first step to Photogrammetry processing is feature extraction and camera alignment.

Meshroom loads with all the Photogrammetry nodes you need. Don’t be put off by how many nodes there are. You don’t really need to know what each does to be able to able to get good results. But with this tutorial, you’ll understand what each “cluster” of nodes do.

I spaced out the alignment nodes cluster for clarity.

Camera alignment nodes cluster

For simple subjects like the brick fireplace subject we picked, where the subject is not reflective, the surrounding is visible and full of features, and I systematically moved between each image and ensured overlap between each image, we should be confident that the images should all properly align.

Step 2: Right-click on the StructureFromMotion node and click Compute. Meshroom will process all the nodes up until that node.

If successful, you should have all your images align (in my case, 97 images) and produce a sparse point cloud.

all cameras aligned!

If you have less than 300 images, Meshroom recommends that you increase the FeatureExtraction node “Describer Preset” to High from Normal. All this means is that the feature extraction algorithm will find and use more features per image to find precise alignment of the cameras in the scene (which also means it will take more time to process, but for small datasets, it’s not very significant).

More features, mean more chance of photos aligning

If you’re having alignment problems,

In FeatureExtraction, increase the Describer preset to High or Ultra.

In FeatureExtraction, turn on Guided Matching setting.

In the FeatureExtraction, FeatureMatching, and StructureFromMotion nodes, change the Describer Types to AKAZE (default: SIFT). Meshroom documentation says “it may improve results from some surfaces like skin as it is more affine invariant than SIFT and can help recover connections when you have not enough images in input”.

Depthmaps and Meshing

The next step after camera alignment is to generate depth maps and create a mesh (the geometry of the Photogrammetry scene).

NOTE: Generating DepthMaps ONLY works for Nvidia equipped PC’s, so for everyone else, you’ll have to use Draft Meshing and skip generating Depthmaps.

Again, I’ve arranged the nodes in clusters for the sake of this tutorial.

Step 3: Right click on Meshfiltering and click Process. Go and have some coffee, this step will take the longest.

Meshing nodes cluster

Once it’s done it should look like this:

default meshing settings

As you can see, it produced a mesh with 3.9mil polygons and a fair bit of the surrounding environment (which we don’t really need).

Improving Mesh Quality

If you’d like to improve the meshing quality of the subject, there are a few settings to tweak.

To decrease meshing the surrounding,

in the Meshing node, you can increase Min Observations Angle for SfM Space Estimation much higher to discard meshing the surrounding

To have a denser mesh reconstruction (higher poly count),

in the Meshing node, you can increase Max Input Points and Max Points (this is limited by your PC’s RAM. 16GB RAM users might need to lower these parameters)

In the DepthMap node, you can decrease the Downscale setting from 2 [default] to 1 (note: this will increase processing times by 4x)

If you’re getting holes in your mesh, in DepthMapFilter, you can reduce the Min Consistent Cameras to 2 and Min Consistent Cameras Bad Similarity to 3

if your mesh is still noisy generally more noisy than you’d like, you can increase the MeshFiltering Smoothing iteration amount.

However, after some iteration and tweaking settings, I wasn’t getting a mesh that looked all that different from the default settings. This might be tied to the limited quality of my camera and the small size of the scene.

The biggest improvement in mesh quality I observed was setting the DepthMap downscale factor to 1 (default 2).

DepthMap Downscale 1

Mesh retopology and quick editing / crop

It’s recommended to edit the higher poly mesh to have a “master mesh” (the result before MeshDecimate), so that when you create lower poly derivatives, you don’t need to redo all your work.

Step 5: Launch Meshmixer to edit the mesh

Drag and drop the mesh you’d like to edit (recall that you can right-click on the last node you processed and select “Open Folder” to find your mesh)

If there are floaty triangles that you’d like to get rid of, select the “Select” tool (shortcut “S” key), double click on the largest component, then press “I” to invert your selection to everything that’s not connected to the mesh. (note there, the MeshFiltering node also has an option to “Keep Only the Largest Mesh” which I could’ve used)

editing floaty triangles

Now let’s crop out the surroundings with a clean cut. Select Edit > Plane Cut. Left-click and drag a line to set the location of the plane cut. Flip the direction of the cut if need be (as seen in the gif bellow).

cropping the surrounding

There might be some parts of the mesh you’d like to smooth out. I use the Sculpt>Robust Smooth tool as seen in the gif bellow.

Smoothing out some noisy bits

If you’re happy with your edits, export is as an OBJ to any accessible folder (doesn’t need to be your Meshroom project’s directory) and use this mesh as your “Master Mesh”.

Re-import mesh into Meshmixer

We’re almost there! We just need to decimate and texture, and we’ll be done.

The mesh re-import process can be done on any Node that accepts meshes as an input. You just need to disconnect the link from the previous node so you can type in the directory of the mesh you’d like to import.

Step 6: re-import mesh into Meshroom

Create a MeshDecimate node and connect it between Meshfiltering and Texturing

MeshDecimate node

Now, to import the Meshmixer edited “master mesh”, disconnect the MeshDecimate Input link, select MeshDecimate, and drag and drop your edited OBJ onto the Input Mesh text field (as seen in the gif bellow).

Importing a mesh into MeshMixer

Select MeshDecimate, and change Max Vertices to 50,000 (equivalent to 100k poly)

Texturing Settings

Final Step: Select Texturing node, and click compute.

The default Texturing node settings will produce a high quality result, as seen in the screenshot bellow.

The problem is, textures take can take lot of space, as this is 43.6 MB

Default texturing settings. [Downscale 2, four 8k textures, Basic unwrap]

To meet Sketchfab’s basic account’s upload limit (50mb), I changed the settings to the following, totaling 23.9 MB.

[EDIT: I originally wrote this thinking the limit was 30 mb, not 50. However it’s still useful to find ways of minimizing file space while minimizing reducing quality]

Texture Downscale: 1 (so it uses the full resolution of the images)

Texture Size: 4k

Unwrap method: LCSM (which limits to 1 texture map)

Downscale 1, one 4k texture, LCSM unwrap

Best Texturing settings:

Downscale: 1 (default is 2)

Unwrap: Default (will produce multiple textures)

Max number of Images For Fusion: 3 or bellow (the less images per fusion, the sharper the image, but there’s a tradeoff if certain areas dont have enough images)

Let’s do a quick comparison between a few texturing settings.