Repository: nerfstudio-project/nerfstudio

Issue:
FFMPEG autorotates source images without user control
FFMPEG autorotates images with Orientation EXIF tag. It happens automatically without user control during the first usage of FFMPEG.
In many cases, this does not influence data processing (because of the behavior described below).  We need to have ways to control this behavior.
This behavior can give more issues and unexpected results, especially for data import (RealityCapture as an example) 
Note: FFMPEG autorotates images with the Orientation EXIF tag only the first time. All images that are already saved with FFMPEG have removed part of the EXIF tags including the Orientation tag. So it does not autorotate already saved images.


The current behavior in the case of using source images :
1. Currently we use FFMPEG  for all folders we created with ns-process-data  (all created folders: images. images_2, images_4, images_8). This is normal behavior for image downscaling. For the original size folder (images) that don't require border pixel cropping, we do not expect to use FFMPEG. But we always use it because of the bug described below.
2.  In process_data.py we have bug [here](https://github.com/nerfstudio-project/nerfstudio/blob/7466788b2e0a4fa54b9d7283f7d1c8899fac726a/nerfstudio/process_data/process_data_utils.py#L178) . 
```
        # Current code
        num_frames = len(copy_images_list(image_paths, image_dir, verbose))
        # Should be:
        num_frames = len(copy_images_list(image_paths, image_dir, verbose=verbose))
``` 
This bug forces to use FFMPEG here with crop_border_pixels=0  in [copy_images_list](https://github.com/nerfstudio-project/nerfstudio/blob/7466788b2e0a4fa54b9d7283f7d1c8899fac726a/nerfstudio/process_data/process_data_utils.py#L143).  This is not expected behavior. BUT this is a "lucky" bug that prevents nerfsudio data processing to be broken :) If we just fix it we can get data inconsistency (different image orientation) between the original and downscaled folders. At the current moment all are consistent because we use FFMPEG with the original image in any case.
3. We need  to fix the bug above and decide how to control autorotation 
4. I made several tests but unfortunately don't find any practical way to switch off autorotation in FFMPEG for images (tested in Windows).  Please give me an idea of how to switch autorotation off if anybody have success with this.
5. As a possible option we can control Orientation EXIF tag. We can add a function to remove this tag in copied images. And also  function to check Orientation EXIF tag status. PIL can read EXIF or we can use exiftool utility for this as a subprocess.





Discussion:
Supposedly there is an option for recent ffmpeg `-noautorotate` -- does that help?
For some reason, I don't see this `-noautorotate` option in my Windows ffmpeg. 
Instead, I see:
`-autorotate         automatically insert correct rotate filters`
Ok , I see in doc:
-autorotate
Automatically rotate the video according to file metadata. Enabled by default, use -noautorotate to disable it.


Resolving patch:
diff --git a/nerfstudio/process_data/process_data_utils.py b/nerfstudio/process_data/process_data_utils.py
--- a/nerfstudio/process_data/process_data_utils.py
+++ b/nerfstudio/process_data/process_data_utils.py
@@ -144,7 +144,7 @@ def copy_images_list(
         file_type = image_paths[0].suffix
         filename = f"frame_%05d{file_type}"
         crop = f"crop=iw-{crop_border_pixels*2}:ih-{crop_border_pixels*2}"
-        ffmpeg_cmd = f"ffmpeg -y -i {image_dir / filename} -q:v 2 -vf {crop} {image_dir / filename}"
+        ffmpeg_cmd = f"ffmpeg -y -noautorotate -i {image_dir / filename} -q:v 2 -vf {crop} {image_dir / filename}"
         run_command(ffmpeg_cmd, verbose=verbose)
 
     num_frames = len(image_paths)
@@ -209,7 +209,7 @@ def downscale_images(image_dir: Path, num_downscales: int, verbose: bool = False
             for f in files:
                 filename = f.name
                 ffmpeg_cmd = [
-                    f"ffmpeg -y -i {image_dir / filename} ",
+                    f"ffmpeg -y -noautorotate -i {image_dir / filename} ",
                     f"-q:v 2 -vf scale=iw/{downscale_factor}:ih/{downscale_factor} ",
                     f"{downscale_dir / filename}",
                 ]


Test patch: