Repository: nerfstudio-project/nerfstudio

Issue:
Feature extraction/matching doesn't work when using all the frames from a video with COLMAP and ns-process-data
**Describe the bug**
COLMAP isn't able to run feature extraction and subsequent bundle adjustment when I use all the frames from a video. Everything works fine when I use the default ~300 input frames. Have anynone experienced the same?

For every image it tries to extract features from it exits with `ERROR: Failed to read image file format.`, although the file format is PNG. I see that the resulting files are 2-3 times larger (totalling ~11Mb) than what the ones extracted with the thumbnail-option is. Is there possibly an issue with convert_video_to_images and the ffmpeg_cmd?

**To Reproduce**
Steps to reproduce the behavior:
1. Run `ns-process-data video` with an input-video and `--num-frames-target` equal (or approx. equal) to the video's number of frames. And `--verbose`
2. This results in `spacing` in `process_data_utils.py` to be 1
3. Observe that all phases before Feature extraction works as expected
4. Observe that for each line of "Processed file" during feature extraction, an error `ERROR: Failed to read image file format.` is logged.
5. Observe that `/colmap/sparse/0` isn't generated which causes the Bundle adjustment step to fail.

**Expected behavior**
Feature extraction should generate features and match them, resulting in a `/colmap/sparse/0`-dir that bundle adjustment can operate on.

**Screenshots**
![image](https://user-images.githubusercontent.com/27623888/205260865-432a5f98-1eb8-469f-b316-da8a2e1641f2.png)
![image](https://user-images.githubusercontent.com/27623888/205260889-42aae792-6538-4f17-bd3c-a921f1e52648.png)
![image](https://user-images.githubusercontent.com/27623888/205260911-ff5cddb2-b9e2-432a-81ce-e86e41a995b5.png)


**Additional context**
- COLMAP version: 3.6
- FFMPEG version: 4.2.2
- The video contains 1416 frames, captured at 30Hz.
- I have tried running the ffmpeg_cmd with -r 30 


Discussion:
I just tried, but was unable to replicate this error. Do you find the images located in the output folder?
Yes, the images are successfully grabbed from the video and downsampled. It works when `spacing = 2`, but not when `spacing = 1`.

Here's the last two frames in the folder where it **fails**:
<img width="524" alt="image" src="https://user-images.githubusercontent.com/27623888/206149633-09917b94-71c7-4b7d-b279-bacdb1c5c586.png">

Here's the last two frames in the folder where it **works**:
<img width="518" alt="image" src="https://user-images.githubusercontent.com/27623888/206149581-e258a893-26e7-4086-bbbb-c6a63346cee8.png">

I'm still struggling with this one. I've uploaded the input file if you'd like to test for yourself.
Input-file: https://drive.google.com/file/d/13AMoMUO05gQzKYUKckuK7qZd8RsOgIY7/view?usp=sharing
Thanks for sharing the data, sorry it's taken awhile to get back to this. Turns out the issue is that ffmpeg is storing the images as 16bit rather than 8 bit when the thumbnail command isn't used. Ill have a fix soon.

Resolving patch:
diff --git a/nerfstudio/process_data/process_data_utils.py b/nerfstudio/process_data/process_data_utils.py
--- a/nerfstudio/process_data/process_data_utils.py
+++ b/nerfstudio/process_data/process_data_utils.py
@@ -94,6 +94,7 @@ def convert_video_to_images(
             ffmpeg_cmd += f" -vf thumbnail={spacing},setpts=N/TB -r 1"
         else:
             CONSOLE.print("[bold red]Can't satify requested number of frames. Extracting all frames.")
+            ffmpeg_cmd += " -pix_fmt bgr8"
 
         ffmpeg_cmd += f" {out_filename}"
 


Test patch: