How should I fill null values for pictures

Jordan_Howell: <p>Hello,</p>
<p>I have a model that will take results from three different models based on the types of pictures and combine them for a final binary classification.  The data is pulled based on the location of a photo pulled from a dataframe.</p>
<p>Here is a snapshot:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/e/ae2a98da10e05d776f7df5588938fa449987db6d.png" data-download-href="https://discuss.pytorch.org/uploads/default/ae2a98da10e05d776f7df5588938fa449987db6d" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/e/ae2a98da10e05d776f7df5588938fa449987db6d.png" alt="image" data-base62-sha1="oQKlj5wfyQNd4fHAvPm9N5pf5cV" width="690" height="235" data-dominant-color="F1F1F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1010×344 13.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Is there  a way that I can skip the NaN values or fill them with a tensor of 0s the size (224x224)?</p>
<p>If the above is possible would I do that in the model object or in the custom data pull object?</p>
<p>Is there a better way to do this?</p>
<p>I was thinking:</p>
<pre><code class="lang-auto">class image_Dataset(Dataset):
    '''
    image class data set   
    
    '''
    def __init__(self, data, transform = None):
        '''
        Args:
        ------------------------------------------------------------
            data = dataframe
            image = column in dataframe with absolute path to the image
            label = column in dataframe that is the target classification variable
            numerical_columns =  numerical columns from data
            categorical_columns = categorical columns from data
            policy = ID variable
            
        '''
        self.image_frame = data
        self.transform = transform
        
    def __len__(self):
        return len(self.image_frame)
    
    def __getitem__(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()
         
        label = self.image_frame.loc[idx, 'target']
        
        if self.image_frame[self.image_frame[idx, 'Roof'].isna() == True]:
            pic = np.ones(3,224,224)
        else:
            pic = Path(self.image_frame.loc[idx,'Roof'])
        img = Image.open(pic)
        if self.transform:
            image = self.transform(img)
        
        policy = self.image_frame.loc[idx, 'policy']
        
        numerical_data = self.image_frame.loc[idx, numerical_columns]
        
        numerical_data = torch.tensor(numerical_data, dtype = torch.float)
        
        for category in cat_columns:
            self.image_frame[category] = self.image_frame[category].astype('category')
            
            self.image_frame[category] = self.image_frame[category].astype('category').cat.codes.values
        
            
        categorical_data = self.image_frame.loc[idx, cat_columns]
        categorical_data = torch.tensor(categorical_data, dtype = torch.int64)
            
        return image, label, policy, categorical_data , numerical_data
</code></pre>
<p>But this still throws and error when there is a NaN value.</p>
<p>I also tried passing a <code>pass</code> statement when it encountered a NaN value.  No luck.</p>

ptrblck: <p>I’m not sure I understand the use case completely.<br>
Do you need to load all three columns and would like to return a constant fake image for the NaN values?<br>
Would it work, if you only load the valid images, as the other two don’t seem to be very useful?</p>

Jordan_Howell: <p>Use case is this:</p>
<p>I have observations with different number and type of pictures.  Each observation can have from 1 to 8 pictures across three different types (roof, roof hazard and front) of photos. I want to use all photos but it creates a  dataframe with null values as seen above.</p>
<p>For your second question, I guess I could run three different models, one for each picture and concatenate either, the predictions or the feature sets they are reduced too.  I’m not sure if i would take the max or average of the results for observations with more than one photo though dynamically.  Any thoughts on that?</p>

ptrblck: <p>Are these picture types (roof, roof hazard and front) representing the target?<br>
If so, then your approach to create separate models for these images won’t work, as you won’t know the target for new images (and would probably want to predict them).</p>
<p>I’m still unsure if I understand the use case.<br>
You’ve created a pandas <code>DataFrame</code> and would now like to load all pictures at once (ignoring the NaN entries)?</p>

Jordan_Howell: <p>The target is a binary 1 or 0 I’m trying to predict from the picture. There could be multiple pictures per target.</p>
<p>I was thinking of three different models then take a weighted average of each model’s prediction per unique ID. I’m not sure what to do if the ID is missing a picture like roof hazard.</p>
<p>The dataframe has the location for the images for loader to grab.  So the loader iterates 1 row at a time grabbing pictures.</p>
<p>Jordan</p>