question: Example in domain_templates: computer_vision_fine_tuning
context: I found that in the code below sigmoid activation applied before binary_cross_entropy_with_logits loss:
  
    
      
        pytorch-lightning/pl_examples/domain_templates/computer_vision_fine_tuning.py
      
      
          Lines 196 to 231
        in
        71b4611
      
    
    
      
  
          
            
             def __build_model(self): 
          
  
          
            
                 """Define model layers & loss.""" 
          
  
          
            
              
          
  
          
            
                 # 1. Load pre-trained network: 
          
  
          
            
                 model_func = getattr(models, self.backbone) 
          
  
          
            
                 backbone = model_func(pretrained=True) 
          
  
          
            
              
          
  
          
            
                 _layers = list(backbone.children())[:-1] 
          
  
          
            
                 self.feature_extractor = nn.Sequential(*_layers) 
          
  
          
            
              
          
  
          
            
                 # 2. Classifier: 
          
  
          
            
                 _fc_layers = [ 
          
  
          
            
                     nn.Linear(2048, 256), 
          
  
          
            
                     nn.ReLU(), 
          
  
          
            
                     nn.Linear(256, 32), 
          
  
          
            
                     nn.Linear(32, 1), 
          
  
          
            
                 ] 
          
  
          
            
                 self.fc = nn.Sequential(*_fc_layers) 
          
  
          
            
              
          
  
          
            
                 # 3. Loss: 
          
  
          
            
                 self.loss_func = F.binary_cross_entropy_with_logits 
          
  
          
            
              
          
  
          
            
             def forward(self, x): 
          
  
          
            
                 """Forward pass. Returns logits.""" 
          
  
          
            
              
          
  
          
            
                 # 1. Feature extraction: 
          
  
          
            
                 x = self.feature_extractor(x) 
          
  
          
            
                 x = x.squeeze(-1).squeeze(-1) 
          
  
          
            
              
          
  
          
            
                 # 2. Classifier (returns logits): 
          
  
          
            
                 x = self.fc(x) 
          
  
          
            
              
          
  
          
            
                 return torch.sigmoid(x) 
          
  
          
            
              
          
  
          
            
             def loss(self, logits, labels): 
          
  
          
            
                 return self.loss_func(input=logits, target=labels) 
          
      
    
  
  
  From the documentation https://pytorch.org/docs/stable/generated/torch.nn.BCEWithLogitsLoss.html#torch.nn.BCEWithLogitsLoss:
  This loss combines a Sigmoid layer and the BCELoss in one single class. 
  Is that performed intentionally? Or it's just a bug?
answer: Yes, the sigmoid should be removed from the forward, because BCEWithLogits already contains the sigmoid.
  Do you want to send a PR to update the example?