init_slurm_connection causing hostname errors
Problem
Can you update this function to support checking if MASTER_ADDR and MASTER_PORT are already in os.environ?  Running into some weird errors where this code adds host to MASTER_ADDR and crashes the code.

  
    
      pytorch-lightning/pytorch_lightning/core/lightning.py
    
    
         Line 903
      in
      0697dd3
    
  
  
    

        
          
           def _init_slurm_connection(self) -> None: 
        
    
  


Solution
Do you prefer a pull request?  Here's the check I put it
def _init_slurm_connection(self) -> None:
        """
        Sets up environment variables necessary for pytorch distributed communications
        based on slurm environment.
        """

        if 'MASTER_PORT' not in os.environ:
            # use slurm job id for the port number
            # guarantees unique ports across jobs from same grid search
            try:
                # use the last 4 numbers in the job id as the id
                default_port = os.environ['SLURM_JOB_ID']
                default_port = default_port[-4:]

                # all ports should be in the 10k+ range
                default_port = int(default_port) + 15000

            except Exception:
                default_port = 12910

            # if user gave a port number, use that one instead
            try:
                default_port = os.environ['MASTER_PORT']
            except Exception:
                os.environ['MASTER_PORT'] = str(default_port)

        # figure out the root node addr
        if 'MASTER_ADDR' not in os.environ:
            try:
                root_node = os.environ['SLURM_NODELIST'].split(' ')[0]
            except Exception:
                root_node = '127.0.0.1'

            root_node = self.trainer.resolve_root_node_address(root_node)
            os.environ['MASTER_ADDR'] = root_node