A peephole LSTM unit with input (i.e. 
  
    
      
        i
      
    
    {\displaystyle i}
  
), output (i.e. 
  
    
      
        o
      
    
    {\displaystyle o}
  
), and forget (i.e. 
  
    
      
        f
      
    
    {\displaystyle f}
  
) gates. Each of these gates can be thought as a "standard" neuron in a feed-forward (or multi-layer) neural network: that is, they compute an activation (using an activation function) of a weighted sum. 
  
    
      
        
          i
          
            t
          
        
        ,
        
          o
          
            t
          
        
      
    
    {\displaystyle i_{t},o_{t}}
  
 and 
  
    
      
        
          f
          
            t
          
        
      
    
    {\displaystyle f_{t}}
  
 represent the activations of respectively the input, output and forget gates, at time step 
  
    
      
        t
      
    
    {\displaystyle t}
  
.  The 3 exit arrows from the memory cell 
  
    
      
        c
      
    
    {\displaystyle c}
  
 to the 3 gates 
  
    
      
        i
        ,
        o
      
    
    {\displaystyle i,o}
  
 and 
  
    
      
        f
      
    
    {\displaystyle f}
  
 represent the peephole connections. These peephole connections actually denote the contributions of the activation of the memory cell 
  
    
      
        c
      
    
    {\displaystyle c}
  
 at time step 
  
    
      
        t
        −
        1
      
    
    {\displaystyle t-1}
  
, i.e. the contribution of 
  
    
      
        
          c
          
            t
            −
            1
          
        
      
    
    {\displaystyle c_{t-1}}
  
 (and not 
  
    
      
        
          c
          
            t
          
        
      
    
    {\displaystyle c_{t}}
  
, as the picture may suggest). In other words, the gates 
  
    
      
        i
        ,
        o
      
    
    {\displaystyle i,o}
  
 and 
  
    
      
        f
      
    
    {\displaystyle f}
  
 calculate their activations at time step 
  
    
      
        t
      
    
    {\displaystyle t}
  
 (i.e., respectively, 
  
    
      
        
          i
          
            t
          
        
        ,
        
          o
          
            t
          
        
      
    
    {\displaystyle i_{t},o_{t}}
  
 and 
  
    
      
        
          f
          
            t
          
        
      
    
    {\displaystyle f_{t}}
  
) also considering the activation of the memory cell 
  
    
      
        c
      
    
    {\displaystyle c}
  
 at time step 
  
    
      
        t
        −
        1
      
    
    {\displaystyle t-1}
  
, i.e. 
  
    
      
        
          c
          
            t
            −
            1
          
        
      
    
    {\displaystyle c_{t-1}}
  
.  The single left-to-right arrow exiting the memory cell is not a peephole connection and denotes 
  
    
      
        
          c
          
            t
          
        
      
    
    {\displaystyle c_{t}}
  
.  The little circles containing a 
  
    
      
        ×
      
    
    {\displaystyle \times }
  
 symbol represent an element-wise multiplication between its inputs. The big circles containing an S-like curve represent the application of a differentiable function (like the sigmoid function) to a weighted sum.  There are many other kinds of LSTMs as well.[5]