  Tensor network states have been a very prominent tool for the study of
quantum many-body physics, thanks to their physically relevant entanglement
properties and their ability to encode symmetries. In the last few years, the
formalism has been extended and applied to theories with local symmetries to -
lattice gauge theories. In the contraction of tensor network states as well as
correlation functions of physical observables with respect to them, one uses
the so-called transfer operator, whose local properties dictate the long-range
behaviour of the state. In this work we study transfer operators of tensor
network states (in particular, PEPS - projected entangled pair states) in the
context of lattice gauge theories, and consider the implications of the local
symmetry on their structure and properties. We focus on the Wilson loop - a
nonlocal, gauge-invariant observable which is central to pure gauge theories,
whose long range decay behaviour probes the confinement or deconfinement of
static charges. Using the symmetry, we show how to handle its contraction, and
formulate conditions relating local properties to its decay fashion.
