The reference was to the fact that the crucifix over the altar faces the harbour (rather than the street as with most churches) - this is the wooden tower.  The statue above the church is said to be another reference in the song - the Lady of the Harbour (ambiguous in the third stanza as to whether it is this Lady or Suzanne herself).  I remember hearing these things on  a Montreal radio station in the early '70s, but can't give a reference.