The mariner's tale begins with his ship departing on its journey. Despite initial good fortune, the ship is driven south by a storm and eventually reaches Antarctic waters. An albatross appears and leads them out of the ice jam where they are stuck, but even as the albatross is praised by the ship's crew, the mariner shoots the bird:
With my cross-bow,

I shot the albatross.
The crew is angry with the mariner, believing the albatross brought the south wind that led them out of the Antarctic. However, the sailors change their minds when the weather becomes warmer and the mist appears:
'Twas right, said they, such birds to slay,

That bring the fog and mist.
They soon find that they made a grave mistake in supporting this crime, as it arouses the wrath of spirits who then pursue the ship "from the land of mist and snow"; the south wind that had initially led them from the land of ice now sends the ship into uncharted waters near the equator, where it is becalmed.
Day after day, day after day,

We stuck, nor breath nor motion;

As idle as a painted ship

Upon a painted ocean.

Water, water, every where,

And all the boards did shrink;

Water, water, every where,

Nor any drop to drink.

The very deep did rot â Oh Christ!

That ever this should be.

Yea, slimy things did crawl with legs,

Upon the slimy sea.
The sailors change their minds again and blame the mariner for the torment of their thirst. In anger, the crew forces the mariner to wear the dead albatross about his neck, perhaps to illustrate the burden he must suffer from killing it, or perhaps as a sign of regret:
Ah! Well a-day! What evil looks

Had I from old and young!

Instead of the cross, the albatross

About my neck was hung.
Eventually, the ship encounters a ghostly hulk. On board are Death (a skeleton) and the "Night-mare Life-in-Death", a deathly-pale woman, who are playing dice for the souls of the crew. With a roll of the dice, Death wins the lives of the crew members and Life-in-Death the life of the mariner, a prize she considers more valuable. Her name is a clue to the mariner's fate: he will endure a fate worse than death as punishment for his killing of the albatross.
One by one, all of the crew members die, but the mariner lives on, seeing for seven days and nights the curse in the eyes of the crew's corpses, whose last expressions remain upon their faces. Eventually, this stage of the mariner's curse is lifted after he appreciates the sea creatures swimming in the water. Despite his cursing them as "slimy things" earlier in the poem, he suddenly sees their true beauty and blesses them ("a spring of love gush'd from my heart, and I bless'd them unaware"); suddenly, as he manages to pray, the albatross falls from his neck and his guilt is partially expiated. The bodies of the crew, possessed by good spirits, rise again and help steer the ship. In a trance, the mariner hears two spirits discussing his voyage and penance, and learns that the ship is being powered preternaturally:
The air is cut away before,

And closes from behind.
Finally the mariner comes in sight of his homeland, but is initially uncertain as to whether or not he is hallucinating.
Oh! Dream of joy! Is this indeed

The light-house top I see?

Is this the hill? Is this the kirk?

Is this mine own countree?

We drifted o'er the harbour-bar,

And I with sobs did prayâ

O let me be awake, my God!

Or let me sleep alway.
The rotten remains of the ship sink in a whirlpool, leaving only the mariner behind. A hermit on the mainland had seen the approaching ship and had come to meet it with a pilot and his boy, in a boat. When they pull him from the water, they think he is dead, but when he opens his mouth, the pilot has a fit. The hermit prays, and the mariner picks up the oars to row. The pilot's boy goes crazy and laughs, thinking the mariner is the devil, and cries, "The Devil knows how to row". As penance for shooting the albatross, the mariner, driven by guilt, is forced to wander the earth, telling his story over and over, and teaching a lesson to those he meets:
He prayeth best, who loveth best

All things both great and small;

For the dear God who loveth us,

He made and loveth all.
After relaying the story, the mariner leaves, and the wedding guest returns home, and wakes the next morning "a sadder and a wiser man".
The poem received mixed reviews from critics, and Coleridge was once told by the publisher that most of the book's sales were to sailors who thought it was a naval songbook. Coleridge made several modifications to the poem over the years. In the second edition of Lyrical Ballads, published in 1800, he replaced many of the archaic words.

Question: What is the Mariner's penance for shooting the albatross?
Answer: He has to wander the Earth, telling his story and being a lesson for everyone he meets. 
Alternative answer: The guilt ridden mariner has to tell his story to everyone he meets.