Q:

"All that is gold does not glitter"

"All that is gold does not glitter" is the first line of a poem from the Lord of the Rings and it's supposed to mean "not all gold glitters" but I'm struggling to see how this can be deduced. 
If all that is gold does not glitter then it follows that "gold never glitters".  If all gold doesn't glitter then there's no such thing as gold that glitters.  This is quite different from the actual meaning. " Some that is gold does not glitter" would make more sense.  Am I missing something?  Does "all" actually mean "not all" in this context?  It's really confusing. 

It looks like "Not all that is gold does glitter" would mean the same thing, but in logic it has a different meaning. 

A:

The original was actually Shakespeare: all that glisters is not gold, but that needn't concern us here. 
OP has simply misparsed the sentence - it actually means "Not everything that is gold glitters" (which is to say, "There are some things which are gold that don't glitter"). 
You can always Google "every x is not y" for more discussion of why this type of construction should be treated with caution.  As it happens, I already knew what it means in this particular case (and I knew it was originally glisters), but I think the bottom line is the statement is inherently ambiguous, so you have to go for the interpretation that makes most sense in context. 

Tolkien experimented with several variants of the "quirky inversion" of Shakespeare's original before finally settling on the The Riddle of Strider version (that appears twice in The Fellowship of the Ring).  But I quite like this somewhat more "pithy" earlier draft...

All that is gold does not glitter;
  all that is long does not last;
  All that is old does not wither;
  not all that is over is past. 

(I don't know whether the punctuation/capitalisation there was actually what Tolkien wrote). 

A:

Shakespeare's line is the best known example of this general phenomenon where a universal quantifier scoping over negation gives a counterintuitive meaning.  The expected meaning of:

All that glisters is not gold. 

Would be:

For each thing that glisters, it is not gold. 

Instead, the meaning to be understood is:

Not everything that glisters is gold. 

See Laurence Horn's excellent discussion of this phenomenon in Chapter 4 of his book A Natural History of Negation. ( University of Chicago Press, 1989).  Other examples Horn draws attention to are (p.226--7):

All things are lawful unto me, but all
  things are not expedient. ( 1Cor.6:12)
  Every one cannot make music. ( Walton)
  Tout Ie monde n' est past fait pour l' art. ( Rolland)
  Thank heaven, all scholars are not like
  this. ( Richardson)
  All is not lost. ( Milton, Shelley)
  Each man kills the thing he loves/Yet each
  man does not die. ( Wilde)

Horn points out, even more interestingly, that while the English can be paraphrased into the intuitive:

Not all that glitters is gold. 

In French, however, the "counterintuitive" version:

Tout ce qui reluit n'est pas or. 

does not have an "intuitive" grammatical paraphrase.  The following is ungrammatical in French:

*Pas tout ce qui reluit est or. 

A:

It means not everything that is gold glitters.  Tolkien undoubtedly was borrowing from Shakespeare here, specifically the poem that one of Portia's suitors discovers when he reads the scroll associated with the golden chest that he has (to his loss) chosen:

“All that glisters is not gold—
  Often have you heard that told. 
  Many a man his life hath sold
  But my outside to behold. 
  Gilded tombs do worms enfold. 
  Had you been as wise as bold,
  Young in limbs, in judgment old,
  Your answer had not been inscrolled. 
  Fare you well.  Your suit is cold—
  Cold, indeed, and labor lost.”   

The meaning is that even if a thing "glisters" (glistens) it is not necessarily true gold — plenty exists that is false.  Tolkien offers an inversion of this when he says, in effect, that some things that are pure gold do not glitter.  
Edit
Some meddlesome individuals reversed the wording in my first sentence so that it meant the exact opposite of what I said (and what is correct).  I have changed it back.