Tokenizer outputs same offsets for different tokens.
Author: Fil-onto
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.30.2
- Platform: Linux-6.8.0-49-generic-x86_64-with-glibc2.35
- Python version: 3.10.13
- Huggingface_hub version: 0.27.1
- Safetensors version: 0.5.0
- PyTorch version (GPU?): 2.2.1+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import AutoTokenizer

tokenizer = tokenizer = AutoTokenizer.from_pretrained("Gladiator/roberta-large_ner_conll2003")
text = """"One Ring to rule them all, One Ring to find them, One Ring to bring them all and in the darkness bind them." —The Ring's inscription, translated


The One Ring, also known as the Ruling Ring, Master Ring, Ring of Power, and Isildur's Bane, was among the most powerful artifacts ever created in Middle-earth. It was crafted by the Dark Lord Sauron in the fire of Orodruin, also known as Mount Doom, in the Second Age. Sauron's intent was to enhance his own power and exercise control over the other Rings of Power, which had been made by Celebrimbor and the Gwaith-i-Mírdain with Sauron's assistance. In this way, Sauron hoped to gain lordship over the Elves and all other peoples of Middle-earth.


History[]


Origin and War[]


Sauron forges the One Ring, by Ted Nasmith The Ring saw its origin at the core of a plot by Sauron to bring the peoples of Middle-earth under his dominion. To do this, Sauron conceived of a set of magic Rings that he would aid the Elves in forging, and then by way of a singular, more powerful Ring, he would be able to bend the wills of those who wore the Rings to his own. Once the Elves were under his sway, he assumed that the other peoples of Middle-earth could be either persuaded to join him or be conquered with relative ease. As such, he spent a significant amount of time amongst the Elves, taking on the fair guise of Annatar, Lord of Gifts and instructing them on how to create the Rings. After the sixteen planned upon Rings had been forged, Sauron returned to Mordor around the year SA 1600, and within the Sammath Naur upon Mount Doom, fashioned the One Ring.


Sauron had known since the beginning of his endeavor that, in order to control the other Rings, the One would have to be an object of surpassing potency. To give the One the power necessary to fulfill its function, he concentrated within it a great part of his own fëa (soul). In this way, Sauron's fate became bound to that of the Ring. If it were damaged or destroyed, so too would be Sauron's strength and power.


Soon afterwards, Sauron attempted to use it to subjugate the Elven wielders of the other Rings. However, when Sauron placed the One Ring on his finger, the Elves who bore the other Rings became immediately aware of him and, guessing his ill intent, removed their Rings. Correctly assuming that his attempt to gain lordship over the Elves had been thwarted, Sauron marshaled his armies to seize the Rings of Power by force. The conflict, which became known as the War of the Elves and Sauron, began in SA 1693. Initially, the war went very well for Sauron. He captured Eregion in short order and took back the Nine Rings that were kept there, and also Celebrimbor, the maker of the Elven Rings of Power. He tortured Celebrimbor until he divulged the location of the Seven Rings, and discovered too that Celebrimbor had made an additional Three Rings without him, each of significantly greater power than any of the Seven or the Nine. Celebrimbor died under torment by Sauron, refusing to reveal what he had done with the Three Rings, which he valued most. After the destruction of Eregion, Sauron was able to conquer most of western Middle-earth fairly quickly, driving the Ñoldor under Gil-galad to the Grey Havens and besieging Imladris. But in SA 1700, as the Elves were nearing defeat, Tar-Minastir of Númenor sent a great armada to Middle-earth and, together with Gil-galad, completely destroyed Sauron's armies, forcing Sauron to return to Mordor to regroup.


In SA 3261, Ar-Pharazôn, the last and most powerful of the Kings of Númenor, landed at Umbar at the head of an even more gigantic army to do battle with Sauron, in contention of Sauron's self-proclaimed title as Overlord of Middle-earth and King of Men. The sheer size and might of the Númenórean army was enough to cause Sauron's own forces to flee. Understanding that he could not overcome the Númenóreans through military power, Sauron surrendered to Ar-Pharazôn and was taken back to Númenor as a prisoner. However, Sauron's surrender was both "voluntary and cunning",[1] and allowed him to gain access to the people of Númenor. The Elves had not revealed to the Númenóreans the existence of the Rings of Power, and so Ar-Pharazôn was unaware of the One Ring's existence and capabilities. Ascending rapidly to become the King's most trusted counsellor, Sauron was able to use the Númenóreans' fear of death as a way to turn them against the Valar, and toward worship of Melkor.


The Ring on Sauron's finger, in the films Although Sauron's body was destroyed in the Fall of Númenor, his spirit was able to bear the Ring back to Middle-earth and he wielded it in his renewed war against the Last Alliance of Elves and Men between SA 3429 and 3441.[2]


The Ring was cut from Sauron's hand by Isildur at the end of the Siege of Barad-dûr in SA 3441, and he in turn lost it in the River Anduin (at the Gladden Fields) just before he was killed in an Orc ambush (TA 2). Since it indirectly caused Isildur's death by slipping from his finger and revealing him to the Orcs, it was known in Gondor's lore as Isildur's Bane.


The Third Age[]


Discovery by Sméagol and Déagol[]


Déagol observing the Ring for the first time Déagol/Gollum meets Bilbo The Ring remained hidden in the riverbed for almost two and a half millennia until a Stoor named Déagol discovered it on a fishing trip in TA 2463. His friend and cousin Sméagol murdered Déagol and stole the Ring. Sméagol was changed by the Ring’s influence over four and a half centuries into the creature known as Gollum. Gollum, after being exiled from his home, sought shelter deep beneath the Misty Mountains. There, he and it remained for nearly five hundred years, its power corrupting his mind and soul and deforming him into a pathetic physical form until the Ring abandoned him and fell off his finger, landing on the floor of one of the tunnels. This is thought to be an example of one of the more noticeable powers of the Ring; the ability to change size at will. It also displayed that the Ring was sentient, having part of the spirit of Sauron inside it.


Possession by Bilbo Baggins[]


Bilbo finding the Ring in The Hobbit: An Unexpected Journey As is told in The Hobbit, Bilbo Baggins found the Ring a few hours after Gollum lost it, while he was alone in the caverns of the Misty Mountains. Shortly after discovering it, Bilbo came upon Gollum himself, who had intended to eat the lost Hobbit. Bilbo managed to get Gollum to agree to a riddle-game to determine his own fate; if he lost, Gollum could eat him, and if he won, Gollum would have to show him an exit from the caves. Gollum lost the game but had no intention of letting Bilbo leave. He went to retrieve the Ring in order to use its powers of invisibility to help him kill Bilbo, but flew into a rage when he found it missing. Deducing that Bilbo had it from his last question — "What have I got in my pocket?" — Gollum chased him through the caves, not knowing that Bilbo had discovered the Ring's powers of invisibility and was following him to the cave's exit. At one point as he neared the exit, Bilbo was presented with an opportunity to easily kill Gollum, but relented out of pity for Gollum's wretched condition.


Bilbo escaped from Gollum and the Orcs that inhabited the Misty Mountains by remaining invisible, and managed to regroup with Thorin's company and Gandalf. Upon meeting them, he claimed to have escaped the goblins simply by being very agile in the dark of the caverns. Of the Ring, he claimed to Gandalf to have received it from Gollum as a prize for winning the aforementioned riddle-game. Gandalf quickly became suspicious both of Bilbo's story and of the Ring itself, which he immediately recognized as one of the Rings of Power due to the retarding effects it had had on Gollum's aging process. However, he appeared to have believed it to have been one of the lesser Rings. He knew that the smiths of the Gwaith-i-Mírdain had created such Rings as "essays in the craft", and knew nothing of the specific appearance of the One. At the time however, he was quite focused on finding a way to eliminate Smaug before Sauron grew powerful enough to influence him and as such, he did not concern himself overmuch with the Ring.


A few years after Bilbo's return to the Shire, Gandalf decided to address the issue of Bilbo's story with him once again. He managed to coerce from Bilbo the true account of how the Ring had come into his possession but somewhat counter-intuitively, the truth turned out to have been quite innocent. It was also so similar to Bilbo's initial fabrication that Gandalf saw no real reason why Bilbo would have lied about it in the first place, save perhaps to put his claim to the Ring beyond any possible doubt. Gandalf concluded that the Ring had an "unwholesome" effect on its owner that set to work almost immediately, as it was not in Bilbo's nature to lie, particularly about something so apparently trivial. However, he saw no real danger in letting Bilbo keep the Ring despite the Hobbit's strangely possessive attitude towards it. In addition, he would later tell Frodo that he believed that he had had no real right to take the Ring from Bilbo in any event, and so let the matter lie.


In the sixty years that Bilbo owned the Ring he seldom used it, although he kept it on his person at nearly all times. This lack of use meant its malign effects were slow to take hold, the most noticeable being that Bilbo retained a relatively young appearance even past 100 years of age. Gandalf later theorized that Bilbo's virtuous mindset upon having acquired the Ring, specifically his pity for Gollum, had also played a significant role in slowing the Ring's malignancy. Additionally, the few times he did use the Ring appeared to have lacked any malicious intent, such as an occasion where he used it to avoid notice by the Sackville-Bagginses, or when he theatrically vanished at the conclusion of his Farewell Birthday Party speech. This stood in stark contract to Sméagol's early uses of the Ring, which were primarily to commit theft or spy upon people.


Bilbo considers giving up the Ring prior to leaving Bag End, making him the first wearer to abandon it of own free will In TA 3001, Bilbo concocted a plan to leave the Shire for Rivendell, and both he and Gandalf had initially intended for Bilbo's nephew and adopted heir Frodo to inherit both Bilbo's estate and the Ring. As the time came for Bilbo to give it up however, he became extremely reluctant to pass the Ring to his nephew, and his obstinacy over the issue led Gandalf to confront him directly about the Ring. At this point, though Gandalf did not yet know exactly what the Ring was, he could tell that it was both evil and gaining a great deal of influence over his old friend. As such, he advised Bilbo in the strongest terms to give the Ring to Frodo. However, Bilbo became uncharacteristically angry with Gandalf, and argued intensely with him over the matter. After a very stern warning from Gandalf, Bilbo realized that the wizard was right about the Ring's negative impact on his well-being. With Gandalf's aid and advice, Bilbo managed to give up the Ring willingly. He then departed from the Shire, and Frodo came into possession of the Ring.


The truth of the Ring[]


Frodo and the One Ring Bilbo's strangely hostile reaction to giving up the Ring greatly disturbed Gandalf. Troubled by both his encounter with Bilbo and recent events abroad, Gandalf began to consider the possibility that the Ring might be more dangerous than he had suspected. He initially considered revealing his concerns to Saruman, head of the White Council and the Istari. Saruman had devoted many years of his life to the study of the Rings of Power, ostensibly for the purpose of countering Sauron's use of them should he resurface. As such, he was supremely knowledgeable with regard to the properties, powers and natures of the Rings. However, Saruman's pride had increased in tandem with his knowledge over the years, and this combined with a somewhat nebulous feeling of wariness on Gandalf's part led him to keep his own counsel.


Instead, he considered finding and interrogating Gollum in order to help him further understand the nature of the Ring. He searched for news of Gollum, and managed to determine that Gollum had indeed left the Misty Mountains to locate the Ring. However, lulled by the fact that Saruman had told the White Council on at least one occasion that the Ring was beyond finding, he decided to let Gollum be. This proved to be a nearly catastrophic oversight on Gandalf's part, because Gollum's long possession of the Ring had incidentally left him open to an otherworldly summons from Sauron. For at the time, the Dark Lord had been putting forth his power to draw as many evil beings as possible to Mordor to rebuild his forces. As such, Gollum was pulled away from his search for the Shire by Sauron's will and came at last to Mordor.


Gollum was eventually captured while skulking around the borders of the Black Land, and was taken to the rebuilt Barad-dûr. Here, Sauron too recognized the effects of a Ring of Power on Gollum. As the other Rings were all accounted for; being in his possession, destroyed, or in the hands of the Elves, he knew that Gollum must have at some point possessed the One. Under torture, Gollum revealed the existence of Bilbo and the Shire.


Around this time, Gandalf requested of his close friend Aragorn and his fellow Dúnedain Rangers to keep an extremely close watch on the Shire. They soon reported that an inordinately large number of creatures not native to the Shire were being used to keep watch on it at the behest of an unknown party. It would turn out to be Saruman who was responsible for the spying, but at the time, he was trusted by all those who opposed Sauron. Extremely worried at this discovery, Gandalf decided at last to locate Gollum. He requested Aragorn's aid in hunting Gollum, as Aragorn's woodcraft was prodigious even amongst the Rangers. However, Gollum was at the time in Sauron's custody, and so the search was in vain. After months of fruitless wandering, Gandalf gave up on finding Gollum, and left the seemingly hopeless search to Aragorn.


Gandalf discovers Isildur's account of the Ring Desperate for information, Gandalf realized while thinking of Saruman's Ring-lore that the only source from which he could have obtained knowledge of the One was some sort of account left by Isildur, as he was the only person known to have possessed the Ring besides Sauron. Gandalf then traveled to Minas Tirith in search of any records Isildur might have left behind concerning the Ring. After a thorough search, he finally came across a short manuscript by Isildur concerning the Ring's properties, including an important note that the Ring, when heated, seemed to manifest fiery writing on its outer band. Armed with this knowledge, Gandalf began a return trek to the Shire when he learned that, against all odds, Aragorn had somehow managed to find and capture Gollum. Gollum then revealed that he had been to Mordor, but would not tell Gandalf anything about what he had endured there. Gandalf was able to deduce that there was some terror upon Gollum greater than any he himself could induce, and inferred from that that Sauron must have interrogated Gollum personally. As such, Sauron almost certainly knew nearly everything Gandalf knew about the Ring's whereabouts.


Gandalf then hastened to the Shire and confirmed his suspicions; the Ring was indeed the One. Aware that Sauron would use every means at his disposal to get it back, Gandalf instructed Frodo to flee to Rivendell with it, as it was the closest safe haven. Gandalf had intended to accompany them, but was lured to Isengard and imprisoned by Saruman, who wanted the location of the Ring so that he could take it for himself. Before he had departed however, he had given a letter to Barliman Butterbur, the innkeeper of The Prancing Pony in Bree, with instructions that it was to be delivered to Frodo immediately. The letter contained a warning to Frodo that he needed to leave the Shire at once, and had also contained a bit of information about Aragorn, whom Gandalf had instructed to watch for the hobbits and aid them if he could. However, the letter was never delivered, and as such, Frodo delayed his departure in the hope that Gandalf was simply late. Eventually however, Frodo decided that he could no longer wait, and with his companions, Samwise Gamgee, Peregrin Took, and Meriadoc Brandybuck set out without him for Rivendell.


The Hobbits' delay however, had allowed Sauron's servants, a number of strange black-clad horsemen, to journey to the Shire and begin searching for the Ring. The hobbits had a number of close encounters with these black riders, but managed to stay out of their grasp. Within a few days, they reached the village of Bree and encountered Aragorn, who revealed to them that he knew of their quest and was a friend of Gandalf's. He offered to guide them to Rivendell, but the hobbits were wary of his intentions. Fortunately however, Butterbur revealed Gandalf's letter to the hobbit party and they accepted his offer. By then, Gandalf had managed to escape from Isengard, and had begun desperately seeking for Frodo. He quickly reached the Shire, and discovered to his dismay that Frodo had only left a few days before. Having intended for Frodo to have left weeks prior, he realized that Butterbur had almost certainly not delivered his letter.


Furious, he arrived at Bree a mere day after the hobbits had left it, and learned from Butterbur that Aragorn had found them and was guiding them to Rivendell. Greatly comforted by this knowledge, Gandalf set out for Rivendell the next morning. With virtually no hope of finding Aragorn and the hobbits in the wilderness, Gandalf set out for Weathertop, hoping that Aragorn would make for it as well. However, he was attacked at Weathertop by all nine of the Nazgûl, and though he managed to fend them off, he was ultimately forced to flee. In fleeing, he managed to draw four of the Nine into pursuing him, hoping that it would give the hobbits a greater chance of reaching Rivendell. His ploy worked to a degree, as the Nazgûl would later attack Aragorn and the hobbits at Weathertop, but with only five of their number. The Witch-king managed to stab Frodo in the shoulder with a Morgul-knife, but Aragorn managed to drive them off. Weeks later Aragorn and the Hobbits, with Glorfindel's aid, reached Rivendell. There, Frodo was saved by Elrond's healing arts from a fragment of the knife which had remained lodged in his shoulder, and the Ring was made temporarily secure.


Quest of the Ring[]


The Council of Elrond, as depicted in The Lord of the Rings: The Fellowship of the Ring (2001) At Rivendell, a Council overseen by Elrond was convened, which incidentally included members of every free race in Middle-earth, to decide what to do with the Ring. Numerous members of the Council advocated for hiding the Ring, sending it to Aman, or using it against Sauron, but all were ultimately rejected as possibilities. It was also debated whether or not it could be given to a safe guardian, such as Tom Bombadil, or left in Rivendell. However, these options too were rejected, as no place or guardian could be entrusted to withstand Sauron's full power when he came at last to take the Ring. As long as the Ring existed, Sauron's power would continue to grow, and the strength of the remaining Elves and Edain of Middle-earth was far less than it once had been. As things stood, Sauron would not need to regain the Ring to achieve eventual victory. Nor could it be used against Sauron, for the Ring was of Sauron and thus wholly evil. As such, any with the strength to effectively wield the One against Sauron would ultimately become a Dark Lord in turn, eliminating any point in having used it in the first place.


It was eventually decided that the Ring needed to be taken to Mordor and cast into Mount Doom, where it could be destroyed. This action would cripple Sauron's powers, and would end his ability to influence the world forever. Frodo volunteered to carry the Ring there, and a company of eight companions were chosen to accompany him. The Fellowship of the Ring, as the group came to be known, began the long trek to Mordor.


Following their adventure through Moria, during which Gandalf fell, and their time in Lothlórien, the Fellowship was scattered when Frodo and Sam split off from the rest of the group after an Uruk-hai attack. They continued on from Nen Hithoel to Mordor alone, without any clear sense of how to get there. Frodo and Sam soon became lost in the Emyn Muil, where they encountered Gollum, who had been shadowing them ever since Moria. Frodo and Sam managed to capture him, and after swearing an oath to serve the Ring's owner (in the immediate case Frodo), Gollum was ordered to lead them to Mordor, as he had been there before and knew the way.


Passing through the Dead Marshes, the hobbits came to the Black Gate and prepared to enter Mordor. However Gollum, learning only upon their arrival of Frodo's intent to actually enter Mordor, revealed that there was another way in; the pass of Cirith Ungol. On their way to the pass, the hobbits encountered Faramir and a group of Ithilien Rangers. Learning of their goal, Faramir aided them, providing supplies of food and water, instead of taking the Ring as his father would have wished.


Reaching Minas Morgul, the hobbits began their climb up the winding Stairs of Cirith Ungol to a long tunnel. Here, Gollum betrayed them, for inside the tunnel dwelt the monstrous spider Shelob, who sought to consume the two hobbits. Sam and Frodo nearly escaped, but Frodo was stung by Shelob and captured by her. Sam managed to drive the gigantic spider off, but believing that Frodo was dead, he took the Ring and resolved to finish the quest himself. However, Frodo had simply been paralyzed by Shelob's venom, making him appear dead. He was captured by a group of Orcs while he was unable to move but Sam, who had overheard the Orcs discussing Frodo's condition, followed them to the Tower of Cirith Ungol. Shortly thereafter, Sam rescued Frodo in the aftermath of a mutinous battle among the Orcs who had captured him, which had resulted in nearly the tower's entire garrison killing each other. Returning the Ring to Frodo, the two began the arduous trek towards Mount Doom.


Destruction[]


After a few days, Frodo and Sam reached the volcano, using Sauron's Road to climb it, but were ambushed by Gollum. Fending him off, Frodo continued to the Crack of Doom. But throughout his quest, the Ring had continued to tighten its hold on Frodo's mind, growing more and more powerful the closer it came to the place of its making. Entering and coming to the Cracks of Doom, Frodo claimed the Ring for his own and put it on. Sauron immediately spotted him and, realizing the magnitude of his folly, sent the Nazgûl on winged mounts to retrieve it. As fortune would have it however, Gollum, who had been spared moments before by Sam, attacked Frodo and bit the Ring and most of the finger it had been on off of Frodo's hand. Then, dancing with joy over retrieving the Ring, Gollum took one misstep and toppled over the side of a cliff into the Crack of Doom (this was one of Eru's few interventions in the events of Middle-earth).[3] There, the Ring was swiftly melted, undoing Sauron's power and finally vanquishing him.


Properties[]


Effects[]


The One Ring as portrayed for the films When a typical person put on the Ring, they would be partly "shifted" out of the physical realm into an unseen realm, walking its threshold. A side-effect (though usually the first effect noticed) was invisibility to physical beings like living Men, but this brought high visibility to other unseen beings, such as Ringwraiths.


For mortals, the Ring had several side-effects, most of them negative. Perhaps the first was that the bearer soon developed a strong attachment to it, becoming increasingly reluctant to relinquish it. Like the other great Rings of Power, it would extend the lifespan of a mortal bearer indefinitely, but the bearer would not grow or obtain more vitality. Instead the bearer would merely "continue" until existence became unbearably wearisome. Gandalf also held that if the Ring was worn too frequently, the wearer would become wraith-like over time and become entirely subsumed into the spirit world. The rate at which this occurred depended largely on the wearer's inner nature. Race also seemed to perhaps play some part in it, as Hobbits in particular seemed to "fade" rather reluctantly. Merely keeping the Ring without using it greatly slowed most of the negative effects of the Ring, but it could not do so indefinitely.


Additionally, prolonged life came to its bearer independently of their use of the Ring. Bilbo and Gollum hardly ever used the Ring, yet aged remarkably well (even in Gollum's case, though his lifestyle and extreme age warped his body). The circumstances of initially finding the Ring also played a profound role in how quickly the Ring's negative effects began to take hold of the bearer, as did the bearer's own disposition. Gollum, for instance, obtained the Ring by murdering his close friend, and as such the negative effects of the Ring on his mind in particular manifested quickly. Bilbo, on the other hand, not only found and obtained the Ring by chance, but when confronted with an opportunity to slay Gollum minutes after finding it, chose to spare him out of pity for his wretchedness. Due to this, Bilbo was ultimately able to give it up (sixty years later) of his own free will, which Gollum would have been wholly unable to do.


One potentially positive effect was that the Ring may have granted the wearer some understanding of the speech of evil creatures: while wearing it, Samwise Gamgee was able to understand the Black Speech of the Orcs in Mordor and Bilbo was able to understand the Mirkwood Spiders. It also sharpened one's senses.


Frodo holding the One Ring The Ring, being essentially an extension of Sauron himself, was evil in nature. It seemed to possess at least a limited will of its own, and could "call out" subliminally to other people, in an attempt to get them to pick it up or possibly kill the current holder. After being cut from Sauron's hand, the Ring was, as Gandalf put it, "trying to get back to its master" and used all of its power and influence to find its way back to the Dark Lord. It was also capable of changing sizes and could easily slip off of a finger where once it had been tight, with no obvious explanation as to why. Frodo Baggins was warned by Bilbo of this oddity, and kept it on a chain to make sure it never got lost.


The Ring's power grew throughout Frodo's journey, particularly on the last stage of the quest when Sam and Frodo entered Mordor and approached Mount Doom. Inside the Sammath Naur, the Ring was so powerful that not even a bearer that intended it harm would be able to do so, as was the case with Frodo Baggins.


Although the Ring was crafted by Sauron and served as a remote vessel of his essence and power, its loyalty for its maker was not absolute. Gandalf implies that if he or any other exceptionally powerful being (such as Saruman or Galadriel) were to take and use the Ring to enhance their power enough to destroy Sauron, the Ring would not lose its own power and instead consider its bearer its new master. However, the Ring would retain its corrupting nature and twist whomever had destroyed Sauron to evil.


Powers[]


"War is upon us and all our friends, a war in which only the use of the Ring could give us surety of victory." —Gandalf[4]


The Ring's primary power was control over the other Rings of Power, including "mastery over [their] powers" and domination of the wills of their users. However, its effectiveness in this manner proved limited, as the wielders of the Three never used them while Sauron held the One, and the Dwarves to whom the Seven had been given proved too tough for Sauron's mental influence to take hold. By extension, the Ring also conferred the power to dominate the wills of other beings whether they were wearing Rings or not. However, this was its least accessible power, since it granted this ability in proportion to the user's natural capacity.


Perhaps most usefully, the Ring was capable of augmenting the abilities and powers of whatever being held it. Also, the wielder of the One Ring might not have possessed most if not all the powers of the Ring. Only Sauron or someone similar in power and experience with the Ring would be able to wield its full power. Varying powers or effects of the Ring are seen in the wielder depending on their power, skill, abilities, natural capacity, and situation. When Samwise Gamgee held the Ring he could hear far better and with more acute accuracy while trying to sneak around Cirith Ungol to evade the Orcs and save Frodo. Frodo was able to see for miles when wearing it.


In all cases, a mortal wearing the Ring became invisible, except to anyone able to perceive the non-physical world, with only a thin, shaky shadow discernible in sunlight. Whether immortals would be made invisible by it is unknown, as the only immortal being who ever wore the Ring was Tom Bombadil, over whom the Ring had absolutely no power whatsoever. However, Bombadil appeared to have been unique in that regard, as both Gandalf and Saruman were susceptible to the Ring's influence, and Bombadil was anomalous in many other ways.


Another power of the Ring was the ability to project a false vision of its wearer to observers. When Sam encountered an Orc in the Tower of Cirith Ungol while holding the Ring, he appeared to the Orc as a powerful warrior cloaked in shadow "[holding] some nameless menace of power and doom." The Orc was so terrified of this illusion that it fled from the otherwise unintimidating Sam. Similarly at Mount Doom, when Frodo and Sam were attacked by Gollum, Frodo grabbed the Ring and appeared as "a figure robed in white... [and] it held a wheel of fire." In this scene, Frodo (or perhaps the Ring itself) spoke "with a commanding voice" foretelling the destruction of Gollum.


However, the Ring did not offer the wielder protection from physical harm. While wearing the Ring, Frodo was still seriously injured by the Witch-king and his Morgul-knife, and lost a finger when Gollum bit it off. Sauron himself suffered the destruction of his physical body at the hands of Gil-galad and Elendil while wearing the Ring.


As it contained the better part of Sauron's native power, it seemed to exhibit a malevolent, but limited, form of sentience. While separated from Sauron, the Ring would strive to return to him, both by impelling its bearer to yield to Sauron or his servants, or by abandoning its possessor at key moments. The Ring had the ability to change size. As well as adapting to fingers of varying size, from Sauron's to Frodo's, it sometimes suddenly expanded to escape from its wearer. For example, it slipped off of Gollum's finger when the time was right for it to be brought back into the world at large. Sauron was also capable of sensing the location of the Ring if someone put it on for any extended period of time, even if that person was hundreds of miles away from him.


To fully master all of the Ring's abilities, a wielder of the Ring would need an extremely disciplined and well-trained mind, a strong will, and a high degree of spiritual development. Even for those with the necessary prerequisites it would have taken time to master the Ring's powers to the point at which they would be strong enough to overthrow Sauron, and, hypothetically, bring peace. While this is a tantalizing prospect for some, in the end, the Ring's inherent corruption would twist its bearer into another Dark Lord as evil as Sauron was, or worse, regardless of their intentions at the outset. This result was apparently inevitable no matter how well-intentioned the bearer, as even fellow Maiar like Gandalf feared to so much as possess the Ring lest its power begin to take hold.


Despite its powerful qualities, neither the Ring's innate power nor its power over others was absolute. Three times Sauron suffered military defeat with it in his possession, first by Tar-Minastir in the SA 1700, and again by Ar-Pharazôn in SA 3262 when the Númenóreans' might so overawed his armies that they deserted him. He was defeated militarily once more at the end of the Second Age by the Last Alliance of Elves and Men, which culminated in his personal defeat at the hands of Gil-galad, Elendil and Isildur.


Ring-bearers[]


In total, the One Ring existed for c. 4867 years and was held by nine people, five of whom were hobbits. Sauron was by far the one to carry it for the most time (c. 1850 years), followed by Gollum (478 years), Bilbo (60 years), Frodo (17 years) and Isildur (2 years). Tom Bombadil wore the Ring temporarily on September 26, 3018 but was apparently unaffected by it. Of those who ever wore it, only Samwise Gamgee, Bilbo Baggins, Frodo Baggins and Tom Bombadil gave it up willingly. "Bearing" the Ring does not seem to be synonymous with merely touching or carrying it, since Gandalf refused to bear the Ring but was willing to handle it for a few seconds in Bag End.


Order


Name of Holder


From


Until


Duration


1st


Sauron


About SA 1600, when it was forged


SA 3441


about 1850 years


2nd


Isildur


SA 3441, after the defeat of Sauron; equivalent to TA 1


October 5, TA 2


about 2 years


---


the Ring was lost in the Anduin River


October 5, TA 2


TA 2463


2461 years


3rd


Déagol


Unknown date, TA 2463


The same day, TA 2463


A few minutes


4th


Gollum (Sméagol)


TA 2463


July, TA 2941


478 years


5th


Bilbo Baggins


July, TA 2941


September 22, TA 3001


60 years


6th


Frodo Baggins


September 22, TA 3001


March 14, TA 3019


About 17 years, 6 months


7th


Gandalf


TA 3018


TA 3018


A few seconds


8th


Tom Bombadil


September 26, TA 3018


September 26, TA 3018


A few minutes


9th


Samwise Gamgee


March 14, TA 3019


March 15, TA 3019


1 day


---


Frodo Baggins


March 15, TA 3019


March 25, TA 3019


10 days


---


Gollum (Sméagol)


March 25, TA 3019


March 25, TA 3019


A few seconds


Fate[]


Of the several bearers of the One Ring, three were still alive following the One Ring's destruction: Bilbo Baggins, Frodo Baggins, and Samwise Gamgee. Bilbo, having borne the Ring longest of the three, had reached a very advanced age for a Hobbit. Frodo suffered both physical and psychological scars from his strenuous quest to destroy the Ring. Samwise, having only briefly kept the Ring was affected the least and appeared to carry on a normal life following the Ring's destruction.


In consideration of the trials the Ring-bearers had endured, special dispensation was granted them by the Valar to travel to Tol Eressëa, the Elvenhome; though not the Undying Lands themselves, where it was hoped they could find rest and healing. At the close of The Return of the King, Bilbo and Frodo embark for the voyage to the West along with Galadriel, Elrond, and many of their folk, as well as Gandalf. Near the end of his life, Samwise is also said to have sailed to Eressëa. Tolkien in one of his letters described the process as a period of extended life and healing, after which, their spiritual scars cured, they would die in peace — if they so wished. Otherwise no mortal could set foot in the Undying Lands.


Description[]


Ash nazg durbatulûk, ash nazg gimbatul, ash nazg thrakatulûk, agh burzum-ishi krimpatul.


The Ring appeared to be made of real gold, but was impervious to damage. Even dragon-fire was said to be inadequate to harm the One Ring. It could only be destroyed by someone whose smithcraft was as great as Sauron's, or, by the fires within Mount Doom, where it had been created. Like the lesser rings forged by the Elves as "essays in the craft" before the Great Rings, it bore no gem, but when heated, it displayed a fiery Tengwar inscription in Elvish script. The lines were later taken up into a rhyme of lore describing the Rings, but they were evidently part of the spell that imbued the One Ring with power, since the Elves heard Sauron utter the same words during the Ring's creation, whereupon they took off their own Rings and foiled his plan.


Normally, the One Ring appeared perfectly plain and featureless, but when cast into fire the inscription appeared in fiery letters inside and outside the Ring.[5] A transliteration was seen when Gandalf read the ring-inscription during the Council of Elrond.[6]


Ash nazg durbatulûk, ash nazg gimbatul, ash nazg thrakatulûk, agh burzum-ishi krimpatul.—The inscription on the One Ring Roughly translated, the words mean:


One Ring to rule them all, One Ring to find them, One Ring to bring them all and in the darkness bind them.


When the Ring was first forged, Sauron spoke these words aloud, and Celebrimbor, maker of the Three Rings of the Elves, heard him from afar and was aware of his now-revealed purposes. The inscription uses Elvish lettering because all forms of writing J.R.R. Tolkien describes at that time were invented by the Elves.


Some recent editions of The Fellowship of the Ring accidentally omit the first two clauses of this phrase from Chapter 2, an error that was corrected by the time of the 50th Anniversary editions. The first four lines of the verse introduce three of the races inhabiting Middle-earth, as well as the eponymous title character, the Lord of the Rings:


Three Rings for the Elven-kings under the sky, Seven for the Dwarf-lords in their halls of stone, Nine for Mortal Men doomed to die, One for the Dark Lord on his dark throne In the Land of Mordor where the Shadows lie. One Ring to rule them all, One Ring to find them, One Ring to bring them all and in the darkness bind them In the Land of Mordor where the Shadows lie.


Gandalf first learned of the Ring-inscription when he read the account that Isildur had written before marching north to his death and the loss of the Ring. When Isildur had cut the Ring from Sauron's hand, it was burning hot, and so Isildur was able to transcribe the inscription before it faded. When Gandalf subsequently heated the Ring that Bilbo Baggins had found and passed on to Frodo the inscription appeared, the wizard had no doubt that it was the One Ring.


Symbolism and meaning[]


"The ring that Frodo carries is so real that it can't be tossed away and forgotten about. The consequences of doing so would be so disastrous that it would be an unthinkable act......The ring is not a fantasy, a figment of the mind, an urge, a desire, or an opinion which can be changed, it is a fact which must be faced, borne and finally overcome." —Joseph Pearce on the objectivity of the One Ring[7]


Sauron's Ring represents the addiction to power, the idolization of material things, and the contravening of human nature (in giving its bearer a wholly unnatural lifespan).


J.R.R. Tolkien held, however, that his works should not be seen as strict symbolism (he in fact believed that an author telling a reader how to think or feel about a story acts as a kind of tyrant, as seen in The Letters of J. R. R. Tolkien). The Andvarinaut in the Völsunga saga is considered to be one of the inspirations for the Ring, though Tolkien himself credited many other myths that revolve around the separation of a part of oneself to grant immortality.


In The Philosophy of Tolkien, Peter Kreeft explains how the One Ring, in meaning and utility, is the opposite of the Christian cross.


Earlier in the 20th century, many readers thought the Ring was an allegorical symbol of the Atomic bomb used in World War II.


In other versions[]


When The Hobbit was written, Tolkien had not yet conceived the Ring's sinister back-story. Thus, in the first edition of The Hobbit, Gollum surrenders the Ring to Bilbo as a reward for winning the Riddle Game. However, as Tolkien changed the nature of the Ring to fit into the legendarium of Middle-earth, he realized that the Ring's grip on Gollum would never permit him to give it up willingly. Therefore, Tolkien revised this chapter in the second edition of The Hobbit, having Gollum offer to show Bilbo the way out instead of offering to give up the Ring. Tolkien then decided that the first edition's version of events was how Bilbo had originally told the story to Gandalf and the Dwarves of Thorin's company, rather than what had actually occurred.


Trivia[]


The 13th Century Welsh king of Gwynedd, Llywelyn the Great, had a ring made for his wife-to-be Joan, Lady of Wales, the illegitimate daughter of King John of England. The ring had an inscription which read: Un fodrwy i ddangos ein cariad; Un fodrwy i'n clymu.


This inscription translates as:


One ring to show our love; One ring to bind us.


Llywelyn and Joan married in 1206. The inscription would have been known to Tolkien who had ready access to the Red Book of Hergest and the White Book of Rhydderch, the two main sources of the Welsh legends of the Mabinogion, in which Llywelyn the Great features and is known to have been part of Tolkien's library.


The sequence in Magyk by Angie Sage in which the main character Septimus finds the Dragon Ring is oddly similar to when Bilbo finds the Ruling Ring. An imitation of the One Ring makes an appearance in the animated movie Shrek 2; this version is a new wedding ring that Shrek has forged for his new wife Fiona, which displays 'I love you' in fiery writing once on her finger. The One Ring bears many similarities to the magic ring in Richard Wagner's Der Ring des Nibelungen cycle. Both rings' powers were bestowed upon them by dark magic, both are desired by all who do not possess them yet bring no contentment to their owners, and both have caused an individual to kill someone close to them in order to gain possession of it, an act for which they are cursed and eventually turned into a grotesque creature. Fans of both Lord of the Rings and Harry Potter have compared the One Ring to the Horcruxes used by Lord Voldemort in Harry Potter, as both the Ring and Horcruxes contain the essence of their creators and serve to protect the one who created them from permanent death by preserving a part of the creator's soul in another object. So powerful was the One Ring that even looking at it could inspire "Dragon Fever" Boromir attacked Frodo because of it although he redeemed himself by dying in defense of the hobbits from the Orcs; Saruman and Gollum were another victims of "Dragon Fever" because they both desired the One Ring. Saruman because he desired power for himself; Gollum because he killed his cousin Deagol for possession of it; at the end Frodo Baggins despite knowing the consequences of not destroying the One Ring fell under its power and proclaimed it was his; Gollum fought Frodo and biting off his finger regained possession of the One Ring until he falls into the Cracks of Doom; the ring survives him by only a few seconds before it is destroyed. Of all those who had possession of the One Ring only three were not affected very much by "Dragon Fever"; Bilbo Baggins because he showed mercy to the wretched Gollum; Deagol had the ring for a few minutes; Samwie Gamgee had it but never let it dominate his mind-in fact he took it from Frodo because he did not want the Orcs to have possession of it after Frodo was spider bitten and captured by Goblins; Sam in fact gives the ring back to Frodo and reminds Frodo not to forget their quest at the Crack of Doom. A variation on "Dragon-Sickness" is greed: In Tolkien poem "The Hoard" an old elven treasure hoard is claimed by a dwarf; a dragon and a King-each in turn becomes consumed with the possession of the treasure until each in turn is destroyed by someone even more greedier than the present owner.[8]


See also[]


Rings of Power


Translations[]


Foreign Language


Translated name


Arabic


الخاتم الأوحد


Afrikaans


Een Ring


Albanian


Një Unazë


Amharic


ነጠላ ቀለበት


Aragonese


Aniello Unico


Armenian


Իշխանության մատանի


Assamese


একক আঙঠি


Basque


Eraztun Bakarra


Belarusian Cyrillic


Адзіны Пярсцёнак


Bengali


একক রিং


Bulgarian Cyrillic


Един пръстен


Cambodian


ចិញ្ចៀនតែមួយ


Catalan


Anell Únic


Chinese (Hong Kong/Taiwan)


至尊魔戒


Chinese (Mainland)


指环王


Cornish


Huni Besow


Croatian


Jedinstveni Prsten


Czech


Jeden Prsten


Danish


Herskerringen


Dutch


De Ene Ring


Estonian


Üks Sõrmus


Esperanto


Unu Ringo


Faroese


Einkringur


Filipino


Ang Iisang Singsing


Finnish


Sormusten sormus


French


Anneau Unique


Galician


Anel Único


Georgian


ერთი ბეჭედი


German


Der eine Ring


Greek


Το ένα δαχτυλίδι


Gujarati


એક વીંટી


Hebrew


הטבעת האחת


Hindi


एक रिंग


Hmong


Ib lub nplhaib


Hungarian


Egy Gyűrű


Icelandic


Hringurinn eini


Irish Gaelic


Fáinne Amháin


Italian


Unico Anello


Interlingua


Anello Unic


Japanese


一つの指輪


Kannada


ಒಂದೇ ಉಂಗುರ


Kazakh


Бір сақина (Cyrillic) Bir saqïna (Latin)


Konkani


एकूच रिंग


Korean


절대반지


Kurdish


تاکە ئەڵقە (Sorani) Yek Qulp (Kurmanji)


Kyrgyz Cyrillic


Бир шакек


Laotian


ແຫວນຫນຶ່ງ


Latin


Unus Anulus


Latvian


Viens Gredzens


Lithuanian


Vienas žiedas


Luxembourgish


Een Ring


Macedonian Cyrillic


Оне Ринг


Maithili


एकल अंगूठी


Malayalam


ഒരു മോതിരം


Malaysian


Satu Cincin


Maltese


Ċirku Wieħed


Manx


Un Fainey


Maori


Kotahi te mowhiti


Marathi


एक अंगठी


Mongolian Cyrillic


Нэг бөгж


Nepalese


एकल औंठी


Norwegian


Ene Ringen


Pashto


واحد گوته


Persian


حلقه یگانه


Polish


Jedyny Pierścień


Portuguese (Portugal and Latin America)


Um Anel


Querétaro Otomi


'Nar Anillo


Romanian


Inelul Suprem , Unul


Russian


Кольцо Всевластья


Samoan


Tasi Mama


Sanskrit


एकवलयम्


Scots


Ane Ring


Scottish Gaelic


Aon Fàinne


Serbian


Један Прстен (Cyrillic) Jedan Prsten (Latin)


Shona


Mumwe Mhete


Sicilian


Unu Aneddu


Sinhalese


එක් මුදුවක්


Slovak


Jeden Prsteň


Slovenian


Prstan Mogote


Spanish (Spain and Latin America)


Anillo Único


Sundanese


Hiji Ngirining


Swahili


Pete Moja


Swedish


Härskarringen


Tamil


ஒன்று மோதிரம்


Tatar


Бер боҗра


Telugu


ఒక ఉంగరం


Thai


เอกธำมรงค์


Turkish


Tek Yüzük


Ukrainian Cyrillic


Єдиний Перстень


Uzbek


Битта узук (Cyrillic) Bitta uzuk (Latin)


Welsh


Un Fodrwy


Xhosa


Nye Indandatho


Yiddish


איין רינג


Yoruba


Oruka Kan


Zazaki


Engıştaneyo Gırd


References[]


↑ The Letters of J. R. R. Tolkien, Letter 211


↑ The Letters of J.R.R. Tolkien, Letter 211


↑ The Letters of J. R. R. Tolkien, Letter 192


↑ The Lord of the Rings, The Two Towers, Book Three, Chapter V: "The White Rider"


↑ The Lord of the Rings, The Fellowship of the Ring, Book One, Chapter II: "The Shadow of the Past"


↑ The Lord of the Rings, The Fellowship of the Ring, Book Two, Chapter II: "The Council of Elrond"


↑ Tolkien: A Celebration, 9: Joseph Pearce, "Tolkien and the Catholic Literary Revival"


↑ Tolkien Gateway - The Hoard


Categories


Categories:


Rings and jewels


Magical objects


Languages


Deutsch Español Suomi Français Italiano Nederlands Polski Português do Brasil Русский Slovenčina Українська


Community content is available under CC-BY-SA unless otherwise noted.


More Fandoms


Fantasy Lord of the Rings


Advertisement


Fan Feed


More The One Wiki to Rule Them All


1 Sauron


2 Morgoth


3 Adar


Explore properties


Fandom


Muthead


Fanatical


Follow Us


Overview


What is Fandom?


About


Careers


Press


Contact


Terms of Use


Privacy Policy


Digital Services Act


Global Sitemap


Local Sitemap


Cookie Preferences


Community


Community Central


Support


Help


Do Not Sell or Share My Personal Information


Advertise


Media Kit


Contact


Fandom Apps


Take your favorite fandoms with you and never miss a beat.


The One Wiki to Rule Them All is a FANDOM Movies Community.


View Mobile Site


Follow on IG


TikTok


Join Fan Lab"""
input_ids = tokenizer.encode_plus(
                text,
                return_offsets_mapping=True,
                return_token_type_ids=False,
                return_attention_mask=False,
                add_special_tokens=False
            )
print(input_ids.data["offset_mapping"][10882:10885])

### Expected behavior

Three separate tokens have the same start and end offset: (44923, 44924), (44923, 44924), (44923, 44924).
Expected behavior each token has different start or end offset. 
Problem might be that word is non-english.

Comments:
Hi @fil-onto, can you shorten the example text a lot and identify which characters are triggering this issue?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.