It may have originated from seed hemp cultivated there for making oil, bird feed or human food. In other countries varieties grown for seed production were sometimes psychoactive. Another possibility is that potent cannabis was introduced from India or Southeast Asia, where cannabis smoking has a long history. Until 1948 cannabis could be legally imported into Japan from other countries. Marijuana, hashish or viable seeds could be shipped from India and Thailand to Japan.
Japan has a long history of trading with China, which in turn sent ships as far as Africa.
In the 8th century Buddhism reached Japan from India via Nepal and China. Later Japan was trading with Portugal which had a base in Goa, India and with the Dutch who colonized Indonesia, places where marijuana use was not uncommon. The famous Silk Road carried trade from Arabia, Persia, Afghanistan all the way to Japan. It would not be surprising if the smoking materials travelled along the same route as the smoking instruments. In fibre hemp on the other hand the most common cannabinoid is CBD, which is not psychoactive.