A high-strength heavy-walled stainless steel seamless tube or pipe having a wall thickness of 12.7 mm or more and less than 100 mm, an excellent low-temperature toughness of absorbed energy of 50 J or more at a test temperature of -10°C in Charpy impact test at the wall thickness center position and a yield strength of 654 MPa or more, characterized by comprising a chemical composition containing, on a percent by mass basis, Cr: 15.5% to 18.0%, C: 0.030 to 0.050%, Si: 1.00% or less, Mn: 0.20% to 1.80%, Ni: 1.5% to 5.0%, Mo: 2.0% to 3.5%, N: 0.01% to 0.15%, O: 0.006% or less, P: 0.03% or less, S: 0.005% or less, optionally V: 0.02% to 0.20%, optionally at least one group selected from Group A to Group D below, Group A: Al: 0.002% to 0.050% Group B: at least one selected from Cu: 0.8% to 3.5%, W: 0.5% to 3.5%, and REM: 0.3% or less Group C: at least one selected from Nb: 0.2% or less, Ti: 0.3% or less, and Zr: 0.2% or less Group D: at least one selected from Ca: 0.01% or less and B: 0.01% or less, and the remainder composed of Fe and incidental impurities, and a steel microstructure containing a ferritic phase at an area fraction of 20% or more, a martensitic phase at an area fraction of 50% to 80%, an austenitic phase at an area fraction of 15% or less, and the ferritic phase, the martensitic phase, and the austenitic phase sum up to an area fraction of 100%, wherein the maximum value of the areas of the ferrite grains in the steel microstructures in a circumferential direction cross-section and an L direction (rolling direction) cross-section of the steel tube or pipe is 3,000 μm 2 or less and the content of ferrite grains having areas of 800 μm 2 or less is 50% or more on an area fraction basis, where when adjacent ferrite grains are present in the steel microstructure and the crystal misorientation between one ferrite grain and the other ferrite grain is 15° or more, the adjacent grains are assumed to be grains different from each other, the circumferential direction cross-section and the L direction cross-section of the steel tube or pipe referring to a wall thickness central portion at a center in the rolling direction of the steel tube or pipe. characterized by comprising a chemical composition containing, on a percent by mass basis, Cr: 15.5% to 18.0%, C: 0.030 to 0.050%, Si: 1.00% or less, Mn: 0.20% to 1.80%, Ni: 1.5% to 5.0%, Mo: 2.0% to 3.5%, N: 0.01% to 0.15%, O: 0.006% or less, P: 0.03% or less, S: 0.005% or less, optionally V: 0.02% to 0.20%, optionally at least one group selected from Group A to Group D below, Group A: Al: 0.002% to 0.050% Group B: at least one selected from Cu: 0.8% to 3.5%, W: 0.5% to 3.5%, and REM: 0.3% or less Group C: at least one selected from Nb: 0.2% or less, Ti: 0.3% or less, and Zr: 0.2% or less Group D: at least one selected from Ca: 0.01% or less and B: 0.01% or less, and the remainder composed of Fe and incidental impurities, and a steel microstructure containing a ferritic phase at an area fraction of 20% or more, a martensitic phase at an area fraction of 50% to 80%, an austenitic phase at an area fraction of 15% or less, and the ferritic phase, the martensitic phase, and the austenitic phase sum up to an area fraction of 100%, wherein the maximum value of the areas of the ferrite grains in the steel microstructures in a circumferential direction cross-section and an L direction (rolling direction) cross-section of the steel tube or pipe is 3,000 μm 2 or less and the content of ferrite grains having areas of 800 μm 2 or less is 50% or more on an area fraction basis, where when adjacent ferrite grains are present in the steel microstructure and the crystal misorientation between one ferrite grain and the other ferrite grain is 15° or more, the adjacent grains are assumed to be grains different from each other, the circumferential direction cross-section and the L direction cross-section of the steel tube or pipe referring to a wall thickness central portion at a center in the rolling direction of the steel tube or pipe.