The time for grilling swordfish depends, like most everything you grill, on the thickness of the fish. An average of 8 to 10 minutes total for a 1-inch steak is about right, with the heat source being about 4 inches away from the grill.