I think the speech should be seen in the context of the struggle against slavery.

This was a battle against substantial commercial vested interests and deeply held racism/race hatred. Wilberforce and other abolitionists argued from biblical belief that all had been made in the image of God. They believed (and spoke and wrote) that slavery was contrary to Christianity, and that to fail to fight slavery was unchristian. Additionally, as in the United States, the use of slaves by the ruling class was held to be an attack on the economic position of the working class, so there was a popular sentiment against slavery which was not moral or anti-racist but instead self interested. Most English people thought that if blatant slavery were allowed they would loose their livelihoods and would either starve or end as slaves themselves. Note: there were many practices in England at this time that would be described as slavery now, but they stopped short of the extraction of free labour via violence as per field slaves in the USA and the Caribbean.

The position of slavery in law was tenuous and attacked by the racism of the establishment and the vast wealth of slave owning and slave trading interests. It rested on the common law assertion of the rights of individuals in England. These rights had emerged over 700 or so years as a bargain between the English ruling class, the King and the peasantry and were the basis of civil society. Undermining them could (and had cf. France, the English Civil War, the Peasants Revolt) led to mass slaughter, including of noble people and members of parliament. They were not explicit or clear.

The position at the time of the speech was that it was vaguely agreed that if someone was in England they could not be a slave. Some people included non-Caucasians in the "someone" category, but there were a lot of people who would exclude non-Caucasians (and other groups like Irish people, Scottish people, people who were poor and so on) from the definition of humanity. There were many examples of slaves being owned and no one doing anything about it. This is very similar to the position in law of domestic violence in the 20th century, it was quite possible to assert that it was banned completely, but in fact it was allowed so long as no one really kicked up a fuss, which if you were very powerful would be never - although if you beat your wife to death you would probably get in trouble.

The question, for the pro-slavers was, where to drive the wedge so as to avoid the nasty "my house is on fire and there are 1000 people outside with sticks" episodes that other ruling classes had been faced with.

So for Wilberforce it was fundamental to assume that Slavery was utterly banned in England, and that it was unthinkable and inadmissible to consider it. Tacitly the argument was that if you drew the line anywhere at all then the position of the elite would be unsustainable. Wilberforce believed that all humans were made in the image of God and were God's children equal in his eyes but was trying to sell the proposition that having slaves in England was impossible and the slave trade was necessarily wicked.

It was not established at all that slavery outside of England and the protection of the Common Law as enforced by God's representative (the King of England) was not allowed. And no one believed that slaves in the USA or any other god forsaken hole would lead to a revolution in England. Many racists didn't care at all that non-Caucasians were enslaved, and didn't see anything wrong with it.

Additionally some silly behaviour in Boston harbour and the subsequent brilliantly diplomatic and subtle disengagement from the territory of the USA by the British (we ran away) meant that there was no prospect of the English parliament being able to assert a ban on slavery in the USA. Also a direct assault on the economics of the sugar plantations in Jamaica would have stirred some strong opposition that the abolitionists were keen to avoid.

The propensity of the English working classes to violence could not be invoked to attack the slave trade or ownership overseas. They could be relied on to prevent the admission in public debate that anyone wanted to establish mass slave ownership in England. The evident cruelty of some of the practices of slavery were widely used to attack it, but it was argued that these were not part of the institution of slavery, any more than cruelty to livestock was part of farming. The cruel practices of slavery could be characterised as aberrant and amenable to remedy. But Wilberforce is arguing that the institution of the Trade of Slaves is necessarily cruel, and these cruelties are not amenable to remedy.

This is why Wilberforce is attacking the trade and not the practice.