The simple answer is ownership however there are also two other main areas of difference.
Firstly the treatment of VAT on commercial vehicles is generally different, with full vat on the cost of the goods payable in advance on Hire Purchase and VAT payable on the monthly payments, spreading the VAT across each rental.
Secondly some leases may become ‘off balance sheet’, which means that neither the asset nor the liability appear on the balance sheet. Just the net payment, treated as a direct overhead cost. HP is always ‘on balance sheet’ meaning the asset appears in your Fixed Assets and the balance outstanding in your liabilities. The HP interest is an allowable expense of the business.
Some finance leases, because you have a benefit in the residual sale proceeds, are treated very similarly to Hire Purchase and capitalised as leased assets. Realistically you should speak to your accountant for the best advice specific to your circumstances.