In Scotland they are the Bank of Scotland, the Royal Bank of Scotland and the Clydesdale Bank. In larger towns and cities banks will normally stay open until at least 4.30pm and some are open on Saturday mornings. Many of these are available to use 24 hours a day, but some do still close for a few hours during the night. If you can not find a bank, most larger supermarkets and some petrol stations will also have ATMs available where you can withdraw cash. But in general terms, it is not particularly cheap.
Costs also vary regionally, with London being the most expensive place to visit.
This is almost always included in the prices you will see in shops, pubs, restaurants and hotels. There is a scheme called the Retail Export Scheme, but not all shops participate and services such as hotel services are excluded. If a store participates in the scheme it will normally display a sign in the window. Then you present the goods, form and invoice to customs as you depart from the EU, the form is returned to the store and a refund forwarded to you, minus a small charge. You should be able to get these from student unions, student travel agencies or hostelling associations.