While a physical wallet that we keep in our pocket or purse may contain paper money or coins, when it comes to cryptocurrencies, the currency is actually never held by the owner but rather remains on the blockchain and can never leave the blockchain to go anywhere else. However, your proof of ownership of that digital currency is held on a digital wallet.
Each digital wallet will also have a public identifier that will appear when you transact using a cryptocurrency. No visible records of who did what transaction with who, only the identifier of a digital wallet. So, cryptocurrencies are not truly anonymous, but rather pseudonymous.
Digital wallets can be mobile, web-based, desktop, computer-based, stored on a secure physical device, or even physically-stored.