No. You have total ownership over your assets as long as their private keys are safely stored in your Ledger wallet. Once you transfer funds from your Ledger wallet to your CL account, they are no longer under self-custody.