Alexei was born into the noble Orlov family in Lyubini in Tver Oblast on , the son of Grigory Ivanovich Orlov, governor of Novgorod, and brother of Grigory Grigoryevich Orlov. He entered the Preobrazhensky Regiment and by 1762 had reached the rank of sergeant. He distinguished himself in the Seven Years' War and was wounded at the Battle of Zorndorf. He was described as a giant of a man, over two meters tall, and a celebrated duellist, with a scar across his cheek. The scar earned him the nickname 'scarface'.