Basically, you need an RF stage and an IF stage or two, and then a conversion of the analog signal to bits. Once the signal is in digital form, demodulation can be performed in software. Lots of work can be done in this area by the ham radio hobbyist. In fact, you could invent your own modulation scheme. Additional work can be applied to new forms of weak signal operation.
Digital technology can often work at extremely low power levels for a given communications path.
By manipulating the bits, in software, reliable communications can be maintained under difficult path conditions. This field is ripe for experimentation. Messaging Hams have been operating low speed packet radio networks since the early 1980s. Back then, a 1200 baud telephone modem was exciting - 1200 bps packet radio was pretty cool. But with 28,800 bps telephone modems now common place and inexpensive, that old 1200 bps packet radio looks pretty dated.