That's actually happened multiple times. There used to be a company called On2 which basically specialized in making proprietary, royalty-free design-arounds of other patented video codecs. Yes, those words sound confusing. The implementation was proprietary but the codec itself was designed to not bear a patent royalty. They made VP3 (later donated as Theora), VP6 (bought by Adobe to use as the fancy new Flash Player 8 video codec), and then got bought out by Google around the time they were making VP8.

Google basically had them continue what they were doing, but on a completely FOSS basis. They took VP8, bundled it with Ogg Vorbis and the Matroska container, and rebranded it as webm. They released VP9, and then started work on VP10 up until the whole H.265 licensing debacle happened. At this point all the holdouts for ITU/ISO standard codecs threw up their hands and decided to form a new standards consortium to build a new royalty-free video codec. Google, xiph, Cisco, and a few other companies basically put together some of their research projects and built AV1. This codec was more or less funded by their own research, and their business models are such that just having decoder ubiquity is worth more than patent royalties they could otherwise charge.

Video codec research is difficult, but mostly because it requires specialized knowledge of a lot of different related fields of mathematics and computer science. For various historical reasons, the way ITU (H.26x) and ISO (MPEG-n) paid for their research was by deliberately shipping patent-encumbered standards. Essentially, they just used whatever coding tools worked the best and assumed the economics would work out in favor of forming a patent pool under uniform-fee licensing rules. For example, despite the name, MPEG-LA has literally nothing to do with the ISO MPEG working group, or ITU VCEG, despite being the primary means by which their standards are licensed.

This sort of arrangement occasionally works out amazingly well (MPEG-4, H.264) or blows up spectacularly in their faces (H.263, H.265).

If you had the requisite knowledge I mentioned above, you probably could discover at least some small improvement upon the state-of-the-art for video coding techniques. However, I could not guarantee that it would be unencumbered by patents. Hell, MPEG even tried building a patent-free video codec of their own, but it failed completely because of some... rather interesting particularities of how standards bodies handle patent claims. You see, for various legal reasons, people proposing standards that they may have a patent interest in need to be very up front about what parts they will own. In ISO standards speak, these patent declarations come in three flavors:

Option 1, which means you agree to license your patents without a patent royalty (the AV1 patent model).

Option 2, or "FRAND" (Fair, Reasonable, And Non-Discriminatory), which means you agree to license patents for a royalty payment to any entity (the MPEG-LA patent model). (Note that RMS would prefer you call this "Uniform Fee Only" and not FRAND)

Option 3, which means you are refusing to license the patent under any terms suitable for a standard (the "get off my lawn" patent model).

An interesting peculiarity is that ISO will accept patent declarations under Option 1 or Option 2 that do not specifically name the patents potentially being infringed by the standard. This is not the case for Option 3; for obvious reasons. If a standards entity just accepted vague claims of infringement without any offer of licensing, then it would not be able to actually make standards. However, the acceptance of vague Option 2 claims meant that MPEG actually couldn't ascertain what part of their attempt at a royalty-free codec was infringing, so it was a standard dead in the water.

This problem isn't unique to MPEG either; vague claims of infringement are perfectly fine until you actually have to file a lawsuit, and at that point it's far too late to rewrite a standard. So if you do write your own video codec, you still have to have a team of lawyers examine everything about it against basically every patent even remotely relating to video codecs. You don't get the benefit of ISO-style patent disclosures either, as none of the patent owners are working with you and thus have no legal reason to tell you about the patents. Even if you managed to do the legal research yourself, it's entirely possible that another company had a patent application before you wrote your codec that you weren't aware of.

That is known as a "submarine patent" and it's why Microsoft and Apple were initially quite skeptical of VP8 and WebM. Having a large international standards process with basically the entire industry on-board means pretty much everyone with patents will need to disclose and license their patents (albeit only under Option 2). Having all that money into one large standard further creates economic incentives in favor of forming a patent pool. On2/Google going it alone means pretty much any patent claim will be some kind of submarine patent asserted against you far after you had any hope of changing the spec to avoid it. Google gets around this with lots of money and lawyers, which you don't have.