From what I've seen, outlook.com does seem to have some special case for mail from people you've sent to. I did some experiments a few months ago, and again a week or two ago with mail delivery, and outlook.com was one of the targets. Here's what I did.

I have a domain, which I'll refer to as @mine in the following. It's a .net domain that I've had for 20+ years. It has never been used by anyone other than me. Before I had it, it was owned by my employer which had bought it for me as a gift and transferred it to me. They were the first to ever register it, but never used it for anything.

@mine has never sent spam, or any commercial email at all except for either replies to received commercial email or to contact businesses I was a customer of or wanted to be a customer of at their support or sales addresses.

In the early 2000s I think spammers occasionally forged @mine emails, back before SPF was a thing. I added an SPF record when those became normal, and later DKIM and DMARC records. The various online checking tools tell me that these are all set up correctly. I've never had a hit checking for @mine with the various online blacklist checking tools.

I did some experiments a few months ago, when @mine was handled by a server of my own, running on a VM at Rackspace. All the test messages were short, with no URLs or images, mentions of money, or anything else suspicious. Thing's like "Bob's birthday party will be Saturday instead of Sunday. Can you make it?"...i.e., innocuous innocent things that ordinary people might send to other ordinary people.

@mine -> gmail.com: went to inbox. It shows that SPF, DKIM, and DMARC all pass. I'm not sure that this was a valid test though, because I'm pretty sure that Google knows that me@mine is the same person as me@gmail, so they might give it preferential treatment.

@mine -> yahoo.com: went to spam. I marked it as not spam, and sent another one. That and subsequent messages went to inbox.

@mine -> outlook.com: went to spam. I marked it as not spam, and sent another one. Went to spam. Repeat the mark and send thing several times. Still goes straight to spam.

A couple months or so later, the Linux distro I ran my server on was rapidly approaching EOL for long term support, and I moved @mine to Fastnet.com and cancelled my Rackspace account. (The only other thing @mine was a very low volume web server...I moved that to Amazon Lightsail. $3.50/month for that + $3.61/month for Fastmail is less than 1/3 of what the cheapest Rackspace VM cost (and I was on a grandfathered plan there that was cheaper than any of the current plans).

I repeated the experiment with @mine now at Fastmail, to see if the problem at outlook.com was that I was running my own server at a public cloud.

@mine ->gmail and yahoo still went to inbox, just as they did at the end of the first experiment.

@mine -> outlook.com: still to spam. A few more cycles of mark not spam and try again did not change this.

I own another domain, my ham radio call sign in .us. I grabbed that when I got my license, and have never done anything with it. It just had whatever standard parking page Namecheap uses and whatever standard mail handling they provide when you register a domain there and don't do anything with it.

I switched @ham mail handling over the Fastmail, and tried the experiment again.

Same results as the first @mine experiment. Gmail never went to spam, yahoo went to spam once, and then to inbox after I told it that one was not spam. Outlook straight to spam despite repeated attempts to train it otherwise.

I also tried marking @ham as a "safe sender" in outlook, which is supposed to bypass the spam checking. Nope. Still straight to spam. Some Googling turned up reports on MS support forums of "safe sender" not working, and it appears that this has been a long term problem. (I also found that sometimes my "safe sender" list would disappear. Outlook.com, at least via the web interface, seems rather flaky). (I saw some other posts that suggested that using rules was also insufficient...apparently rules processing takes place after spam processing?)

Finally, I sent a mail from outlook.com to @ham. After that, mail from @ham went to inbox.

It would be nice to try now with a third new domain, this time sending from outlook to @third before ever trying to send to outlook, and then seeing what happens to the first mail from @third to outlook. Actually, it would be best to also do this with @fourth, with the first mail from @third being a reply to the mail from outlook, and the first mail from @fourth not being a reply.

But I don't have any other domains. Maybe the next time I see a really cheap domain promotion at Namecheap I'll grab a couple and try this. They've got .xyz for $1/year right now. Maybe if I get bored enough with the COVID-19 "stay at home" I'll grab a couple and give it a try.