A Bitcoin-style blockchain is a combination of two things:

1) A public ledger where each new entry includes the hash of the previous one, so you can efficiently attest to history and items cannot be silently removed from history, even if they're signed by the same participant. This is an old idea - Merkle trees are a slightly more advanced take on this idea. Present-day use cases, which are not blockchains in the conventional sense, include Git and Certificate Transparency.

2) A mechanism whereby the ability to add something to the blockchain requires the expenditure of computational power, as a way to solve the "double-spend" problem - namely, if you're storing financial accounts in your ledger, how do you make sure one person doesn't claim to transfer the same dollar to two different people? Bitcoin's innovation is to require some "proof of work" to record the operation and to make "miners" record operations (instead of participants adding things themselves), such that maintaining two branches of the ledger requires you to turn into the most powerful miner, which is computationally prohibitive.

You only have the double-spend problem if you have an object that cannot be used twice. Git does not have the double-spend problem because there's nothing wrong with having two commits branching off the same commit (in fact, it's quite common/encouraged). Certificate Transparency does not have the double-spend problem because CAs can sign an unlimited number of certificates, so if you see two different signed claims on two branches they're both valid. The two ledgers can be trivially merged together (unlike two branches of where money went).

Timestamping has the double-spend problem in its own way: either something happened or didn't happen on a day, and you don't want multiple, different records of what happened on that day. If I want to attest that something happened on June 1st, you need to be confident that I didn't make a second history that diverged at July 30 and merge it back in.

Most people do not have the double-spend problem. Fundamentally, you need a finite resource (like time or money) and you need to track the resource itself on the blockchain for the double-spend problem to exist.

A lot of people think a blockchain is useful for provenance tracking. But private records of provenance do not have the double-spend problem - you're not tracking the thing itself, you're tracking claims about where the thing was. If you see signed claims from me saying both "Lot X consists of high-grade steel that's been inspected" and "Lot X consists of faulty steel that needs to be discarded as scrap," you know I'm lying somehow. You don't need to arbitrarily pick one of those statements. (You actually don't want to arbitrarily pick one, because, again, the ledger here is just a record and not the thing itself: you want to get on my case for lying and find the truth. For the currency-itself use case of Bitcoin, it's fine to arbitrarily pick one.)

A lot of people are interested in so-called "permissioned blockchains" / "private blockchains." For the financial use cases, these usually don't have the double-spend problem because they're just a reconciliation layer on top of a slower payments system, and given a public record, you can just hold someone responsible for their double-spending, because again the ledger is just a record and not the thing itself. (This is largely similar to how "sidechains" like Lightning Network don't need to use the full complexity of the blockchain - because what they really are are records of promises to perform transactions on the primary blockchain. If someone makes two promises, hold them accountable for both.) For the non-financial use cases, you probably don't have anything resembling the double-spend problem in the first place: what you want is, at most, something like signed commits in Git.