I like to share my experience:
Writing tests from day 1 is essential. Think of tests as a road for your business logic, which is a car. It might seems expensive to write tests but bad quality code slows you down within weeks rather than months.

Unit tests are important tests and are very useful for engineers. As others mentioned, thinking about test-ability keeps code simple, which is a must when you want to move fast. Test-ability (have the TDD mindset and thinking, if you don't then apply TDD until you think tests while writing your code) usually translates to readable and maintainable code. When you write good unit tests, then most assumptions are documented in an automated way and tested at every execution of the unit tests. When a unit test fails, then the error message and the test name should tell you precisely the assumption and what is broken, that ultimately makes you really fast in maintaining and (with a good overall design) changing code.

When you want to be really fast then have fewer or no integration tests. That is because integration tests usually span over several parts of your software. So a change in any part will lead to probably several changes in the corresponding integration tests. That makes it more expensive to change things fast. On top of that, many integration tests are more complex to write because of the work in mocking adjacent parts of the software.

System/Black box tests. Those test are very important for evaluating business value. One tests common happy paths from your customers and validate that business value/ feature x still works. Again like with the system tests, the process of doing, deciding and prioritising will lead to clear business values and very important a better shared vision between the developers and business. The software serves the purpose to be used by a customer, the system tests are ultimately a distilled version of what you the business is selling and it's an automated process to ensure that this value is delivered.

Even the best software developers do mistakes and forget. I have not yet seen a team which profits from not writing tests. It takes some time to write good software -> testing and approaches like TDD are some of the ways to start climbing that hill.

About your questions:

A robust deployment pipeline is the right way to go and can be even better leveraged with tests. Example: you can connect your system tests to the roll-back functionality and re-deploy the last working state.

You are right about your worries about Spaghetti code and technical dept, from my experience they are always an issue from early on. But also keep in mind that no one is perfect so even if it's only you writing code, you will find code written from you, which gives you goose bumps. A nice approach to code quality is the in-place refactoring, that is you don't do re-factoring as a task. Each time you touch code and it's not very clear or the structure/ design is not ideal then you re-factor with each and every task (piece by piece). A rule of thumb: When a 1 day tasks adds 3 days of re-factoring (only fixing the touched code), then you know that you are sliding the slope of lower productivity and you need to act and going back up the slope usually takes a lot of discipline in the team (that's an extreme example, personally I act much earlier).

Your current idea has some risks associated with it. I like to expand with an example: Usually a code base changes and grows with new features, rarely it gets smaller. When your code base grows and your team grows -> productivity declines (people: communication overhead, technology: overall more complex/bigger system, probably many more factors). In the first weeks you might deliver very fast without tests but in the future that is not given. Let's assume that you and your team are unicorns and will produce perfect code all the time. Your productivity declines naturally by having a bigger system and more people. In the case of declining productivity it is very unusual to add workload like writing tests. It might be easier in your project/startup, but usually it's hard to add more workload. The future business will probably be under constant pressure to deliver fast and follow the market.
Now back to the real world, you will probably face a more pronounced slowdown than in the unicorn case. Several risks are coming from your decision, if you think you can manage them then try and keep your eyes peeled and learn from every mistake along the way (btw that's always good advice for every part of your journey ;) ).

Tip use a tools like SonarQube (https://www.sonarqube.org/) to asses and measure obvious code quality.

Another point I like to share: Keep in mind that good tests don't stop bugs, there will be enough bugs even with good tests. Tests help developers/engineers to understand code and assumptions, they help to reduce bugs from being introduced, they help you doing the right design decisions and much more. You can even write strongly unreadable code with a very good rating from SonarQube. Maintainability, readability and change-ability of your code are human related problems and only slightly technology problems.

Software Quality often relates to tests, if you replaces tests with software quality in your question this article has some good talking points: https://martinfowler.com/articles/is-quality-worth-cost.html