I work in the corporate performance test / performance tuning space in a brownfield environment with a lot of co-dependent systems supported by a lot of different teams. My background prior to this was a programmer / DBA.

It's all hard - but it's not any harder than building and testing end to end solution working in the equivalent environment - there's a lot of similarities, benefits and also disadvantages. Overall performance testing provides

"Performance Testing" often covers quite a few different areas:

- User Experience (more difficult with SPA and Mobile applications)

- Server Response

- Capacity Planning / Utilization

- Performance Tuning

- Enterprise / Systems Architecture

- Production Support for Performance Problems

- Monitoring (both for testing purposes and planning monitoring for production)

Like programming, most the time actual coding / scripting isn't the challenge. Some of the bigger challenges I deal with are:

- Test Data Setup, Availability, Size, Distribution, Consistency, Security, Backup/Restore over multiple systems

- Understanding / Confirming Scenarios

- Complex Application Authentication (more secure authentication methods can be harder to test or work-around)

- Data integrity / security (difficult to use SaaS solutions for regulatory reasons)

- Environments, Sizes, Accuracy (Prod Like), Support (Application and Infrastructure), Monitoring, Limitations, Architecture

- Load Planning/Estimation/Reporting (both in Performance and Production)

- Deployment Planning / Management

- Investigating Performance Problems

- Working with lots of different development teams.

A lot of the recent trends in DevOps / Agile / IaaS / PaaS are helping improve the situation with significant improvements to deployments and scaling.

Good commercial tools and OSS tools are available. The move to web / HTTP based systems has actually made testing much simpler because there's really only one protocol to target these days (HTTP) - older applications platforms using non-HTTP based protocols often make things more difficult. There are still some smaller niche areas for industries such as call centres where there are physical requirements (such as generating call traffic), and Desktop Virtulization / Remote Access (due to COVID) that don't fit the more common testing patterns. Mobile App performance testing (which is closer to User Experience testing) is also relatively immature compared to SPA and HTTP testing. There are options out there though.

I also view there being (at least) two different traditional paths into performance testing - Functional Testing (Non Technical) and Developers/Tech Admin (Technical) who bring different strengths and weaknesses to large performance testing efforts.

I'm a fairly big fan of trying to push most performance testing down into individual application teams as they know the technical details of the applications and often have better technical skills than a lot of performance testers. However, you still need cross-system / whole environment perf testing where you have complex interconnected systems and dependencies.

It's difficult trying to balance everything between purely scientific / methodological and pragmatic (just prove it's capable of meeting the volumes) approaches.