Repository: Automattic/jetpack
Pull request #43259: Docker: add support for multiplugin 'integration' PHP unit tests
Author: login: kraftbj
Labels: name: Enhancement
description: Changes to an existing feature — removing, adding, or changing parts of it
name: [Tools] Development CLI
description: The tools/cli to assist during JP development.
name: [Tests] Includes Tests
name: [Plugin] Wpcomsh
Created: 2025-04-27T16:29:59Z | Merged: 2025-04-27T21:21:06Z

<!--- Provide a general summary of your changes in the Title above -->
<!-- Would you like this feature to be tested by Beta testers?
Please add testing instructions to projects/plugins/jetpack/to-test.md in a new commit as part of your PR. -->
<!-- a12s: If you have an expected version that you're aiming for the PR to add, please use the Milestone field to communicate it. If you leave it blank, that indicates there isn't a preference. -->

I'm working on a bug that impacts Boost while wpcomsh is running and want to be able to test to confirm the bug and the fix. I don't believe there's a way to run unit tests with multiple plugins running.

This iteration is meant to only run wpcomsh or jetpack unit tests (as they're written to be executed with a full WordPress stack) while including other plugins as active.

It's not the most robust solution and likely incomplete. My thought here is to get something working for being able to test wpcomsh and boost while providing a starting point for more iteration.

## Proposed changes:
<!--- Explain what functional changes your PR includes -->
* Add a `jetpack docker phpunit-integration` command.

### Other information:

- [ ] Have you written new tests for your changes, if applicable?
- [ ] Have you checked the E2E test CI results, and verified that your changes do not break them?
- [ ] Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?

## Jetpack product discussion
<!-- If you're an Automattician, include a shortlink to the p2 discussion with Jetpack Product here. -->
<!-- Make sure any changes to existing products have been discussed and agreed upon -->
n/a

## Does this pull request change what data or activity we track or use?
<!--- If so, please add the "[Status] Needs Privacy Updates" label and explain what changes there are. -->
<!--- Check existing Jetpack support documents for a preview of the information we need. -->
No.

## Testing instructions:
<!-- If you were reviewing this PR, how would you like the instructions to be presented? -->
<!-- Please include detailed testing steps, explaining how to test your change. -->
<!-- Bear in mind that context you working on is not obvious for everyone.  -->
<!-- Adding "simple" configuration steps will help reviewers to get to your PR as quickly as possible. -->
<!-- "Before / After" screenshots can also be very helpful when the change is visual. -->

* Ensure docker is running ({jp|jetpack} docker up -d )
* Run {jp|jetpack} docker phpunit-integration wpcomsh boost
* Same command with just wpcomsh
* Both should pass, noting the conditional in the wpcomsh test to check if boost is active or not based on the env vars that are set by the command.



Diff:
diff --git a/projects/plugins/wpcomsh/changelog/add-multi-plugin-phpunit b/projects/plugins/wpcomsh/changelog/add-multi-plugin-phpunit
new file mode 100644
index 000000000000..75c9986e4da7
--- /dev/null
+++ b/projects/plugins/wpcomsh/changelog/add-multi-plugin-phpunit
@@ -0,0 +1,4 @@
+Significance: patch
+Type: added
+
+Added ability to test with other Jetpack monorepo plugins active
diff --git a/projects/plugins/wpcomsh/tests/WpcomshTest.php b/projects/plugins/wpcomsh/tests/WpcomshTest.php
index ab164a4fab81..942a04f57e47 100644
--- a/projects/plugins/wpcomsh/tests/WpcomshTest.php
+++ b/projects/plugins/wpcomsh/tests/WpcomshTest.php
@@ -60,4 +60,20 @@ public function test_wpcomsh_make_content_clickable() {
 
 		$this->assertEquals( $expected_output, wpcomsh_make_content_clickable( $original_content ) );
 	}
+
+	/**
+	 * Tests if Jetpack Boost plugin is active, to test the integreation setup.
+	 *
+	 * This is for the `jp docker phpunit-integration` command to verify it works.
+	 *
+	 * @return void
+	 */
+	public function test_is_jetpack_boost_active() {
+		$plugins = getenv( 'JP_MONO_INTEGRATION_PLUGINS' );
+		if ( $plugins && strpos( $plugins, 'boost' ) !== false ) {
+			$this->assertTrue( is_plugin_active( 'boost/jetpack-boost.php' ) );
+			return;
+		}
+		$this->assertFalse( is_plugin_active( 'boost/jetpack-boost.php' ) );
+	}
 }
diff --git a/projects/plugins/wpcomsh/tests/bootstrap.php b/projects/plugins/wpcomsh/tests/bootstrap.php
index b8eff09a68df..b78bc4ae3126 100644
--- a/projects/plugins/wpcomsh/tests/bootstrap.php
+++ b/projects/plugins/wpcomsh/tests/bootstrap.php
@@ -60,10 +60,40 @@ function _manually_load_plugin() {
 	if ( file_exists( WPMU_PLUGIN_DIR . '/wpcomsh-loader.php' ) ) {
 		return;
 	}
+
 	require_once dirname( __DIR__ ) . '/wpcomsh.php';
 }
 tests_add_filter( 'muplugins_loaded', '_manually_load_plugin' );
 
+/**
+ * Add the various plugins to the active options array.
+ */
+function _manually_load_other_plugins( $active_plugins ) {
+	// Get the JP_MONO_INTEGRATION_PLUGINS env var to include other plugins.
+	$plugins = getenv( 'JP_MONO_INTEGRATION_PLUGINS' );
+	if ( $plugins ) {
+		$plugins = explode( ',', $plugins );
+		require_once ABSPATH . 'wp-admin/includes/plugin.php';
+		$all_plugins = get_plugins();
+		foreach ( $plugins as $plugin ) {
+			// Ignore wpcomsh and jetpack for now.
+			if ( 'wpcomsh' === $plugin || 'jetpack' === $plugin ) {
+				continue;
+			}
+			// Find the main plugin file for this plugin directory
+			foreach ( $all_plugins as $plugin_file => $plugin_data ) {
+				if ( strpos( $plugin_file, $plugin . '/' ) === 0 ) {
+					$active_plugins[] = $plugin_file;
+					break;
+				}
+			}
+		}
+	}
+	return $active_plugins;
+}
+
+tests_add_filter( 'option_active_plugins', '_manually_load_other_plugins' );
+
 // Override WP_TESTS_CONFIG_FILE_PATH via environment.
 // Important for monorepo CI, if you don't do this then different test runs might collide!
 if ( false !== getenv( 'WP_TESTS_CONFIG_FILE_PATH' ) ) {
diff --git a/tools/cli/commands/docker.js b/tools/cli/commands/docker.js
index 06443655c3ff..0f7f241a3acc 100644
--- a/tools/cli/commands/docker.js
+++ b/tools/cli/commands/docker.js
@@ -441,6 +441,31 @@ const buildExecCmd = argv => {
 			],
 		};
 		opts = buildPhpUnitTestCmd( argv, opts, unitTestArgs );
+	} else if ( cmd === 'phpunit-integration' ) {
+		// Only run tests for wpcomsh and jetpack, but always set JP_MONO_INTEGRATION_PLUGINS to the full list
+		const plugins = argv.plugins || argv._.slice( 2 );
+		const integrationPluginsEnv = plugins.join( ',' );
+		const allCmds = [];
+		// Only run for these plugins
+		const testablePlugins = [ 'wpcomsh', 'jetpack' ];
+		for ( const plugin of testablePlugins ) {
+			if ( ! plugins.includes( plugin ) ) continue;
+			let envVars = [ `JP_MONO_INTEGRATION_PLUGINS=${ integrationPluginsEnv }` ];
+			if ( plugin === 'wpcomsh' ) {
+				envVars = envVars.concat( [
+					'WP_TESTS_DIR=/tmp/wordpress-develop/tests/phpunit',
+					'WP_CORE_DIR=/var/www/html',
+					'WP_CONTENT_DIR=/var/www/html/wp-content',
+				] );
+			}
+			const unitTestArgs = {
+				plugin,
+				envVars,
+			};
+			const pluginOpts = buildPhpUnitTestCmd( argv, [ 'exec', 'wordpress' ], unitTestArgs );
+			allCmds.push( buildComposeFiles().concat( pluginOpts ) );
+		}
+		return allCmds;
 	} else if ( cmd === 'wp' ) {
 		const wpArgs = argv._.slice( 2 );
 		// Ugly solution to allow interactive shell work in dev context
@@ -840,6 +865,33 @@ export function dockerDefine( yargs ) {
 					handler: async argv => {
 						await generateConfig( argv );
 					},
+				} )
+				.command( {
+					command: 'phpunit-integration <plugins...>',
+					description:
+						'Run PHPUnit for each specified plugin with all specified plugins activated (integration mode)',
+					builder: yargCmd =>
+						defaultOpts( yargCmd ).positional( 'plugins', {
+							describe: 'Plugin slugs to activate and test',
+							type: 'string',
+							array: true,
+						} ),
+					handler: argv => {
+						// Get all the commands to run
+						const allCmds = buildExecCmd( argv );
+						let hasFailure = false;
+						for ( const opts of allCmds ) {
+							try {
+								composeExecutor( argv, opts, buildEnv( argv ) );
+							} catch ( e ) {
+								console.error( chalk.bgRed( `Error: ` ) + e );
+								hasFailure = true;
+							}
+						}
+						if ( hasFailure ) {
+							process.exit( 1 );
+						}
+					},
 				} );
 		},
 	} );


Reviews:
user: login: jeherve
body: This looks good to me, let's give it a try!
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-27T19:51:59Z
user: login: anomiex
body: I'm not really liking this change. It adds a `jetpack docker phpunit-integration` command that claims to generically run tests with different combinations of plugins active. What it actually does is run jetpack and/or wpcomsh tests only, and only the wpcomsh tests get run with the specified other plugins enabled (and even there it refuses to enable jetpack).

As it is, it seems to me this would have been better done as an option to `jetpack docker phpunit-wpcomsh` than as a separate command, since that's the only thing it works for.

If we actually want an "integration mode", it looks like we'd have to somehow make it a common feature of every plugins' boostrapping to recognize this environment variable specifying which other plugins to activate. And then actually run the tests for all the specified plugins (or, maybe better, just the first one) instead of only jetpack and wpcomsh. And then consider how exactly we want to communicate that it's likely various tests will fail because they assume other plugins _aren't_ enabled, and that's not necessarily a problem.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-28T15:29:28Z

Review comments:
diff_hunk: @@ -60,10 +60,40 @@ function _manually_load_plugin() {
 	if ( file_exists( WPMU_PLUGIN_DIR . '/wpcomsh-loader.php' ) ) {
 		return;
 	}
+
 	require_once dirname( __DIR__ ) . '/wpcomsh.php';
 }
 tests_add_filter( 'muplugins_loaded', '_manually_load_plugin' );
 
+/**
+ * Add the various plugins to the active options array.
+ */
+function _manually_load_other_plugins( $active_plugins ) {
+	// Get the JP_MONO_INTEGRATION_PLUGINS env var to include other plugins.
+	$plugins = getenv( 'JP_MONO_INTEGRATION_PLUGINS' );
+	if ( $plugins ) {
+		$plugins = explode( ',', $plugins );
+		require_once ABSPATH . 'wp-admin/includes/plugin.php';
+		$all_plugins = get_plugins();
+		foreach ( $plugins as $plugin ) {
+			// Ignore wpcomsh and jetpack for now.
+			if ( 'wpcomsh' === $plugin || 'jetpack' === $plugin ) {
+				continue;
+			}
+			// Find the main plugin file for this plugin directory
+			foreach ( $all_plugins as $plugin_file => $plugin_data ) {
+				if ( strpos( $plugin_file, $plugin . '/' ) === 0 ) {
path: projects/plugins/wpcomsh/tests/bootstrap.php
user: login: anomiex
body: This would probably be better done with `str_starts_with()`. Or `substr() ===` if that can't be used for some reason, but I think it can here.
created_at: 2025-04-28T15:19:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -60,4 +60,20 @@ public function test_wpcomsh_make_content_clickable() {
 
 		$this->assertEquals( $expected_output, wpcomsh_make_content_clickable( $original_content ) );
 	}
+
+	/**
+	 * Tests if Jetpack Boost plugin is active, to test the integreation setup.
path: projects/plugins/wpcomsh/tests/WpcomshTest.php
user: login: anomiex
body: Typo, "integration"
created_at: 2025-04-28T15:19:29Z
author_association: CONTRIBUTOR
diff_hunk: @@ -60,4 +60,20 @@ public function test_wpcomsh_make_content_clickable() {
 
 		$this->assertEquals( $expected_output, wpcomsh_make_content_clickable( $original_content ) );
 	}
+
+	/**
+	 * Tests if Jetpack Boost plugin is active, to test the integreation setup.
+	 *
+	 * This is for the `jp docker phpunit-integration` command to verify it works.
+	 *
+	 * @return void
+	 */
+	public function test_is_jetpack_boost_active() {
+		$plugins = getenv( 'JP_MONO_INTEGRATION_PLUGINS' );
+		if ( $plugins && strpos( $plugins, 'boost' ) !== false ) {
+			$this->assertTrue( is_plugin_active( 'boost/jetpack-boost.php' ) );
path: projects/plugins/wpcomsh/tests/WpcomshTest.php
user: login: anomiex
body: I note this will probably only pass within the monorepo environment, because everywhere else it'd be "jetpack-boost/jetpack-boost.php".

That's not necessarily a problem, since that's probably also about the only environment we support running these tests in, but it may be worth noting.
created_at: 2025-04-28T15:21:21Z
author_association: CONTRIBUTOR
diff_hunk: @@ -60,4 +60,20 @@ public function test_wpcomsh_make_content_clickable() {
 
 		$this->assertEquals( $expected_output, wpcomsh_make_content_clickable( $original_content ) );
 	}
+
+	/**
+	 * Tests if Jetpack Boost plugin is active, to test the integreation setup.
+	 *
+	 * This is for the `jp docker phpunit-integration` command to verify it works.
+	 *
+	 * @return void
+	 */
+	public function test_is_jetpack_boost_active() {
+		$plugins = getenv( 'JP_MONO_INTEGRATION_PLUGINS' );
+		if ( $plugins && strpos( $plugins, 'boost' ) !== false ) {
path: projects/plugins/wpcomsh/tests/WpcomshTest.php
user: login: anomiex
body: It'd probably be a bit more robust to do like `strpos( ",$plugins,", ',boost,' ) !== false` to avoid the possibility that we someday have a "foo-for-boost" plugin or the like.

Or explode the string and check with `in_array()`.
created_at: 2025-04-28T15:23:55Z
author_association: CONTRIBUTOR
diff_hunk: @@ -60,10 +60,40 @@ function _manually_load_plugin() {
 	if ( file_exists( WPMU_PLUGIN_DIR . '/wpcomsh-loader.php' ) ) {
 		return;
 	}
+
 	require_once dirname( __DIR__ ) . '/wpcomsh.php';
 }
 tests_add_filter( 'muplugins_loaded', '_manually_load_plugin' );
 
+/**
+ * Add the various plugins to the active options array.
+ */
+function _manually_load_other_plugins( $active_plugins ) {
+	// Get the JP_MONO_INTEGRATION_PLUGINS env var to include other plugins.
+	$plugins = getenv( 'JP_MONO_INTEGRATION_PLUGINS' );
+	if ( $plugins ) {
+		$plugins = explode( ',', $plugins );
+		require_once ABSPATH . 'wp-admin/includes/plugin.php';
+		$all_plugins = get_plugins();
+		foreach ( $plugins as $plugin ) {
+			// Ignore wpcomsh and jetpack for now.
path: projects/plugins/wpcomsh/tests/bootstrap.php
user: login: anomiex
body: wpcomsh I understand, since this _is_ wpcomsh and we probably don't want to activate it twice. But why jetpack?
created_at: 2025-04-28T15:25:20Z
author_association: CONTRIBUTOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-04-27T16:30:53Z
body: <!-- wpcom-reminder-comment -->
Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.

- To test on WoA, go to the Plugins menu on a WoA dev site. Click on the "Upload" button and follow the upgrade flow to be able to upload, install, and activate [the Jetpack Beta plugin](https://jetpack.com/download-jetpack-beta/). Once the plugin is active, go to Jetpack > Jetpack Beta, select your plugin (WordPress.com Site Helper), and enable the `add/multi-plugin-phpunit` branch.


*Interested in more tips and information?*

- In your local development environment, use the `jetpack rsync` command to sync your changes to a WoA dev blog.
- Read more about our development workflow here: PCYsg-eg0-p2
- Figure out **when your changes will be shipped to customers** here: PCYsg-eg5-p2
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-04-27T16:31:02Z
body: **Thank you for your PR!**

When contributing to Jetpack, we have [a few suggestions](https://github.com/Automattic/jetpack/blob/trunk/.github/PULL_REQUEST_TEMPLATE.md) that can help us test and review your patch:<br>
- :white_check_mark: Include a description of your PR changes.<br>
- :white_check_mark: Add a "[Status]" label (In Progress, Needs Review, ...).<br>
- :white_check_mark: Add a "[Type]" label (Bug, Enhancement, Janitorial, Task).<br>
- :white_check_mark: Add testing instructions.<br>
- :white_check_mark: Specify whether this PR includes any changes to data or privacy.<br>
- :white_check_mark: Add changelog entries to affected projects<br>


This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation :robot:

******

**Follow this PR Review Process:**

1. Ensure all required checks appearing at the bottom of this PR are passing.
2. Make sure to test your changes on all platforms that it applies to. *You're responsible for the quality of the code you ship*.
3. You can use [GitHub's Reviewers functionality](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/requesting-a-pull-request-review) to request a review.
4. When it's reviewed and merged, you will be pinged in Slack to deploy the changes to WordPress.com simple once the build is done.

If you have questions about anything, reach out in #jetpack-developers for guidance!
******

**Wpcomsh plugin:**

- Next scheduled release: _Atomic deploys happen twice daily on weekdays (p9o2xV-2EN-p2)_


If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

author_association: CONTRIBUTOR
user: login: jp-launch-control[bot]
created_at: 2025-04-27T16:34:05Z
body: ### Code Coverage Summary

This PR did not change code coverage!

That could be good or bad, depending on the situation. Everything covered before, and still is? Great! Nothing was covered before? Not so great. 🤷



[Full summary](https://jetpackcodecoverage.atomicsites.blog/prs/43259/) · [PHP report](https://jetpackcodecoverage.atomicsites.blog/prs/43259/php/)
author_association: NONE
user: login: kraftbj
created_at: 2025-04-28T22:13:04Z
body: All very fair feedback. I'll add to my list to re-work this. My initial thought was to allow it to run any combo of plugin tests (without the plugin's bootstraps needing to do anything), but ran into issues since the plugin tests aren't expecting WordPress itself to be running like usual (either mocking files or wordbless). So, ran into obvious problems pretty quickly. 

The pivot to leach the wpcomsh test worked enough for my immediate desire to test changes quickly via a test, but you're right, it's not ready for primetime.
author_association: CONTRIBUTOR