Repository: wp-cli/i18n-command
Pull request #439: Add ability to pass extensions to `make-json` command
Author: login: michaelw85
Labels: name: command:i18n-make-json
description: Related to 'i18n make-json' command
Created: 2025-04-25T11:42:50Z | Merged: 2025-04-25T21:11:48Z

Resolves #231 

Linked issue #231: Search for translation sources with custom extensions via make-json
`make-json` extract JavaScript strings from PO files with .js and .min.js files extensions while generating individual JSON files like it should. But I have JS files with other extensions (for example .tag), which ignored this way.

In [/i18n-command/blob/master/src/MakeJsonCommand.php](https://github.com/wp-cli/i18n-command/blob/master/src/MakeJsonCommand.php) start at 134 line I found code which responsible for it. So there inside iteration it check for each extension and return source if was found. Off course, I can "hardcode" and this will work. Something like:
```
if ( substr( $file, - 4 ) === '.tag' ) {
    return $file;
}
```
But my request is in core update from your side, because I think a lot of people will use with feature for their own purposes and there can be a great variety of similar extensions. Feature must be multipurpose for any extension and, desirable, provide this opportunity to programmers at their own discretion (like `--extensions=tag` argument or something similar). I.e additional to .js and .min.js will search for extensions specified in `--extensions` argument.

Or maybe you have any other suggestions for resolving my request? I am open to discussion and ready to listen to any suggestions.

Thanks!
Linked issue comments: user: login: olegglocc
created_at: 2021-02-12T06:00:04Z
body: Hi!

Have you got some updates about this issue?
author_association: NONE
user: login: swissspidy
created_at: 2021-02-12T08:44:21Z
body: No update I‘m afraid. I personally haven‘t had time to look into it but it sounds like a feasible enhancement and PRs are more than welcome.

You do have some exotic file extensions I must say :-)
author_association: MEMBER
user: login: michaelw85
created_at: 2025-04-25T12:09:29Z
body: @olegglocc @swissspidy I've created a [PR](https://github.com/wp-cli/i18n-command/pull/439) to add this feature.

I was wondering, how do you end up with a PO file containing custom JS extensions? 
Is this already supported by the create pot command?
author_association: CONTRIBUTOR
user: login: swissspidy
created_at: 2025-04-25T12:35:17Z
body: > Is this already supported by the create pot command?

The `wp i18n make-pot` command only supports `.js` and `.jsx` files.

But they might be using something else to extract the strings.


author_association: MEMBER
user: login: michaelw85
created_at: 2025-04-25T12:39:32Z
body: > > Is this already supported by the create pot command?
> 
> The `wp i18n make-pot` command only supports `.js` and `.jsx` files.
> 
> But they might be using something else to extract the strings.

Yes, I was also thinking something along those lines. I think we should add a new ticket to add support to the make pot although I don't know how complex it is to add this feature.
author_association: CONTRIBUTOR

Diff:
diff --git a/features/makejson.feature b/features/makejson.feature
index 948265f0..20f33461 100644
--- a/features/makejson.feature
+++ b/features/makejson.feature
@@ -984,3 +984,137 @@ Feature: Split PO files into JSON files.
       """
     And the return code should be 0
     And the foo-theme/my-custom-domain-de_DE-557240f2080a0894dbd39f5c2f559bf8.json file should exist
+
+  Scenario: Should only process js/min.js extensions by default
+    Given an empty foo-theme directory
+    And a foo-theme/de_DE.po file:
+      """
+      # Copyright (C) 2018 Foo Theme
+      # This file is distributed under the same license as the Foo Plugin package.
+      msgid ""
+      msgstr ""
+      "Project-Id-Version: Foo Plugin\n"
+      "Report-Msgid-Bugs-To: https://wordpress.org/support/plugin/foo-plugin\n"
+      "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
+      "Language-Team: LANGUAGE <LL@li.org>\n"
+      "Language: de_DE\n"
+      "MIME-Version: 1.0\n"
+      "Content-Type: text/plain; charset=UTF-8\n"
+      "Content-Transfer-Encoding: 8bit\n"
+      "POT-Creation-Date: 2018-05-02T22:06:24+00:00\n"
+      "PO-Revision-Date: 2018-05-02T22:06:24+00:00\n"
+      "X-Domain: foo-theme\n"
+      "Plural-Forms: nplurals=2; plural=(n != 1);\n"
+
+      #: foo-theme.js:15
+      msgid "Foo Theme"
+      msgstr "Foo Theme"
+
+      #: bar.min.minified.min.js:15
+      msgid "Foo Theme"
+      msgstr "Foo Theme"
+
+      #: foo-theme.ts:15
+      msgid "Foo Theme"
+      msgstr "Foo Theme"
+
+      #: foo-theme.tag:15
+      msgid "Foo Theme"
+      msgstr "Foo Theme"
+      """
+
+    When I run `wp i18n make-json foo-theme`
+    Then STDOUT should contain:
+      """
+      Success: Created 2 files.
+      """
+    And the return code should be 0
+    And the foo-theme/foo-theme-de_DE-557240f2080a0894dbd39f5c2f559bf8.json file should exist
+    And the foo-theme/foo-theme-de_DE-a9c6627f5fe96185c0a0d0ddd8fa0216.json file should exist
+
+  Scenario: Allows processing custom file extensions
+    Given an empty foo-theme directory
+    And a foo-theme/de_DE.po file:
+      """
+      # Copyright (C) 2018 Foo Theme
+      # This file is distributed under the same license as the Foo Plugin package.
+      msgid ""
+      msgstr ""
+      "Project-Id-Version: Foo Plugin\n"
+      "Report-Msgid-Bugs-To: https://wordpress.org/support/plugin/foo-plugin\n"
+      "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
+      "Language-Team: LANGUAGE <LL@li.org>\n"
+      "Language: de_DE\n"
+      "MIME-Version: 1.0\n"
+      "Content-Type: text/plain; charset=UTF-8\n"
+      "Content-Transfer-Encoding: 8bit\n"
+      "POT-Creation-Date: 2018-05-02T22:06:24+00:00\n"
+      "PO-Revision-Date: 2018-05-02T22:06:24+00:00\n"
+      "X-Domain: foo-theme\n"
+      "Plural-Forms: nplurals=2; plural=(n != 1);\n"
+
+      #: foo-theme.js:15
+      msgid "Foo Theme"
+      msgstr "Foo Theme"
+
+      #: foo-theme.ts:15
+      msgid "Foo Theme"
+      msgstr "Foo Theme"
+
+      #: foo-theme.tag:15
+      msgid "Foo Theme"
+      msgstr "Foo Theme"
+      """
+
+    When I run `wp i18n make-json foo-theme --extensions=ts,tag`
+    Then STDOUT should contain:
+      """
+      Success: Created 3 files.
+      """
+    And the return code should be 0
+    And the foo-theme/foo-theme-de_DE-557240f2080a0894dbd39f5c2f559bf8.json file should exist
+    And the foo-theme/foo-theme-de_DE-a4e9f6529ffa4750907c140158b834b9.json file should exist
+    And the foo-theme/foo-theme-de_DE-df975addfaa8d6579df3d3133999691a.json file should exist
+
+  Scenario: Should use extensions, strip spaces and . from notation
+    Given an empty foo-theme directory
+    And a foo-theme/de_DE.po file:
+      """
+      # Copyright (C) 2018 Foo Theme
+      # This file is distributed under the same license as the Foo Plugin package.
+      msgid ""
+      msgstr ""
+      "Project-Id-Version: Foo Plugin\n"
+      "Report-Msgid-Bugs-To: https://wordpress.org/support/plugin/foo-plugin\n"
+      "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
+      "Language-Team: LANGUAGE <LL@li.org>\n"
+      "Language: de_DE\n"
+      "MIME-Version: 1.0\n"
+      "Content-Type: text/plain; charset=UTF-8\n"
+      "Content-Transfer-Encoding: 8bit\n"
+      "POT-Creation-Date: 2018-05-02T22:06:24+00:00\n"
+      "PO-Revision-Date: 2018-05-02T22:06:24+00:00\n"
+      "X-Domain: foo-theme\n"
+      "Plural-Forms: nplurals=2; plural=(n != 1);\n"
+
+      #: foo-theme.js:15
+      msgid "Foo Theme"
+      msgstr "Foo Theme"
+
+      #: foo-theme.ts:15
+      msgid "Foo Theme"
+      msgstr "Foo Theme"
+
+      #: foo-theme.tag:15
+      msgid "Foo Theme"
+      msgstr "Foo Theme"
+      """
+
+    When I run `wp i18n make-json foo-theme --extensions=".ts, .tsx"`
+    Then STDOUT should contain:
+      """
+      Success: Created 2 files.
+      """
+    And the return code should be 0
+    And the foo-theme/foo-theme-de_DE-557240f2080a0894dbd39f5c2f559bf8.json file should exist
+    And the foo-theme/foo-theme-de_DE-a4e9f6529ffa4750907c140158b834b9.json file should exist
diff --git a/src/MakeJsonCommand.php b/src/MakeJsonCommand.php
index 7b662a0e..164ba06e 100644
--- a/src/MakeJsonCommand.php
+++ b/src/MakeJsonCommand.php
@@ -41,6 +41,9 @@ class MakeJsonCommand extends WP_CLI_Command {
 	 * [--domain=<domain>]
 	 * : Text domain to use for the JSON file name. Overrides the default one extracted from the PO file.
 	 *
+	 * [--extensions=<extensions>]
+	 * : Additional custom JS extensions, comma separated list. By default searches for .min.js and .js extensions.
+	 *
 	 * [--purge]
 	 * : Whether to purge the strings that were extracted from the original source file. Defaults to true, use `--no-purge` to skip the removal.
 	 *
@@ -82,6 +85,12 @@ public function __invoke( $args, $assoc_args ) {
 		$update_mo_files = Utils\get_flag_value( $assoc_args, 'update-mo-files', true );
 		$map_paths       = Utils\get_flag_value( $assoc_args, 'use-map', false );
 		$domain          = Utils\get_flag_value( $assoc_args, 'domain', '' );
+		$extensions      = array_map(
+			function ( $extension ) {
+				return trim( $extension, ' .' );
+			},
+			explode( ',', Utils\get_flag_value( $assoc_args, 'extensions', '' ) )
+		);
 
 		if ( Utils\get_flag_value( $assoc_args, 'pretty-print', false ) ) {
 			$this->json_options |= JSON_PRETTY_PRINT;
@@ -119,7 +128,7 @@ public function __invoke( $args, $assoc_args ) {
 		/** @var DirectoryIterator $file */
 		foreach ( $files as $file ) {
 			if ( $file->isFile() && $file->isReadable() && 'po' === $file->getExtension() ) {
-				$result        = $this->make_json( $file->getRealPath(), $destination, $map, $domain );
+				$result        = $this->make_json( $file->getRealPath(), $destination, $map, $domain, $extensions );
 				$result_count += count( $result );
 
 				if ( $purge ) {
@@ -228,13 +237,15 @@ static function ( $value ) {
 	 * @param string     $destination Path to the destination directory.
 	 * @param array|null $map         Source to build file mapping.
 	 * @param string     $domain      Override text domain to use.
+	 * @param array      $extensions  Additional extensions.
 	 * @return array     List of created JSON files.
 	 */
-	protected function make_json( $source_file, $destination, $map, $domain ) {
+	protected function make_json( $source_file, $destination, $map, $domain, $extensions ) {
 		/** @var Translations[] $mapping */
 		$mapping      = [];
 		$translations = new Translations();
 		$result       = [];
+		$extensions   = array_merge( [ 'js' ], $extensions );
 
 		PoExtractor::fromFile( $source_file, $translations );
 
@@ -251,18 +262,13 @@ protected function make_json( $source_file, $destination, $map, $domain ) {
 
 			// Find all unique sources this translation originates from.
 			$sources = array_map(
-				static function ( $reference ) {
-					$file = $reference[0];
-
-					if ( substr( $file, - 7 ) === '.min.js' ) {
-						return substr( $file, 0, - 7 ) . '.js';
-					}
-
-					if ( substr( $file, - 3 ) === '.js' ) {
-						return $file;
-					}
+				static function ( $reference ) use ( $extensions ) {
+					$file      = $reference[0];
+					$extension = pathinfo( $file, PATHINFO_EXTENSION );
 
-					return null;
+					return in_array( $extension, $extensions, true )
+						? preg_replace( "/.min.{$extension}$/", ".{$extension}", $file )
+						: null;
 				},
 				$this->reference_map( $translation->getReferences(), $map )
 			);


Reviews:
user: login: swissspidy
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-25T14:06:19Z
user: login: swissspidy
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-04-25T14:12:54Z
user: login: michaelw85
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-25T14:16:24Z
user: login: michaelw85
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-25T14:17:03Z
user: login: michaelw85
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-25T14:17:25Z
user: login: michaelw85
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-25T14:17:58Z
user: login: swissspidy
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-25T14:57:35Z
user: login: swissspidy
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-25T14:58:16Z
user: login: michaelw85
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-25T15:02:09Z
user: login: michaelw85
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-25T18:38:25Z
user: login: michaelw85
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-25T18:38:37Z
user: login: michaelw85
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-25T18:39:07Z
user: login: michaelw85
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-25T18:39:16Z
user: login: swissspidy
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-25T21:11:39Z

Review comments:
diff_hunk: @@ -0,0 +1,101 @@
+<?php
+
+
+namespace WP_CLI\I18n\Tests;
+
+use ReflectionClass;
+use ReflectionMethod;
+use WP_CLI\I18n\MakeJsonCommand;
+use WP_CLI\Tests\TestCase;
+use WP_CLI\Utils;
+
+class MakeJsonTest extends TestCase {
+	/** @var string A path files are located */
+	private static $base;
+	/** @var \PHPUnit\Framework\MockObject\MockObject MakeJsonCommand partial mock */
+	private static $mock = null;
+	/** @var \ReflectionMethod make_json reflection (private) */
+	private static $make_json = null;
+
+	public function set_up() {
+		parent::set_up();
+
+		/**
+		 * PHP5.4 cannot set property with __DIR__ constant.
+		 * Shamelessly stolen from @see IterableCodeExtractorTest.php
+		 */
+		self::$base = Utils\normalize_path( __DIR__ ) . '/data/make_json/';
path: tests/MakeJsonTest.php
user: login: swissspidy
body: We don't support PHP 5.4 (anymore)

```suggestion
		self::$base = Utils\normalize_path( __DIR__ ) . '/data/make_json/';
```
created_at: 2025-04-25T14:06:18Z
author_association: MEMBER
path: foo/empty.mo
user: login: swissspidy
body: Please remove this file
created_at: 2025-04-25T14:06:41Z
author_association: MEMBER
path: foo/translations.mo
user: login: swissspidy
body: Please remove this file
created_at: 2025-04-25T14:06:44Z
author_association: MEMBER
diff_hunk: @@ -251,18 +262,12 @@ protected function make_json( $source_file, $destination, $map, $domain ) {
 
 			// Find all unique sources this translation originates from.
 			$sources = array_map(
-				static function ( $reference ) {
+				static function ( $reference ) use ( $extensions ) {
 					$file = $reference[0];
 
-					if ( substr( $file, - 7 ) === '.min.js' ) {
-						return substr( $file, 0, - 7 ) . '.js';
-					}
+					$extension = pathinfo( $file, PATHINFO_EXTENSION );
 
-					if ( substr( $file, - 3 ) === '.js' ) {
-						return $file;
-					}
-
-					return null;
+					return in_array( $extension, $extensions, true ) ? str_replace( '.min.', '.', $file ) : null;
path: src/MakeJsonCommand.php
user: login: swissspidy
body: This also removes `.min.` in the middle of the file name. That's why the `substr` stuff is there, so it really only matches `.min.js` and nothing else.
created_at: 2025-04-25T14:09:19Z
author_association: MEMBER
path: foo/empty.mo
user: login: michaelw85
body: These files are still used in the unit tests
created_at: 2025-04-25T14:16:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -251,18 +262,12 @@ protected function make_json( $source_file, $destination, $map, $domain ) {
 
 			// Find all unique sources this translation originates from.
 			$sources = array_map(
-				static function ( $reference ) {
+				static function ( $reference ) use ( $extensions ) {
 					$file = $reference[0];
 
-					if ( substr( $file, - 7 ) === '.min.js' ) {
-						return substr( $file, 0, - 7 ) . '.js';
-					}
+					$extension = pathinfo( $file, PATHINFO_EXTENSION );
 
-					if ( substr( $file, - 3 ) === '.js' ) {
-						return $file;
-					}
-
-					return null;
+					return in_array( $extension, $extensions, true ) ? str_replace( '.min.', '.', $file ) : null;
path: src/MakeJsonCommand.php
user: login: michaelw85
body: Yes you are right that's why I added two dots to make this less likely but I will update and improve this code
created_at: 2025-04-25T14:17:03Z
author_association: CONTRIBUTOR
path: foo/translations.mo
user: login: michaelw85
body: File is used by the unit test
created_at: 2025-04-25T14:17:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,101 @@
+<?php
+
+
+namespace WP_CLI\I18n\Tests;
+
+use ReflectionClass;
+use ReflectionMethod;
+use WP_CLI\I18n\MakeJsonCommand;
+use WP_CLI\Tests\TestCase;
+use WP_CLI\Utils;
+
+class MakeJsonTest extends TestCase {
+	/** @var string A path files are located */
+	private static $base;
+	/** @var \PHPUnit\Framework\MockObject\MockObject MakeJsonCommand partial mock */
+	private static $mock = null;
+	/** @var \ReflectionMethod make_json reflection (private) */
+	private static $make_json = null;
+
+	public function set_up() {
+		parent::set_up();
+
+		/**
+		 * PHP5.4 cannot set property with __DIR__ constant.
+		 * Shamelessly stolen from @see IterableCodeExtractorTest.php
+		 */
+		self::$base = Utils\normalize_path( __DIR__ ) . '/data/make_json/';
path: tests/MakeJsonTest.php
user: login: michaelw85
body: Ah yes, should have known. I will remove this and also in the file where I copy/pasted it from
created_at: 2025-04-25T14:17:58Z
author_association: CONTRIBUTOR
path: foo/empty.mo
user: login: swissspidy
body: Then these files should be moved to the `tests/data` folder like the others.
created_at: 2025-04-25T14:57:35Z
author_association: MEMBER
path: foo/empty.mo
user: login: swissspidy
body: Alternatively we can also just keep the Behat test and remove the unit test. Less need for creating these dummy files.
created_at: 2025-04-25T14:58:16Z
author_association: MEMBER
path: foo/empty.mo
user: login: michaelw85
body: I did place these files in the data dir, weird. Guess these were generated by accident. I will check what happened and see if I will remove the unit tests
created_at: 2025-04-25T15:01:50Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,101 @@
+<?php
+
+
+namespace WP_CLI\I18n\Tests;
+
+use ReflectionClass;
+use ReflectionMethod;
+use WP_CLI\I18n\MakeJsonCommand;
+use WP_CLI\Tests\TestCase;
+use WP_CLI\Utils;
+
+class MakeJsonTest extends TestCase {
+	/** @var string A path files are located */
+	private static $base;
+	/** @var \PHPUnit\Framework\MockObject\MockObject MakeJsonCommand partial mock */
+	private static $mock = null;
+	/** @var \ReflectionMethod make_json reflection (private) */
+	private static $make_json = null;
+
+	public function set_up() {
+		parent::set_up();
+
+		/**
+		 * PHP5.4 cannot set property with __DIR__ constant.
+		 * Shamelessly stolen from @see IterableCodeExtractorTest.php
+		 */
+		self::$base = Utils\normalize_path( __DIR__ ) . '/data/make_json/';
path: tests/MakeJsonTest.php
user: login: michaelw85
body: Unit tests have been removed in favor of behat tests
created_at: 2025-04-25T18:38:25Z
author_association: CONTRIBUTOR
path: foo/empty.mo
user: login: michaelw85
body: Removed file
created_at: 2025-04-25T18:38:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -251,18 +262,12 @@ protected function make_json( $source_file, $destination, $map, $domain ) {
 
 			// Find all unique sources this translation originates from.
 			$sources = array_map(
-				static function ( $reference ) {
+				static function ( $reference ) use ( $extensions ) {
 					$file = $reference[0];
 
-					if ( substr( $file, - 7 ) === '.min.js' ) {
-						return substr( $file, 0, - 7 ) . '.js';
-					}
+					$extension = pathinfo( $file, PATHINFO_EXTENSION );
 
-					if ( substr( $file, - 3 ) === '.js' ) {
-						return $file;
-					}
-
-					return null;
+					return in_array( $extension, $extensions, true ) ? str_replace( '.min.', '.', $file ) : null;
path: src/MakeJsonCommand.php
user: login: michaelw85
body: Improved check, specifically replacing .min.{ext} at the end of the string now.
created_at: 2025-04-25T18:39:07Z
author_association: CONTRIBUTOR
path: foo/translations.mo
user: login: michaelw85
body: Removed file
created_at: 2025-04-25T18:39:16Z
author_association: CONTRIBUTOR

PR comments:
user: login: codecov[bot]
created_at: 2025-04-25T11:43:54Z
body: ## [Codecov](https://app.codecov.io/gh/wp-cli/i18n-command/pull/439?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=wp-cli) Report
All modified and coverable lines are covered by tests :white_check_mark:

:loudspeaker: Thoughts on this report? [Let us know!](https://github.com/codecov/feedback/issues/255)
author_association: NONE
user: login: swissspidy
created_at: 2025-04-25T12:39:04Z
body: Thanks for opening this PR, appreciate the initiative!

Please note that most tests here are Behat tests instead of PHPUnit tests, see `features/makejson.feature`. We should follow that in this PR.
author_association: MEMBER
user: login: michaelw85
created_at: 2025-04-25T12:48:21Z
body: > Thanks for opening this PR, appreciate the initiative!
> 
> Please note that most tests here are Behat tests instead of PHPUnit tests, see `features/makejson.feature`. We should follow that in this PR.

Alright, I thought it would be pretty easy to unit test (did struggle a bit with mocking though 😅 ) I will have a look at the Behat tests and add some.
author_association: CONTRIBUTOR
user: login: michaelw85
created_at: 2025-04-25T13:30:55Z
body: @swissspidy I've added some basic scenarios similar to the unit tests and everything passes 🥳 
author_association: CONTRIBUTOR
user: login: michaelw85
created_at: 2025-04-25T18:41:18Z
body: @swissspidy I've updated the PR, resolved your feedback. Removed files and the new unit tests in favor of Behat tests, and lastly improved removing `.min` from minified files. Let me know if you see anything else that needs improvement. 
author_association: CONTRIBUTOR