# United States **Court of Appeals**  **for the Federal Circuit**

______________________ MID CONTINENT STEEL & WIRE, INC., Plaintiff

- Appellee

v.

UNITED STATES, Defendant

- Appellee

PT ENTERPRISE INC., PRO

- TEAM COIL NAIL ENTERPRISE INC., UNICATCH INDUSTRIAL CO., LTD., WTA INTERNTIONAL CO., LTD., ZON MON CO., LTD., HOR LIANG INDUSTRIAL CORPORATION, PRESIDENT INDUSTRIAL INC., LIANG CHYUAN INDUSTRIAL CO., LTD., Defendants

- Appellants

______________________

2021

- 1747

______________________ Ap peal from the United States Court of International Trade in No s . 1:15

- cv

- 00213

- CRK, 1:15

- cv

- 00220

- CRK, Judge Claire R. Kelly.

______________________
Decided: April 21, 2022

______________________
A DAM H. G ORDON , The Bristol Group PLLC, Washington, DC, argued for plaintiff

- appellee. Also represented by P ING G ONG

. Case: 21-1747 Document: 48 Page: 1 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 2

M IKKI C OTTET , Appellate Staff, Civil Division, United States Department of Justice, Washington, DC, argued for defendant

- appellee. Also repre sented by B RIAN M. B OYNTON , J EANNE D AVIDSON , P ATRICIA M. M C C ARTHY ; V

ANIA W ANG , Office of the Chief Counsel for Trade Enforcement and Compliance, United States Department of Commerce, Washington, DC.

N ED H. M ARSHAK , Grunfeld, Desiderio, Lebowitz, S ilverman & Klestadt LLP, New York, NY, argued for defendants

- appellants. Also represented by M AX F. S CHUTZMAN ; D HARMENDRA N ARAIN C HOUDHARY , A NDREW T HOMAS S CHUTZ , Washington, DC.

______________________
Before N EWMAN , L OURIE , and T ARANTO , Circuit Judges

. T ARANTO , Circuit Judge

. In 2015, the United States Department of Commerce issued an antidumping duty order covering steel nails from Taiwan

. In 2019, w e ordered a remand to Commerce for further explanation of o ne aspect of the methodology it had adopted to determine whether there was “ a pattern of export prices .

. . that differ significantly among purchasers, regions, or periods of time” under 19 U.S.C. § 1677f

- 1(d)(1)(B)(i). Mid Continent Steel & Wire, Inc. v. United States , 940 F.3d 662, 675 (Fed. Cir. 2019) ( CAFC 2019 Op. ). The present appeal involves Commerce’s redetermination on remand from our 2019 decision.

I n this proceeding, as in other s , Commerce, in order to

assess the significance of the difference between the prices of two groups of sales, stated that it was using a widely known statistical measure called the Cohen’s d coefficient

. As applied to groups of sales, t hat coefficient is a ratio whose numerator is the diffe rence between means of the prices of the two groups and whose denominator is a figure

, reflecting the general dispersion of the pricing data

, that Case: 21-1747 Document: 48 Page: 2 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 3

serves as a benchmark against which to judge the significance of the difference stated in the numerator. Com merce used, for th at benchmark, a figure based on the standard deviations of the prices in the two groups; it squared the standard deviations of the prices of each group (yielding the variances) , added them together and divided by two, then took the square root. The middle step —a dding together and dividing by tw o— is “simple averaging,” which gives equal weight in the average to each group, even if they are very different in size ( e.g. , if the first group reflects sales of 5 units and the second group reflects sales of 95 units ). A “weighted average” approach, in contrast, would

, at the middle step, assign weights proportionate to each group’s share of the total ( e.g. , multiplying the f irst group’s variance by 5 and the second by 95, then dividing the sum by 100, thus giving 5/100 weight to the first group and 95/100 weight to the second group). In 2019, w e held that Commerce did not adequately explain why it was reasonable to use simpl e averaging

. Id. at 673

– 75. On remand from our decision, Commerce again chose to use simple averaging for its version of a Cohen’s d denominator.

The Court of International Trade (Trade Court) upheld Commerce’s decision. Mid Continent Steel & Wire, In c. v. United States , 495 F. Supp. 3 d 1298 , 1308 (Ct. Int’l Trade 2021) ( CIT 2021 Op. ). The Taiwanese producer s and exporter s of the steel nails at issue appeal. W e conclude that the relevant statistical literature cited by Commerce uniformly uses weighted averaging in the Cohen’s d denominator calculation and that Commerce has not offered a reasonable justification for its departure from the cited literature . We therefore vacate the Trade Court’s decisionand require a remand to Commerce for further consideration of its methodology for applying § 1677f

- 1(d)(1)(B)(i) here. Case: 21-1747 Document: 48 Page: 3 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 4

I

AIn an antidumping duty investigation, when Commerce seeks to determine whether the foreign-originated merchandise of a foreign producer or exporter is being sol d in the United States at less than fair value, see 19 U.S.C. § 1673, it must compare the home

- country “normal value” (often the sale price in t he home country) with the actual or constructed “export price” reflecting the price at which the merchandise is sold into the United States

. CAFC 2019 Op.

, 940 F.3d at 665. That comparison usually calls for use of an “average

- to

- average” method . W hen the normal value is based on home

- country sales prices of a foreign producer or exporter who is a respondent in the proceeding

, the average

- to

- average method compares “the weighted average of the respondent’s sales prices in its home country during the investigation period to the weighted average of the respondent’s sales prices in the United States during the same period.” Stupp Corp. v. United States , 5 F.4th 1341, 1345 (Fed. Cir. 2021); CAFC 2019 Op. , 940 F.3d at 666; see also 19 U.S.C. § 1677f

- 1(d)(1); 19 C.F.R. § 351.414(b)(1), (c)(1). But that average

- to

- average co mparison is not the only authorized method : two other methods are authorized, of which one is at issue here

. The statute permits comparisons on a “ transaction

- to

transaction ” basis in unusual circumstances

, 19 U.S.C. § 1677f

- 1(d)(1)(A)(ii); 19 C.F.R. § 35 1.414(c)(2) , but that method is not at issue here

. What is at issue is a third method authorized by Congress u nder certain circumstances

— an “average

- to

- transaction” method . This method calls for the “weighted average of normal values” in the home country to be compared to the “export values (or constructed export values) of individual transactions” in the United States. 19 U.S.C. § 1677f

- 1(d)(1)(B); 19 C.F.R. § 351.414(b)(3). The object is to uncover “targeted ” dumping, a label for an exporter ’s unduly low pricing in Case: 21-1747 Document: 48 Page: 4 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 5

portions (less than all) of its overall U.S. sales

, which would be “masked” (offset) by the exporter’s other, higher

- priced sales if only overall averages are considered

. See Stupp , 5 F.4th at 1345. Co ngress directed that Commerce may use the “average

- to

- transaction” method only if (i) there is a pattern of export prices (or constructedexport prices) for comparable merchandise that differ significantly among purchasers, regions, or periods of time, and (ii) the administering authority explains why such differences cannot be taken into account using a method described in paragraph (1)(A)(i) [average

to

- average] or (ii) [transaction

- to

- transaction]. 19 U.S.C. § 1677f

- 1(d)(1)(B).The statute does not specify how Commerce should determi ne whether those conditions are met. Stupp , 5 F.4th at 1346. Starting in 2014, Commerce has used a two

- stage “differential pricing” analysis. See Differential Pricing Analysis; Request for Comments, 79 Fed. Reg. 26

, 720, 26
, 72 2 , (May 9, 2014) (Differenti al Pricing RFC ); see also Stupp , 5 F.4th at 1346

– 48. The first stage of that process corresponds to the inquiry in paragraph (i)

— whether “there is a pattern of export prices .

. . that differ significantly among purchasers, regions, or periods of time”

— an d itself has two parts : the “Cohen’s d test
, ” followed by the “ratio test.” Differential Pricing RFC at 26
, 722

– 23. Thesecond (final) stage involves a “meaningful difference” assessment to make the determination required in paragraph (ii)

. Id. The present case involves the Cohen’s d test

— thefirst part of the first stage of Commerce’s overall differential pricing analysis.

Under t he method as described in 2014

, Commerce

, considering all sales in the United States by an exporter, is
to select a s pecific purchaser, region, or period of time

, form a “test group” consisting of all the exporter’s sales meeting Case: 21-1747 Document: 48 Page: 5 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 6

that criterion, and put all the exporter’s remaining U.S. sales in the “comparison group

. ” Id. at 26
, 722. That is, Commerce is to compare sal es to one purchaser to sales to all others, sales in one region to sales in all others, and sales in one period to sales in all others

— in fact, to do so for each purchaser , each region, and each period

. For each such test group, Commerce is to compute the Cohen’s d coefficient by comparing the average price of sales within thetest group to the average price of sales within the corresponding comparison group. Id. 1 How Commerce did that comparison to calculate the Cohen’s d in this matter

— which appears t o be representative of its general approach

— is the subject of the dispute before us.

Commerce exp lained that it started with the following formula from Cohen’s textbook to calculate d :

dd = mm AA − mm BB

σ J.A. 1079 (quoting , w ith font changes, Jacob Cohen, Statistical Power Analysis for the Behavioral Sciences 20 ( 2d e d. 1988) ( Cohen )). 2 In that formula, mm AA is the mean of the test group (here, the weighted average of the prices of sales in

1 “The Department calculates the Cohen’s d coefficient with respect to comparable merchandise if the testand comparison groups of data each have at least two observations, and if the sales quantity for the comparison group accounts for at least five percent of the total sales quantity of the comparable merchandise.” Id.

2 It appears that Commerce may have use d the “two

tailed” version of the test to account for differences in either direction ( mm AA > mm BB or mm AA < mm BB ) , taking the absolute value of the coefficient

, which is not shown in the formula in the
text. See Stupp , 5 F.4th at 1346 ; Cohen at 20 . That choice is not in dispute here , and the issue before us is unaffected by the presence or absence of absolute value signs in the formula

. Case: 21-1747 Document: 48 Page: 6 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 7

the group )

, mm BB is the mean of the comparison group (here, the weighted average of the prices of sales in that group)

, and σ is “ the standard deviation of either population [ thetest group or the comparison group] (since they are assumed equal).” Cohen at 20. Where, as here, the grou ps consist of sales at known prices, mm AA − mm BB is in price units ( e.g. , dollars per kilogram ), and so is σ , so the ratio d is a pure (unitless) number.

Commerce then changed the d enominator to a figure

, also drawn from Cohen

, designed to be applied when the two groups , though of the same size, have different standard deviations. Specifically, f or this new denominator σ′

, Commerce used the following formula :

σ′ = �
σ A 2 + σ B 2

2

J.A. 1080 ( quo tin g Cohen at 44 ). In this formula, σ A 2 and σ B 2

a re the squared standard deviations (variances ) of the prices in the test and comparison groups, respectively . T he simple average is used under the square

- root sign (with no weighting by the sizes of groups A and B) , reflecting the fact that, in the situation add ressed in the section of Cohen containing this formula, groups A and B are of the same size: “ n A = n B . ” Cohen at 43. Th is formula involves “pooling ” the data from the two groups , and the name “pooled standard deviation” is used for both the above formula and also the variation where a weighted average is used instead of a simple average

. E.g.

, CIT 2021 Op. , 495 F. Supp. 3d at 1300 ; see also CAFC 2019 Op. , 940 F.3d at 673 (referring to the expression as the “pooled variance” because σ A 2 and σ B 2 are the variances of the prices in the two groups ).

The disputed feature of th e formula is that it does not use the size of the groups to weight the two figures ( squared standard deviation s, i.e.

, variance s ) being av erage d

. It is undisputed that, w he n the groups are of the same size, simple averaging equals weighted averaging. Case: 21-1747 Document: 48 Page: 7 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 8

But Commerce used the formula without group

- size weighting even whe n , unlike in the situation described in the Cohen section from which the formula is borrowed , the groups are of differen t sizes. In that circumstance, it is undisputed, simple averaging does not equal weighted averaging

. Commerce noted: “To be sure, the use of a simple versus weight[ed] average yields v ery different results.” J.A. 667.

The steps following the calculation of Cohen’s d in Commerce’s analysis are not in dispute. Nor, we note, has Commerce relied on those steps to help justify the simple

averaging choice it has made for the denominator at th e first step. We briefly summarize the remaining steps

. Upon calculating d for a test group of sales, Commerce described the test group as having “passed” the Cohen’s d
test if d for that group exceed ed 0.8, i.e.

, if the difference in
means was at least 80% of the pooled standard deviation. See Mid Continent Steel & Wire, Inc. v. United States , 219 F. Supp. 3d 1326, 1338

– 39 (Ct. Int’l Trade 2017) ( CIT 2017 Op. ). 3 Commerce then compute d , for the sales of the subject merchandise of a given respond ent, the ratio of (a) the total value of those sales which were part of any group that passed the Cohen’s d test (whether by a purchaser , region, or period comparison) to (b) the total value of all the respondent’s sales being studied by Commerce

. Id. at 1343 n.24

. Because that “ratio test” produced a ratio between 33 and 66 percent in this matter

, Commerce tentatively decided to use average

- to

- transaction comparisons in part

. See CAFC 2019 Op. , 940 F.3d at 671

– 72.

3 A “pass” thus indicates that the test group’s prices are sufficiently different from the comparison group’s prices to contribute to a finding of targeted dumping. In this way, the label means the opposite of the word’s usual connotation of success in avoiding trou ble

. Case: 21-1747 Document: 48 Page: 8 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 9

To make its final determination whether to use an average

- to

- transaction method , Commerce asked, pursuant to § 1677f

- 1(d)(1)(B)(ii)

, whether the pricing differences found “cannot be taken into account using” average

- to

- average or transaction

- to

- transaction comparison s

. For that determination, Commerce asked whether using a comparison other than average

- to

- transaction would make a “meaningful difference” in the result. Commerce found that there would be such a difference and so adopted the
average

- to

- transaction meth od. See CAFC 2019 Op. , 940 F.3d at 672.

B

1 In response to a petition by Mid Continent Steel &Wire, Inc., Commerce initiated an antidumping duty investigation of certain steel nails from Taiwan and certain other countries. See CAFC 2019 Op. , 940 F.3d at 665. The investigation of nails from Taiwan

— for the period April 1, 2013, to March 31, 2014

— was broken out separately, and Commerce selected PT Enterprises, Inc. and its affiliated producer Pro

- Team Coil Nail Enterprise Inc. as mandatory respondents. In May 2015, Commerce issued an affirmative final determination of less

- than

- fair

- value sales in the United States and determined that the appropriate weighted- average dumping margin for those respondents was 2.24%. Certain Steel Nails from Taiwan: Final De termination of Sales at Less Than Fair Value , 80 Fed. Reg. 28,959, 28,961 (Dep’t of Commerce May 20, 2015) (Final Determination). Following the I nternational Trade Commission’s affirmative determination of material injury to a domestic injury, Commerce issued an antidumping duty order

. In 2017, following an appeal to the Trade Court, Commerce revised the dumping margin for the respondents to 2.16%. The all

- others rate was also set at 2.16%.
Those respondents and other Taiwanese producers and

e xporters ( collectively, PT ) and Mid Continent brought Case: 21-1747 Document: 48 Page: 9 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 10

actions in the Trade Court to challenge Commerce’s determination. The Trade Court sustained Commerce’s application of the Cohen’s d test in determining whether “there is a pattern of export prices .

. . for comparable merchandisethat differ significantly among purchasers, regions, or periods of time ,” 19 U.S.C. § 1677f

- 1(d)(1)(B)(i), and in particular approved Commerce’s decision “to use a simple average to calculate the pooled standard deviation in th e Cohen’s d test of the differential pricing analysis.” CIT 2017 Op. , 219 F. Supp. 3d at 1330. In 2019, we mostly affirmed th e Trade Court’s decision , but we vacated it in part, holding that Commerce’s explanation of its use of “a simple average, rather t han a weighted average, to calculate the pooled variance used in the Cohen’s d calculation” was insufficient, requiring a remand to Commerce “for further explanation.” CAFC 2019 Op. , 940 F.3d at 673, 675

. Specifically, we noted that (1) “Commerce said that it was simply using a widely accepted statistical test; yet it did not acknowledge that the only cited literature source for the relevant aspect of the test itself calls for the use of weighted averages”; (2) Commerce’s statement that weighted averaging produces “skewing” was a “ mere conclusion ” without independent explanation of what the statute calls for ; (3) Commerce’s rebuttal of PT’s argument against the simple average was unsupported and also was
not itself an affirmative argument for simple averaging ; and (4) Commerce’s “predictability” concern seemed tied to the manipulability of reporting sales by number of transaction s and Commerce did not indicate why the concern would be present if the average u sed weighting by quantities or weight of nails sold (nails seemingly being priced per kilogram)

. Id. at 674 (cleaned up)

. We did not preclude Commerce from making the same decision on remand if it supplied adequate reasoning in support. Id. at 675. Case: 21-1747 Document: 48 Page: 10 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 11

2

In December 2019, the Trade Court remand ed the matter to Commerce in accordance with our decision. In early March 2020 , Commerce issued a draft redetermination decision , again opt ing to use the simple average to calculate the pooled standard deviation, J.A. 660

– 76, and attaching portions of three statistics references : Cohen

, J.A. 723

– 61 ; Paul D. Ellis, The Essential Guide to Effect Sizes (2010) ( Ellis ), J.A. 678

– 721 ; and Robert Coe, It’s the Effect Size Stupid: What E ffect S ize I s and W hy I t I s I mportant , Paper presented at the Ann ual Conf. of Brit ish Educ ational Research Ass’n (Sept. 2002) ( Coe ), J.A. 7 63

– 73.

In response, PT submitted comments in mid

- March 2020 , J.A. 780

– 1004, arguing that “use of simple averaging is both mathematically and stati stically inaccurate

, ” J.A. 781

. PT pointed to section s of Cohen (at 67), of Coe (at 6 ), and of Ellis (at 10 , 26, 27 ), all of which set forth formula s that clearly use weighted averages when comparing groups that have both different sizes and different standard deviations (and hence variances). J.A. 790 –9 8. 4 PT proposed a modification, under which the variances of the two groups (test group, comparison group) are weighted by the total weight

, in kilograms

, of the goods in each group, so the denominator would be � WW aa WW

aa + WW bb σ aa 2 + WW bb WW

aa + WW bb σ bb 2

J.A. 791

– 92 . I n that formula

, WW aa and WW bb are the kilogram weight s of the test

- group goods and comparison

- group goods , respectively (and σ aa 2 and σ bb 2 again refer to the variances of the sale prices in the test and comparison groups,

4 The Coe reference, at 6 (question 7), is the reference discussed in our 2019 opinion. CAFC 2019 Op. , 940 F.3d at 673

– 74. Case: 21-1747 Document: 48 Page: 11 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 12

respectively). This formula differs in minor ways from the specific formulas in Cohen

, Coe , and Ellis

, which involve details of weighted averaging appropriate for sampling when not all population data is known. Commerce did not object to PT’s formula on the ground that it departed from those models, but rather on the ground that it used weighted averages rather than simple averages

. In May 2020, Mid Continent submitted comments arguing for the simple

- average approach

. J.A. 1005

– 70

. It i ncluded in its comments a discussion of a portion of Cohen to which Commerce, in its draft redetermination, had not pointed. J.A. 1022

– 24 ( citing Cohen at 3 60

– 61)

. Mid Contin ent pointed to a statement in Cohen

— discussing an example involving a researcher’s creating equal

- size samples of the groups under study, even though some of the groups are a much smaller share of the overall population than the others— about treat ing a group’s characteristic as an “abstract effect quite apart from the relative frequency with which that effect

. . . occurs in the population.” Id.
In June 2020, Commerce published its final redetermination. J.A. 1073 –11 21 . Commerce continued to use a simple average, and it “provid[ed] further explanation of [its] methodology as requested.” J.A. 1073. Commerce explained that to determine whether there was a pa ttern of export prices that “differ significantly” among purchasers, regions, or periods, it used the widely accepted Cohen’s d test to measure the “effect size” on price associated with sales to certain purchasers, in certain regions, or during certain periods of time, and it relied on Ellis

, Cohen , and Coe for elaboration

. See J.A. 1077

– 80 . It noted that the denominator of the Cohen’s d coefficient was a “yardstick to gauge the significance of the difference of the means,” J.A. 1079, and it stated that the statistical literature presented different methods for computing the denominator,“including the square root of the simple average of the variances within each group,” J. A

. 1080 (citing Cohen at 44). Case: 21-1747 Document: 48 Page: 12 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 13

To justify its decision to use the simple average to calculate the denominator, Commerce wrote: [T]he purpose of Commerce’s Cohen’s d test is to determine whether U.S. prices differ significantly among purchasers, regions, or time periods – i.e.

, do prices to each purchaser, region, or time perioddiffer significantly from all other prices of the comparable merchandise. Although these are all prices in the U.S. market made by the respondent, this analysis requires that these prices be subdivided into separate distinct groups to consider separately whether the respondent’s pricing behavior for salesto one specific group differs from its pricing behavior for all other sales. In other words, these prices, all of which are used to evaluate: 1) a re spondent’s pricing behavior in the U.S. market; and 2)whether the respondent is dumping, are now considered to represent two distinct pricing behaviors which may differ significantly. For the purpose of this particular analysis, Commerce finds that these two distinct pricing behaviors are separate andequally rational, and each is manifested in the individual prices within each group. Therefore, each warrants an equal weighting when determiningthe “standard deviation” used to gauge the significance of t he difference in the means of the prices of comparable merchandise between these two groups. Because Commerce finds that each of these pricing behaviors are equally genuine when considering the distinct pricing behaviors between a given purchaser, region, or time period and all other sales, an equal weighting is justified when calculating the “standard deviation” of the Cohen’s d coefficient. To do otherwise and use an average weighted by sales volume, sales value, or number of transactions would give pre ference to one pricing behavior over the other, and therefore would bias Case: 21-1747 Document: 48 Page: 13 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 14

the “yardstick” by which Commerce measures the observed difference in prices between the test and comparison groups.

J.A. 1080

– 81. In responding to comments, Commerce referred to th e “abstract effect” idea invoked by Mid Continent. J.A. 1112, 1116

– 17. It also pointed to the difference between this context, in which Commerce has the complete population data

poo l ( and each pairwise comparison involves the entire population), and the context of the cited literature involving sampling from a population. J.A. 1109. Commerce further said that PT’s challenge of the simple average relied on conclusory allegations of “skewed” results, J. A

. 1081 , incorrect assumptions about the relationship between standard deviation and group size, J.A. 1083

– 84, and “cherry

- picked” data, J.A. 1084

– 85 . It added that the simple average provid es “predictability” because “the importance given to each pricing behavior will be the same for all products,” and it concluded that the use of a simple average was “notonly a reasonable approach but a more accurate and consistent measurement

. ” J.A. 1087

. 3

The matter returned to the Trade Court. PT submitted comments that included extensive attachments containing the sales information before Commerce and figures that, according to PT

, show ed why weighted averaging is substantially better than simple averaging at capturing thoseinstances in which a test group’s prices are noticeably outside the dispersion of prices gen erally. J.A. 1122 –1 373. The government responded, arguing, among other things, that PT failed to exhaust administrative remedies as to some of what PT presented. J.A. 1397 –1 428.In January 2021, the Trade Court sustained Commerce’s determination. CIT 2021 Op. , 495 F. Supp. 3d at 1300. It accepted Commerce’s explanation that a weighted Case: 21-1747 Document: 48 Page: 14 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 15

average would “inappropriately move the pooled standard deviation toward the pricing behavior of either the test or comparison group,” id. at 1304, and agreed that an equal weighting was justified because the prices in each test and comparison group “separ ately and equally represent the respondent’s pricing behavior,” id. at 1308 (quoting J.A. 1108). The Trade Court did not refer to the “abstract effect” idea invoked by Mid Continent and Commerce. 5 PT timely appealed to this court. We have jurisdiction

u nder 28 U.S.C. § 1295(a)(5).

II

A

We review Commerce’s decisions using the same standard of review applied by the Trade Court, while carefully considering the Trade Court’s analysis. CAFC 2019 Op.

, 940 F.3d at 667. Commerce’s selection of a methodology f or implementing the statutory directive of § 1677f

- 1(d)(1)(B) is “an interpretation of that statutory language” that we review for reasonableness. Stupp , 5 F.4th at 1352

– 53; see Ningbo Dafa Chem. Fiber Co. v. United States , 580 F.3d 1247, 1256 (Fed. Cir. 2009) (“It is well established that statutory interpretations articulated by Commerce duringits antidumping proceedings are entitled to judicial deference under Chevron .” (cleaned up)).

5 The Trade Court reached its conclusion without having to determine which if any submissions by PT were objectionable under the exhaustion requirement , because the court concluded that all of the submissions were, in any event, answered by the just

- noted ra tionale. Id. at 1306

– 08. Our decision does not rely on the materials that were the subject of the exhaustion dispute, which we therefore need not address

. Case: 21-1747 Document: 48 Page: 15 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 16

“Commerce has discretion to make reasonable choices within statutory constraints.” CAFC 2019 Op. , 940 F.3d at 667; see also Stupp , 5 F.4 th at 1353, 1354

. Commerce’s “special expertise in administering antidumping duty law” is one recognized basis for the “significant deference” embodied in the reasonableness standard

. Ningbo Dafa , 580 F.3d at 1256 ; see also Wheatland Tube Co. v. United States

, 495 F.3d 1355, 1361 (Fed. Cir. 2007) . E xpertise enabl es an agency to identify a reasonable interpretation and to set forth an adequate justification for choosing it over others

, but it remains a judicial obligation to ensure that the agency has done so, while avoiding judicial usurpation ofagency authority to make pertinent factual and policy determinations. See Burlington Truck Lines, Inc. v. United States , 371 U.S. 156, 167 –69 (1962); CS Wind Vietnam Co. v. United States , 832 F.3d 1367, 1377 (Fed. Cir. 2016). For us to fulfill that obligation, we must ensure that Commerce provide s “ an explanation that is adequate to enable the court to determine whether the choices are in fact reasonable, including as to calculation methodologies.” CAFC 2019 Op. , 940 F.3d at 667; Stupp , 5 F.4 th at 1357

. Last year, in Stupp , we held that Commerce had provided an inadequate explanation of the reasonableness of its use of Cohen’s d in its differential

- pricing analysis in circumstances where that use seemingly departed from what the statistical literature taught. Stupp , 5 F.4 th at 1357

– 60. What was unjustified there was Commerce’s use of Cohen’s d “in adjudications in which the data groups being compared are small, are not normally distributed, and have disparate variances.” Id. at 1357. We remanded for further consideration.
On the record presented to us here

, we do the same, focusing on a different feature of Commerce ’s use of Cohen’s d . W e hold that Commerce has not adequately justified its adoption of simple averaging for the Cohen’s d denominator . Commerce has departed from the methodology described in all the cited statistical literature Case: 21-1747 Document: 48 Page: 16 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 17

governing Cohen’s d , but it has not justifie d that departure as reasonable

. We again remand for further consideration.

B

1

Commerce recognized that the function of the denominator in the Cohen’s d coefficient is to be a “yardstick to gauge the significance of the difference of the means” of the sales prices of the test and comparison groups

. J.A. 1079. Th e numerator of Cohen’s d is the difference in weighted average sales prices between the test and comparison groups. Without further context, i.e. , without a basis for comparison, it is impossible to say whether that difference is “significant,” under 19 U.S.C. § 1677f

- 1(d)(1)(B)(i) or otherwise

. Th e central purpose of using the Cohen’s d ratio is
to provide the missing basis of comparison

— the “yardstick.” Cohen’s d relates

, by d ivision

, the difference in mean prices of the two particular groups to a figure representing the magnitude of differences in ( dispersion of ) the prices in the data pool more generally

. S ee CAFC 2019 Op.

, 940 F.3d at 671

. If the mean

- price difference is large enough compared to the more general dispersion measure ( i.e. , the ratio of the two is at least 0.8), “Commerce deemsthe sales prices in the test group to be significantly different from the sales price s in the comparison group.” Stupp

, 5 F.4th at 1347 ; see Differential Pricing RFC at 26
, 722 (“ The Department finds that the difference is significant, and that the sales of the test group pass the Cohen’s d test, if the calculated Cohen’s d coefficient is equal to or ex ceeds the large threshold.”). The cited literature makes clear that one way to form the more general data -pool dispersion figure for the denominator

— seemingly the preferred way if the full set of population data is available

— is to use the standard deviatio n for the entire population. But the references recognize that entire population data may be unavailable, in which case an alternative is needed, and the alternative is chosen with Case: 21-1747 Document: 48 Page: 17 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 18

the object of estimating (approximat ing ) the unavailable population standard deviation . Thus, Ellis states: To calculate the difference between two groups we subtract the mean of one group from the other (M 1 – M 2 ) and divide the result by the standard deviation (SD) of the population from which the groups were sampled . The only tricky part in this calculation is figuring out the population standard deviation. If this number is unknown, some approximate value must be used instead.

Ellis at 10 (emphasis added). Coe presents the formula for measuring effect size as

[ MMMMMMMM oooo eeee eeee ee ee gg gg ]

− [ MMMMMM MM oooo cc gg gg ]
SS DD DD and then states:

The “standard deviation” is a measure of the spread of a set of values. Here it refers to the standard deviation of the population from which the different treatment groups were taken . In practice, however, this is almost never known, so itmust be estimated either from the standard deviation of the control group, or from a “pooled” value from both groups

. . . . Coe at 2 (emphasis added). And Cohen similarly indicates that the ideal denominator is the full population’s standard deviation

, which may be approximated by a pooled estimate. See Cohen at 27 (dividing by “the common within

population standard deviation”); Cohen at 67 (noting that the denominator is “the usual pooled within sample estimate of the population standard deviation”

— indicating that the pooling method, based on the standard deviation s of each of the two groups, aims to estimate the standard deviation of the overall population)

. When the full population data set is unavailable, all of the cited literature point s to use of a “pooled standard deviation” of the two particular Case: 21-1747 Document: 48 Page: 18 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 19

groups at is sue to form the denominator. Cohen at 67; Ellis at 10, 26

– 27; Coe at 6.In this matter, Commerce did not use the standard deviation of all the data for its denominator. It made that choice even while recognizing that it had the full set of data for U.S. sales for the period Commerce was reviewing . J.A. 1109 (“Commerce’s anal ysis is based on all of the U.S. sales data for the respondent .

. . . Commerce does not sample the respondent’s U.S. sales data used in the Cohen’s d test, and the calculated means and variances of the U.S. prices are the actual values of the entire population of U.S. sales and are not estimates of those values.”) . I ndeed, in each
test

- group/comparison

- group pair, the test and comparison groups together make up “the entire universe, i.e. , population, of the available data

, ” J.A. 1115 , because for each te st group, the comparison group is all other sales data

. Rather than use the population standard deviation in the d enominator, Commerce used a “pool ed standard deviation,” pooling the standard deviations for each pair of test and comparison groups. As discussed above , it used simple averaging to do the pooling —even where the test and comparison groups have different sizes. In making that choice to use simple averaging, however, Commerce departed from, r ather than followed, the cited statistical literature. As we have described above, Commerce’s formula for the denominator, �
σσ A 2 + σσ B 2

2

comes from a section of Cohen tha t addresses a situation in which the two groups at issue are of the same size. Cohen
at 43

– 44; id. at 43 ( “C ASE 2: σ A ≠ σ B , n A = n B ”). By contrast, when the sampled groups have unequal sizes, the cited literature uniformly teaches us e of a pooled standard deviation estimate that involves weighted averaging

. See Cohen at 67; Ellis at 26

– 27; Coe at 6. Case: 21-1747 Document: 48 Page: 19 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 20

The section of Cohen (at 359

– 61 ) cited by Mid Continent and Commerce for its “abstract effect” language is no exception. It nowhere recites use of a simple average forcalculating a pooled standard deviation from groups of unequal size

. The discussion in that section involves f, an effect size index that is related to, but not the same as, the Cohen’s d coefficient , applicable when there are arbitrarily many groups to compare, rather than just two

. See Cohen at 274

– 80. It expressly s ets forth a simple average formulafor when the groups are equal in size but a weighted average formula for when the groups are of different size. Id. at 359

– 60. The language of “abstract effect” is used in a discussion of forming certain equal

- size grou ps for the comparative analysis : in the example given, if the object was to identify differences in viewpoint on a topic (attitudes toward the United Nations) among three groups (Jews, Protestants, Catholics), the researcher could form equal
groups even th ough random sampling from a population would produce different

- size groups. Id. at 360

– 61. Nothing in the section applies simple averaging to pooled standard deviation estimates for different

- size groups.

2

Commerce offered one principal reason for departing from the teaching of all the cited statistical literature. It said that the data in each group (the test and comparison groups) represent “equally rational” and “equally genuine” pricing choices and tha t, therefore, each group “warran tsan equal weighting” for calculating the pooled standard deviation

. J.A. 1080

– 81

. We see no basis for questioning, here or generally, the premise of equal rationality of the pricing behavior (and equal genuineness, if that is different, which is not cl ear). But Commerce has not offered an adequate explanation of why that premise supports the particular step Commerce must justify: a choice of how to form the denominator in the Cohen’s d formula

. Case: 21-1747 Document: 48 Page: 20 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 21

The fact that the seller is acting rationally and genuine ly in its pricing choices in both the test and comparison
groups provides no apparent reason for assigning equal weight to each group’s standard deviation when computing the pooled standard deviation. Th e rationality and genuineness of the seller’s pricing choices have no evident connection to the undisputed purpose of the denominator figure

— to provide a dispersion figure for the more general pool that serves as a yardstick for deciding on the significance of the difference in mean prices of the two groups. Both the numerator and denominator take the behavior asa given and form certain statistical measures from the objective data that are then related in the ratio that is Cohen’s d. Commerce has not identified anything in the statistical measure at issue that depends on considerations of rationality and genuineness of the conduct that gave rise to the objective data

. Indeed, Commerce has not shown that the numerous real

- world examples used in Cohen to illustrate the methods taught are different in th e respect Commerce now features

, i.e.

, Commerce has not shown that the Cohen examples (generally or, perhaps, ever) involve sampled groups of data that reflect behavior that is
not “rational” and “genuine

. ” Thus, Commerce has not adequately justified, through its central rationale, its departure from the statistical literature’s description of the Cohen’s d coefficient.

Commerce also assert ed that a simple average provides “predictab [ ility ] ” in that “th e importance given to each pricing behavior will be the same for all products.” J.A. 1087. But Commerce did not suggest that this basis would suffice for its denominator choice without the principal basis we have just discussed and found inadequate. And in anyevent, Commerce has not provided a reasonable explanation for this predictability assertion. I t is not clear from Commerce’s language, including its “importance given to each pricing behavior” language, what meaning Commerce was ascribing to “pred ictability” independent of its equal ity Case: 21-1747 Document: 48 Page: 21 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 22

(of rationality and genuineness ) basis. If Commerce was referring , as “predictability” would suggest, to the ability to predict the consequences for the dumping analysis based on the ability to predict the weighting of a sale (the “importance” component of the analysis), Commerce did not explain why simple averaging has greater predictability than weighted averaging (let alone than using the full population’s standard dev iation for every d calculation). The mathematical formula s hav e no identified element s of discretion , or other components, that distinguish them with respect to prediction . Specifically, Commerce provided no basis for an assertion of lesser “predictabili ty” if weighted averaging is done on the basis of weight (or dollars orunits), not transactions, as we discussed in our 2019 opinion. See CAFC 2019 Op. at 674. Not having provided an adequate explanation of “predictability,” Commerce also did not provid e an adequate explanation of what significance this consideration should have in the overall choice of denominator for Cohen’s d

. In its final redetermination

, Commerce invoked the “abstract effect ” idea mentioned in the section of Cohen discussed above. J.A. 1112, 1116

– 17. As we have noted, t hat section does not call for simple averaging for unequal size groups in the denominator of Cohen’s d or in the formula for the related f figure. And Commerce has not explained how such simple averaging could be derived from the “abstract effect” idea itself . W e do not understand Commerce

, in invoking this idea , to be saying anything other than that the statutory “differ significantly” analysis foc uses on the difference between the test and comparison groups for its own sake

, rather than for what it indicates about the overall population. One difficulty with this observation is that Commerce has not explain ed how it affects comparisons, such as tho se Commerce makes in its differential

- pricing analysis , where the groups together make up the entire population (which was not the case in the section of Cohen at issue)

. More broadly and fundamentally, Commerce has Case: 21-1747 Document: 48 Page: 22 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 23

not explain ed why th e fact that the focus is being placed on the difference between the groups distinguishes the teaching of the cited literature

— which, as discussed, uses the Cohen’s d coefficient precisely to provide a yardstick for determining the significance of the difference in group mean s. Thus, Commerce has not explained why that focus calls for

a simple

- averaging yardstick figure for determining the significance of the difference when calculating Cohen’s d (or , even, the f statistical measure) for different

- size
groups

. Commerce observes that the cited literature discuss es

“ sampling ” from a population, whereas Commerce has the entire population data and each of its test

- comparison group pairs involves the entire population . J.A. 1109. In Stupp , we stated that Commerce h ad not explained how this difference bears on the reasonableness of Commerce’s use of Cohen’s d in certain respects not at issue in the present matter. 5 F.4th at 1360. Here, too, although it is undisputed that sampling for estimation of an unknown

overa ll population figure requires certain minor alterations of the formula for weighted averaging not needed in the present context, compare , e.g. , Cohen at 67, with J.A. 792 (PT proposal), Commerce has not explained why the basic choice of weighted averaging of unequal

- size group s fails to apply to the present context. The cited literature nowhere suggest s simple averaging for unequal

- size groups. Indeed, when the entire population is known, the cited literature points toward using the standard deviation of the entire population as the denominator in Cohen’s d

— which Commerce has not done.

3 Commerce’s job is not to follow a statistical test as explained in published literature for its own sake, but to implement the statutory mandate to determine when prices of certain groups “differ significantly.” 19 U.S.C. § 1677f

1(d)(1)(B)(i). In implementing a statutory mandat e, an Case: 21-1747 Document: 48 Page: 23 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 24

agency is not duty

- bound to follow published literature when, e.g. , the literature is inapplicable to the specifi c problem before the agency or is not itself well grounded. But here Commerce embraced the Cohen’s d statistics measure and relied on the literature for that measure in making its statutory significance assessment

— and that embrace extends beyond the first step and is the foundation of the remaining steps

. A fter the calculation of Cohen’s d , the next step in Commerce’s analysis is to declare what number is high enough to be significant (constituting “passing” the Cohen’s d test), and the number it uses is 0.8, the threshold for a “large ” effect size stated in Cohen

. See Cohen at 26; J.A. 1080; Differential Pricing RFC at 26
, 722 ; Stupp

, 5 F.4 th at 1347. The “passing” sales then determine the results of the next “ratio test” step.

In this situation, Commerce needs a reasonable justification for departing from what the acknowledged literature teaches about Cohen’s d

. It has departed from those teachings about how to calculate the denominator of Cohen’s d

, specifically in deciding to use sim ple averaging when the groups differ in size. And its explanations for doing so fail to meet the reasonableness threshold (a deferential one, in recognition of expertise) for the reasons we have set forth. We must remand for further proceedings before C ommerce in light of the identified deficiencies

— as we did in this matter in 2019 regarding the simple

- averaging choice and as we did in Stupp regarding other aspects of Commerce’s use of Cohen’s d

. Commerce must either provide an adequate explanation for its choice of simple averaging or make a different choice, such as use of weigh t ed averaging or use of the standard deviation for the entire population. 6

6 Mid Continent argues that, if weighted averaging is to be done, the weighting should be based on the number Case: 21-1747 Document: 48 Page: 24 Filed: 04/21/2022MID CONTINENT STEEL & WIRE v. US 25

III For the foregoing reasons, we vacate the decision of the Trade Court and remand for further proceedings consistent with this opinion.

No costs.

VACATED AND REMANDED

of transactions, rather than on a measure of how much is sold ( e.g. , number of nails, weight of nails, dollars paid). Mid Continent Br. 28

– 29. But Commerce rejected weighted averaging altogether, so we do not have before us for review a choic e of one basis of weighting rather thananother. We make two observations relevant to Commerce’s consideration of that choice if it adopts weighted averaging on remand. First, when it uses the average

- to

average method, Commerce computes average prices b y quantity sold , not by transaction. See J.A. 1111. Second, in our earlier opinion, we recognized that Commerce had criticized weighting by the number of transactions as susceptible to manipulation, and we noted that weighting by quantity appears to addr ess that issue. CAFC 2019 Op.

, 940 F.3d at 674. Case: 21-1747 Document: 48 Page: 25 Filed: 04/21/2022