Slip Op. 23-23

UNITED STATES COURT OF INTERNATIONAL TRADE

STUPP CORPORATION ET AL., 

Plaintiffs and Consolidated 

Plaintiffs, 

and 

MAVERICK TUBE CORPORATION, 

Plaintiff-Intervenor and 

Consolidated Plaintiff-Intervenor, 

v. 

UNITED STATES, 

 Defendant, 

and 

SEAH STEEL CORPORATION AND 

HYUNDAI STEEL COMPANY, 

Defendant-Intervenors and 

Consolidated DefendantIntervenors. 

 Before: Claire R. Kelly, Judge 

 Consol. Court No. 15-00334 

 

OPINION AND ORDER 

[Sustaining the U.S. Department of Commerce’s third remand redetermination in the 

less-than-fair-value investigation of welded line pipe from the Republic of Korea.] 

 Dated: February 24, 2023 

Jeffrey M. Winton and Jooyoun Jeong, Winton and Chapman PLLC, of Washington, 

D.C., argued for plaintiff SeAH Steel Corporation. 
Consol. Court No. 15-00334 Page 2 

Robert R. Kiepura, Trial Attorney, Commercial Litigation Branch, Civil Division, 

U.S. Department of Justice, of Washington, D.C., argued for defendant. With him on 

the brief were Claudia Burke, Assistant Director, Patricia M. McCarthy, Director, 

and Brian M. Boynton, Principle Deputy Assistant Attorney General. Of Counsel 

was Mykhaylo Gryzlov, Senior Counsel, Office of the Chief Counsel for Trade 

Enforcement and Compliance, U.S. Department of Commerce, of Washington, D.C. 

Jeffrey D. Gerrish, Schagrin Associates, of Washington, D.C., argued for plaintiff 

Welspun Tubular LLC USA. With him on the brief were Roger B. Schagrin and Saad 

Y. Chalchal. 

Kelly, Judge: Before the court is the U.S. Department of Commerce’s 

(“Commerce”) third remand redetermination in its 2015 less-than-fair-value 

investigation of welded line pipe imported from the Republic of Korea (“Korea”). See 

Final Results of Redetermination Purs. Ct. Remand, April 4, 2022, ECF No. 208 

(“Remand Results”); see also Welded Line Pipe From [Korea], 80 Fed. Reg. 61,366 

(Dep’t Commerce Oct. 13, 2015) (final determination of sales at less than fair value), 

as amended by Welded Line Pipe From [Korea], 80 Fed. Reg. 69,637 (Dep’t Commerce 

Nov. 10, 2015) (“Amended Final Determination”) and accompanying Issues & 

Decisions Memo, A-580-876, (Oct. 5, 2015), ECF No. 30-3 (“Final Decision Memo”). 

In Stupp Corporation v. United States, the Court of Appeals for the Federal Circuit 

vacated this court’s opinion, remanding to Commerce to further explain why it is 

reasonable to apply the Cohen’s d test as part of its differential pricing analysis if 

certain statistical assumptions have not been met. Stupp Corporation v. United 

States, 5 F.4th 1341 (Fed. Cir. 2021) (“Stupp III”). For the following reasons, the court 

sustains Commerce’s third remand redetermination. 
Consol. Court No. 15-00334 Page 3 

BACKGROUND 

The court presumes familiarity with the facts of this case as set out in this 

court’s previous opinions, as well as the Court of Appeals’ decision in Stupp III, and 

now recounts only the facts relevant to the court’s review of the Remand Results. On 

November 14, 2014, Commerce initiated an antidumping duty investigation of welded 

line pipe from Korea. Welded Line Pipe From [Korea], 79 Fed. Reg. 68,213, 68,213 

(Dep’t Commerce Nov. 14, 2014) (initiation of less-than-fair-value investigation). 

Commerce published its final determination on October 5, 2015 and, finding that 

39.72% of SeAH Steel Corporation’s (“SeAH”) U.S. sales passed the Cohen’s d test, 

applied the average-to-transaction method to those sales. Final Decision Memo. at 4. 

Commerce accordingly calculated a 2.53% dumping margin for SeAH. Amended 

Final Determination at 69,638. SeAH appealed, arguing that Commerce’s 

differential pricing analysis and application of the Cohen’s d test were contrary to 

law and unsupported by substantial evidence. See Stupp Corp. v. United States, 359 

F. Supp. 3d 1293, 1302 (Ct. Int’l Tr. 2019) (“Stupp I”), reconsideration denied, 365 F. 

Supp. 3d 1373 (Ct. Int’l Tr. 2019). SeAH also argued that Commerce improperly 

rejected its case brief, which contained citations to certain academic texts not part of 

the administrative record. Id. at 1300–03; Letter from Commerce Rejecting SeAH’s 

Sept. 1, 2015 Case Br., 1–2, PD 384, bar code 3302027-01 (Sept. 3, 2015); [SeAH’s] 

Case Br., PD 377–79, bar codes 3301610-01–03 (Sept. 1, 2015) (“SeAH’s Rejected 

Brief”). 
Consol. Court No. 15-00334 Page 4 

This court sustained Commerce’s determinations with respect to its use of 

differential pricing analysis and rejection of SeAH’s case brief. Stupp I, 359 F. Supp. 

3d at 1299–1306. Specifically, the court found that Commerce correctly rejected 

SeAH’s brief because the academic authorities cited in the brief constituted new 

factual information intended to advance SeAH’s arguments. Id. at 1301. The court 

also found that Commerce’s differential pricing analysis was supported by 

substantial evidence because, among other reasons, Commerce was not required to 

apply the Cohen’s d test in accordance with academic literature. Id. at 1302–06. 

The Court of Appeals remanded, instructing Commerce to further explain why 

its use of the Cohen’s d test was reasonable in light of “significant concerns” related 

to application of the test. Stupp III, 5 F. 4th at 1357. Specifically, the Court of Appeals 

questioned the reasonableness of Commerce’s application of Cohen’s d test to data 

failing to satisfy the statistical criteria of normality, equal variance, and sufficient 

observation size. Id. 1357–60. Citing to academic literature examining the use of 

Cohen’s d test to measure effect size, the Court of Appeals expressed concern that 

Commerce’s failure to satisfy the statistical criteria assumed by Cohen’s test could 

“undermine the usefulness of the interpretive cutoffs,” resulting in artificially 

inflated dumping margins. Id. at 1357. The Court of Appeals affirmed the remaining 

issues from Stupp I, including this court’s decision to uphold Commerce’s rejection of 

SeAH’s case brief. Id. at 1344. 
Consol. Court No. 15-00334 Page 5 

JURISDICTION AND STANDARD OF REVIEW

The court has jurisdiction pursuant to 28 U.S.C. § 1581(c) (2018), which grants 

the court authority to review actions initiated under 19 U.S.C. § 1516a(a)(2)(B)(i)1

contesting the final determination in an antidumping duty order. The court will 

uphold Commerce’s determination unless it is “unsupported by substantial evidence 

on the record, or otherwise not in accordance with law.” 19 U.S.C. § 1516a(b)(1)(B)(i). 

“The results of a redetermination pursuant to court remand are also reviewed ‘for 

compliance with the court’s remand order.’” Xinjiamei Furniture Co. v. United 

States, 968 F. Supp. 2d 1255, 1259 (Ct. Int’l Tr. 2014). 

DISCUSSION 

 On remand, SeAH challenges Commerce’s application of the Cohen’s d test on 

the grounds that (1) assumptions underlying the test have not been met, (2) the large 

cutoff prescribed by the test is arbitrary, and (3) random variables such as exchange 

rates can cause “false positives.” See Cmts. of [SeAH] on Final Determ. on Remand, 

5–36, June 14, 2022, ECF No. 216 (“SeAH’s Cmts.”). Defendant and Welspun Tubular 

LLC (“Welspun”) counter that (1) the assumptions are inapplicable, (2) Commerce’s 

application of Cohen’s d test leads to reasonable results, (3) the cutoff is supported by 

statistical literature, (4) SeAH cannot introduce non-record documents for the first 

time on remand, and (5) SeAH failed to exhaust administrative remedies for its 

1 Further citations to the Tariff Act of 1930, as amended, are to the relevant 

provisions of Title 19 of the U.S. Code, 2018 edition. 
Consol. Court No. 15-00334 Page 6 

exchange rate-related arguments. See Def.’s Corr. Resp. to Cmts., 9–34, Sept. 22, 

2022, ECF No. 230 (“Def.’s Reply”); [Welspun’s] Reply [SeAH’s] Cmts. on Remand 

Redeterm., 18–32, Aug. 15, 2022, ECF No. 218 (“Welspun’s Reply”). For the following 

reasons, the court sustains the results of Commerce’s remand redetermination. 

I. SeAH’s Non-Record Documents 

 SeAH’s comments to the Remand Results reference several pieces of academic 

literature which were not included in the administrative record. See SeAH’s Cmts. 

at 6–36. Welspun and Defendant argue that the court should disregard these 

materials, as judicial review is limited to the agency record. Welspun’s Reply at 19–

20; Def.’s Reply at 10–12. SeAH argues that the court may take judicial notice, or 

otherwise consider, these materials to better understand the statistical principles 

behind Cohen’s d test. Reply of [SeAH] to Responses by Def. and [Welspun], 10–11, 

Sept. 28, 2022, ECF No. 236 (“SeAH’s Reply”). For the following reasons, the court 

need not take judicial notice of SeAH’s non-record documents to understand the 

statistical principles they illustrate. 

 Judicial review is generally limited to the administrative record before the 

agency at the time it rendered its decision. See Camp v. Pitts, 411 U.S. 138, 142 

(1973). “The purpose of limiting review to the record actually before the agency is to 

guard against courts using new evidence to ‘convert the “arbitrary and capricious” 

standard into effectively de novo review.’” Axiom Res. Mgmt, Inc. v. United States, 
Consol. Court No. 15-00334 Page 7 

564 F.3d 1374, 1380 (Fed. Cir. 2009) (quoting Murakami v. United States, 46 Fed. Cl. 

731, 735 (2000), aff’d, 398 F.3d 1342 (Fed. Cir. 2005)). 

This court previously upheld Commerce’s decision to reject SeAH’s non-record 

documents, on the grounds that the submissions constituted new factual information 

not on the administrative record. Stupp I, 359 F. Supp. 3d at 1299–1306. SeAH’s 

submissions primarily cited academic articles relating to application of the Cohen’s d

test under certain conditions. SeAH’s Rejected Brief at 26–33. On appeal, the Court 

of Appeals affirmed this court’s decision rejecting the non-record information, 

concluding that SeAH’s materials were not introduced to correct inaccuracies in 

Commerce’s reporting, but to support its argument challenging Commerce’s use of 

Cohen’s d test. Stupp III, 5 F.4th at 1350. In Stupp III, the Court of Appeals 

nevertheless referenced and quoted from several of the non-record texts introduced 

by SeAH.2 Id. at 1357–59. On remand, Commerce asked SeAH to place the 

2 The Federal Circuit cited the following five works: Grissom, Robert and Kim, John, 

Effect Sizes for Research: Univariate and Multivariate Applications (2nd ed. 2012), A580-876, PRRD 8, bar code 4181776-01 (Nov. 12, 2021) (“Grissom & Kim”); Coe, 

Robert, It’s the Effect Size Stupid: What Effect Size Is and Why It Is Important, paper 

presented at the Annual Conference of the British Educational Research Association 

(September 2002), A-580-876, PRRD 8, bar code 4181776-01 (Nov. 12, 2021) (“Coe”); 

Lane, David, et al., Introduction to Statistics, Online Edition, A-580-876, PRRD 8, 

bar code 4181776-01 (Nov. 12, 2021) (“Lane”); Algina, James, Keselman, H.J., and 

Penfield, Randall, An Alternative to Cohen’s Standardized Mean Difference Effect 

Size: A Robust Parameter and Confidence Interval in the Two Independent Groups 

Case, 10 Psychological Methods (2005), A-580-876, PRRD 8, bar code 4181776-01 

(Nov. 12, 2021) (“Algina”); Li, Johnson Ching-Hong, Effect Size Measures in a TwoIndependent-Samples Case With Nonnormal and Nonhomogenous Data, Behavioral 

Research (2015), A-580-876, PRRD 8, bar code 4181776-01 (Nov. 12, 2021) (“Li”). 
Consol. Court No. 15-00334 Page 8 

previously-rejected materials on the record, which SeAH did. See Letter from 

[Commerce] to Interested Parties, A-580-876, PRRD 1, bar code 4176823-01 (Oct. 29, 

2021); SeAH Submission of Publications Requested, A-580-876, PRRD 8, bar code 

4181776-01 (Nov. 12, 2021). In its comments on the remand redetermination, SeAH 

again cites new academic sources not on the record, arguing that the court may 

consider the underlying statistical principles which the texts discuss.3 See SeAH’s 

Cmts. at 6–25. SeAH claims that the Court of Appeals considered SeAH’s previous 

academic sources in Stupp III, despite upholding Commerce’s rejection of SeAH’s 

brief which contained these materials. SeAH’s Reply at 10–11. Although SeAH 

states that the Court of Appeals took judicial notice of the texts, it later clarified that 

the court may consider the statistical principles regardless of whether the texts 

themselves are on the record. Response of [SeAH] to Def’s Sur-Reply, 2–3, Nov. 14, 

2022, ECF No. 247 (“SeAH’s Sur-Reply”).4

3 SeAH cites to the following six non-record sources in its comments: Todd D. Little, 

Oxford Handbook of Quantitative Methods in Psychology (2013); Ricca and Blaine, 

Notes on a Nonparametric Estimate of Effect Size, 90:1 Journal of Experimental 

Education 249 (2022); Hedges and Olkin, Overlap Between Treatment and Control 

Distributions as an Effect Size Measure in Experiments, 21:1 Psychological Methods 

61 (2016); Huberty and Lohman, Group Overlap as a Basis for Effect Size, 60:4 

Educational and Psychological Measurement 543 (2000); J. Cohen, A Power Primer, 

112:1 Psychological Bulletin 155 (1992); F. Alvarez, A. Atkeson, and P. Kehoe, If 

Exchange Rates Are Random Walks, Then Almost Everything We Say about 

Monetary Policy is Wrong, Federal Reserve Bank Of Minneapolis Research 

Department Staff Report 388 (2007). 

4 SeAH argues that the Court of Appeals’ decision “stands for the proposition that, 

when an agency purports to be using a statistical test in accordance with widelyadopted statistical practice, the courts may consider non-record academic materials 

to evaluate that claim.” SeAH’s Sur-Reply at 2. 
Consol. Court No. 15-00334 Page 9 

Consistent with the approach of the Court of Appeals, the court may recognize 

the basic statistical principles discussed in these texts. The idea, for example, that a 

skewed statistical sample may yield inaccurate results is inductive reasoning—not 

an assertion of fact. The Court of Appeals’ references to academia do not render its 

reasoning dependent on academic sources. Thus, the court considers Commerce’s 

Cohen’s d methodology in the same way it would review any other methodology, and 

may make logical inferences without taking judicial notice of SeAH’s literature. 

II. Administrative Exhaustion 

 SeAH argues that random fluctuations of exchange rates can affect the Cohen’s 

d test, and lead to inaccurate results. SeAH’s Cmts. at 24–28. Welspun and 

Defendant argue SeAH failed to properly exhaust this argument. Welspun’s Reply 

at 20–21; Def.’s Reply. at 27–28. For the following reasons, the court concludes that 

SeAH has exhausted this argument. 

Pursuant to 28 U.S.C. § 2637(d), the court “shall, where appropriate, require 

the exhaustion of administrative remedies,” including at the preliminary 

determination stage before the agency. 28 U.S.C. § 2637(d); 19 C.F.R § 351.309(c)(2). 

Section 2637(d) grants the court “discretion to identify circumstances where 

exhaustion of administrative remedies does not apply.” ABB, Inc. v. United States, 

920 F.3d 811, 818 (Fed. Cir. 2019) (quoting Consol. Bearings Co. v. United States, 

348 F.3d 997, 1003 (Fed. Cir. 2003)). The court may also excuse exhaustion in certain 
Consol. Court No. 15-00334 Page 10 

circumstances, such as when a party is raising a “pure question of law.” Agro Dutch 

Indus. Ltd. v. United States, 508 F.3d 1024, 1029 (Fed. Cir. 2007). 

 In its comments on the draft redetermination, SeAH argued that random 

factors, such as exchange rates, could cause the standard deviation of test populations 

to vary significantly. SeAH’s Cmts. on Draft Redetermination, 17–20, A-580-876, 

PRRD 30, bar code 4224356-02 (March 21, 2022). SeAH did not provide an exchange 

rate table, or assert that its actual sales during the period of review were affected by 

these factors. See id. Subsequently, in its comments on the final remand results, 

SeAH again argues that Cohen’s d could be significantly affected by random factors 

where the population of data is not normally distributed. SeAH’s Cmts. at 24–28. 

SeAH adds that its sales were, in fact, affected by fluctuations in the exchange rate 

between the U.S. dollar and Korean won, because its inland freight expenses were 

denominated in won. Id. at 24. Welspun counters that SeAH failed to raise its 

exchange rate argument and supporting factual information during the draft 

redetermination. Welspun’s Reply at 26. 

SeAH has exhausted its exchange rate argument. SeAH’s exchange rate 

examples provide an illustration of how it believes random factors can render the 

Cohen’s d test inaccurate when values are not normally distributed. Normal 

distributions is one of the three assumptions that the Court of Appeals remanded to 

Commerce to explain. See Stupp III, 5 F. 4th at 1360. Therefore, Welspun’s argument 

that Commerce had no opportunity to address SeAH’s exchange rate calculations 
Consol. Court No. 15-00334 Page 11 

misses the point; these calculations are not a new argument, but an illustration of 

the same scenario Commerce was directed to explain. 

SeAH separately argues Commerce must “ignore” fluctuations in exchange 

rates pursuant to 19 U.S.C. 1677b-1(a). SeAH’s Br. at 8. SeAH concedes that it did 

not raise this argument in its comments to the draft remand results; nevertheless, it 

argues that this argument may be considered as a “pure question of law.” Oral 

Argument at 0:27:39–0:27:53. Defendant argues that Commerce had no opportunity 

to consider this argument on remand, and Welspun characterizes the argument as a 

mixed question of law and fact. Oral Argument at 0:24:15–0:26:33, 0:27:56–0:28:39. 

Whether § 1677b-1(a) is pertinent to Commerce’s differential pricing analysis is a 

matter of statutory interpretation, not subject to exhaustion requirements. See Agro 

Dutch Indus., 508 F.3d at 1029. However, SeAH’s argument that § 1677b-1(a) directs 

Commerce to compensate for exchange rate variations is inapposite. In its full 

context the statute directs Commerce to use the exchange rate “in effect on the date 

of sale” for valuation of merchandise, and to ignore fluctuations on that particular 

date. 19 U.S.C. 1677b-1(a). The plain language does not mandate that Commerce 

compensate for a respondent’s decision to report expenses in a foreign currency, as 

SeAH suggests. 

III. Differential Pricing Analysis 

In Stupp III, the Court of Appeals remanded for further explanation of 

Commerce’s application of the Cohen’s d test as part of its differential pricing 
Consol. Court No. 15-00334 Page 12 

analysis. Stupp III, 5 F.4th at 1360. On remand, SeAH renews its argument that 

Commerce’s application of Cohen’s d test is flawed because it fails to take into account 

assumptions of sample size, distribution, and variance underlying the test, and 

further argues Commerce’s choice of Cohen’s large cutoff is arbitrary. SeAH’s Cmts. 

at 6–24. SeAH also claims random fluctuations in exchange rates can affect the d

coefficient, causing even test groups with identical prices to pass. Id. at 24–28. 

Commerce counters that its Cohen’s d analysis does not operate in a vacuum, and 

must be considered with the ratio test and meaningful difference test. See Remand 

Results at 26, 28, 30–31, 41–42, 54–60. Commerce also argues the cutoffs are tied 

to real-world criteria, that small fluctuations in price will not lead to “false positives” 

in Cohen’s test, and that use of the 0.8 threshold results in reasonably infrequent 

application of alternative methodologies. Remand Results at 16–19, 32, 54–60. For 

the following reasons, Commerce has adequately addressed Court of Appeals’ 

concerns.

 When investigating whether subject merchandise is being sold at less than fair 

value, Commerce typically compares “the weighted average of the normal values to 

the weighted average of the export (and constructed export prices) for comparable 

merchandise” unless it determines another method is appropriate. 19 U.S.C. § 1677f1(d)(1)(A)(i); 19 C.F.R. § 351.414(c)(1). This average-to-average (“A-to-A”) method 

compares the weighted average of a respondent’s home country sales prices during 

the investigation period to the weighted average of the respondent’s sales prices in 
Consol. Court No. 15-00334 Page 13 

the United States during the same period. See 19 C.F.R. § 351.414(b)(1). One concern 

with the A-to-A method is that it may allow a foreign producer or exporter to engage 

in “targeted dumping,” which occurs when an exporter sells at a dumped price to 

particular customers or regions, while selling at higher prices to other customers or 

regions. See Apex Frozen Foods Priv. Ltd. v. United States, 862 F.3d 1337, 1341 (Fed. 

Cir. 2017) (“Apex II”). As a result, higher-priced products can mask dumped products 

when Commerce averages the sales using the A-to-A method. 

Congress addressed concerns over targeted dumping with the passage of 19 

U.S.C. § 1677f-1(d)(1)(B). See Apex II, 862 F.3d at 1342. Section 1677f-1(d)(1)(B) 

allows Commerce to compare “the weighted average of the normal values to export 

prices . . . of individual transactions for comparable merchandise if (i) there is a 

pattern of export prices . . . for comparable merchandise that differ significantly 

among purchasers, regions or periods of time, and (ii) [Commerce] explains why such 

differences cannot be taken into account using [the A-to-A method or transaction-totransaction method5].” 19 U.S.C. § 1677f-1(d)(1)(B)(i)–(ii). Targeted dumping is more 

likely when export prices fit a pricing model that differs significantly across different 

market segments. Apex II, 862 F.3d at 1341–42. Congress has not provided a method 

for Commerce to use to determine whether a pattern of significantly different prices 

5 Commerce’s regulations provide that the transaction-to-transaction method, which 

compares prices of individual transactions, will be employed only in rare cases, “such 

as when there are very few sales of subject merchandise and the merchandise sold in 

each market is identical or very similar or is custom-made.” 19 C.F.R. § 351.414(c)(2). 
Consol. Court No. 15-00334 Page 14 

exists. However, the Statement of Administrative Action (“SAA”) of the Uruguay 

Round Agreements Act explains that Commerce should proceed “on a case-by-case 

basis, because small differences may be significant for one industry or one type of 

product, but not for another.” Uruguay Round Agreements Act, Statement of 

Administrative Action, H.R. Doc. No. 103-316, vol. 1, at 842–43 (1994), reprinted in 

1994 U.S.C.C.A.N. 4040, 4178.6 

 To determine whether the criteria set forth in § 1677f-1(d)(1)(B) are met, 

Commerce conducts a “differential pricing analysis” of a respondent’s sales. See 

Differential Pricing Analysis; Request for Comments, 79 Fed. Reg. 26,720, 26,722 

(Dep’t of Commerce May 9, 2014). This analysis contains three tests. First, 

Commerce applies to respondent’s sales what it refers to as the “Cohen’s d test,” 

described in more detail below, which measures the degree of price disparity between 

groups of sales. Id. Commerce then counts the percentage of sales by value which 

“pass” the Cohen’s d test, and applies its “ratio test.” Id. at 26,722–23. If 33% of 

respondent’s sales or less pass, Commerce uses the A-to-A method, and if 66% or 

more pass, Commerce uses the A-to-T method. Id. If the total percentage of passing 

sales is between 33% and 66%, Commerce takes a hybrid approach, applying the Ato-T method to those sales passing the test, and the A-to-A method to the remainder. 

Id. Finally, if Commerce has not selected the A-to-A method for all sales, it applies 

6 The SAA is an “authoritative expression by the United States concerning the 

interpretation and application” of the Uruguay Rounds Agreement Act. 19 U.S.C. 

§ 3512(d). 
Consol. Court No. 15-00334 Page 15 

the “meaningful difference” test to determine whether the A-to-A method could 

nevertheless account for the disparate pricing. Id. at 26,723. Commerce applies the 

test by comparing a respondent’s dumping margin using both A-to-A and the selected 

method. Id. If the A-to-A margin is below the de minimis threshold and the margin 

from the selected method is not, or if both margins are above the threshold and differ 

by 25% or more, Commerce continues to use the selected method; otherwise, 

Commerce applies the A-to-A method for all sales. Id. 

 As applied by Commerce, the Cohen’s d test involves comparing the productspecific prices of “test groups” of a respondent’s sales to a “comparison group” by 

region, purchaser, and time period. Stupp III, 5 F.4th at 1346. For each category, 

Commerce segregates sales into subsets, with one subset becoming the test group, 

and the remaining subsets being combined as the comparison group. Id. Commerce 

then calculates the means and standard deviations of the test and comparison groups. 

Id. Commerce then calculates a Cohen’s d coefficient by dividing the difference in the 

groups’ means by the groups’ standard deviation.7 Id. Each subset is thus tested 

against the remaining subsets across each category, and assigned a d coefficient by 

7 Thus, d = |mean of test group – mean of control group| ÷ standard deviation. 

Commerce uses a modified version of this formula, substituting the square root of the 

simple average of the groups’ variances for standard deviation. The Cohen’s d test 

solves for a coefficient representing “effect size.” See generally Cohen, Jacob, 

Statistical Power Analysis for the Behavioral Sciences, (2nd ed. 1988), A-580-876, 

PRRD 8, bar code 4181776-01 (Nov. 12, 2021) (“Cohen”). “Effect size quantifies the 

size of the difference between two groups, and may therefore be said to be a true 

measure of the significance of the difference.” Coe at 7. 
Consol. Court No. 15-00334 Page 16 

solving Cohen’s ratio. If the d value of a test group is equal to or greater than the 

“large threshold,” or 0.8, the observations within that group are said to have “passed” 

the Cohen’s d test. Id. at 1347. 

 In Stupp III, the Court of Appeals expressed concern that Commerce’s 

application of Cohen’s d under certain circumstances could undermine the usefulness 

of the test in less-than-fair-value determinations. Specifically, the Court of Appeals 

identified three potential scenarios in which use of Cohen’s d could be problematic: 

first, when the distribution of a respondent’s sales data is not normal, second, when 

the test groups have few data points, and third, when there is minimal variance in a 

respondent’s sales. Stupp III, 5 F.4th at 1357–59. The assumption of normality is 

satisfied when a fixed percentage of the population falls within each standard 

deviation from the mean—in other words, that a population density graph generally 

shows a symmetrical, bell-shaped curve. See Starnes, Yates, and Moore, Statistics 

through Applications, 116 (2005), A-580-876, PRRD 8, bar code 4181776-01 (Nov. 12, 

2021). The assumption of size is satisfied when the population is sufficiently large. 

See Cohen at 21. The assumption of homogeneous variances is satisfied when the 

standard deviations of test and comparison groups are similar. See Grissom at 68–

69. Commerce argued in Stupp III, as it does now, that the three assumptions are 

only relevant as a matter of statistical significance, and do not apply when analyzing 

a whole population; the Court of Appeals concluded that this explanation did not fully 

address its concerns. Stupp III, 5 F.4th at 1360. 
Consol. Court No. 15-00334 Page 17 

 The Court of Appeals illustrated the problems it identified with the Cohen’s d

test through two hypotheticals. First, the Court of Appeals considered a situation in 

which Commerce analyzed a group of only eight export sales across four groups, such 

that each test group would consist of only two sales. Stupp III, 5 F. 4th at 1358–59. 

With groups of such small numbers, the Court of Appeals pointed out that there 

would be some upward bias in effect size, such that the test would produce more 

“passing” results, and potentially exaggerate dumping margins. Id. at 1359. The 

Court of Appeals also observed that a group of only two sales would lack normality. 

Id. Second, the Court of Appeals described a test group of five sales of about $100 

each, which differed from one-another by up to two cents. Id. Because the standard 

deviation of such a group would be so small, the Court of Appeals pointed out that 

the denominator in Cohen’s ratio would be drastically reduced, again causing an 

increase in effect size, and inflating the resulting dumping margin. Id. The Court of 

Appeals noted that an objective examiner looking at these similar sales prices “would 

be unlikely to conclude that they embody a ‘pattern’ of prices which ‘differ 

significantly.’” Id. (citing 19 U.S.C. § 1677f-1(d)(1)(B)(i)). 

Commerce reasonably explains that Cohen’s d test does not operate in a 

vacuum, but as part of the differential pricing analysis as a whole.8 Turning first to 

8 The parties devote a significant part of their briefings to discussion of (1) the 

permissibility of using Cohen’s d test on full populations, and (2) questions of 

(footnote continued) 
Consol. Court No. 15-00334 Page 18 

the assumptions of population size and normalcy, the Court of Appeals questioned 

whether small sample sizes without normal distributions could “exaggerate” 

dumping margins by introducing an “upward bias” to effect size. Stupp III, 5 F.4th at 

1359.9 Addressing the Court of Appeals’ concerns about population size, Commerce 

statistical significance versus practical significance. See Remand Results at 11–16, 

43–51; SeAH’s Cmts. at 7–10; Def.’s Br. at 12–20; Welspun’s Br. at 21–23; SeAH’s 

Reply at 17–27; Def’s Sur-Reply at 12–29, 34–35; SeAH’s Sur-Reply at 16–24. Neither 

question is determinative of whether Commerce’s methodology is reasonable. Both 

arguments have already been raised before the Court of Appeals, which concluded 

that they did not resolve its concern over whether the absence of certain assumptions 

forecloses Commerce’s use of Cohen’s d test. Stupp III, 5 F.4th at 1360. 

Commerce correctly asserts that a “t-test” for statistical significance is used 

with sampled data, and that Dr. Cohen considered normal distribution and equal 

variance as necessary assumptions in a t-test. See Cohen at 19; Remand Results at 

12–16. However, Commerce improperly reasons that because there is no need for a 

t-test, there is no basis for the assumptions. Remand Results at 14. Commerce also 

asserts that SeAH’s assumptions are only relevant as a matter of statistical 

significance, and that they do not apply because Cohen’s d test determines practical 

significance. Remand Results at 14, 43–45. That these assumptions are required 

for questions of statistical significance does not answer the question of whether they 

are also needed to determine practical significance, as the Court of Appeals suggests. 

See Stupp III, 5 F.4th at 1360. 

Although SeAH claims that academic sources do not support Commerce’s use 

of Cohen’s d in its differential pricing analysis, this argument is inapposite. SeAH’s 

decision to substantially advance its arguments using labels taken from statistical 

literature does not alter the court’s obligation on review. See Soc Trang Seafood Joint 

Stock Co. v. United States, 321 F. Supp. 3d 1329, 1339 n.13 (2018) (“the fact that 

Commerce has adopted a methodology based upon a statistical tool known as Cohen’s 

d, and chooses to refer to this methodology as Cohen’s d, does not diminish the 

discretion granted to Commerce”); see also Mid Continent Steel & Wire, Inc. v. United 

States, 31 F.4th 1367 (Fed. Cir. 2022) (“Commerce’s job is not to follow a statistical 

test as explained in published literature for its own sake, but to implement the 

statutory mandate to determine when prices of certain groups “differ significantly”). 

9 Although the parties dispute whether such results are really “false positives,” it is 

undisputed that in at least some instances, groups with as few as two sales have 

(footnote continued) 
Consol. Court No. 15-00334 Page 19 

explains that its Cohen’s d analysis does not stand alone, and operates together with 

the ratio test and meaningful difference test. See Remand Results at 26, 28, 30–31, 

41–42, 54–60. Thus, even if the Cohen’s d values of small test groups were less 

accurate than for large test groups, this difference does not by itself render 

Commerce’s use of Cohen’s test unreasonable, because the ratio test and meaningful 

difference test compensate for inaccuracies. See id. Commerce’s differential pricing 

analysis looks at the frequency and impact of effect size to detect targeted dumping—

not the effect size alone. See Cohen at 8; Remand Results at 26–28. As Commerce 

points out in its remand redetermination, the “sole purpose of the Cohen’s d test” is 

to determine whether prices “differ significantly” across region, time period, or 

customer. Remand Results at 41. The “pattern” of export prices which Commerce 

must find under 19 U.S.C. § 1677f-1(d)(1)(B)(i) is then determined by the ratio test. 

Id. at 42. The ratio test has already been approved by the Court of Appeals, which 

found that Commerce’s choice of the 33% and 66% thresholds was a “reasonable 

choice.” Stupp III, 5 F.4th at 1355. SeAH’s attacks on Cohen’s d test presuppose that 

what SeAH claims are “false positives” automatically affect the accuracy of 

passed Cohen’s test. See Stupp III, 5 F.4th at 1357; SeAH’s Cmts. at 17–19; see 

Remand Results at 55, 58–59. Identifying results as “false” positives begs the 

question of what is a false positive. See Remand Results at 59 (“To label this result 

a ‘false-positive’ does not render the variances inaccurate or erroneous”). SeAH 

illustrates this situation using data from its own sales, showing how a group of only 

two sales to a single customer passed Cohen’s test, despite SeAH’s observation that 

a visual comparison of the groups on a graph showed those sales to be near the 

average price. SeAH’s Cmts. at 18. Commerce counters that a visual inspection may 

be inadequate in situations involving complex calculations. Remand Results at 59. 
Consol. Court No. 15-00334 Page 20 

Commerce’s differential pricing analysis, when in fact Commerce has allowed for 33% 

positives before there is any potential effect on a respondent’s dumping margins. 

 Commerce also addresses the Court of Appeals’ concern whether samples 

without normal distributions will produce an inappropriate number of passes. SeAH 

points to numerous academic sources which it claims confirm the usefulness of 

Cohen’s test is compromised when comparing data sets with non-normal 

distributions. See SeAH’s Cmts. at 7, n.19 (citing Cohen at 13); Id. at 12 (citing Ellis 

at 41); Id. at 13 (citing Starnes, Yates, and Moore at 135). The Court of Appeals has 

acknowledged some of these sources. See Stupp III, 5 F.4th at 1357–59 (citing Cohen 

at 21, Grissom & Kim at 66, Coe at 13, Lane at 645, Algina at 318, and Li at 1571). 

The court need not opine on the relevance of these academic observations;10 however, 

it logically follows that a relatively large-tailed distribution (i.e., with large standard 

deviation) in a test group would tend to decrease Cohen’s d coefficient, while the 

opposite would result in an increase. See Remand Results at 29 (“in other words, the 

fat-tailed distribution may undervalue the significance of effect”) (emphasis in 

10 The task of the court is not to interpret the meaning of literature treating with 

correct application of Cohen’s d. Rather, the court must determine whether 

Commerce’s methodology is reasonable in light of considerations that run counter to 

its decision. See Motor Vehicle Mfrs. Ass’n of U.S. v. State Farm Mut. Auto. Ins. Co., 

463 U.S. 29, 43 (1983); Ceramica Regiomontana, S.A. v. United States, 636 F. Supp. 

961, 966 (Ct. Int’l Tr. 1986), aff’d, 810 F.2d 1137, 1139 (Fed. Cir. 1987) (“As long as 

the agency’s methodology and procedures are reasonable means of effectuating the 

statutory purpose, and there is substantial evidence in the record supporting the 

agency’s conclusions, the court will not impose its own views as to the sufficiency of 

the agency’s investigation or question the agency’s methodology”). 
Consol. Court No. 15-00334 Page 21 

original). SeAH focuses on the second of these two scenarios, arguing that even 

inputting random data, such as exchange rates, can cause test groups to frequently 

pass Cohen’s d test. SeAH’s Cmts. at 25–28. SeAH further argues that the ratio test 

does not account for such random fluctuations.11 Oral Argument at 0:42:57–0:43:43. 

Commerce addresses these arguments by explaining that even if Cohen’s test can 

produce positive results under unusual circumstances, this possibility does not mean 

its use of Cohen’s d is unreasonable when combined with the ratio test and 

meaningful difference test. See Remand Results at 26, 28, 30–31, 41–42, 54–60. 

The Court of Appeals also specifically asked Commerce to explain why it can 

use the 0.8 threshold identified by Dr. Cohen as a measure of a significant price 

difference, when Commerce evaluates data which fails to meet statistical 

assumptions of normality, size and variance.12 Stupp III, 5 F.4th at 1360. Although 

Commerce reiterates those assumptions are irrelevant, see Remand Results at 11–

11 Commerce explains that, even if exchange rate fluctuations do affect prices, this 

effect is not “random” because a respondent can control in which currency it 

denominates its prices. Remand Results at 45; Oral Argument at 0:49:11–0:50:05. 

12 Although the Court of Appeals approved the 0.8 cutoff in Mid Continent Steel & 

Wire, Inc. v. United States, it explained in Stupp III that it had yet to consider the 

reasonableness of the 0.8 cutoff value when the assumptions in question have not 

been met. Stupp III, 5 F.4th at 1356–57(“We held that . . . it is reasonable to adopt 

that [0.8] measure where there is no better objective measure of effect size. We did 

not, however, address SeAH’s second argument [on assumptions] in Mid Continent”) 

(citation omitted) discussing Mid Continent Steel & Wire, Inc. v. United States, 940 

F.3d 662, 673 (Fed. Cir. 2019); see also Oral Argument at 1:39:45–1:40:30. More 

specifically, SeAH argues that it is unreasonable to compare its prices, which are not 

normally distributed, using a subjective benchmark that was derived from a 

normally-distributed population. SeAH’s Cmts. at 10–12. 
Consol. Court No. 15-00334 Page 22 

16, it also explains its choice of the 0.8 threshold as a function of its differential 

pricing analysis. First, Commerce explains that it employs the 0.8 threshold to 

identify where prices “differ significantly” pursuant to 19 U.S.C. § 1677f-1(d)(1)(B)(i). 

Remand Results at 11. Second, Commerce states the 0.8 measurement “represents a 

difference which is ‘grossly perceptible.’” Remand Results at 52. Reasonably 

discernible from this statement is that Commerce considers a significant difference 

to be grossly perceptible in the same way that Dr. Cohen identified a large threshold 

as one that is “grossly perceptible.” See Cohen at 27. The SAA to the Uruguay Round 

Agreements Act directs Commerce to proceed “on a case-by-case basis, because small 

differences may be significant for one industry or one type of product, but not for 

another,” SAA at 842–43; thus, Commerce’s choice of a measurement that is a 

function of standard deviation as a uniform approach to identify differences as 

significant is reasonable, even if the absolute difference in means is small. 

Commerce’s approach tailors the question of what is a significant difference in price 

to the pricing parameters of different products. Third, Commerce adequately 

explains its adoption of Cohen’s widely-recognized choice of 0.8 as a large threshold 

as significant. Remand Results at 18. It explains that it chose the 0.8 standard 

because it was “a conservative standard to determine that the observed price 

differences are significant.” Id. Commerce summarizes its reasoning by explaining 

that “[u]sing Dr. Cohen’s thresholds is a reasonable approach to interpret whether 

the difference in the prices is significant and the further interpretation of the 
Consol. Court No. 15-00334 Page 23 

difference in the prices in the context of the calculation of dumping margins ensures 

the reasonable and limited application of the alternative comparison methodology.” 

Id. at 33. Thus, Commerce chose a threshold it predicted would result in limited 

application of the alternative methodology. 

Although Commerce adopted this yardstick from Dr. Cohen, and did so because 

it was widely acknowledged in the statistical literature, Commerce does not rely on 

the prominence of this yardstick alone. Commerce elaborates that its “actual 

application of the Cohen’s d test in the context of the differential pricing analysis 

resulted in the application of an alternative comparison methodology to a relatively 

small number of respondents.” Remand Results at 32. Discernible from Commerce’s 

explanation is that the 0.8 cutoff produces reasonable passing rates once the ratio 

and meaningful difference tests are applied. SeAH challenges Commerce’s reliance 

on the 0.8 threshold as large, arguing that Commerce’s only basis for using the 

threshold is that it is widely accepted. SeAH’s Cmts. at 10–11. However, in addition 

to relying on a widely-accepted standard for “grossly perceptible” to determine what 

is significant, Commerce defines “significant” with reference to the impact a price 

difference has on a respondent’s dumping margins. Remand Results at 32. Finding 

that the 0.8 threshold leads to relatively few determinations of targeted dumping, 

Commerce concludes that its choice is reasonable. Id. 

Congress delegated to Commerce the authority to determine where a price 

difference is significant. 19 U.S.C. § 1677f-1(d)(1)(B)(i). Congress also made clear 
Consol. Court No. 15-00334 Page 24 

that the definition of a “significant price difference” would depend on the product at 

issue. See SAA at 842–43. Thus, Congress entrusted Commerce to use its expertise 

and knowledge of pricing to gauge price distinctions. Cf. Fujitsu General Ltd. v. 

United States, 88 F.3d 1034, 1039 (Fed. Cir. 1996) (granting Commerce significant 

deference in determinations “involv[ing] complex economic and accounting decisions 

of a technical nature”). Commerce’s decision to adopt Cohen’s 0.8 (“large”) threshold 

as a measure of significance because it is widely accepted in the statistical literature 

does not undermine the reasonableness of that choice, if it is based on Commerce’s 

expertise and Commerce demonstrates the reasonableness of that choice with 

reference to the impact it has on the differential pricing analysis. Thus, Commerce’s 

reference to Cohen’s work does not circumscribe its discretion to choose the same 

values in a new context, because that choice is itself reasonable. 

 Commerce addresses the Court of Appeals’ concern that prices with small 

variances, which hover around the same value, will produce inaccurate results on 

Cohen’s test. As an initial matter, Commerce explains that results which pass 

Cohen’s test under these circumstances are not “false positives,” as small differences 

in average prices will mean that variances, too, will be small. Remand Results at 59; 

Oral Argument at 1:06:32–1:07:02. Thus, it is discernable that a small variance 

means a small difference in price will be more significant, and a passing result under 

these circumstances is not necessarily “erroneous.” Remand Results at 59. 

Nevertheless, the Court of Appeals observed that an objective examiner considering 
Consol. Court No. 15-00334 Page 25 

a group of sales where prices differed by only a few cents would be unlikely to conclude 

that they show a “pattern” of prices that “differ significantly” under the statute. 

Stupp III, 5 F.4th at 1359. Commerce responds to this issue by pointing out that an 

examiner would indeed conclude that there was no pattern—because Commerce does 

not look for a pattern at this stage of its differential pricing analysis. Remand Results 

at 41. Again, Commerce explains that the ratio test determines whether a pattern 

exists, while Cohen’s d test only shows whether there are significant price differences. 

Id. at 41–42. Thus, Cohen’s test would need to generate enough “false positives” to 

overcome the 33% threshold, at minimum, and there is no evidence on the record 

suggesting that price patterns, such as that proposed by the Court of Appeals, occur 

with frequency in SeAH’s sales. 

 Additionally, to specifically address the hypothetical proposed by the Court of 

Appeals, Commerce explains that, in addition to the ratio test, the meaningful 

difference test would prevent low-variance sales which pass Cohen’s d test from 

impacting a respondent’s dumping margins. See Remand Results at 30–31. Adopting 

the Court of Appeals’ example in which all of a respondent’s prices hovered around 

$100 and passed Cohen’s test, Commerce explains that even in this extreme scenario, 

the respondent would still be assessed under the A-to-A method. Id. Choosing a 

normal value for comparison equal to the highest sales price, and thus maximizing 

the respondent’s theoretical dumping margin, Commerce observes the margin would 

still be under the 2% de minimis threshold. Id.; see 19 U.S.C § 1673d(a)(4), 
Consol. Court No. 15-00334 Page 26 

1673b(b)(3). SeAH argues that Commerce’s reliance on the meaningful difference 

test is misplaced, because even changes of less than 2% in a respondent’s dumping 

margin can cross the de minimis threshold and result in a “meaningful difference” 

finding. SeAH’s Sur-Reply at 15. Specifically, SeAH argues that when the Cohen’s 

d results from small-variance data sets of different products are cumulated, 

Commerce may find that a respondent’s sales pass the thresholds for both ratio test 

and the meaningful difference test, even if price differences are negligible. Oral 

Argument at 1:13:37–1:16:06.13 This argument overstates Commerce’s position. It is 

reasonably discernable that Commerce does not rely on the meaningful difference 

test to prevent all “inappropriate” passes from affecting a respondent’s dumping 

margins. Commerce has explained the meaningful difference test compensates for a 

specific concern with low-variance sales which the Court of Appeals identified. See 

Stupp III, 5 F.4th at 1359; Remand Results at 30–31. Moreover, SeAH’s argument is 

misplaced, because the question before the court is not whether it is possible to 

construct an unusual scenario where Cohen’s d test can result in an alternative 

comparison method. Rather, the question is whether Commerce’s use of Cohen’s test, 

when applied as a component of its differential pricing analysis, is reasonable. See 

Ceramica Regiomontana, S.A. v. United States, 636 F. Supp. 961, 966 (Ct. Int’l Tr. 

13 SeAH does not argue that it received an alternative method because its own 

combined sales inappropriately passed the Cohen’s d test. Rather it offers a 

hypothetical to challenge the reasonableness of Commerce’s methodology more 

generally. SeAH’s Cmts. at 8, 14–21. 
Consol. Court No. 15-00334 Page 27 

1986). Thus, for the forgoing reasons, Commerce has adequately explained how its 

methodology is reasonable. 

CONCLUSION 

 For the foregoing reasons, Commerce’s remand results are supported by 

substantial evidence and comply with the court’s Order, Oct. 8, 2021, ECF No. 192, 

in conformity with the Court of Appeals’ Mandate, Oct. 8, 2021, ECF No. 191, and are 

therefore sustained. Judgment will enter accordingly. 

 /s/ Claire R. Kelly 

 Claire R. Kelly, Judge 

Dated: February 24, 2023 

 New York, New York