62850 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices 

333See id.; see also FINRA Rules 12904(f) and 

13904(f). 

334See FINRA August Letter at 7 (citing FINRA 

By-Laws, Article V, Sections 2(c), 3(a) and 3(b)). 

335See id. at 7 n.30. 

336FINRA Rule 8312 (FINRA BrokerCheck 

Disclosure) governs the information FINRA releases 

to the public through its BrokerCheck system. 

Information available to investors through 

BrokerCheck includes, among other things, 

information reported on the most recently filed 

‘‘Registration Forms’’ (with limited exceptions) for 

both member firms and registered individuals, and 

summary information about certain arbitration 

awards against the firm involving a securities or 

commodities dispute with a public customer; see 

also FINRA Rule 8312(b)(2)(A) (using the term 

‘‘Registration Forms’’ to refer collectively to Form 

U4, the Uniform Termination Notice for Securities 

Industry Registration (Form U5), the Uniform 

Disciplinary Action Reporting Form (Form U6), the 

Uniform Application for Broker-Dealer Registration 

(Form BD), and the Uniform Request for BrokerDealer Withdrawal (Form BDW)). The BrokerCheck 

website is available at brokercheck.finra.org. 

337 15 U.S.C. 78o–3(b)(6). 

338 15 U.S.C. 78s(b)(2). 

339 17 CFR 200.30–3(a)(12). 

1 15 U.S.C. 78s(b)(1). 

2 17 CFR 240.19b–4. 

3See Securities Exchange Act Release No. 92844 

(January 4, 2023), 88 FR 1438. 

4 15 U.S.C. 78s(b)(2). 

5See Securities Exchange Act Release No. 96963, 

88 FR 12710 (February 28, 2023). 

6See Securities Exchange Act Release No. 97263, 

88 FR 22498 (April 13, 2023). 

7 15 U.S.C. 78s(b)(2). 

8See Securities Exchange Act Release No. 97844, 

88 FR 44423 (July 12, 2023). 

9All comments received by the Commission on 

the proposed rule change are available on the 

Commission’s website at: https://www.sec.gov/ 

comments/sr-nasdaq-2022-079/srnasdaq2022079.

htm. 

relevant context.333 In addition, FINRA 

stated that after a panel dismisses a case 

at the conclusion of the case-in-chief, 

the firm must file an amended Uniform 

Application for Securities Industry 

Registration or Transfer (‘‘Form U4’’) for 

the associated person to report the final 

disposition of the case as dismissed.334

FINRA stated that along with the final 

disposition, an associated person can 

provide a brief summary or add context 

on Form U4 regarding the circumstances 

leading to the customer arbitration, as 

well as the current status or final 

disposition.335 This updated 

information is subsequently disclosed 

on the associated person’s BrokerCheck 

report, which is publicly available to 

investors.336

The Commission believes that this 

proposed rule change should promote 

transparency about FINRA’s arbitration 

process and help ensure consistent 

treatment of awards. Specifically, the 

proposed rule change equally requires 

all arbitration awards, including awards 

granting a motion to dismiss all claims, 

to be published. These published 

awards should provide current and 

future parties to an arbitration with data 

that could help inform the 

administration of their cases. The 

Commission acknowledges the 

commenter’s concern that a published 

award granting a motion to dismiss all 

claims may not reflect any defense 

raised by respondents. However, these 

concerns should be ameliorated by the 

fact that the Codes permit arbitrators to 

include a rationale underlying the 

award, providing relevant context to the 

dismissal of the claim such as the 

circumstances under which the claim 

was dismissed. In addition, an 

associated person may provide context 

on Form U4 regarding the circumstances 

leading to the customer arbitration, as 

well as the claim’s current status or final 

disposition. For these reasons, the 

Commission finds that this proposed 

rule change is reasonably designed to 

protect investors and the public interest. 

IV. Conclusion 

For the reasons set forth above, the 

Commission finds that the proposed 

rule change, as modified by Amendment 

No. 1, is consistent with Section 

15A(b)(6) of the Exchange Act, which 

requires, among other things, that 

FINRA rules be designed to prevent 

fraudulent and manipulative acts and 

practices, promote just and equitable 

principles of trade, and, in general, 

protect investors and the public 

interest.337

It is therefore ordered pursuant to 

Section 19(b)(2) of the Exchange Act 338

that the proposed rule change (SR– 

FINRA–2022–033), as amended by 

Amendment No. 1, be, and hereby is, 

approved. 

For the Commission, by the Division of 

Trading and Markets, pursuant to delegated 

authority.339

Sherry R. Haywood, 

Assistant Secretary. 

[FR Doc. 2023–19729 Filed 9–12–23; 8:45 am] 

BILLING CODE 8011–01–P 

SECURITIES AND EXCHANGE 

COMMISSION 

[Release No. 34–98321; File No. SR– 

NASDAQ–2022–079] 

Self-Regulatory Organizations; The 

Nasdaq Stock Market LLC; Notice of 

Filing of Amendment No. 2 and Order 

Granting Accelerated Approval of a 

Proposed Rule Change, as Modified by 

Amendment No. 2, To Amend Rules 

4702(b)(14) and (b)(15) Concerning 

Dynamic M–ELO Holding Period 

September 7, 2023. 

I. Introduction 

On December 21, 2022, The Nasdaq 

Stock Market LLC (‘‘Nasdaq’’ or 

‘‘Exchange’’) filed with the Securities 

and Exchange Commission 

(‘‘Commission’’), pursuant to Section 

19(b)(1) of the Securities Exchange Act 

of 1934 (‘‘Act’’) 1 and Rule 19b–4 

thereunder,2 a proposed rule change to 

replace the static holding period 

requirements for Midpoint Extended 

Life Orders and Midpoint Extended Life 

Orders Plus Continuous Book with 

dynamic holding periods. The proposed 

rule change was published for comment 

in the Federal Register on January 10, 

2023.3 On February 22, 2023, pursuant 

to Section 19(b)(2) of the Act,4 the 

Commission designated a longer period 

within which to approve the proposed 

rule change, disapprove the proposed 

rule change, or institute proceedings to 

determine whether to disapprove the 

proposed rule change.5 On March 9, 

2023, the Exchange filed Amendment 

No. 1 to the proposed rule change, 

which amended and superseded the 

proposed rule change as originally filed. 

On April 7, 2023, the Commission 

provided notice of filing of Amendment 

No. 1 and instituted proceedings to 

determine whether to approve or 

disapprove the proposed rule change, as 

modified by Amendment No. 1.6 On 

July 6, 2023, pursuant to Section 

19(b)(2) of the Act,7 the Commission 

designated a longer period on 

proceedings to determine whether to 

approve or disapprove the proposed 

rule change.8 On July 18, 2023, the 

Exchange filed Amendment No. 2 to the 

proposed rule change, which amended 

and superseded the proposed rule 

change as amended by Amendment No. 

1. The Commission received comments 

on the proposed rule change.9 The 

Commission is publishing this Notice 

and Order to solicit comment on 

Amendment No. 2 in Sections II and III 

below, which sections are being 

published verbatim as filed by the 

Exchange, and to approve the proposed 

rule change, as modified by Amendment 

No. 2, on an accelerated basis. 

II. Self-Regulatory Organization’s 

Statement of the Terms of Substance of 

the Proposed Rule Change 

The Exchange proposes to amend 

Rules 4702(b)(14) and (b)(15) of the 

Exchange’s Rulebook to replace the 

static holding period requirements for 

Midpoint Extended Life Orders and 

Midpoint Extended Life Orders Plus 

Continuous Book with dynamic holding 

periods. This Amendment No. 2 


ddrumheller on DSK120RN23PROD with NOTICES1
Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices 62851 

10See SR–Nasdaq–2022–079 Amendment No. 1 

(March 9, 2023), at https://www.sec.gov/comments/ 

sr-nasdaq2022-079/srnasdaq2022079-20159016- 

327215.pdf. 

11See Securities Exchange Act Release No. 34– 

82825 (March 7, 2018), 83 FR 10937 (March 13, 

2018) (SR–NASDAQ–2017–074) (‘‘M–ELO 

Approval Order’’). 

12See Securities Exchange Act Release No. 34– 

86938 (September 11, 2019), 84 FR 48978 

(September 17, 2019) (SR–NASDAQ–2019–048) 

(‘‘M–ELO+CB Approval Order’’). 

13See Securities Exchange Act Release No. 34– 

88743 (April 24, 2020), 85 FR 24068 (April 30, 

2020) (SR–NASDAQ–2020–011) (‘‘M–ELO Timer 

Approval Order’’). 

14The Exchange examined each of its historical 

M–ELO executions to determine at what Midpoints 

of the NBBO the M–ELOs would have executed if 

their Holding Periods had been shorter than onehalf second (500 milliseconds). After examining the 

historical effects of shorter Holding Periods of 

between 10 milliseconds and 400 milliseconds, the 

Exchange determined that a reduction of the M– 

ELO Holding Period to as short as 10 milliseconds 

would have caused an average impact on mark-outs 

of only 0.10 basis points (across all symbols). In 

other words, compared to the execution price of an 

average M–ELO with a one-half second Holding 

Period, the Exchange found that a M–ELO with a 

10 millisecond Holding Period would have had an 

average post-execution impact that was only a tenth 

of a basis point per share—a difference in protective 

effect that is immaterial. See Nasdaq, ‘‘The 

Midpoint Extended Life Order (M–ELO); M–ELO 

Holding Period,’’ available at https:// 

www.nasdaq.com/articles/the-midpoint-extendedlife-order-m-elo%3A-m-elo-holding-period-2020-02- 

13 (analyzing effects of shortened Holding Periods 

on M–ELO performance). 

supersedes the original filing and 

Amendment No. 1 10 in their entireties. 

The text of the proposed rule change 

is available on the Exchange’s website at 

https://listingcenter.nasdaq.com/ 

rulebook/nasdaq/rules, at the principal 

office of the Exchange, and at the 

Commission’s Public Reference Room. 

III. Self-Regulatory Organization’s 

Statement of the Purpose of, and 

Statutory Basis for, the Proposed Rule 

Change 

In its filing with the Commission, the 

Exchange included statements 

concerning the purpose of and basis for 

the proposed rule change and discussed 

any comments it received on the 

proposed rule change. The text of these 

statements may be examined at the 

places specified in Item IV below. The 

Exchange has prepared summaries, set 

forth in sections A, B, and C below, of 

the most significant aspects of such 

statements. 

A. Self-Regulatory Organization’s 

Statement of the Purpose of, and 

Statutory Basis for, the Proposed Rule 

Change 

1. Purpose 

The Exchange proposes to amend 

Rules 4702(b)(14) and (15) of the 

Exchange’s Rulebook to replace the 

static 10 millisecond holding period 

requirements for its Midpoint Extended 

Life Order (‘‘M–ELO’’) and Midpoint 

Extended Life Order Plus Continuous 

Book (‘‘M–ELO+CB’’) Order Types with 

dynamic holding periods (‘‘Dynamic M– 

ELO and M–ELO+CB’’ or collectively, 

‘‘Dynamic M–ELO’’). 

Background 

In 2018, the Exchange introduced the 

M–ELO, which is a Non-Displayed 

Order priced at the Midpoint between 

the National Best Bid and Offer 

(‘‘NBBO’’) and which is eligible for 

execution only against other eligible M– 

ELOs and only after a minimum of onehalf second passes from the time that 

the System accepts the order (the 

‘‘Holding Period’’).11 In 2019, the 

Exchange introduced the M–ELO+CB, 

which closely resembles the M–ELO, 

except that a M–ELO+CB may execute at 

the midpoint of the NBBO, not only 

against other eligible M–ELOs (and M– 

ELO+CBs), but also against NonDisplayed Orders with Midpoint 

Pegging and Midpoint Peg Post-Only 

Orders (‘‘Midpoint Orders’’) that rest on 

the Continuous Book for at least onehalf second and have Trade Now 

enabled.12

When the Exchange designed M–ELO, 

it originally set the length of the 

Holding Period at one-half second 

because it determined that this time 

period would be sufficient to ensure 

that likeminded investors would 

interact only with each other, and with 

minimal market impacts. The Exchange 

believed that the longer length of the M– 

ELO Holding Period and its simplicity 

in design would provide greater 

protection for participants than they 

could achieve through competing delay 

mechanisms. 

In 2020, however, the Exchange 

shortened the length of the Holding 

Period to 10 milliseconds.13 The 

Exchange did so after studying two 

years of actual use and performance of 

M–ELOs, as well as customer feedback. 

That is, the Exchange came to 

understand that, while users of M–ELO 

and M–ELO+CB are less concerned with 

achieving rapid executions of their 

Orders than are other participants, they 

are not indifferent about the length of 

time in which their M–ELOs and M– 

ELO+CBs must wait before they are 

eligible for execution. Indeed, 

participants informed the Exchange that 

in certain circumstances, such as when 

they sought to trade symbols that on 

average had a lower time-to-execution 

than a half-second, they were reticent to 

enter M–ELOs or M–ELO+CBs. They 

indicated that the associated Holding 

Periods for these Order Types were 

longer than necessary to achieve the 

desired protections and that, during the 

residual portion of the Holding Periods, 

they risked losing out on favorable 

execution opportunities that would 

otherwise be available to them had they 

placed a non-MELO order. 

Based upon this feedback, the 

Exchange studied the potential effects of 

reducing the length of the Holding 

Periods for both M–ELOs and M– 

ELO+CBs (as well as for Midpoint 

Orders that would execute against M– 

ELO+CBs). Ultimately, the Exchange 

determined that it could reduce the 

Holding Periods to 10 milliseconds 

without compromising the protective 

power that M–ELO and M–ELO+CB are 

intended to provide to participants and 

investors.14 Thus, the Exchange 

determined that shortening the Holding 

Periods to 10 milliseconds for M–ELOs 

and M–ELO+CBs would increase the 

efficacy of the mechanism while not 

undermining the power of those Order 

Types to fulfill their underlying purpose 

of minimizing market impacts. At the 

same time, the Exchange determined 

that a reduction in the Holding Periods 

to 10 milliseconds would dramatically 

add to the circumstances in which M– 

ELOs and M–ELO+CBs would be useful 

to participants. In its M–ELO Timer 

Approval Order, the Commission agreed 

with the Exchange: 

The Commission notes that, with the 

proposed ten-millisecond Holding 

Period and Resting Period, M–ELOs and 

M–ELO+CBs would continue to be 

optional order types that are available to 

investors with longer investment time 

horizons, including institutional 

investors. The Commission also believes 

that the proposal could make M–ELOs 

and M–ELO+CBs more attractive for 

securities that on average have a timeto-execution of less than one-half 

second and, for investors who currently 

do not use M–ELOs and M–ELO+CBs 

for these securities, provide optional 

order types that could enhance their 

ability to participate effectively on the 

Exchange. The Commission notes that, 

if market participants determine that the 

proposal would make M–ELOs and M– 

ELO+CBs less attractive for their 

particular investment objectives, such 

market participants may elect to reduce 

or eliminate their use of these optional 

order types. Moreover, as noted above, 

the Exchange will continue to conduct 

real-time surveillance to monitor the use 

of M–ELOs and M–ELO+CBs to ensure 

that such usage remains appropriately 

tied to the intent of the order types. If, 

as a result of such surveillance, the 

Exchange determines that the shortened 


ddrumheller on DSK120RN23PROD with NOTICES1
62852 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices 

15M–ELO Timer Approval Order, supra, at 85 FR 

24069. 

16See Diana Kafkes et al., ‘‘Applying Artificial 

Intelligence & Reinforcement Learning Methods 

Towards Improving Execution Outcomes,’’ SSRN, 

October 19, 2022, available at https:// 

papers.ssrn.com/sol3/papers.cfm?abstract_

id=4243985 (attached hereto [sic] without 

modification from the prior version as Exhibit 3(a)) 

(the ‘‘White Paper’’). 

17Although the AI Core Development Group 

acknowledges that an optimal Holding Period 

would update with every incoming order, it 

determined that training a reinforcement learning 

model on every order would be too difficult to 

program and too difficult to implement given the 

nanosecond latency requirements of the Exchange. 

The Group then investigated more feasible update 

cadences and determined the point at which 

optimal outcomes were best balanced with the level 

of programming and implementation difficulty to be 

between 15 and 30 second updates. Ultimately, the 

Group chose a 30 second update cadence to give the 

model the greatest opportunity to learn between 

potential actions. 

18As the White Paper explains, the Group 

developed a model to simulate activity on the 

Exchange involving M–ELOs and M–ELO+CBs 

during the Training Period. See White Paper, supra, 

at 10. 

19See id. 

20The AI Core Development Group experimented 

with a range of permissible Holding Period 

durations. Ultimately, it concluded that it could 

produce better outcomes for M–ELO and M– 

ELO+CB participants than the existing approach 

using Holding Periods as low as 0.25 milliseconds 

and as high as 2.5 milliseconds, under normal 

market conditions. 

21Nasdaq attaches a full list of these data 

elements (attached hereto [sic] as ‘‘Exhibit 3(b))’’, 

along with an observation of the strength of the 

correlations that currently exist between changes to 

those data values and decisions the system makes 

to set the duration of Holding Periods at any given 

time. The Exchange notes that the version of this 

list attached to this Amendment No. 2 supersedes 

prior versions attached to prior versions of this 

filing. This version of the list includes expanded 

explanations of the terminology used therein. See 

also White Paper, supra, at 31, for a description of 

these features. 

22The AI Core Development Group also applied 

to the model a paradigm called ‘‘retraining’’ to 

combat the degradation of model performance that 

can otherwise occur as the reference data it uses for 

initial comparison becomes stale. Finally, the AI 

Core Development group added a stability 

protection mechanism to the model to provide 

maximum production to participants in the event 

that the model observes extraordinary levels of 

instability in the National Best Bid and Offer during 

the prior three seconds as compared to reference 

data. When the model detects such instability, it is 

Holding Period does not serve its 

intended purpose or adversely impacts 

market quality, the Exchange would 

seek to make further recalibrations.15

For similar reasons and with even 

better potential results for participants, 

the Exchange now proposes to further 

refine the length of the Holding Periods 

for M–ELOs and M–ELO+CBs, this time 

through the application of innovative 

and patent pending machine learning 

technology. 

Dynamic M–ELO 

After receiving feedback from 

participants that even 10 millisecond 

Holding Periods for M–ELO and M– 

ELO+CB may, at times, exceed what is 

necessary to accomplish the underlying 

intent of these Order Types, the 

Exchange began to experiment with 

making further refinements to the 

duration of the Holding Periods. 

Ultimately, the Exchange concluded 

that shorter Holding Periods could 

achieve the same, if not better results for 

participants in terms of mark-outs, but 

not in all circumstances. That is, where 

prices of the underlying securities are 

stable, and not subject to imminent 

unfavorable changes, M–ELOs and M– 

ELO+CBs face lower risks of confronting 

spread-crossing orders, such that shorter 

Holding Periods could suffice to protect 

M–ELOs and M–ELO+CB from such 

orders. In periods of heightened price 

volatility, however, M–ELOs and M– 

ELO+CBs also face heightened risks, 

such that longer Holding Periods would 

continue to be beneficial in protecting 

M–ELOs and M–ELO+CBs from such 

risks. Thus, the Exchange determined 

that another across-the-board reduction 

of the static 10 millisecond Holding 

Periods would be sub-optimal because it 

could impact the performance of the M– 

ELO and M–ELO+CB Order Types 

during periods of heightened volatility. 

In light of these observations, the 

Exchange tasked its artificial 

intelligence and machine learning 

laboratory (the ‘‘AI Core Development 

Group’’) to explore whether it could 

employ these innovative technologies to 

optimize the length of M–ELO and M– 

ELO+CB Holding Periods during various 

states of price volatility, and then to 

vary the lengths of the Holding Periods 

dynamically during the lifecycles of M– 

ELOs and M–ELO+CBs, with the 

objectives of improving the performance 

of these Order Types while also further 

reducing opportunity costs. 

As the Exchange explains in greater 

depth in the attached White Paper,16 the 

AI Core Development Group proceeded 

to develop an artificial intelligencebased timer control system that will 

achieve these objectives.17 The AI Core 

Development Group did so by using 

reinforcement learning techniques— 

machine learning paradigms which 

develop optimal solutions to problems 

over time by taking actions to solve 

them, generating feedback on the results 

of such actions, applying that feedback 

to direct and improve the next round of 

solutions, and then repeating the 

feedback loop until the paradigm 

achieves optimized solutions. 

In this instance, the AI Core 

Development Group applied 

reinforcement learning techniques to a 

simulation of the M–ELO Book that it 

constructed using a representative data 

set from the first quarter of 2022 (the 

‘‘Training Period’’). The Training Period 

data consisted of 380 out of the 6,257 

symbols on the M–ELO Book 

(accounting for approximately 67 

percent of M–ELO volume). The 

symbols chosen reflect both activelytraded and thinly-traded securities, and 

both low-priced and high-priced 

securities. 

The AI Core Development Group then 

developed a machine learning model 

and applied it to the Training Period 

data. The Group programmed the model 

to value the achievement of higher fill 

rates or lower mark-outs than that 

which occurred in a historical 

simulation of M–ELOs and M–ELO+CBs 

involving the Training Period data.18

The Group then programmed the model 

to seek to achieve its goals by taking one 

of five possible actions with respect to 

the duration of the Holding Periods at 

30 second intervals 19 for each symbol 

during each trading day of the Training 

Period. That is, at each 30 second 

internal, the model evaluated market 

conditions for each symbol over the 

prior 30 second period and either kept 

the Holding Periods the same, 

increased/decreased them by 0.25 

milliseconds, or increased/decreased 

them by 0.50 milliseconds.20 After each 

decision-making round, the model 

utilized the results to inform its actions 

at the next 30 second increment. 

In making its decisions, the model 

(again, drawing upon a combination of 

historical SIP and M–ELO-specific data) 

considered 142 categories of data 

points.21 A confluence of data points 

that correlated with an increase in 

volatility tended to cause the model to 

increase the durations of Holding 

Periods, including increases in the 

standard deviation of NBBO prices, the 

number of unique participants placing 

sell orders on M–ELO and M–ELO+CB, 

and the volume-weighted average of the 

NBBO spread. Conversely, a confluence 

of data points that correlated with 

greater price stability tended to cause 

the model to decrease the durations of 

Holding periods, such as an increase in 

the median and max number of shares 

per trade and the number of resting bids 

left in the M–ELO and M–ELO+CB 

Book. 

The AI Core Development Team 

produced variations of its model that 

prioritized achievement of the lowest 

mark-outs, the highest fill rates, and a 

blend of these two objectives.22 Through 


ddrumheller on DSK120RN23PROD with NOTICES1
Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices 62853 

programmed to increase the length of the Holding 

Period to 12 milliseconds for a period of 750 

milliseconds. 

23See White Paper, supra, at 22. 

24See id. 

25As set forth in the proposed rule text, the 

phrase ‘‘proprietary assessment of market 

conditions’’ refers to the Exchange’s evaluation of 

prevailing market conditions for a given symbol 

using an algorithm programmed to set a Holding 

Period duration which, at each Change Event, 

achieves an optimal blend of two objectives: 

maximization of M–ELO fill rates; and 

minimization of M–ELO mark-out rates. As the rule 

text states and as is discussed below, the algorithm 

ingests and analyzes 142 data points, which the 

Exchange identifies and describes in Exhibit 3b 

hereto. The Exchange derives these data from a 

combination of public data and M–ELO data feeds. 

Furthermore, the Exchange conducts weekly retrainings of the algorithm, outside of Market Hours, 

to improve its performance relative to the 

immediately preceding period (in terms of the two 

aforementioned objectives). The Exchange deploys 

a retrained version of the algorithm only if it 

determines that doing so will, in fact, improve its 

performance relative to the immediately preceding 

period. The Exchange provides further information 

about the algorithm and the retraining process in a 

White Paper attached hereto [sic] as Exhibit 3a. 

26For purposes of this Rule, the System 

determines that ‘‘extraordinary instability’’ for a 

symbol exists through observations it makes 

following every change in the NBBO for that symbol 

that occurs during the trading day. When the NBBO 

changes, the System looks back at the prior three 

seconds of trading and measures the difference 

between the highest and the lowest NBBO midpoint 

values that occurred during that period, and then 

it compares that measurement to a threshold value 

for the symbol. The System concludes that 

extraordinary instability exists for a symbol if the 

measurement exceeds the threshold value. The 

threshold value for a symbol, in turn, is the 

difference between the highest and the lowest 

NBBO midpoint values for the symbol that, if 

applied to its trading activity during the prior 

trading day, would have caused the System to deem 

trading in the symbol to be extraordinarily unstable 

for as close to one percent of that day as possible. 

a process of learning and 

experimentation involving a 

combination of historical and simulated 

data, the AI Core Development Group 

settled on a Dynamic M–ELO model that 

achieved substantial simulated 

performance improvements for users of 

M–ELO and M–ELO+CB—both in terms 

of mark-outs and fill rates—as compared 

to the static 10 millisecond Holding 

Periods. As the White Paper explains in 

greater detail, Dynamic M–ELO yielded 

an average combined volume-weighted 

(simulated) improvement of 31.7 

percent, including a 20.3 percent 

increase in fill rates and a 11.4 percent 

reduction in mark-outs.23 The White 

Paper provides a more fulsome 

explanation of these improvements.24

Based upon these exciting results, the 

Exchange now proposes to amend Rule 

4702(b)(14) and (15) to replace the static 

10 millisecond timers applicable to M– 

ELO and M–ELO+CB with Dynamic M– 

ELO Holding Periods. Using the 

Exchange’s ‘‘proprietary assessment of 

market conditions’’ 25 and patent 

pending technology, the Dynamic M– 

ELO system will evaluate and, as it 

deems necessary, adjust the length of 

the Holding Periods for each symbol 

comprising M–ELOs and M–ELO+CBs 

(and Midpoint Orders on the 

Continuous Book that opt to interact 

with M–ELO+CBs after resting on the 

Book) every 30 seconds throughout the 

Market Hours (each such 30 second 

interval, a ‘‘Change Event’’). In so doing, 

Dynamic M–ELO will help participants 

to achieve a more optimized blend of 

the underlying purposes of the M–ELO 

and M–ELO+CB Order Types: 

protection against adverse selection 

(low mark-outs) without sacrificing 

opportunities to achieve high-quality 

executions (high fill rates). 

A proposed M–ELO or M–ELO+CB 

with a Dynamic Holding Period will 

operate as follows. At the outset of 

Market Hours (approximately 9:30:00 

a.m.), the Exchange will impose initial 

Holding Periods of 1.25 milliseconds for 

M–ELOs and M–ELO+CBs in all 

symbols. Thereafter, Holding Periods for 

a given symbol will become eligible to 

change dynamically from the initial 

duration beginning at 9:30:30 a.m. and 

then at 30 second intervals thereafter 

during Market Hours. The Exchange 

will then apply to the M–ELO or M– 

ELO+CB Order a Holding Period that is 

of the duration that prevailed at the time 

of entry. For example, if participant A 

enters a M–ELO for symbol XYZ at 

9:30:25 a.m., then Holding Period for 

that M–ELO will be 1.25 milliseconds. 

If at 9:30:30:00 a.m., the System decides 

to lower the duration of the Holding 

Period by 0.50 milliseconds, and then 

participant B enters a M–ELO for 

symbol XYZ at 9:30:45 a.m., then the 

System will assign a 0.75 millisecond 

Holding Period to participant B’s M– 

ELO. To be clear, the System will 

determine Dynamic M–ELO Holding 

Periods independently for M–ELOs and 

M–ELO+CBs in each symbol. 

During normal market conditions, the 

range of potential Holding Period 

durations for M–ELOs and M–ELO+CBs 

will be between 0.25–2.50 milliseconds, 

with the Holding Period duration being 

eligible to change by increments of 

either 0.25 or 0.50 milliseconds at each 

Change Event. Thus, if the Holding 

Period for a M–ELO in symbol XYZ is 

set at 0.75 milliseconds at 2:22:11 p.m., 

and at 2:22:41 p.m., the System 

determines to increase the duration of 

the Holding Period, it may do so only 

by 0.25 or 0.50 milliseconds during that 

event. 

When a Change Event occurs, and the 

System determines to adjust the 

duration of a Holding Period for a 

symbol, that adjustment will apply, not 

only to all M–ELOs and M–ELO+CBs for 

that symbol entered within the 30 

second period after the Change Event 

occurs, but also to M–ELOs and M– 

ELO+CBs entered prior to the Change 

Event with unexpired Holding Periods 

(with applicability retroactive to the 

time of Order acceptance). Thus, if a 

participant enters a M–ELO in symbol 

XYZ at 1:14:299 p.m., and the prevailing 

Holding Period applicable to that M– 

ELO is 2 milliseconds, and at 1:14:30 

p.m., the System modifies the Holding 

Period to be 1.5 milliseconds, then the 

M–ELO will become eligible to execute 

at 1:14:3005 p.m. This is the case 

because the M–ELO will have already 

expended 1 millisecond of its Holding 

Period as of the time of the Change 

Event; thereafter, the M–ELO will need 

to rest only another 0.5 milliseconds to 

become eligible to execute under the 

new 1.5 millisecond Holding Period (as 

measured from 1:14:299 p.m.). This last 

feature ensures that the M–ELO Book 

maintains time priority among M–ELOs 

and M–ELO+CBs in a dynamic 

environment. That is, it ensures that no 

M–ELO or M–ELO+CB with an 

unexpired Holding Period at the time of 

a Change Event will end up becoming 

eligible to execute later than a M–ELO 

entered after the Change Event which 

has a shorter Holding Period applicable 

to it. 

If at any time, the System detects 

extraordinary instability in a symbol, 

then the System will activate a ‘‘stability 

protection mechanism’’ to provide an 

extra layer of protection to M–ELO and 

M–ELO users from the heightened risks 

of adverse selection that exists during 

such periods of instability.26 The 

stability protection mechanism will 

override the prevailing Holding Periods 

for M–ELOs and M–ELO+CBs in a 

symbol experiencing extraordinary 

instability and immediately increase the 

duration of those Holding Periods to 12 

milliseconds for a period of 750 

milliseconds. The System may activate 

the stability protection mechanism even 

between Change Events. The System 

will evaluate, at each NBBO update, 

whether market conditions remain 

extraordinarily unstable and, if so, it 

will restart the 750 millisecond Stability 

Protected Period and maintain the 12 

millisecond Holding Period until 

conditions stabilize. Once the System 

determines that market conditions have 

stabilized (i.e., all measurements for the 

symbol are at or below the threshold 

value throughout the duration of the 

prevailing Stability Protected Period), 

the System will revert the duration of 

the Holding Periods to that which 

prevailed as of the Change Event that 


ddrumheller on DSK120RN23PROD with NOTICES1
62854 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices 

27Prior to commencement of a new 12 

millisecond Holding Period for a new or pending 

M–ELO or M–ELO+CB following a Halt, the System 

will first determine whether the M–ELO or M– 

ELO+CB is or remains eligible for execution. That 

is, the Holding Period will commence only if, upon 

commencement of trading following the Halt, the 

midpoint price for the Order is within the limit set 

by the participant. If not, the System will hold the 

Order until the midpoint falls within the limit set 

by the participant, at which time the 12 millisecond 

Holding Period will commence. 

28Also as a safeguard, the System will apply a 

default Holding Period of 12 milliseconds to a M– 

ELO or M–ELO+CB if ever it fails to receive a signal 

during a Change Event as to whether the System 

should adjust or maintain the duration of the 

prevailing Holding Period. The System will 

continue to apply the default 12 millisecond 

Holding Period until the next Change Event where 

the signal is restored and the System is able to act 

dynamically again. 

29 During periods where the model is not 

undergoing retraining, the System will behave 

predictably from day to day, such that its decisions 

when presented with given set of facts and 

circumstances in a given security on day 1 should 

be the same as they would be on day 2. 

30 In addition to the proposed changes described 

above, the Exchange proposes to delete an 

extraneous reference in Rule 4702(b)(15) to M– 

ELO+CB being eligible to execute against a 

Midpoint Order on the Continuous Book if the 

Continuous Book order has the ‘‘Midpoint’’ Trade 

Now Attribute enabled. In a prior filing, the 

Exchange folded the concept of ‘‘Midpoint Trade 

Now’’ into the general ‘‘Trade Now’’ Attribute. See 

Securities Exchange Act Release No. 34–92180 

(June 15, 2021), 86 FR 33420 (June 24, 2021)(SR– 

NASDAQ–2021–044). 

occurred immediately prior to the 

activation of the stability protection 

mechanism or, if the stability protection 

mechanism was active when a Change 

Event occurred, to the duration selected 

at the immediately preceding Change 

Event. The System will then proceed to 

reevaluate the duration of the Holding 

Periods as per the regular schedule of 

Change Events. 

The following is an illustration of the 

operation of the stability protection 

mechanism. At 11:10:04 a.m., the 

prevailing Holding Period for M–ELOs 

in symbol XYZ is 1.5 milliseconds. At 

the same time, the NBBO for symbol 

XYZ updates. The System looks back at 

the prior three seconds of trading in 

symbol XYZ and finds that during that 

period, the highest observed NBBO 

midpoint was $10.05, and the lowest 

was $10.00, such that the difference 

between these two values is a range of 

$0.05. The System then looks back at 

trading behavior for symbol XYZ during 

the immediately preceding trading day. 

In doing so, the System calculates the 

value of the threshold that would have 

caused the symbol to be deemed 

extraordinarily unstable for one percent 

of the trading day; the System 

determines that this threshold value is 

a range of $0.03. The System then 

compares the $0.03 threshold to its 

measurement of the prior three seconds 

of NBBO changes ($0.05), and concludes 

that over these past three seconds, the 

symbol is extraordinarily unstable. 

Accordingly, the System activates the 

stability protection mechanism and the 

Holding Period for M–ELOs in symbol 

XYZ immediately increases to 12 

milliseconds for a period of 750 

milliseconds. However, 5 milliseconds 

after the Stability Protection Period 

commences, the NBBO updates again, 

thus prompting the System to repeat its 

assessment of the stability of the symbol 

in light of the update. This reassessment 

reveals that the symbol remains 

unstable, such that a new Stability 

Protection Period of 750 milliseconds 

begins at that time (overriding the preexisting Period). Over the course of this 

new Stability Protection Period, the 

NBBO shifts two more times, but each 

of the ensuing reassessments indicate 

that the NBBO ranges for the symbol 

have fallen below the $0.03 threshold. 

The Stability Protection Period elapses 

750 milliseconds after it began with the 

symbol remaining stable. Thus, the 

Holding Period reverts to 1.5 

milliseconds. 

If the Exchange halts trading in a 

symbol, then upon resumption of 

trading, any new M–ELO or M–ELO+CB 

in that symbol and any pending M–ELO 

or M–ELO+CB in that symbol with an 

unexpired Holding Period will be 

subject to a new 12 milliseconds 

Holding Period (running from the time 

when trading resumes) until the next 

scheduled Change Event, at which point 

the System may determine to adjust that 

Holding Period to a duration within the 

range applicable under normal market 

conditions.27 If, however, the System 

determines that extraordinary instability 

in the symbol exists, it will instead 

determine to activate the stability 

protection mechanism and maintain the 

duration of the Holding Period at 12 

milliseconds for another 750 

milliseconds. This design will help to 

ensure that M–ELOs and M–ELO+CBs 

receive added protection coming out of 

halt conditions.28

The Exchange notes that same 

dynamic process described above will 

also apply to and govern the time 

periods during which Midpoint Orders 

on the Continuous Book must rest 

before they will become eligible to 

interact with M–ELO+CBs (provided 

that participants have opted for their 

Midpoint Orders to interact with M– 

ELO+CBs). Thus, the same Holding 

Period duration that the System sets for 

a M–ELO+CB in a symbol during 

Regular Market Hours will also be the 

length of time that a Midpoint Order 

must rest on the Continuous Book must 

rest before it may interact with a M– 

ELO+CB. 

Apart from these impacts of Dynamic 

Holding Periods, M–ELOs and M– 

ELO+CBs will continue to behave as 

they do now in all respects, and as set 

forth in Rules 4702(b)(14) and (15). 

It is important to note that within the 

parameters discussed herein and in the 

White Paper, the Exchange will 

continue to re-train Dynamic M–ELO 

and M–ELO+CB on a weekly basis 

(outside of market hours) so that the 

model will continue to learn from and 

act upon the basis of more recent SIP 

and M–ELO book data sets, and further 

improve its performance over time. The 

retraining process should not result in 

dramatic or unpredictable changes to 

the behavior of Dynamic M–ELO. The 

retraining process will not retrain the 

model from scratch each week; rather, it 

will retain the model’s existing data 

inputs, knowledge base, and 

objectives—all without alteration. 

Retraining will result in new behaviors 

only as needed to address new scenarios 

that the model did not confront 

previously, and even then, only in a 

manner designed to further optimize 

outcomes, i.e., reduce mark-outs or 

increase fill rates. If the System assesses 

that a retrained model would be worse 

than the existing model in achieving its 

objectives, then the System will 

continue to use the existing model and 

discard the retrained model. This 

retraining process is a standard and 

accepted practice for use of deep 

learning models; it helps to ensure that 

deep learning models not only work 

well, but that they continue to work 

well in dynamic circumstances.29

The Exchange will not modify the 

underlying structure of Dynamic M– 

ELO and M–ELO+CB without first 

obtaining the Commission’s approval to 

do so, including modifications to the 

data elements the model considers in 

making decisions about Holding Period 

durations, the conditions under which 

the model may adjust the duration of 

Holding Periods, the frequency with 

which the model my adjust the Holding 

Periods, the range of Holding Period 

durations available to M–ELOs and M– 

ELO+CBs, the increments by which 

Holding Periods may change at any 

given Change Event, and the procedures 

for triggering, maintaining, and ending 

12 millisecond Holding Periods during 

times of extraordinary instability.30

Although the Exchange will seek 

Commission approval prior to changing 

any of the data elements that the model 

considers, the Exchange will not seek 

Commission approval prior to retraining 


ddrumheller on DSK120RN23PROD with NOTICES1
Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices 62855 

31See http://www.nasdaqtrader.com/Trader.

aspx?id=MELOSymbolData. 

32See, e.g., https://www.nasdaq.com/docs/MELO-Monthly-Report. Nasdaq understands that 

current users of M–ELO and M–ELO independently 

monitor the performance of these Order Types. 

Nasdaq often receives feedback from such users 

about M–ELO and M–ELO+CB performance, which 

Nasdaq then factors into decisions about 

improvements and enhancements. Nasdaq expects 

that this feedback loop will continue after 

implementation of Dynamic M–ELO. 

33 17 CFR 242.1000 et seq. As set forth in Reg. 

SCI, the term ‘‘SCI Systems’’ means ‘‘means all 

computer, network, electronic, technical, 

automated, or similar systems of, or operated by or 

on behalf of, an SCI entity that, with respect to 

securities, directly support trading, clearance and 

settlement, order routing, market data, market 

regulation, or market surveillance.’’ Id. at 242.1000. 

An ‘‘SCI Entity’’ means ‘‘an SCI self-regulatory 

organization, SCI alternative trading system, plan 

processor, exempt clearing agency subject to ARP, 

or SCI competing consolidator.’’ Id. 

34 15 U.S.C. 78f(b). 

35 15 U.S.C. 78f(b)(5). 

36M–ELO Approval Order, supra 83 FR at 10938– 

39; M–ELO+CB Approval Order, supra, 84 FR at 

48980. 

37See note 6, supra. 

38To be clear, performance statistics for Dynamic 

M–ELO cited herein and in the White Paper are 

based upon data derived from weekly, not daily 

retrainings. 

39As discussed above, Nasdaq will not seek 

Commission approval prior to allowing the model, 

as part of its re-training process, to vary the 

weighting of the data elements it ingests. Nasdaq 

believes this is appropriate because such variance 

will only occur to the extent that it will improve 

the model’s performance with respect to predefined objectives. Nasdaq will alert traders if the 

retraining process would result in substantial 

performance changes, and it will also publish 

statistics to help participants to assess performance 

themselves. Moreover, Nasdaq will retain historical 

iterations of its models for the Commission’s 

review, should it wish to examine how these 

models have changed over time. 

the model to adjust the weighting it 

applies to those data elements. 

To aid investors in understanding and 

evaluating Dynamic M–ELO, Nasdaq 

will continue to publish weekly and 

monthly transparency statistics on 

Nasdaqtrader.com, as it does now, about 

the performance of its M–ELOs and M– 

ELO+CBs, including statistics listing the 

weekly numbers of shares and trades in 

M–ELOs by symbol, weekly aggregated 

M–ELO share and trade data, and 

monthly aggregated block data.31

Nasdaq also will continue to disclose 

monthly data on Nasdaq.com, as it does 

now (the M–ELO Monthly Report), 

about M–ELO and M–ELO+CB markouts (quote stability by time horizon) 

and fill rates.32 Moreover, Nasdaq will 

add statistics to the M–ELO Monthly 

Report about how frequently, on 

average, the System changes Holding 

Period durations for the top decile, 

median, and bottom decile of symbols, 

as measured by monthly M–ELO and 

M–ELO+CB trading volumes. Nasdaq 

will retain copies of each historical 

iteration of its models as part of its 

books and records, and make them 

available to the Commission upon 

request, should it wish to examine them 

to understand how the model changes 

over time. Furthermore, Nasdaq will 

publish an equity trader alert in advance 

of deploying a retrained version of 

Dynamic M–ELO whenever Nasdaq has 

reason to anticipate that the retrained 

version will produce results that differ 

materially from the prior version, i.e., a 

projected change in mark-outs or fillrates of 10% or more in either direction. 

The Exchange acknowledges that 

systems necessary to implement 

Dynamic M–ELO, including the systems 

proposed that include model 

development and retraining processes, 

are ‘‘SCI Systems’’ within the meaning 

of Regulation Systems Compliance and 

Integrity (‘‘Reg. SCI’’),33 and that the 

Exchange, as an SCI Entity, remains 

responsible for compliance with all 

requirements of Reg. SCI, including, 

without limitation, to have policies and 

procedures reasonably designed to 

ensure that its SCI Systems operate in a 

manner that complies with the Act and 

the rules and regulations thereunder 

and Exchange’s rules and governing 

documents, among them a plan for 

assessments of the functionality of SCI 

Systems designed to detect systems 

compliance issues, including by 

responsible SCI personnel and by 

personnel familiar with applicable 

provisions of the Act and the rules and 

regulations thereunder and Exchange’s 

rules and governing documents. 

Implementation 

The Exchange intends to make the 

proposed change effective for M–ELOs 

and M–ELO+CBs in the Second or Third 

Quarter of 2023, but that time frame is 

subject to change. The Exchange will 

publish a Trader Alert in advance of 

making the proposed change effective. 

2. Statutory Basis 

The Exchange believes that its 

proposal is consistent with Section 6(b) 

of the Act,34 in general, and furthers the 

objectives of Section 6(b)(5) of the Act,35

in particular, in that it is designed to 

promote just and equitable principles of 

trade, to remove impediments to and 

perfect the mechanism of a free and 

open market and a national market 

system, and, in general to protect 

investors and the public interest, by 

allowing for more widespread use of M– 

ELOs and M–ELO+CBs. 

When the Commission approved the 

M–ELO and the M–ELO+CB, it 

determined that these Order Types are 

consistent with the Act because they 

‘‘could create additional and more 

efficient trading opportunities on the 

Exchange for investors with longer 

investment time horizons, including 

institutional investors, and could 

provide these investors with an ability 

to limit the information leakage and the 

market impact that could result from 

their orders.’’ 36 Nothing about the 

Exchange’s proposal should cause the 

Commission to revisit or rethink this 

determination. Indeed, the proposal will 

not alter the fundamental design of 

these Order Types, the manner in which 

they operate, or their effects. 

Even with Dynamic M–ELO Holding 

Periods, M–ELOs and M–ELO+CBs will 

continue to provide their users with 

protection against information leakage 

and adverse selection—and they will do 

so at levels which are substantially 

undiminished from that which they 

provide now.37

At the same time, however, the 

proposal will benefit market 

participants and investors by reducing 

the opportunity costs of utilizing M– 

ELOs and M–ELO+CBs. The proposal, 

in other words, will re-calibrate the 

lengths of the Holding Periods so that 

M–ELOs and M–ELO+CBs will operate 

in the ‘‘Goldilocks’’ zone—their Holding 

Periods will not be so short as to render 

them unable to provide meaningful 

protections against information leakage 

and adverse selection, but the Holding 

Periods also will not be too long so as 

to cause participants and investors to 

miss out on favorable execution 

opportunities. Nasdaq believes the 

proposal will render M–ELOs and M– 

ELO+CBs more useful and attractive to 

market participants and investors, and 

this increased utility and attractiveness, 

in turn, will spur an increase in M–ELO 

and M–ELO+CB use cases on the 

Exchange, both from new and existing 

users of M–ELOs and M–ELO+CBs. 

Ultimately, the proposal should 

enhance market quality by increasing 

opportunities for midpoint executions 

on the Exchange. 

As Nasdaq explained above, the 

Proposal will operate within strict, welldefined, and transparent parameters. 

Although it will undergo weekly 

retraining (outside of market hours),38

such retraining will aim to improve the 

performance of the model in achieving 

its twin objectives; retraining will not 

alter the inputs, objectives, or basic 

design parameters of Dynamic M–ELO 

without prior Commission approval.39

Moreover, the Exchange will not deploy 

a retrained model if it fails to achieve 

performance improvements. To aid 


ddrumheller on DSK120RN23PROD with NOTICES1
62856 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices 

40Beyond this grouping of participants, the 

activity levels of other individual M–ELO 

participants were so small as to be insignificant. In 

many cases, these participants entered only a 

handful of M–ELOs during the study period. As 

such, the Exchange believes it is reasonable to 

exclude such participants from its analysis to avoid 

their data distorting the results. 

41The Exchange will review its AI model 

periodically to affirm that it continues to perform 

in accordance with the Exchange’s rules and has 

not introduced any harmful bias in favor of or 

against any participant or category of participants. 

42See 17 CFR 240.3b–16(a)(2) (‘‘(a) An 

organization, association, or group of persons shall 

be considered to constitute, maintain, or provide ‘a 

market place or facilities for bringing together 

purchasers and sellers of securities or for otherwise 

performing with respect to securities the functions 

commonly performed by a stock exchange,’ as those 

terms are used in section 3(a)(1) of the Act, (15 

U.S.C. 78c(a)(1)), if such organization, association, 

or group of persons: (1) Brings together the orders 

for securities of multiple buyers and sellers; and (2) 

Uses established, non-discretionary methods 

(whether by providing a trading facility or by 

setting rules) under which such orders interact with 

each other, and the buyers and sellers entering such 

orders agree to the terms of a trade.’’). 

43See Securities Exchange Act Release No. 40760 

(December 8, 1998), 63 FR 70844, 70850 (December 

22, 1998). 

44See id. at 70900 (‘‘an essential indication of the 

non-discretionary status of rules and procedures is 

that those rules and procedures are communicated 

to the systems users’’ and ‘‘[t]hus, participants have 

an expectation regarding the manner of execution— 

that is, if an order is entered, it will be executed 

in accordance with those procedures and not at the 

discretion of a counterparty or intermediary.’’). 

45Cf. id. at 70851 (explaining that a traditional 

block trading desk is an example of a system that 

does not use established, non-discretionary 

methods because the operators of such desks do not 

act according to fixed procedures known to their 

customers, but instead shop orders around for 

potential counterparties and make their own 

determinations as to whether and how to execute 

block orders, including by sometimes deciding to 

take a proprietary position in part of the block 

order). 

46See id. at 80755 (describing an example of a 

system that would be non-discretionary in nature: 

‘‘System I permits participants to enter a range of 

ranked contingent buy and sell orders at which they 

are willing to trade securities. These orders are 

matched based on a mathematical algorithm whose 

priorities are designed to achieve the participants’ 

objectives. System I does not display orders to any 

participants. System I is included under Rule 3b– 

16.’’); see also Securities Exchange Act Release No. 

34–89686 (August 20, 2020), 85 FR 54438, at 54445, 

n.92 (September 1, 2020) (Order approving SR–IEX– 

2019–15) (rejecting argument that IEX’s D-Limit 

order time is an exercise of discretion because ‘‘DLimit orders will not allow IEX to exercise any 

discretion on any particular order by deviating from 

the CQI and D-Limit functionality, which is 

hardcoded in the IEX rulebook.’’; Securities 

Exchange Act Release No. 34–78101 (June 17, 

2016), 81 FR 41141, at 41153(June 17, 2016) (Order 

approving IEX Form 1 and D-Peg Order Type) (‘‘the 

Commission does not believe that the hardcoded 

conditionality of the IEX proposed ‘‘discretionary’’ 

peg order type provides IEX with actual discretion 

or the ability to exercise individualized judgment 

when executing an order. Rather, if IEX’s fixed 

formula determines the quote to be stable, the 

discretionary peg order can execute up to the 

midpoint; if it does not deem the quote to be stable, 

then it will hold the order to its pegged price. As 

such, IEX would not exercise discretion over the 

routing and execution of a resting order’’). Nasdaq 

does not believe that it is necessary to codify its 

mathematical formula for Dynamic M–ELO in its 

Rules because Nasdaq has disclosed sufficient 

information in its Rules and in its filing to inform 

the public as to the possible and expected behaviors 

associated with Dynamic M–ELO, as well as a 

means for the Commission and/or investors to 

verify whether Dynamic M–ELO is performing 

appropriately. Much as the Commission does not 

investors in evaluating Dynamic M– 

ELO, the Exchange will publish 

statistics about its performance, 

including as to mark-outs and fill rates, 

as well as statistics about how 

frequently the System changes Holding 

Period durations. To further facilitate 

accountability, the Exchange will retain 

each historical iteration of its model as 

part of its books and records, and make 

such information available to the 

Commission, upon request. The 

Exchange will also publish equity trader 

alerts whenever retraining will result in 

a performance change of 10% or more. 

Nasdaq notes that the twin objectives 

it prescribes for the model involve the 

absolute values of mark-outs and fill 

rates; they are not designed to further 

the performance of any participant or 

any category of participant. 

Furthermore, Nasdaq performed internal 

tests of its AI model to detect 

indications of harmful bias in its 

performance results, and such tests 

concluded that no such indications 

exist. That is, the Exchange reviewed 

the impact on fill rates and mark-outs of 

Dynamic M–ELO, as compared to the 

‘‘static’’ M–ELO, for those firms that 

accounted for more than 95% of M–ELO 

activity on the Exchange during Q1 

2022.40 The Exchange analyzed results 

both in an absolute and a relative sense. 

Testing revealed that all participants 

experienced at least some improvements 

in fill rates and mark-outs when using 

Dynamic M–ELO versus static M–ELO, 

with the volume-weighted average 

improvement being aligned with the 

results expressed in the White Paper. 

We detected no material variations that 

might suggest that a particular 

participant or category of participant 

(i.e., nature of firm; size of firm) 

benefitted from Dynamic M–ELO 

functionality to an extent that was 

unreasonably disproportionate to the 

benefits that other participants 

experienced. Thus, Nasdaq believes the 

model is objective, is designed to, and 

does avoid bias and discrimination.41

The Exchange notes that use of 

Dynamic M–ELOs and M–ELO+CBs 

remains voluntary for all market 

participants. Accordingly, if any market 

participant feels that the dynamic 

Holding Periods are still too long or too 

short or because competing venues offer 

more attractive delay mechanisms, then 

the participants are free to pursue other 

trading strategies or utilize other trading 

venues. They need not utilize Dynamic 

M–ELOs or M–ELO+CBs. 

Furthermore, the design of DynamicMELO would constitute an ‘‘established, 

non-discretionary’’ method that is 

consistent with the definition of an 

exchange, as set forth in SEC Rule 3b– 

16.42 The Commission stated as follows 

when it adopted Rule 3b–16: 

A system uses established nondiscretionary methods either by providing a 

trading facility or by setting rules governing 

trading among subscribers. The Commission 

intends for ‘‘established, non-discretionary 

methods’’ to include any methods that 

dictate the terms of trading among the 

multiple buyers and sellers entering orders 

into the system. Such methods include those 

that set procedures or priorities under which 

open terms of a trade may be determined. For 

example, traditional exchanges’ rules of 

priority, parity, and precedence are 

‘‘established non-discretionary methods,’’ as 

are the trading algorithms of electronic 

systems. Similarly, systems that determine 

the trading price at some designated future 

date on the basis of pre-established criteria 

(such as the weighted average trading price 

for the security on the specified date in a 

specified market or markets) are using 

established, non-discretionary methods.43

Nothing in the Reg. ATS Adopting 

Release or in any of its illustrative 

examples suggests that Dynamic M–ELO 

would constitute an exercise of 

discretionary behavior. Dynamic M– 

ELO will handle and execute Orders 

according to published, pre-determined 

rules that are disclosed to the public 

and which provide reasonable notice of 

how the Order Type will behave.44 To 

the extent that the design of the System 

permits variation in the Holding Periods 

for such Orders, it does so by design. 

The range of potential variations, the 

objectives that such variations are 

intended to achieve, and the factors that 

determine when such variations may 

occur are also predetermined and set 

forth in the Exchange’s Rules or 

otherwise disclosed to the public. The 

mere fact that the System may apply 

different weights over time to the factors 

it uses to determine whether and by 

how much to vary a Holding Period 

does not mean that the System will act 

with discretion in the same sense that 

a human being could be said to be 

exercise independent judgment when 

deciding whether and how to handle an 

order.45 Even when the System makes 

decisions about changing the Holding 

Periods, the System will operate 

pursuant to a mathematical algorithm 

from which it cannot deviate—an 

algorithm that is programmed to achieve 

pre-defined and pre-disclosed 

objectives.46


ddrumheller on DSK120RN23PROD with NOTICES1
Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices 62857 

require an exchange to codify the source code it 

uses to effectuate other behaviors or actions that it 

explains in its Rules, including the behaviors of 

other complex Order Types, there is no basis to 

require codification of the Dynamic M–ELO formula 

in this instance. 

47See White Paper, supra. 48 In approving this proposed rule change, the 

Commission has considered the proposed rule’s 

impact on efficiency, competition, and capital 

formation. See 15 U.S.C. 78c(f). One commenter 

questioned whether Nasdaq’s pending patent 

applications for the systems it will use to operate 

Dynamic M–ELO imposes an unnecessary or 

inappropriate burden on competition. See Letter 

from R.T. Leuchtkafer, dated January 21, 2023 

(‘‘Leuchtkafer Letter 1’’), at 3. The Commission does 

not believe that the sole fact that Nasdaq has a 

pending patent application for the technology it has 

developed to operate the Dynamic M–ELO is 

indicative that the operation of Dynamic M–ELO on 

the Exchange would place an inappropriate burden 

on competition. As explained below, Nasdaq has 

provided sufficient public disclosure and analysis 

to explain how Dynamic M–ELO will operate. 

49 In addition to providing a statutory analysis in 

its filing, Nasdaq also acknowledges, above in 

Amendment No. 2, that the systems it will use to 

implement Dynamic M–ELO, including the 

Exchange’s model development and retraining 

processes, are SCI systems under Regulation SCI, 

see 17 CFR 242.1000 et seq., and thus, it will be 

responsible for compliance with Regulation SCI 

with respect to Dynamic M–ELO, including having 

appropriate policies and procedures. See supra note 

33 and accompanying text. 

50 15 U.S.C. 78f(b)(5). 

51 15 U.S.C. 78f(b)(8). 

52See 17 CFR 201.700(b)(3). 

53 15 U.S.C 78s(b)(1). 

The Exchange notes that it will 

continue to conduct real-time 

surveillance to monitor the use of M– 

ELOs and M–ELO+CBs to ensure that 

such usage remains appropriately tied to 

the intent of the Order Types. If, as a 

result of such surveillance, the 

Exchange determines that the Dynamic 

M–ELO Holding Periods do not serve 

their intended purposes, or adversely 

impact market quality, then the 

Exchange will seek to make further recalibrations. 

Nasdaq does not believe that the 

design of Dynamic M–ELO lends itself 

to potential manipulation by a single 

participant or a small group of 

participants because the System makes 

determinations regarding Holding 

Periods based upon prevailing marketwide conditions for a given symbol, 

rather than the behaviors of particular 

participants with respect to that symbol, 

or the activity of participants in M– 

ELOs involving that symbol. 

Manipulation of the System also would 

be difficult to accomplish given the 

large number of variables that factor into 

the System’s decisions to change 

Holding Periods during Change Events, 

as well as the different weights that 

apply to each such factor, which as 

described above, the System may vary 

over time. Any benefits that a 

participant might derive from 

manipulating the duration of Holding 

Periods would likely be small and 

outweighed significantly by the 

difficulty and cost of affecting such 

manipulation. Nevertheless, the 

Exchange will surveil for indications of 

manipulation and act accordingly if it 

detects such indications. 

B. Self-Regulatory Organization’s 

Statement on Burden on Competition 

The Exchange does not believe that 

the proposed rule change will impose 

any burden on competition not 

necessary or appropriate in furtherance 

of the purposes of the Act. To the 

contrary, the Exchange believes that this 

proposal will promote the 

competitiveness of the Exchange by 

rendering its M–ELO and M–ELO+CB 

Order Types more attractive to 

participants. 

The Exchange adopted the M–ELO 

and M–ELO+CB as pro-competitive 

measures intended to increase 

participation on the Exchange by 

allowing certain market participants 

that may currently be underserved on 

regulated exchanges to compete based 

on elements other than speed. The 

proposed change continues to achieve 

this purpose. With Dynamic M–ELO 

Holding Periods, both M–ELOs and M– 

ELO+CBs will afford their users with a 

level of protection from information 

leakage and adverse selection that is 

better from what is achievable at 

present.47 At the same time, the 

Dynamic Holding Periods will increase 

opportunities to interact with other likeminded investors with longer time 

horizons while also lowering the 

opportunity costs for participants that 

utilize M–ELOs and M–ELO+CBs, 

particularly for securities that trade 

within the ‘‘Goldilocks’’ zone. In sum, 

the proposed changes will not burden 

competition, but instead may promote 

competition for liquidity in M–ELOs 

and M–ELO+CBs by broadening the 

circumstances in which market 

participants may find such Orders to be 

useful. With the proposed changes, 

market participants will be more likely 

to determine that the benefits of 

entering M–ELOs and M–ELO+CBs 

outweigh the risks of doing so. 

The proposed change will not place a 

burden on competition among market 

venues, as any market may adopt an 

order type that operates similarly to a 

M–ELO or a M–ELO+CB with Dynamic 

M–ELO Holding Periods. 

C. Self-Regulatory Organization’s 

Statement on Comments on the 

Proposed Rule Change Received From 

Members, Participants, or Others 

No written comments were either 

solicited or received. 

IV. Discussion and Commission 

Findings 

After careful review, the Commission 

finds that the Exchange’s proposal to 

adopt Dynamic M–ELO is consistent 

with the requirements of the Act and the 

rules and regulations thereunder 

applicable to a national securities 

exchange.48 In particular, the 

Commission finds that the proposed 

rule change, as modified by Amendment 

No. 2, is consistent with Sections 6(b)(5) 

and 6(b)(8) of the Act.49 Section 6(b)(5) 

of the Act requires that the rules of a 

national securities exchange be 

designed, among other things, to 

prevent fraudulent and manipulative 

acts and practices, to promote just and 

equitable principles of trade, and, in 

general, to protect investors and the 

public interest, and not be designed to 

permit unfair discrimination between 

customers, issuers, brokers, or dealers.50

Section 6(b)(8) of the Act requires that 

the rules of a national securities 

exchange not impose any burden on 

competition that is not necessary or 

appropriate in furtherance of the 

purposes of the Act.51

Nasdaq’s Obligation To Sufficiently 

Explain Its Proposed Rule Change 

The burden to demonstrate that a 

proposed rule change is consistent with 

the Act and rules and regulations 

thereunder is on the self-regulatory 

organization (‘‘SRO’’) proposing a rule 

change.52 Each proposed SRO rule 

change must be ‘‘accompanied by a 

concise general statement of the basis 

and purpose of such proposed rule 

change.’’ 53 As described in more detail 

below, several commenters argued that 

the proposal did not provide sufficient 

information with respect to the 

operation of Dynamic M–ELO, or that 

the information provided was not ‘‘clear 

and comprehensible,’’ as required by 

Form 19b–4. For the reasons articulated 

below, the Commission believes that 

Nasdaq has provided clear and 

comprehensible information on the 

overall operation of Dynamic M–ELO 

and the role of the machine-learning 

model and demonstrated that the 

proposal is consistent with the Act. 

Several related comments addressed 

this issue; these comments and Nasdaq’s 

responses are discussed below, followed 

by the Commission’s analysis. 

One commenter stated that the initial 

filing would establish ‘‘a dangerously 

vague standard for describing how 

exchange-hosted complex algorithmic 


ddrumheller on DSK120RN23PROD with NOTICES1
62858 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices 

54See Leuchtkafer Letter 1, supra note 48, at 1– 

2. See also Letter from Joseph Saluzzi, Partner, 

Themis LLC, dated January 25, 2023, at 3 (‘‘Themis 

Letter’’) (questioning whether the complexity of 

Dynamic M–ELO is necessary). 

55See Themis Letter, supra note 54, at 2. 

56See Leuchtkafer Letter 1, supra note 48, at 1. 

57See Letter from Brett Kitt, Associate Vice 

President and Principal Associate General Counsel, 

Nasdaq, Inc., dated March 9, 2023, at Appendix A 

(‘‘Nasdaq First Response to Comments’’). 

58See Letter from R.T. Leuchtkafer, dated May 2, 

2023, at 8–9 (‘‘Leuchtkafer Letter 2’’). 

59See id. 

60See Letter from R.T. Leuchtkafer dated May 30, 

2023, at 3–5; 8–9 (‘‘Leuchtkafer Letter 3’’). This 

commenter also cites to the rules governing the 

Crumbling Quote Indicator and D-Limit order type 

on the Investors Exchange (‘‘IEX’’), as well as 

language from the Commission’s approval order for 

the D-Limit order type. See id. at 4. The commenter 

notes the level of detail with regard to how and 

when the D-Limit order type exercises its 

discretionary price-sliding that is set forth in the 

IEX Rulebook. See id. at 4; see also Themis Letter, 

supra note 54, at 2 (‘‘Another exchange, IEX, 

operates a smart logic called CQI (Crumbling Quote 

Indicator) which aims to protect orders from being 

adversely selected. IEX has published detailed 

notes on how the CQI is calculated.’’). Each 

proposal must be evaluated based on the specific 

facts and circumstances before the Commission. In 

this case, the Commission is only reviewing the 

proposed operation of Dynamic M–ELO and its 

machine-learning model. Accordingly, the level of 

detail provided in the IEX Rulebook for the D-Limit 

order type and Crumbling Quote Indicator—or the 

rulebooks for order types on other exchanges—does 

not determine whether Nasdaq has met its burden 

in this proposal. 

61See Letter from Brett Kitt, Associate Vice 

President and Principal Associate General Counsel, 

Nasdaq, Inc., dated May 18, 2023 at Appendix A 

(‘‘Nasdaq Second Response to Comments’’). 

62See supra note 25. 

63See White Paper Section 3.1. 

64See White Paper Section 4.1. 

65See White Paper, Section 7.2. 

66See Leuchtkafer Letter 2, supra note 58, at 3– 

4; Leuchtkafer Letter 3, supra note 60, at 4–5; Letter 

from R.T. Leuchtkafer, dated August 11, 2023, at 8– 

10 (‘‘Leuchtkafer Letter 4’’). 

67See Nasdaq Second Response to Comments, 

supra note 61, at 3–5. 

68See infra notes 91–92 and accompanying text. 

order types operate.’’ 54 In response to 

public comment, Nasdaq added more 

details describing the operation of 

Dynamic M–ELO to both the filing and 

public record since this proposed rule 

change was initially submitted to the 

Commission, and Nasdaq also provided 

additional legal analysis to support 

Dynamic M–ELO’s consistency with the 

Act. Prior to the filing of Amendment 

No. 1, a commenter stated that although 

‘‘Nasdaq shared some of the 142 features 

of their formula,’’ Nasdaq should reveal 

all of these features so that prospective 

users may evaluate how the model 

works.55 Similarly, another commenter 

stated that the public cannot provide 

meaningful comment on the proposal 

without knowing all categories and 

parameters of the proposed Dynamic M– 

ELO.56 In its response to these 

comments, the Exchange, among other 

things, provided the specific 142 data 

elements that will be weighed by the 

machine-learning model as both an 

appendix to its first letter in response to 

comments,57 and as Exhibit 3B to its 

Amendment No. 1 filing. 

In response to the Exchange’s 

disclosures in Exhibit 3B of Amendment 

No. 1, one of these commenters stated 

that the list of data elements was not 

‘‘clear and comprehensible’’ as is 

required by the Form 19–4, but rather 

‘‘vague, confusing, and perfunctory.’’ 58

This commenter also stated that the 

disclosed data elements included 

unexplained terms (e.g., ‘‘baseline 

simulated,’’ ‘‘action simulated,’’ and 

‘‘synthetic mark-out’’).59 In a 

subsequent comment letter, this 

commenter reiterated these points; the 

commenter specified that the 

commenter’s concern is that Nasdaq’s 

rule text does not disclose information 

about its methods for assessing market 

conditions and that ‘‘Nasdaq should 

carefully detail its methods in its 

rulebook, just like other exchanges have 

done, and Nasdaq should also 

thoroughly disclose its methods in its 

filing text.’’ 60 In its second response to 

comments 61 and the revised Exhibit 3b 

to Amendment No. 2, Nasdaq expanded 

and ‘‘simplified’’ the explanation of 

these 142 data elements. The Exchange 

also added to the proposed rule text a 

definition of the term ‘‘proprietary 

assessment of market conditions’’ to 

explain how the machine-learning 

model will evaluate those 142 data 

elements.62

Furthermore, Nasdaq attached, as 

Exhibit 3A to its proposed rule change, 

the White Paper written by its AI Core 

Development Team that explains, 

among other things, how Dynamic M– 

ELO’s machine-learning functions were 

developed and tested. The White Paper 

includes a general discussion of the type 

of model implemented in the proposed 

system, in this case a reinforcement 

learning model,63 as well as citations to 

academic research behind the Double 

Deep Q-Network algorithm that is the 

basis for the algorithm used in Nasdaq’s 

model.64 The White Paper also describes 

the ways in which Nasdaq’s 

implementation of the proposed model 

differs from the model and training in 

the academic research, providing both 

an English summary and a pseudocode 

description of differences in model 

training implemented by Nasdaq.65

One of the commenters stated that the 

White Paper is not easily understood by 

most market participants and that 

referencing the White Paper in the filing 

is an ‘‘unacceptable substitute’’ for a 

‘‘plain English’’ explanation of the 

proposal in Form 19b–4.66 In response, 

Nasdaq explained that it drafted the 

filing to provide a general 

understanding of Dynamic M–ELO and 

how it will behave, and the more 

detailed information and explanation in 

the White Paper are meant to support 

the filing.67

The Commission agrees with 

comments and the Exchange that there 

is an extent to which the proposed 

changes will introduce an unavoidable 

degree of uncertainty with respect to the 

use of these order types. The deep 

reinforcement learning model that will 

determine the dynamic holding periods 

for each symbol for M–ELO and M– 

ELO+CB orders will be implemented 

through established, non-discretionary 

methods,68 but it is so complex that its 

complete details are, for most intents 

and purposes, not readily intelligible, 

and it would be immensely difficult for 

the Exchange or any market participant 

to precisely predict the holding periods 

that will be generated by the model for 

any given symbol at any particular time. 

Nevertheless, as further discussed 

below, the Commission believes that the 

Exchange has provided information 

sufficient for the Commission and 

public to understand the design, 

operation, and limits of the proposed 

changes to these order types, and the 

role of the machine-learning model 

therein. 

While the holding periods under the 

proposal would be dynamic, Nasdaq has 

precisely articulated both the nature of 

changes that would be permissible 

under the proposal, and the limits to 

those changes. Nasdaq described when 

changes might occur (every thirty 

seconds throughout the trading day), the 

initial default holding period for all 

symbols (1.25 milliseconds), the 

permissible increments by which a 

holding period might change in each 

symbol (0.25 or 0.50 milliseconds), and 

the outer bounds of permissible holding 

period lengths (0.25 milliseconds at the 

short end, and 2.50 milliseconds at the 

long end). Nasdaq also described the 

conditions of ‘‘extraordinary instability’’ 

in a symbol when these holding periods 

would not apply, and when the holding 

periods would be overridden by the 

proposed ‘‘stability protection 

mechanism’’ (with a holding period of 

12 milliseconds for at least 750 

milliseconds). The Commission believes 

that these details provide sufficient 

information to understand the range of 

potential holding periods that may be 

applied when M–ELO or M–ELO+CB 

orders are entered or resting on the 

order book, the changes that may occur, 

and the limits to those changes. 


ddrumheller on DSK120RN23PROD with NOTICES1
Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices 62859 

69See supra notes 16–24 and accompanying text. 

70See, e.g., Form 19b–4 at 9, White Paper Section 

5, and Nasdaq First Response to Comments at 2. In 

its White Paper, Nasdaq provides mathematical 

definitions of fill rate for a period of time and markout by trade (White Paper at 5, Equations 1 and 2), 

as well as of the assessment made by the agent in 

the model’s reinforcement learning process (White 

Paper at 11, Equation 3). 

71See White Paper Sections 3–5. 

72See Exhibit 3B. As described below, these data 

elements are also those used in training and 

retraining the model. 

73See Letter from Brett Kitt, Associate Vice 

President and Principal Associate General Counsel, 

Nasdaq, Inc., dated September 6, 2023 (‘‘Nasdaq 

Third Response to Comments’’). 

74Nasdaq affirmed that, while this information 

was not included in all versions of the list of data 

elements, it remains accurate and valid. See id. at 

3. 

75See White Paper Section 5.3. 

76For example, Nasdaq affirmatively states that if 

‘‘a retrained model would be worse than the 

existing model in achieving its objectives, then the 

System will continue to use the existing model and 

discard the retrained model.’’ See Section III.A.1., 

supra. 

77See Nasdaq First Response to Comments, supra 

note 57, at 2–3. See also supra note 30. 

78See Nasdaq First Response to Comments, supra 

note 57, at 2–3. A commenter also noted that it was 

initially unclear when and how frequently the 

machine-learning model would retrain, stating that 

the White Paper set forth an analysis based on daily 

retraining, but the rule filing proposes weekly 

retraining. See Leuchtkafer Letter 2, supra note 58, 

at 4; Leuchtkafer Letter 3, supra note 60, at 2; 

Leuchtkafer Letter 4, supra note 66, at 9. In 

Amendment No. 2, the Exchange affirmatively 

represents that ‘‘the performance statistics for 

Dynamic M–ELO cited herein and in the White 

Paper are based upon data derived from weekly, not 

daily retrainings.’’ See supra note 38. 

79See supra note 46 and accompanying text. 

80See, e.g., Amendment No. 2 at 19–20. 

81See Leuchtkafer Letter 1, supra note 48, at 2; 

Leuchtkafer Letter 2, supra note 58, at 4–6; 

Leuchtkafer Letter 3, supra note 60, at 5–7; 

Leuchtkafer Letter 4, supra note 66, at 4–8. 

82See Leuchtkafer Letter 1, supra note 48, at 2. 

See also Leuchtkafer Letter 3, supra note 60, at 5– 

7. For example, the commenter believes that 

Dynamic M–ELO will ‘‘exercise individualized 

judgment’’ such that it can set a different time-inforce for the very same order presented in the very 

same market conditions on, for example, August 21 

than it set on May 15, depending on the system’s 

undisclosed individualized judgments of market 

conditions and participant behavior from even days 

or weeks in the past. See Leuchtkafer Letter 4, supra 

note 66, at 6. 

83See Leuchtkafer Letter 2, supra note 58, at 5– 

6. See also Leuchtkafer Letter 3, supra note 60, at 

5–7. See also Leuchtkafer Letter 4, supra note 66, 

at 6–9 (’’ . . . . Nasdaq’s rulebook won’t set out the 

‘totality of the discretionary feature’ (I believe it 

can’t, because the totality changes week-to-week 

and even minute-to-minute) and it won’t define the 

‘hardcoded conditionality’ of its feature (again, I 

believe it can’t), and a market participant won’t be 

able to ‘recreate on its own’ what Dynamic M–ELO 

has done (participants can’t—it’s not even clear 

anyone will be able to, as discussed below) . . . . 

Dynamic M–ELO departs from decades of this 

progress. Its behavior will not be deterministic or 

invariant over time, and purposefully so.’’). 

84See Leuchtkafer Letter 2, supra note 58, at 8. 

See also Leuchtkafer Letter 3, supra note 60, at 5– 

7. 

Nasdaq has also described the role of 

the machine-learning model in its 

proposal. The model will determine 

whether, by which increment, and in 

which direction to adjust the holding 

period for each symbol throughout the 

trading day.69 In its Form 19b–4, White 

Paper, and response letters, Nasdaq 

described the goals towards which the 

model is optimized: reducing mark-outs 

and increasing fill rates.70 Nasdaq’s 

White Paper includes a detailed 

discussion of model choice, 

development, and training, including 

citations to relevant other research.71

Nasdaq also provided several iterations 

of a list of data elements that the model 

will ingest and use, including a glossary 

defining terms used in the 

descriptions.72 Nasdaq affirmed that in 

operation during market hours, the data 

used would be calculated based on 

intraday market data.73 One version of 

these lists included Nasdaq’s estimates 

of the tendencies of data elements to 

affect model outcomes.74 Nasdaq’s 

White Paper also included an 

‘‘explainability study’’ that assessed 

both the effects of individual data 

elements on model performance and the 

effects of interactions between 

individual data elements.75 Across its 

filing and incorporated exhibits, aspects 

of the model’s operations and design are 

described in different formats and with 

different levels of specificity—for 

example, the filing and exhibits include 

‘‘plain English’’ descriptions, 

mathematical definitions, and 

pseudocode. Together, this set of 

information allows the Commission to 

understand the type of decision the 

model will implement, the goals the 

model aims to achieve, which model 

type is implemented and how it was 

developed, the range of data types and 

data sources used by the model, and 

estimates of the manner in which those 

data may affect model outcomes. 

Nasdaq also explains how and when 

the machine-learning model will be 

retrained. Nasdaq will retrain the model 

weekly, outside of market hours. 

Retraining will incorporate market data 

obtained during the week from the 

equity consolidated data feeds and M– 

ELO order book. A retrained model will 

only be promoted to production if it 

improves upon the model objectives 

compared to the prevailing model.76

Furthermore, the Exchange explained 

that the machine-learning model is 

consistent in its behavior from day-today during periods when it is not 

undergoing retraining, ‘‘such that its 

decisions when presented with given set 

of facts and circumstances in a given 

security on day 1 should be the same as 

they would be on day 2.’’ 77 The 

Exchange also stated in its initial 

response to comments that ‘‘[e]ven after 

the system undergoes retraining, which 

will occur on a weekly basis (and not 

during market hours), system behavior 

should not change dramatically or in 

unexpected ways from week-toweek.’’ 78 As noted above as well, the 

Exchange also represents that outside of 

set retraining periods, ‘‘the System will 

operate pursuant to a mathematical 

algorithm from which it cannot 

deviate—an algorithm that is 

programmed to achieve pre-defined and 

pre-disclosed objectives.’’ 79 Nasdaq also 

will publish equity trader alerts when it 

anticipates that a model update may 

change mark-outs or fill rate by 10% or 

more in either direction.80 By including 

this set of information, Nasdaq has 

provided the Commission and public 

with information that allows them to 

understand how frequently the model 

will be retrained, the data used for 

retraining, and the criteria that will be 

used to determine whether to update the 

production model based on retraining. 

This information allows the 

Commission to understand when the 

proposed model may change and when 

it will remain constant, the 

circumstances under which a change 

would be implemented, and 

circumstances under which the public 

will receive notice of significant 

changes in the model’s anticipated 

outcomes. 

In addition, a commenter stated their 

belief that the Exchange’s proposal for 

Dynamic M–ELO would result in the 

exercise of discretion by a national 

securities exchange because the 

machine-learning model’s decisions 

would vary over time based on the 

following: (1) varying parameter values; 

and (2) results of retraining cycles.81

The commenter stated that by making it 

possible for Dynamic M–ELO to behave 

differently when confronted by the same 

market conditions before and after the 

model is retrained, Nasdaq’s model 

would be exercising discretion that is 

more akin to a broker than an 

exchange.82

The commenter claimed that Dynamic 

M–ELO would operate outside of 

established non-discretionary methods, 

which require ‘‘fully disclosed 

procedures operating in a strictly linear, 

invariant, and deterministic fashion.’’ 83

Additionally, the commenter stated that 

Nasdaq would be exercising discretion 

with Dynamic M–ELO to alter a 

participant’s material order terms.84 The 

commenter claimed that Nasdaq would 

be using undisclosed data such as the 


ddrumheller on DSK120RN23PROD with NOTICES1
62860 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices 

85See Leuchtkafer Letter 2, supra note 58, at 8. 

See also Leuchtkafer Letter 3, supra note 60, at 5– 

7; Leuchtkafer Letter 4, supra note 66, at 5–7. 

86See supra notes 42–46. 

87See supra note 45 and accompanying text. 

88See Nasdaq Second Response to Comments, 

supra note 61, at 5–9. See also Nasdaq First 

Response to Comments, supra note 57, at 5–7. 

89See supra note 46 and accompanying text. 

90See supra note 29. 

91See supra note 45 and accompanying text. 

92See supra note 46 and accompanying text. 

93See Leuchtkafer Letter 3, supra note 60, at 7– 

8; Leuchtkafer Letter 2, supra note 58, at 5; 

Leuchtkafer Letter 1, supra note 48, at 3. Nasdaq’s 

White Paper includes a ‘‘firm-level analysis’’ that 

‘‘tried to identify patterns and trends that could 

potentially signify a systematic bias towards 

specific firms.’’ White Paper at 24. This analysis 

concluded that ‘‘Dynamic M–ELO will not result in 

systematic-biased execution towards any one firm.’’ 

Id. at 26. 

94See Leuchtkafer Letter 3, supra note 60, at 7– 

8. 

95See Amendment No. 2 at fn. 34. 

buyer and seller counts and recent trade 

sizes.85 Further, the commenter stated: 

that (a) by determining the universe of data 

the system consumes, (b) by programming 

how the system thinks, (c) by controlling and 

supplying the information with which it 

thinks, and (d) by setting the goals and 

programming the nature and extent of its 

actions, and when it does all this to 

determine (e) when and in which prescribed 

intervals to set an ever variable time-in-force 

term for an order, a term which (f) dictates 

when to expose an order to the market to find 

contra-side interest, then without question 

Nasdaq is exercising control, judgment, and 

discretion over its customer orders. 

In Amendment No. 1, Nasdaq added 

language to address these concerns.86

Among other things, Nasdaq stated that 

to the extent that the design of Dynamic 

M–ELO permits variation in the Holding 

Periods for such orders, it does so by 

design, and the ‘‘mere fact that the 

System may apply different weights 

over time to the factors it uses to 

determine whether and by how much to 

vary a Holding Period does not mean 

that the System will act with discretion 

in the same sense that a human being 

could be said to be exercise 

independent judgment when deciding 

whether and how to handle an order.’’ 87

Additionally, Nasdaq stated the 

following in its second response to 

comments: 

It is also worth noting that presently, 

exchanges like Nasdaq already employ nonlinear, non-deterministic functionalities, like 

the randomized timers it uses to resolve 

certain unavoidable race conditions that arise 

in the order handling process. Nasdaq 

employs these functionalities with the 

knowledge of the SEC, and without any 

suggestion that they somehow transform 

Nasdaq into a broker.88

Furthermore, as noted above, the 

Exchange represents that outside of set 

retraining periods, ‘‘the System will 

operate pursuant to a mathematical 

algorithm from which it cannot 

deviate—an algorithm that is 

programmed to achieve pre-defined and 

pre-disclosed objectives.’’ 89 The 

Exchange explains that outside of the 

set retraining periods ‘‘the System will 

behave predictably from day to day, 

such that its decisions when presented 

with given set of facts and 

circumstances in a given security on day 

1 should be the same as they would be 

on day 2.’’ 90

Based on Nasdaq’s representations 

described above, Dynamic M–ELO 

would operate pursuant to predetermined, programmed procedures 

that would dictate order interaction and 

the terms for trading for each Dynamic 

M–ELO order entered on the Nasdaq 

trading facility. While the Exchange’s 

procedures include conditions that, if 

satisfied under certain circumstances, 

might result in different outcomes for 

different M–ELO orders, such 

conditions and circumstances, if predetermined, pre-defined, and 

programmed into the Exchange’s trading 

facility, would be considered 

established and not discretionary. For 

example, according to the Exchange, 

Dynamic M–ELO may apply different 

pre-determined weights over time to 

pre-determined factors it uses to 

determine whether and by how much to 

vary a Holding Period.91 In such an 

event, Dynamic M–ELO will operate 

pursuant to pre-determined procedures 

and programmed mathematical 

algorithm from which it cannot deviate 

to ‘‘achieve pre-defined and predisclosed objectives.’’ 92 Further, the 

procedures governing Dynamic M–ELO 

and use of M–ELO orders will be 

established before the beginning of each 

trading day. For example, Dynamic M– 

ELO will use preset methods to evaluate 

and weigh specific data elements to 

determine the dynamic holding periods. 

Such pre-set methods will be 

established during the prior retraining 

period, and outside regular trading 

hours, and will not vary intra-day until 

adjusted at the next retraining period. 

Given the pre-determined, 

programmed procedures and rules that 

Nasdaq has proposed to dictate trading 

for Dynamic M–ELO, the Commission 

does not believe that Dynamic M–ELO 

is designed to provide Nasdaq with 

judgement and flexibility, and therefore, 

discretion over the handling or 

execution of a M–ELO order entered on 

the Exchange. 

Unfair Discrimination 

Section 6(b)(5) of the Act requires that 

the rules of a national securities 

exchange not be designed to permit 

unfair discrimination between 

customers, issuers, brokers, or dealers. 

In several letters, one commenter stated 

that Nasdaq inadequately explains how 

it will monitor and, if necessary, adjust 

Dynamic M–ELO to ensure no unfair 

discrimination.93 Initially, this 

commenter emphasized what they 

perceived to be silence on the part of 

Nasdaq with regard to whether Dynamic 

M–ELO will discriminate among 

categories of participant types.94 In its 

response to these comments, Nasdaq 

initially added a new representation to 

the filing in Amendment No. 1, stating 

that that Dynamic M–ELO is not 

designed to further the performance of 

any participant or any category of 

participant, but instead has twin 

objectives—the absolute values of markouts and fill rates. In Amendment No. 

2, Nasdaq expanded on this 

representation by adding the following: 

Furthermore, Nasdaq performed internal 

tests of its AI model to detect indications of 

harmful bias in its performance results, and 

such tests concluded that no such indications 

exist. That is, the Exchange reviewed the 

impact on fill rates and mark-outs of 

Dynamic M–ELO, as compared to the ‘‘static’’ 

M–ELO, for those firms that accounted for 

more than 95% of M–ELO activity on the 

Exchange during Q1 2022 . . . . The 

Exchange analyzed results both in an 

absolute and a relative sense. Testing 

revealed that all participants experienced at 

least some improvements in fill rates and 

mark-outs when using Dynamic M–ELO 

versus static M–ELO, with the volumeweighted average improvement being aligned 

with the results expressed in the White 

Paper. We detected no material variations 

that might suggest that a particular 

participant or category of participant (i.e., 

nature of firm; size of firm) benefitted from 

Dynamic M–ELO functionality to an extent 

that was unreasonably disproportionate to 

the benefits that other participants 

experienced. Thus, Nasdaq believes the 

model is objective, is designed to, and does 

avoid bias and discrimination. 

In Amendment No. 2, Nasdaq also 

affirmed that it will periodically review 

its model to ensure that it continues to 

perform in accordance with the 

Exchange’s rules and that it has not 

introduced any harmful bias in favor of 

or against any participant or class of 

participants.95

In response to the above, the 

commenter submitted a fourth comment 

letter, in which they questioned the 

approach Nasdaq took to demonstrate 

that there is not bias against any one 


ddrumheller on DSK120RN23PROD with NOTICES1
Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices 62861 

96See Leuchtkafer Letter 4, supra note 66, at 1– 

4. 

97See id. at 2–3 (‘‘It seems Nasdaq trimmed its 

data before analyzing it for bias and constrained its 

analysis to ‘those firms that accounted for more 

than 95% of M–ELO activity on the Exchange 

during Q1 2022.’ (I assume Nasdaq used the same 

data defined in the Filing as the ‘Training Period’ 

for its analysis. Nasdaq doesn’t say so, however.) 

Nasdaq doesn’t describe the kind of M–ELO 

‘activity’ it filtered the data for, and specifically 

whether it filtered on order or trade counts or order 

or trade volume or some combination of two or 

more of these categories, or on some other factor, 

before removing firms from its analysis.’’) 

98See id. at 2–3. 

99See id. at 5–6. 

100See id. 

101See id. 102See supra note 41. 

103See, e.g., Exhibit 3b. 104According to the ‘‘M–ELO Monthly Report’’ 

published by Nasdaq for July 2023 (available at: 

https://www.nasdaq.com/docs/M-ELO-MonthlyReport (accessed September 2, 2023)), the average 

daily notional volume executed in M–ELO was 

$624,556,748. The average daily notional volume 

executed in July 2023 across the market for NMS 

stocks was about $523,769,246,196. See, e.g., Cboe, 

Historical Market Volume Data, available at: 

https://www.cboe.com/us/equities/market_

statistics/historical_market_volume/. The average 

daily notional volume in M–ELO for that month 

was approximately 0.12% (just over one-tenth of 

one percent) of the average daily notional volume 

across the entire NMS stock market. 

participant or class of participants.96

The commenter, among other things, 

expressed concern about Nasdaq 

conducting its analysis using data for 

firms that accounted for 95% of M–ELO 

activity during Q1 of 2022 rather than 

all M–ELO activity. The commenter 

states that Nasdaq did not describe how 

it determined the 5% of activity during 

that period to exclude from its 

analysis.97 For example, the commenter 

states that it is not clear whether Nasdaq 

excluded firms with large orders and 

trades, and the commenter opines that 

discarding any data could exclude 

activity that has qualitative or 

quantitative differences from the rest.98

In response to this comment, Nasdaq 

represented that it conducted a 

supplemental analysis of the initiallyexcluded data—which were the activity 

of the least-active M–ELO firms from the 

control period of its initial analysis—to 

confirm whether its initial conclusions 

held for those participants.99 Nasdaq 

explains that the individual variations 

among the previously excluded 

participants was higher than that for the 

original batch of data, but that, based on 

simulated data, each of these 

participants would have experienced 

the same or better fill rates during the 

testing period if they had utilized 

Dynamic M–ELO.100 Based on this 

supplemental data analysis, Nasdaq 

concluded that there is no apparent 

biases for the Dynamic M–ELO, even 

among the least active M–ELO 

participants.101

The Commission concludes that 

Nasdaq has adequately demonstrated 

that the proposal is not designed to 

permit unfair discrimination consistent 

with Section 6(b)(5) of the Act. Through 

the White Paper, amendments, and 

response letters, Nasdaq has 

demonstrated that it has analyzed the 

anticipated or simulated effects of the 

proposed change on all current M–ELO 

users, and that this work did not 

indicate that particular firms or classes 

of firms are anticipated to unfairly 

benefit from or be harmed by the 

proposed Dynamic M–ELO 

functionality. 

Prevention of Fraudulent and 

Manipulative Acts and Practices, Just 

and Equitable Principles of Trade, and 

the Protection of Investors and the 

Public Interest 

Section 6(b)(5) of the Act also requires 

that the rules of a national securities 

exchange be designed to prevent 

fraudulent and manipulative acts and 

practices, to promote just and equitable 

principles of trade, and, in general, to 

protect investors and the public interest. 

In Amendment No. 2, Nasdaq addressed 

whether the Dynamic M–ELO is 

designed to prevent fraudulent and 

manipulative acts and practices. Nasdaq 

states that the design of Dynamic M– 

ELO does not lend itself to potential 

manipulation by a single participant or 

a small group of participants because 

the machine-learning model makes 

determinations regarding Holding 

Periods based upon prevailing marketwide conditions for a given symbol, 

rather than the behaviors of particular 

participants with respect to that symbol, 

or the activity of participants in M– 

ELOs involving that symbol. Nasdaq 

further states that manipulation of the 

machine-learning model would be 

difficult to accomplish given the large 

number of variables that factor into the 

machine-learning model’s decisions to 

change Holding Periods during Change 

Events, as well as the different weights 

that apply to each such factor, which as 

described above, may vary over time. 

Furthermore, Nasdaq states that any 

benefits that a participant might derive 

from manipulating the duration of 

Holding Periods would likely be small 

and outweighed significantly by the 

difficulty and cost of effecting such 

manipulation. 

The Exchange, in Amendment No. 2, 

also sets forth representations regarding 

how it will surveil its market after 

Dynamic M–ELO is implemented. First, 

Nasdaq represents that it will review the 

machine-learning functionality and 

operation periodically to affirm that it 

continues to perform in accordance with 

the Exchange’s rules and has not 

introduced any harmful bias in favor of 

or against any participant or category of 

participants.102 Nasdaq also represents 

above that it will surveil for indications 

of manipulation and act accordingly if 

it detects such indications. 

The Commission finds that the 

proposed rule change is designed to 

prevent fraudulent and manipulative 

acts and practices consistent with 

Section 6(b)(5) of the Act. The 

Commission agrees that aspects of the 

Dynamic M–ELO design reduce 

opportunities for manipulation or are 

likely to make manipulation costly or 

difficult. The model’s operation 

depends on 142 data elements, which 

are each likely to have effects on model 

outcomes of differing magnitudes and in 

different directions. Many of these data 

elements are also based on market-wide 

data, in some cases spanning periods of 

days,103 which are likely themselves 

difficult for market participants to 

manipulate. Given these design features, 

it appears likely that manipulating the 

duration of Dynamic M–ELO holding 

periods in any given symbol or group of 

symbols would be an extremely 

complex undertaking. In light of this 

complexity, and the size of M–ELO 

activity relative to the market for NMS 

stocks,104 Nasdaq’s assertion that the 

potential benefits of manipulating the 

dynamic holding periods for these order 

types would be outweighed by the cost 

and complexity of manipulation also 

appears reasonable. Nasdaq has also 

represented that it intends to surveil the 

proposed order types for manipulation. 

This ongoing surveillance, to ensure the 

appropriate use of Dynamic M–ELO by 

Exchange Members and behavior by the 

machine-learning model, is important to 

the successful implementation of 

Dynamic M–ELO and appears 

appropriately tailored to the accomplish 

the intent of the M–ELO and M– 

ELO+CB order types. 

Furthermore, the Commission finds 

that overall structure of Dynamic M– 

ELO—particularly, the static numerical 

constraints set forth in the proposed 

rule text—is designed in general, to 

protect investors and the public interest 

and promote just and equitable 

principles of trade pursuant to Section 

6(b)(5) of the Act. As described above, 

the model will continuously engage in 

dynamic analysis of current market 

conditions during trading hours, and 

outside of market hours, it will retrain 

with the goal of improving the overall 

performance of Dynamic M–ELO. These 

dynamic aspects of the proposal, 


ddrumheller on DSK120RN23PROD with NOTICES1
62862 Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices 

105See Leuchtkafer Letter 1, supra note 48, at 1. 

106See Nasdaq First Response to Comments, 

supra note 57, at 3. 

107See Section III.A.1., supra. 

108See id. 

109See id. 

110See id. 

111See id. 

112See 17 CFR 242.613. 

113The CAT NMS Plan was approved by the 

Commission, as modified, on November 15, 2016. 

See Securities Exchange Act Release No. 79318 

(November 15, 2016), 81 FR 84696 (November 23, 

2016). 

114See 17 CFR 242.613(c)(7). 

115See Leuchtkafer Letter 1, supra note 48, at 2. 

116See Section II.A.1., supra. 

117 15 U.S.C. 78s(b)(1). 

118 17 CFR 240.19b–4. 

119See Section III.A.1., supra. 

120See supra note 30 and accompanying text. 

however, are constrained by the static 

numerical thresholds set forth in the 

proposed rule text. For example, the 

initial Holding Periods for each trading 

day will be 1.25 milliseconds, the 

overall range for any Holding Period 

must be between 0.25 and 2.50 

milliseconds during normal market 

conditions, and the Holding Period can 

only change by either 0.25 or 0.50 

milliseconds at each Change Event 

during normal market conditions. 

Regardless of how the model analyzes 

the current market or changes the 

weighting of the data elements as a 

result of its retraining, Dynamic M–ELO 

cannot operate outside of the static 

numerical ranges and limitations or 

minimums set forth in the rule text. As 

such, the Commission finds that Nasdaq 

has designed Dynamic M–ELO to 

operate in a manner that in general 

protects investors and the public 

interest and promotes just and equitable 

principles of trade in accordance with 

Section 6(b)(5) of the Act. 

Compliance With SRO Recordkeeping 

and Reporting Obligations 

One commenter queried whether 

Nasdaq could maintain an adequate 

audit trail given the potential for 

frequently shifting Holding Periods for 

Dynamic M–ELO.105 In response, 

Nasdaq states that it will retain copies 

of each iteration of its system as part of 

its books and records and will disclose 

publicly statistics relating to Dynamic 

M–ELO performance.106 Nasdaq 

additionally represented that it will 

publish weekly and monthly Dynamic 

M–ELO performance statistics, which 

would include the weekly numbers of 

shares and trades in M–ELOs by symbol, 

weekly aggregated M–ELO share and 

trade data, and monthly aggregated 

block data, on Nasdaqtrader.com.107

Nasdaq also indicated it would add 

statistics to its existing M–ELO Monthly 

Report, which discloses quote stability 

by time horizon, about how frequently, 

on average, its system changes Holding 

Period durations for the top decile, 

median, and bottom decile of symbols, 

as measured by monthly M–ELO and 

M–ELO+CB trading volumes.108

Nasdaq also added a representation to 

the filing, addressing how it would 

comply with its recordkeeping 

obligations.109 Nasdaq states that it will 

retain copies of each historical iteration 

of its models as part of its books and 

records, and make them available to the 

Commission upon request, should it 

wish to examine them to understand 

how the model changes over time.110

Nasdaq also states that it will publish an 

equity trader alert in advance of 

deploying a retrained version of 

Dynamic M–ELO when Nasdaq 

anticipates the retrained version will 

produce results that differ materially 

from the prior version.111 Based on 

these representations, the Commission 

finds that Nasdaq has met its burden to 

demonstrate that it will comply with all 

relevant exchange recordkeeping 

requirements and obligations when it 

implements Dynamic M–ELO. In 

addition, the Commission notes that 

Nasdaq must comply with its reporting 

obligations under Rule 613 of 

Regulation NMS 112 and the National 

Market System Plan Governing the 

Consolidated Audited Trail (‘‘CAT NMS 

Plan’’) 113 with respect to Dynamic M– 

ELO, which requires it to record and 

electronically report to the central 

repository the material terms of each 

order and each reportable event.114

Nasdaq’s Obligation To File Proposed 

Rule Changes Relating to Dynamic M– 

ELO 

Prior to the filing of Amendment No. 

1, a commenter stated that it was 

unclear what types of changes to the 

model would lead Nasdaq to seek 

approval from the Commission via an 

SRO rule filing.115 As explained 

above,116 Nasdaq represents that it will 

not modify the underlying structure of 

Dynamic M–ELO without first obtaining 

the Commission’s approval to do so, 

including modifications to the data 

elements the model considers in making 

decisions about Holding Period 

durations, the conditions under which 

the model may adjust the duration of 

Holding Periods, the frequency with 

which the model may adjust the 

Holding Periods, the range of Holding 

Period durations available to M–ELOs 

and M–ELO+CBs, the increments by 

which Holding Periods may change at 

any given Change Event, and the 

procedures for triggering, maintaining, 

and ending 12 millisecond Holding 

Periods during times of extraordinary 

instability. In contrast, the Exchange 

states that it will not seek Commission 

approval prior to retraining the model to 

adjust the weighting it applies to those 

data elements pursuant to the weekly 

retraining process. 

Section 19(b)(1) of the Act 117 and 

Rule 19b–4 thereunder 118 require an 

SRO to file a proposed rule change with 

the Commission whenever it seeks any 

proposed change in, addition to, or 

deletion from the rules governing the 

SRO and its members’ activities on the 

SRO. As discussed above, the proposal 

sets forth the specific data elements that 

Dynamic M–ELO will use during the 

trading day. Furthermore, the proposed 

rule change sets forth when the 

machine-learning model will retrain and 

the extent to which the retraining can 

and cannot cause the machine-learning 

model to update Dynamic M–ELO’s 

operation during subsequent trading 

days.119 In addition, the proposal sets 

forth the operation of Dynamic M–ELO, 

such as the potential range for a Holding 

Period, how often Dynamic M–ELO 

reevaluates market conditions for a 

given security to adjust a Holding 

Period, and the increment by which a 

Holding Period may be changed. Nasdaq 

represents that it will not change any of 

these aspects of the proposal or any 

other function of Dynamic M–ELO 

without first filing a proposed rule 

change.120 Nasdaq does, however, state 

that it would not file a proposed rule 

change in connection with the operation 

of the machine-learning model’s weekly 

retraining and the results of that 

process. 

Based on the foregoing, the 

Commission believes that Nasdaq has 

adequately responded to the 

commenter’s concern. Nasdaq will need 

to file a proposed rule to make any 

changes, additions, or deletions to the 

operation of Dynamic M–ELO as 

approved herein. Nasdaq has delineated 

when it would file a proposed rule 

change to alter the operation of Dynamic 

M–ELO, and when the machine-learning 

model would retrain and adjust the 

weighting it applies to the data elements 

without it filing a proposed rule change. 

Specifically, Nasdaq’s proposed rule 

change and rule text reflect the 142 data 

elements Dynamic M–ELO will consider 

when determining the Holding Period 

for a security and the goals Nasdaq will 

consider when weighing those data 

elements (i.e., reducing mark-outs and 

increasing fill rates) but does not set 

forth the relative weighting of each 

those individual data elements. Though 


ddrumheller on DSK120RN23PROD with NOTICES1
Federal Register / Vol. 88, No. 176 / Wednesday, September 13, 2023 / Notices 62863 

121 15 U.S.C. 78s(b)(2). 

122 15 U.S.C. 78s(b)(2). 

123 17 CFR 200.30–3(a)(12). 

1 15 U.S.C. 78s(b)(1). 

2 17 CFR 240.19b–4. 

the structure of the proposal does not 

disclose of the exact weighting for each 

of the 142 data elements, it does set 

forth the two goals Nasdaq will consider 

when weighing those data elements 

initially and during each weekly 

retraining, which provides information 

as to how those 142 factors will be used 

in determining the Holding Period for a 

security. Based on how the proposed 

rule sets forth the goals that will govern 

each retraining, the Commission 

believes that Nasdaq’s delineation of 

when it would and would not file a 

proposed rule change to alter the 

operation of Dynamic M–ELO is 

consistent with Nasdaq’s rule filing 

obligation. The Commission agrees that 

the weekly retraining to optimize the 

weighting of the 142 data elements 

considered by Dynamic M–ELO to best 

achieve those goals within the rule’s 

parameters would not necessitate the 

filing of a proposed rule change with the 

Commission because those adjustments 

would be reasonably and fairly implied 

by the proposed rule. However, to the 

extent Nasdaq seeks to change, add to, 

or delete from the rule’s construct in 

connection with the weekly retraining, 

it would first be required to file a 

proposed rule change with the 

Commission. 

V. Solicitation of Comments on 

Amendment No. 2 to the Proposed Rule 

Change 

Interested persons are invited to 

submit written data, views, and 

arguments concerning whether 

Amendment No. 2 is consistent with the 

Act. Comments may be submitted by 

any of the following methods: 

Electronic Comments 

• Use the Commission’s internet 

comment form (https://www.sec.gov/ 

rules/sro.shtml); or 

• Send an email to rule-comments@

sec.gov. Please include file number SR– 

NASDAQ–2022–079 on the subject line. 

Paper Comments 

• Send paper comments in triplicate 

to Secretary, Securities and Exchange 

Commission, 100 F Street NE, 

Washington, DC 20549–1090. 

All submissions should refer to file 

number SR–NASDAQ–2022–079. This 

file number should be included on the 

subject line if email is used. To help the 

Commission process and review your 

comments more efficiently, please use 

only one method. The Commission will 

post all comments on the Commission’s 

internet website (https://www.sec.gov/ 

rules/sro.shtml). Copies of the 

submission, all subsequent 

amendments, all written statements 

with respect to the proposed rule 

change that are filed with the 

Commission, and all written 

communications relating to the 

proposed rule change between the 

Commission and any person, other than 

those that may be withheld from the 

public in accordance with the 

provisions of 5 U.S.C. 552, will be 

available for website viewing and 

printing in the Commission’s Public 

Reference Room, 100 F Street NE, 

Washington, DC 20549 on official 

business days between the hours of 

10:00 a.m. and 3:00 p.m. Copies of the 

filing also will be available for 

inspection and copying at the principal 

office of the Exchange. Do not include 

personal identifiable information in 

submissions; you should submit only 

information that you wish to make 

available publicly. We may redact in 

part or withhold entirely from 

publication submitted material that is 

obscene or subject to copyright 

protection. All submissions should refer 

to File Number SR–NASDAQ–2022– 

079, and should be submitted on or 

before October 4, 2023. 

VI. Accelerated Approval of Proposed 

Rule Change, as Modified by 

Amendment No. 2 

The Commission finds good cause to 

approve the proposed rule change prior 

to the 30th day after the date of 

publication of Amendment No. 2 in the 

Federal Register. Amendment No. 2 

does not include any material changes 

to the operation of the proposed 

Dynamic M–ELO and its machinelearning model. In Amendment No. 2, 

the Exchange: (1) adds the defined term 

‘‘proprietary assessment of market 

conditions’’ to the proposed rule text, 

which consolidates certain details and 

explanations about how the machinelearning model would operate from 

prior versions into a single defined 

term; (2) revises the list of factors 

provided in Exhibit 3b to include 

expanded and ‘‘simplified’’ 

explanations of the terminology used 

therein; (3) adds a representation that 

the systems used to operate Dynamic 

M–ELO and machine-learning model are 

‘‘SCI Systems’’ and thus subject to 

compliance with Regulation SCI; and (4) 

expands the legal analysis to address 

comments regarding unfair 

discrimination and the exercise of 

impermissible discretion by the 

Exchange. 

The Commission finds that 

Amendment No. 2 raises no novel 

regulatory issues that have not 

previously been subject to comment and 

is reasonably designed to prevent 

fraudulent and manipulative acts and 

practices, to promote just and equitable 

principles of trade, and, in general, to 

protect investors and the public interest, 

and not be unfairly discriminatory, or 

impose an unnecessary or inappropriate 

burden on competition. Amendment 

No. 2 does not alter the proposed 

operation or any material features of 

Dynamic M–ELO, which operation and 

features have been subject to two rounds 

of public comment. In response to 

public comment, the revisions to the 

proposal contained within Amendment 

No. 2 provide additional clarification 

and details regarding how Dynamic M– 

ELO and the machine-learning model 

will operate, as well as additional legal 

analysis to support the Exchange’s 

position that the proposal is consistent 

with the Act. Accordingly, pursuant to 

Section 19(b)(2) of the Act,121 the 

Commission finds good cause to 

approve the proposed rule change on an 

accelerated basis prior to the 30th day 

after publication of notice of the filing 

of Amendment No. 2 in the Federal 

Register. 

VII. Conclusion 

It is therefore ordered, pursuant to 

Section 19(b)(2) of the Act,122 that the 

proposed rule change (SR–NASDAQ– 

2022–079), as modified by Amendment 

No. 2, be, and it hereby is, approved on 

an accelerated basis. 

For the Commission, by the Division of 

Trading and Markets, pursuant to delegated 

authority.123

Sherry R. Haywood, 

Assistant Secretary. 

[FR Doc. 2023–19728 Filed 9–12–23; 8:45 am] 

BILLING CODE 8011–01–P 

SECURITIES AND EXCHANGE 

COMMISSION 

[Release No. 34–98320; File No. SR–PHLX– 

2023–41] 

Self-Regulatory Organizations; Nasdaq 

PHLX LLC; Notice of Filing and 

Immediate Effectiveness of Proposed 

Rule Change To Amend Options 3, 

Section 13 Concerning PIXL 

September 7, 2023. 

Pursuant to section 19(b)(1) of the 

Securities Exchange Act of 1934 

(‘‘Act’’),1 and Rule 19b–4 thereunder,2

notice is hereby given that on August 

30, 2023, Nasdaq PHLX LLC (‘‘Phlx’’ or 

‘‘Exchange’’) filed with the Securities 

and Exchange Commission 


ddrumheller on DSK120RN23PROD with NOTICES1