Document Headings

                    Document headings vary by document type but may contain                    the following:                  
                -                       the agency or agencies that issued and signed a document                    
                -                       the number of the CFR title and the number of each part                      the document amends, proposes to amend, or is directly                      related to                    
                -                       the agency docket number / agency internal file number                    
                -                       the RIN which identifies each regulatory action listed in                      the Unified Agenda of Federal Regulatory and Deregulatory                      Actions                    

                    See the                                          Document Drafting Handbook                                        for more details.                  

###### Securities and Exchange Commission

          - [Release No. 34-98321; File No. SR-NASDAQ-2022-079]

## I. Introduction

                    On December 21, 2022, The Nasdaq Stock Market LLC (“Nasdaq” or “Exchange”) filed with the Securities and Exchange Commission (“Commission”), pursuant to Section 19(b)(1) of the Securities Exchange Act of 1934 (“Act”) [1]                                          and Rule 19b-4 thereunder,[2]                                          a proposed rule change to replace the static holding period requirements for Midpoint Extended Life Orders and Midpoint Extended Life Orders Plus Continuous Book with dynamic holding periods. The proposed rule change was published for comment in the                     **Federal Register**                     on January 10, 2023.[3]                                          On February 22, 2023, pursuant to Section 19(b)(2) of the Act,[4]                                          the Commission designated a longer period within which to approve the proposed rule change, disapprove the proposed rule change, or institute proceedings to determine whether to disapprove the proposed rule change.[5]                                          On March 9, 2023, the Exchange filed Amendment No. 1 to the proposed rule change, which amended and superseded the proposed rule change as originally filed. On April 7, 2023, the Commission provided notice of filing of Amendment No. 1 and instituted proceedings to determine whether to approve or disapprove the proposed rule change, as modified by Amendment No. 1.[6]                                          On July 6, 2023, pursuant to Section 19(b)(2) of the Act,[7]                                          the Commission designated a longer period on proceedings to determine whether to approve or disapprove the proposed rule change.[8]                                          On July 18, 2023, the Exchange filed Amendment No. 2 to the proposed rule change, which amended and superseded the proposed rule change as amended by Amendment No. 1. The Commission received comments on the proposed rule change.[9]                                          The Commission is publishing this Notice and Order to solicit comment on Amendment No. 2 in Sections II and III below, which sections are being published verbatim as filed by the Exchange, and to approve the proposed rule change, as modified by Amendment No. 2, on an accelerated basis.                

## II. Self-Regulatory Organization's Statement of the Terms of Substance of the Proposed Rule Change

                    The Exchange proposes to amend Rules 4702(b)(14) and (b)(15) of the Exchange's Rulebook to replace the static holding period requirements for Midpoint Extended Life Orders and Midpoint Extended Life Orders Plus Continuous Book with dynamic holding periods. This Amendment No. 2                           ( print page 62851)                        supersedes the original filing and Amendment No. 1 [10]                                          in their entireties.                

                    The text of the proposed rule change is available on the Exchange's website at                     *https://listingcenter.nasdaq.com/​rulebook/​nasdaq/​rules,*                     at the principal office of the Exchange, and at the Commission's Public Reference Room.                

## III. Self-Regulatory Organization's Statement of the Purpose of, and Statutory Basis for, the Proposed Rule Change

In its filing with the Commission, the Exchange included statements concerning the purpose of and basis for the proposed rule change and discussed any comments it received on the proposed rule change. The text of these statements may be examined at the places specified in Item IV below. The Exchange has prepared summaries, set forth in sections A, B, and C below, of the most significant aspects of such statements.

### A. Self-Regulatory Organization's Statement of the Purpose of, and Statutory Basis for, the Proposed Rule Change

#### 1. Purpose

The Exchange proposes to amend Rules 4702(b)(14) and (15) of the Exchange's Rulebook to replace the static 10 millisecond holding period requirements for its Midpoint Extended Life Order (“M-ELO”) and Midpoint Extended Life Order Plus Continuous Book (“M-ELO+CB”) Order Types with dynamic holding periods (“Dynamic M-ELO and M-ELO+CB” or collectively, “Dynamic M-ELO”).

#### Background

                    In 2018, the Exchange introduced the M-ELO, which is a Non-Displayed Order priced at the Midpoint between the National Best Bid and Offer (“NBBO”) and which is eligible for execution only against other eligible M-ELOs and only after a minimum of one-half second passes from the time that the System accepts the order (the “Holding Period”).[11]                                          In 2019, the Exchange introduced the M-ELO+CB, which closely resembles the M-ELO, except that a M-ELO+CB may execute at the midpoint of the NBBO, not only against other eligible M-ELOs (and M-ELO+CBs), but also against Non-Displayed Orders with Midpoint Pegging and Midpoint Peg Post-Only Orders (“Midpoint Orders”) that rest on the Continuous Book for at least one-half second and have Trade Now enabled.[12] 

When the Exchange designed M-ELO, it originally set the length of the Holding Period at one-half second because it determined that this time period would be sufficient to ensure that likeminded investors would interact only with each other, and with minimal market impacts. The Exchange believed that the longer length of the M-ELO Holding Period and its simplicity in design would provide greater protection for participants than they could achieve through competing delay mechanisms.

                    In 2020, however, the Exchange shortened the length of the Holding Period to 10 milliseconds.[13]                                          The Exchange did so after studying two years of actual use and performance of M-ELOs, as well as customer feedback. That is, the Exchange came to understand that, while users of M-ELO and M-ELO+CB are less concerned with achieving rapid executions of their Orders than are other participants, they are not indifferent about the length of time in which their M-ELOs and M-ELO+CBs must wait before they are eligible for execution. Indeed, participants informed the Exchange that in certain circumstances, such as when they sought to trade symbols that on average had a lower time-to-execution than a half-second, they were reticent to enter M-ELOs or M-ELO+CBs. They indicated that the associated Holding Periods for these Order Types were longer than necessary to achieve the desired protections and that, during the residual portion of the Holding Periods, they risked losing out on favorable execution opportunities that would otherwise be available to them had they placed a non-MELO order.                

                    Based upon this feedback, the Exchange studied the potential effects of reducing the length of the Holding Periods for both M-ELOs and M-ELO+CBs (as well as for Midpoint Orders that would execute against M-ELO+CBs). Ultimately, the Exchange determined that it could reduce the Holding Periods to 10 milliseconds without compromising the protective power that M-ELO and M-ELO+CB are intended to provide to participants and investors.[14]                                          Thus, the Exchange determined that shortening the Holding Periods to 10 milliseconds for M-ELOs and M-ELO+CBs would increase the efficacy of the mechanism while not undermining the power of those Order Types to fulfill their underlying purpose of minimizing market impacts. At the same time, the Exchange determined that a reduction in the Holding Periods to 10 milliseconds would dramatically add to the circumstances in which M-ELOs and M-ELO+CBs would be useful to participants. In its M-ELO Timer Approval Order, the Commission agreed with the Exchange:                

                    The Commission notes that, with the proposed ten-millisecond Holding Period and Resting Period, M-ELOs and M-ELO+CBs would continue to be optional order types that are available to investors with longer investment time horizons, including institutional investors. The Commission also believes that the proposal could make M-ELOs and M-ELO+CBs more attractive for securities that on average have a time-to-execution of less than one-half second and, for investors who currently do not use M-ELOs and M-ELO+CBs for these securities, provide optional order types that could enhance their ability to participate effectively on the Exchange. The Commission notes that, if market participants determine that the proposal would make M-ELOs and M-ELO+CBs less attractive for their particular investment objectives, such market participants may elect to reduce or eliminate their use of these optional order types. Moreover, as noted above, the Exchange will continue to conduct real-time surveillance to monitor the use of M-ELOs and M-ELO+CBs to ensure that such usage remains appropriately tied to the intent of the order types. If, as a result of such surveillance, the Exchange determines that the shortened                           ( print page 62852)                        Holding Period does not serve its intended purpose or adversely impacts market quality, the Exchange would seek to make further recalibrations.[15] 

For similar reasons and with even better potential results for participants, the Exchange now proposes to further refine the length of the Holding Periods for M-ELOs and M-ELO+CBs, this time through the application of innovative and patent pending machine learning technology.

#### Dynamic M-ELO

After receiving feedback from participants that even 10 millisecond Holding Periods for M-ELO and M-ELO+CB may, at times, exceed what is necessary to accomplish the underlying intent of these Order Types, the Exchange began to experiment with making further refinements to the duration of the Holding Periods. Ultimately, the Exchange concluded that shorter Holding Periods could achieve the same, if not better results for participants in terms of mark-outs, but not in all circumstances. That is, where prices of the underlying securities are stable, and not subject to imminent unfavorable changes, M-ELOs and M-ELO+CBs face lower risks of confronting spread-crossing orders, such that shorter Holding Periods could suffice to protect M-ELOs and M-ELO+CB from such orders. In periods of heightened price volatility, however, M-ELOs and M-ELO+CBs also face heightened risks, such that longer Holding Periods would continue to be beneficial in protecting M-ELOs and M-ELO+CBs from such risks. Thus, the Exchange determined that another across-the-board reduction of the static 10 millisecond Holding Periods would be sub-optimal because it could impact the performance of the M-ELO and M-ELO+CB Order Types during periods of heightened volatility.

In light of these observations, the Exchange tasked its artificial intelligence and machine learning laboratory (the “AI Core Development Group”) to explore whether it could employ these innovative technologies to optimize the length of M-ELO and M-ELO+CB Holding Periods during various states of price volatility, and then to vary the lengths of the Holding Periods dynamically during the lifecycles of M-ELOs and M-ELO+CBs, with the objectives of improving the performance of these Order Types while also further reducing opportunity costs.

                    As the Exchange explains in greater depth in the attached White Paper,[16]                                          the AI Core Development Group proceeded to develop an artificial intelligence-based timer control system that will achieve these objectives.[17]                                          The AI Core Development Group did so by using reinforcement learning techniques—machine learning paradigms which develop optimal solutions to problems over time by taking actions to solve them, generating feedback on the results of such actions, applying that feedback to direct and improve the next round of solutions, and then repeating the feedback loop until the paradigm achieves optimized solutions.                

In this instance, the AI Core Development Group applied reinforcement learning techniques to a simulation of the M-ELO Book that it constructed using a representative data set from the first quarter of 2022 (the “Training Period”). The Training Period data consisted of 380 out of the 6,257 symbols on the M-ELO Book (accounting for approximately 67 percent of M-ELO volume). The symbols chosen reflect both actively-traded and thinly-traded securities, and both low-priced and high-priced securities.

                    The AI Core Development Group then developed a machine learning model and applied it to the Training Period data. The Group programmed the model to value the achievement of higher fill rates or lower mark-outs than that which occurred in a historical simulation of M-ELOs and M-ELO+CBs involving the Training Period data.[18]                                          The Group then programmed the model to seek to achieve its goals by taking one of five possible actions with respect to the duration of the Holding Periods at 30 second intervals [19]                                          for each symbol during each trading day of the Training Period. That is, at each 30 second internal, the model evaluated market conditions for each symbol over the prior 30 second period and either kept the Holding Periods the same, increased/decreased them by 0.25 milliseconds, or increased/decreased them by 0.50 milliseconds.[20]                                          After each decision-making round, the model utilized the results to inform its actions at the next 30 second increment.                

                    In making its decisions, the model (again, drawing upon a combination of historical SIP and M-ELO-specific data) considered 142 categories of data points.[21]                                          A confluence of data points that correlated with an increase in volatility tended to cause the model to increase the durations of Holding Periods, including increases in the standard deviation of NBBO prices, the number of unique participants placing sell orders on M-ELO and M-ELO+CB, and the volume-weighted average of the NBBO spread. Conversely, a confluence of data points that correlated with greater price stability tended to cause the model to decrease the durations of Holding periods, such as an increase in the median and max number of shares per trade and the number of resting bids left in the M-ELO and M-ELO+CB Book.                

                    The AI Core Development Team produced variations of its model that prioritized achievement of the lowest mark-outs, the highest fill rates, and a blend of these two objectives.[22]                                          Through                           ( print page 62853)                        a process of learning and experimentation involving a combination of historical and simulated data, the AI Core Development Group settled on a Dynamic M-ELO model that achieved substantial simulated performance improvements for users of M-ELO and M-ELO+CB—both in terms of mark-outs and fill rates—as compared to the static 10 millisecond Holding Periods. As the White Paper explains in greater detail, Dynamic M-ELO yielded an average combined volume-weighted (simulated) improvement of 31.7 percent, including a 20.3 percent increase in fill rates and a 11.4 percent reduction in mark-outs.[23]                                          The White Paper provides a more fulsome explanation of these improvements.[24] 

                    Based upon these exciting results, the Exchange now proposes to amend Rule 4702(b)(14) and (15) to replace the static 10 millisecond timers applicable to M-ELO and M-ELO+CB with Dynamic M-ELO Holding Periods. Using the Exchange's “proprietary assessment of market conditions” [25]                                          and patent pending technology, the Dynamic M-ELO system will evaluate and, as it deems necessary, adjust the length of the Holding Periods for each symbol comprising M-ELOs and M-ELO+CBs (and Midpoint Orders on the Continuous Book that opt to interact with M-ELO+CBs after resting on the Book) every 30 seconds throughout the Market Hours (each such 30 second interval, a “Change Event”). In so doing, Dynamic M-ELO will help participants to achieve a more optimized blend of the underlying purposes of the M-ELO and M-ELO+CB Order Types: protection against adverse selection (low mark-outs) without sacrificing opportunities to achieve high-quality executions (high fill rates).                

A proposed M-ELO or M-ELO+CB with a Dynamic Holding Period will operate as follows. At the outset of Market Hours (approximately 9:30:00 a.m.), the Exchange will impose initial Holding Periods of 1.25 milliseconds for M-ELOs and M-ELO+CBs in all symbols. Thereafter, Holding Periods for a given symbol will become eligible to change dynamically from the initial duration beginning at 9:30:30 a.m. and then at 30 second intervals thereafter during Market Hours. The Exchange will then apply to the M-ELO or M-ELO+CB Order a Holding Period that is of the duration that prevailed at the time of entry. For example, if participant A enters a M-ELO for symbol XYZ at 9:30:25 a.m., then Holding Period for that M-ELO will be 1.25 milliseconds. If at 9:30:30:00 a.m., the System decides to lower the duration of the Holding Period by 0.50 milliseconds, and then participant B enters a M-ELO for symbol XYZ at 9:30:45 a.m., then the System will assign a 0.75 millisecond Holding Period to participant B's M-ELO. To be clear, the System will determine Dynamic M-ELO Holding Periods independently for M-ELOs and M-ELO+CBs in each symbol.

During normal market conditions, the range of potential Holding Period durations for M-ELOs and M-ELO+CBs will be between 0.25-2.50 milliseconds, with the Holding Period duration being eligible to change by increments of either 0.25 or 0.50 milliseconds at each Change Event. Thus, if the Holding Period for a M-ELO in symbol XYZ is set at 0.75 milliseconds at 2:22:11 p.m., and at 2:22:41 p.m., the System determines to increase the duration of the Holding Period, it may do so only by 0.25 or 0.50 milliseconds during that event.

When a Change Event occurs, and the System determines to adjust the duration of a Holding Period for a symbol, that adjustment will apply, not only to all M-ELOs and M-ELO+CBs for that symbol entered within the 30 second period after the Change Event occurs, but also to M-ELOs and M-ELO+CBs entered prior to the Change Event with unexpired Holding Periods (with applicability retroactive to the time of Order acceptance). Thus, if a participant enters a M-ELO in symbol XYZ at 1:14:299 p.m., and the prevailing Holding Period applicable to that M-ELO is 2 milliseconds, and at 1:14:30 p.m., the System modifies the Holding Period to be 1.5 milliseconds, then the M-ELO will become eligible to execute at 1:14:3005 p.m. This is the case because the M-ELO will have already expended 1 millisecond of its Holding Period as of the time of the Change Event; thereafter, the M-ELO will need to rest only another 0.5 milliseconds to become eligible to execute under the new 1.5 millisecond Holding Period (as measured from 1:14:299 p.m.). This last feature ensures that the M-ELO Book maintains time priority among M-ELOs and M-ELO+CBs in a dynamic environment. That is, it ensures that no M-ELO or M-ELO+CB with an unexpired Holding Period at the time of a Change Event will end up becoming eligible to execute later than a M-ELO entered after the Change Event which has a shorter Holding Period applicable to it.

                    If at any time, the System detects extraordinary instability in a symbol, then the System will activate a “stability protection mechanism” to provide an extra layer of protection to M-ELO and M-ELO users from the heightened risks of adverse selection that exists during such periods of instability.[26]                                          The stability protection mechanism will override the prevailing Holding Periods for M-ELOs and M-ELO+CBs in a symbol experiencing extraordinary instability and immediately increase the duration of those Holding Periods to 12 milliseconds for a period of 750 milliseconds. The System may activate the stability protection mechanism even between Change Events. The System will evaluate, at each NBBO update, whether market conditions remain extraordinarily unstable and, if so, it will restart the 750 millisecond Stability Protected Period and maintain the 12 millisecond Holding Period until conditions stabilize. Once the System determines that market conditions have stabilized (                    *i.e.,*                     all measurements for the symbol are at or below the threshold value throughout the duration of the prevailing Stability Protected Period), the System will revert the duration of the Holding Periods to that which prevailed as of the Change Event that                           ( print page 62854)                        occurred immediately prior to the activation of the stability protection mechanism or, if the stability protection mechanism was active when a Change Event occurred, to the duration selected at the immediately preceding Change Event. The System will then proceed to reevaluate the duration of the Holding Periods as per the regular schedule of Change Events.                

The following is an illustration of the operation of the stability protection mechanism. At 11:10:04 a.m., the prevailing Holding Period for M-ELOs in symbol XYZ is 1.5 milliseconds. At the same time, the NBBO for symbol XYZ updates. The System looks back at the prior three seconds of trading in symbol XYZ and finds that during that period, the highest observed NBBO midpoint was $10.05, and the lowest was $10.00, such that the difference between these two values is a range of $0.05. The System then looks back at trading behavior for symbol XYZ during the immediately preceding trading day. In doing so, the System calculates the value of the threshold that would have caused the symbol to be deemed extraordinarily unstable for one percent of the trading day; the System determines that this threshold value is a range of $0.03. The System then compares the $0.03 threshold to its measurement of the prior three seconds of NBBO changes ($0.05), and concludes that over these past three seconds, the symbol is extraordinarily unstable. Accordingly, the System activates the stability protection mechanism and the Holding Period for M-ELOs in symbol XYZ immediately increases to 12 milliseconds for a period of 750 milliseconds. However, 5 milliseconds after the Stability Protection Period commences, the NBBO updates again, thus prompting the System to repeat its assessment of the stability of the symbol in light of the update. This reassessment reveals that the symbol remains unstable, such that a new Stability Protection Period of 750 milliseconds begins at that time (overriding the pre-existing Period). Over the course of this new Stability Protection Period, the NBBO shifts two more times, but each of the ensuing reassessments indicate that the NBBO ranges for the symbol have fallen below the $0.03 threshold. The Stability Protection Period elapses 750 milliseconds after it began with the symbol remaining stable. Thus, the Holding Period reverts to 1.5 milliseconds.

                    If the Exchange halts trading in a symbol, then upon resumption of trading, any new M-ELO or M-ELO+CB in that symbol and any pending M-ELO or M-ELO+CB in that symbol with an unexpired Holding Period will be subject to a new 12 milliseconds Holding Period (running from the time when trading resumes) until the next scheduled Change Event, at which point the System may determine to adjust that Holding Period to a duration within the range applicable under normal market conditions.[27]                                          If, however, the System determines that extraordinary instability in the symbol exists, it will instead determine to activate the stability protection mechanism and maintain the duration of the Holding Period at 12 milliseconds for another 750 milliseconds. This design will help to ensure that M-ELOs and M-ELO+CBs receive added protection coming out of halt conditions.[28] 

The Exchange notes that same dynamic process described above will also apply to and govern the time periods during which Midpoint Orders on the Continuous Book must rest before they will become eligible to interact with M-ELO+CBs (provided that participants have opted for their Midpoint Orders to interact with M-ELO+CBs). Thus, the same Holding Period duration that the System sets for a M-ELO+CB in a symbol during Regular Market Hours will also be the length of time that a Midpoint Order must rest on the Continuous Book must rest before it may interact with a M-ELO+CB.

Apart from these impacts of Dynamic Holding Periods, M-ELOs and M-ELO+CBs will continue to behave as they do now in all respects, and as set forth in Rules 4702(b)(14) and (15).

                    It is important to note that within the parameters discussed herein and in the White Paper, the Exchange will continue to re-train Dynamic M-ELO and M-ELO+CB on a weekly basis (outside of market hours) so that the model will continue to learn from and act upon the basis of more recent SIP and M-ELO book data sets, and further improve its performance over time. The retraining process should not result in dramatic or unpredictable changes to the behavior of Dynamic M-ELO. The retraining process will not retrain the model from scratch each week; rather, it will retain the model's existing data inputs, knowledge base, and objectives—all without alteration. Retraining will result in new behaviors only as needed to address new scenarios that the model did not confront previously, and even then, only in a manner designed to further optimize outcomes,                     *i.e.,*                     reduce mark-outs or increase fill rates. If the System assesses that a retrained model would be worse than the existing model in achieving its objectives, then the System will continue to use the existing model and discard the retrained model. This retraining process is a standard and accepted practice for use of deep learning models; it helps to ensure that deep learning models not only work well, but that they continue to work well in dynamic circumstances.[29] 

                    The Exchange will not modify the underlying structure of Dynamic M-ELO and M-ELO+CB without first obtaining the Commission's approval to do so, including modifications to the data elements the model considers in making decisions about Holding Period durations, the conditions under which the model may adjust the duration of Holding Periods, the frequency with which the model my adjust the Holding Periods, the range of Holding Period durations available to M-ELOs and M-ELO+CBs, the increments by which Holding Periods may change at any given Change Event, and the procedures for triggering, maintaining, and ending 12 millisecond Holding Periods during times of extraordinary instability.[30]                                          Although the Exchange will seek Commission approval prior to changing any of the data elements that the model considers, the Exchange will not seek Commission approval prior to retraining                           ( print page 62855)                        the model to adjust the weighting it applies to those data elements.                

                    To aid investors in understanding and evaluating Dynamic M-ELO, Nasdaq will continue to publish weekly and monthly transparency statistics on Nasdaqtrader.com, as it does now, about the performance of its M-ELOs and M-ELO+CBs, including statistics listing the weekly numbers of shares and trades in M-ELOs by symbol, weekly aggregated M-ELO share and trade data, and monthly aggregated block data.[31]                                          Nasdaq also will continue to disclose monthly data on Nasdaq.com, as it does now (the M-ELO Monthly Report), about M-ELO and M-ELO+CB mark-outs (quote stability by time horizon) and fill rates.[32]                                          Moreover, Nasdaq will add statistics to the M-ELO Monthly Report about how frequently, on average, the System changes Holding Period durations for the top decile, median, and bottom decile of symbols, as measured by monthly M-ELO and M-ELO+CB trading volumes. Nasdaq will retain copies of each historical iteration of its models as part of its books and records, and make them available to the Commission upon request, should it wish to examine them to understand how the model changes over time. Furthermore, Nasdaq will publish an equity trader alert in advance of deploying a retrained version of Dynamic M-ELO whenever Nasdaq has reason to anticipate that the retrained version will produce results that differ materially from the prior version,                     *i.e.,*                     a projected change in mark-outs or fill-rates of 10% or more in either direction.                

                    The Exchange acknowledges that systems necessary to implement Dynamic M-ELO, including the systems proposed that include model development and retraining processes, are “SCI Systems” within the meaning of Regulation Systems Compliance and Integrity (“Reg. SCI”),[33]                                          and that the Exchange, as an SCI Entity, remains responsible for compliance with all requirements of Reg. SCI, including, without limitation, to have policies and procedures reasonably designed to ensure that its SCI Systems operate in a manner that complies with the Act and the rules and regulations thereunder and Exchange's rules and governing documents, among them a plan for assessments of the functionality of SCI Systems designed to detect systems compliance issues, including by responsible SCI personnel and by personnel familiar with applicable provisions of the Act and the rules and regulations thereunder and Exchange's rules and governing documents.                

#### Implementation

The Exchange intends to make the proposed change effective for M-ELOs and M-ELO+CBs in the Second or Third Quarter of 2023, but that time frame is subject to change. The Exchange will publish a Trader Alert in advance of making the proposed change effective.

#### 2. Statutory Basis

                    The Exchange believes that its proposal is consistent with Section 6(b) of the Act,[34]                                          in general, and furthers the objectives of Section 6(b)(5) of the Act,[35]                                          in particular, in that it is designed to promote just and equitable principles of trade, to remove impediments to and perfect the mechanism of a free and open market and a national market system, and, in general to protect investors and the public interest, by allowing for more widespread use of M-ELOs and M-ELO+CBs.                

                    When the Commission approved the M-ELO and the M-ELO+CB, it determined that these Order Types are consistent with the Act because they “could create additional and more efficient trading opportunities on the Exchange for investors with longer investment time horizons, including institutional investors, and could provide these investors with an ability to limit the information leakage and the market impact that could result from their orders.” [36]                                          Nothing about the Exchange's proposal should cause the Commission to revisit or rethink this determination. Indeed, the proposal will not alter the fundamental design of these Order Types, the manner in which they operate, or their effects.                

                    Even with Dynamic M-ELO Holding Periods, M-ELOs and M-ELO+CBs will continue to provide their users with protection against information leakage and adverse selection—and they will do so at levels which are substantially undiminished from that which they provide now.[37] 

At the same time, however, the proposal will benefit market participants and investors by reducing the opportunity costs of utilizing M-ELOs and M-ELO+CBs. The proposal, in other words, will re-calibrate the lengths of the Holding Periods so that M-ELOs and M-ELO+CBs will operate in the “Goldilocks” zone—their Holding Periods will not be so short as to render them unable to provide meaningful protections against information leakage and adverse selection, but the Holding Periods also will not be too long so as to cause participants and investors to miss out on favorable execution opportunities. Nasdaq believes the proposal will render M-ELOs and M-ELO+CBs more useful and attractive to market participants and investors, and this increased utility and attractiveness, in turn, will spur an increase in M-ELO and M-ELO+CB use cases on the Exchange, both from new and existing users of M-ELOs and M-ELO+CBs. Ultimately, the proposal should enhance market quality by increasing opportunities for midpoint executions on the Exchange.

                    As Nasdaq explained above, the Proposal will operate within strict, well-defined, and transparent parameters. Although it will undergo weekly retraining (outside of market hours),[38]                                          such retraining will aim to improve the performance of the model in achieving its twin objectives; retraining will not alter the inputs, objectives, or basic design parameters of Dynamic M-ELO without prior Commission approval.[39]                                          Moreover, the Exchange will not deploy a retrained model if it fails to achieve performance improvements. To aid                           ( print page 62856)                        investors in evaluating Dynamic M-ELO, the Exchange will publish statistics about its performance, including as to mark-outs and fill rates, as well as statistics about how frequently the System changes Holding Period durations. To further facilitate accountability, the Exchange will retain each historical iteration of its model as part of its books and records, and make such information available to the Commission, upon request. The Exchange will also publish equity trader alerts whenever retraining will result in a performance change of 10% or more.                

                    Nasdaq notes that the twin objectives it prescribes for the model involve the absolute values of mark-outs and fill rates; they are not designed to further the performance of any participant or any category of participant. Furthermore, Nasdaq performed internal tests of its AI model to detect indications of harmful bias in its performance results, and such tests concluded that no such indications exist. That is, the Exchange reviewed the impact on fill rates and mark-outs of Dynamic M-ELO, as compared to the “static” M-ELO, for those firms that accounted for more than 95% of M-ELO activity on the Exchange during Q1 2022.[40]                                          The Exchange analyzed results both in an absolute and a relative sense. Testing revealed that all participants experienced at least some improvements in fill rates and mark-outs when using Dynamic M-ELO versus static M-ELO, with the volume-weighted average improvement being aligned with the results expressed in the White Paper. We detected no material variations that might suggest that a particular participant or category of participant (                    *i.e.,*                     nature of firm; size of firm) benefitted from Dynamic M-ELO functionality to an extent that was unreasonably disproportionate to the benefits that other participants experienced. Thus, Nasdaq believes the model is objective, is designed to, and does avoid bias and discrimination.[41] 

The Exchange notes that use of Dynamic M-ELOs and M-ELO+CBs remains voluntary for all market participants. Accordingly, if any market participant feels that the dynamic Holding Periods are still too long or too short or because competing venues offer more attractive delay mechanisms, then the participants are free to pursue other trading strategies or utilize other trading venues. They need not utilize Dynamic M-ELOs or M-ELO+CBs.

                    Furthermore, the design of Dynamic-MELO would constitute an “established, non-discretionary” method that is consistent with the definition of an exchange, as set forth in SEC Rule 3b-16.[42]                                          The Commission stated as follows when it adopted Rule 3b-16:                

                        A system uses established non-discretionary methods either by providing a trading facility or by setting rules governing trading among subscribers. The Commission intends for “established, non-discretionary methods” to include any methods that dictate the terms of trading among the multiple buyers and sellers entering orders into the system. Such methods include those that set procedures or priorities under which open terms of a trade may be determined. For example, traditional exchanges' rules of priority, parity, and precedence are “established non-discretionary methods,” as are the trading algorithms of electronic systems. Similarly, systems that determine the trading price at some designated future date on the basis of pre-established criteria (such as the weighted average trading price for the security on the specified date in a specified market or markets) are using established, non-discretionary methods.[43] 

                    Nothing in the Reg. ATS Adopting Release or in any of its illustrative examples suggests that Dynamic M-ELO would constitute an exercise of discretionary behavior. Dynamic M-ELO will handle and execute Orders according to published, pre-determined rules that are disclosed to the public and which provide reasonable notice of how the Order Type will behave.[44]                                          To the extent that the design of the System permits variation in the Holding Periods for such Orders, it does so by design. The range of potential variations, the objectives that such variations are intended to achieve, and the factors that determine when such variations may occur are also predetermined and set forth in the Exchange's Rules or otherwise disclosed to the public. The mere fact that the System may apply different weights over time to the factors it uses to determine whether and by how much to vary a Holding Period does not mean that the System will act with discretion in the same sense that a human being could be said to be exercise independent judgment when deciding whether and how to handle an order.[45]                                          Even when the System makes decisions about changing the Holding Periods, the System will operate pursuant to a mathematical algorithm from which it cannot deviate—an algorithm that is programmed to achieve pre-defined and pre-disclosed objectives.[46] 
      ( print page 62857)    
The Exchange notes that it will continue to conduct real-time surveillance to monitor the use of M-ELOs and M-ELO+CBs to ensure that such usage remains appropriately tied to the intent of the Order Types. If, as a result of such surveillance, the Exchange determines that the Dynamic M-ELO Holding Periods do not serve their intended purposes, or adversely impact market quality, then the Exchange will seek to make further re-calibrations.

Nasdaq does not believe that the design of Dynamic M-ELO lends itself to potential manipulation by a single participant or a small group of participants because the System makes determinations regarding Holding Periods based upon prevailing market-wide conditions for a given symbol, rather than the behaviors of particular participants with respect to that symbol, or the activity of participants in M-ELOs involving that symbol. Manipulation of the System also would be difficult to accomplish given the large number of variables that factor into the System's decisions to change Holding Periods during Change Events, as well as the different weights that apply to each such factor, which as described above, the System may vary over time. Any benefits that a participant might derive from manipulating the duration of Holding Periods would likely be small and outweighed significantly by the difficulty and cost of affecting such manipulation. Nevertheless, the Exchange will surveil for indications of manipulation and act accordingly if it detects such indications.

### B. Self-Regulatory Organization's Statement on Burden on Competition

The Exchange does not believe that the proposed rule change will impose any burden on competition not necessary or appropriate in furtherance of the purposes of the Act. To the contrary, the Exchange believes that this proposal will promote the competitiveness of the Exchange by rendering its M-ELO and M-ELO+CB Order Types more attractive to participants.

                    The Exchange adopted the M-ELO and M-ELO+CB as pro-competitive measures intended to increase participation on the Exchange by allowing certain market participants that may currently be underserved on regulated exchanges to compete based on elements other than speed. The proposed change continues to achieve this purpose. With Dynamic M-ELO Holding Periods, both M-ELOs and M-ELO+CBs will afford their users with a level of protection from information leakage and adverse selection that is better from what is achievable at present.[47]                                          At the same time, the Dynamic Holding Periods will increase opportunities to interact with other like-minded investors with longer time horizons while also lowering the opportunity costs for participants that utilize M-ELOs and M-ELO+CBs, particularly for securities that trade within the “Goldilocks” zone. In sum, the proposed changes will not burden competition, but instead may promote competition for liquidity in M-ELOs and M-ELO+CBs by broadening the circumstances in which market participants may find such Orders to be useful. With the proposed changes, market participants will be more likely to determine that the benefits of entering M-ELOs and M-ELO+CBs outweigh the risks of doing so.                

The proposed change will not place a burden on competition among market venues, as any market may adopt an order type that operates similarly to a M-ELO or a M-ELO+CB with Dynamic M-ELO Holding Periods.

### C. Self-Regulatory Organization's Statement on Comments on the Proposed Rule Change Received From Members, Participants, or Others

No written comments were either solicited or received.

## IV. Discussion and Commission Findings

                    After careful review, the Commission finds that the Exchange's proposal to adopt Dynamic M-ELO is consistent with the requirements of the Act and the rules and regulations thereunder applicable to a national securities exchange.[48]                                          In particular, the Commission finds that the proposed rule change, as modified by Amendment No. 2, is consistent with Sections 6(b)(5) and 6(b)(8) of the Act.[49]                                          Section 6(b)(5) of the Act requires that the rules of a national securities exchange be designed, among other things, to prevent fraudulent and manipulative acts and practices, to promote just and equitable principles of trade, and, in general, to protect investors and the public interest, and not be designed to permit unfair discrimination between customers, issuers, brokers, or dealers.[50]                                          Section 6(b)(8) of the Act requires that the rules of a national securities exchange not impose any burden on competition that is not necessary or appropriate in furtherance of the purposes of the Act.[51] 

### Nasdaq's Obligation To Sufficiently Explain Its Proposed Rule Change

                    The burden to demonstrate that a proposed rule change is consistent with the Act and rules and regulations thereunder is on the self-regulatory organization (“SRO”) proposing a rule change.[52]                                          Each proposed SRO rule change must be “accompanied by a concise general statement of the basis and purpose of such proposed rule change.” [53]                                          As described in more detail below, several commenters argued that the proposal did not provide sufficient information with respect to the operation of Dynamic M-ELO, or that the information provided was not “clear and comprehensible,” as required by Form 19b-4. For the reasons articulated below, the Commission believes that Nasdaq has provided clear and comprehensible information on the overall operation of Dynamic M-ELO and the role of the machine-learning model and demonstrated that the proposal is consistent with the Act. Several related comments addressed this issue; these comments and Nasdaq's responses are discussed below, followed by the Commission's analysis.                

                    One commenter stated that the initial filing would establish “a dangerously vague standard for describing how exchange-hosted complex algorithmic                           ( print page 62858)                        order types operate.” [54]                                          In response to public comment, Nasdaq added more details describing the operation of Dynamic M-ELO to both the filing and public record since this proposed rule change was initially submitted to the Commission, and Nasdaq also provided additional legal analysis to support Dynamic M-ELO's consistency with the Act. Prior to the filing of Amendment No. 1, a commenter stated that although “Nasdaq shared some of the 142 features of their formula,” Nasdaq should reveal all of these features so that prospective users may evaluate how the model works.[55]                                          Similarly, another commenter stated that the public cannot provide meaningful comment on the proposal without knowing all categories and parameters of the proposed Dynamic M-ELO.[56]                                          In its response to these comments, the Exchange, among other things, provided the specific 142 data elements that will be weighed by the machine-learning model as both an appendix to its first letter in response to comments,[57]                                          and as Exhibit 3B to its Amendment No. 1 filing.                

                    In response to the Exchange's disclosures in Exhibit 3B of Amendment No. 1, one of these commenters stated that the list of data elements was not “clear and comprehensible” as is required by the Form 19-4, but rather “vague, confusing, and perfunctory.” [58]                                          This commenter also stated that the disclosed data elements included unexplained terms (                    *e.g.,*                     “baseline simulated,” “action simulated,” and “synthetic mark-out”).[59]                                          In a subsequent comment letter, this commenter reiterated these points; the commenter specified that the commenter's concern is that Nasdaq's rule text does not disclose information about its methods for assessing market conditions and that “Nasdaq should carefully detail its methods in its rulebook, just like other exchanges have done, and Nasdaq should also thoroughly disclose its methods in its filing text.” [60]                                          In its second response to comments [61]                                          and the revised Exhibit 3b to Amendment No. 2, Nasdaq expanded and “simplified” the explanation of these 142 data elements. The Exchange also added to the proposed rule text a definition of the term “proprietary assessment of market conditions” to explain how the machine-learning model will evaluate those 142 data elements.[62] 

                    Furthermore, Nasdaq attached, as Exhibit 3A to its proposed rule change, the White Paper written by its AI Core Development Team that explains, among other things, how Dynamic M-ELO's machine-learning functions were developed and tested. The White Paper includes a general discussion of the type of model implemented in the proposed system, in this case a reinforcement learning model,[63]                                          as well as citations to academic research behind the Double Deep Q-Network algorithm that is the basis for the algorithm used in Nasdaq's model.[64]                                          The White Paper also describes the ways in which Nasdaq's implementation of the proposed model differs from the model and training in the academic research, providing both an English summary and a pseudocode description of differences in model training implemented by Nasdaq.[65] 

                    One of the commenters stated that the White Paper is not easily understood by most market participants and that referencing the White Paper in the filing is an “unacceptable substitute” for a “plain English” explanation of the proposal in Form 19b-4.[66]                                          In response, Nasdaq explained that it drafted the filing to provide a general understanding of Dynamic M-ELO and how it will behave, and the more detailed information and explanation in the White Paper are meant to support the filing.[67] 

                    The Commission agrees with comments and the Exchange that there is an extent to which the proposed changes will introduce an unavoidable degree of uncertainty with respect to the use of these order types. The deep reinforcement learning model that will determine the dynamic holding periods for each symbol for M-ELO and M-ELO+CB orders will be implemented through established, non-discretionary methods,[68]                                          but it is so complex that its complete details are, for most intents and purposes, not readily intelligible, and it would be immensely difficult for the Exchange or any market participant to precisely predict the holding periods that will be generated by the model for any given symbol at any particular time. Nevertheless, as further discussed below, the Commission believes that the Exchange has provided information sufficient for the Commission and public to understand the design, operation, and limits of the proposed changes to these order types, and the role of the machine-learning model therein.                

                    While the holding periods under the proposal would be dynamic, Nasdaq has precisely articulated both the nature of changes that would be permissible under the proposal, and the limits to those changes. Nasdaq described when changes might occur (every thirty seconds throughout the trading day), the initial default holding period for all symbols (1.25 milliseconds), the permissible increments by which a holding period might change in each symbol (0.25 or 0.50 milliseconds), and the outer bounds of permissible holding period lengths (0.25 milliseconds at the short end, and 2.50 milliseconds at the long end). Nasdaq also described the conditions of “extraordinary instability” in a symbol when these holding periods would not apply, and when the holding periods would be overridden by the proposed “stability protection mechanism” (with a holding period of 12 milliseconds for at least 750 milliseconds). The Commission believes that these details provide sufficient information to understand the range of potential holding periods that may be applied when M-ELO or M-ELO+CB orders are entered or resting on the order book, the changes that may occur, and the limits to those changes.                          ( print page 62859)    

                    Nasdaq has also described the role of the machine-learning model in its proposal. The model will determine whether, by which increment, and in which direction to adjust the holding period for each symbol throughout the trading day.[69]                                          In its Form 19b-4, White Paper, and response letters, Nasdaq described the goals towards which the model is optimized: reducing mark-outs and increasing fill rates.[70]                                          Nasdaq's White Paper includes a detailed discussion of model choice, development, and training, including citations to relevant other research.[71]                                          Nasdaq also provided several iterations of a list of data elements that the model will ingest and use, including a glossary defining terms used in the descriptions.[72]                                          Nasdaq affirmed that in operation during market hours, the data used would be calculated based on intraday market data.[73]                                          One version of these lists included Nasdaq's estimates of the tendencies of data elements to affect model outcomes.[74]                                          Nasdaq's White Paper also included an “explainability study” that assessed both the effects of individual data elements on model performance and the effects of interactions between individual data elements.[75]                                          Across its filing and incorporated exhibits, aspects of the model's operations and design are described in different formats and with different levels of specificity—for example, the filing and exhibits include “plain English” descriptions, mathematical definitions, and pseudocode. Together, this set of information allows the Commission to understand the type of decision the model will implement, the goals the model aims to achieve, which model type is implemented and how it was developed, the range of data types and data sources used by the model, and estimates of the manner in which those data may affect model outcomes.                

                    Nasdaq also explains how and when the machine-learning model will be retrained. Nasdaq will retrain the model weekly, outside of market hours. Retraining will incorporate market data obtained during the week from the equity consolidated data feeds and M-ELO order book. A retrained model will only be promoted to production if it improves upon the model objectives compared to the prevailing model.[76]                                          Furthermore, the Exchange explained that the machine-learning model is consistent in its behavior from day-to-day during periods when it is not undergoing retraining, “such that its decisions when presented with given set of facts and circumstances in a given security on day 1 should be the same as they would be on day 2.” [77]                                          The Exchange also stated in its initial response to comments that “[e]ven after the system undergoes retraining, which will occur on a weekly basis (and not during market hours), system behavior should not change dramatically or in unexpected ways from week-to-week.” [78]                                          As noted above as well, the Exchange also represents that outside of set retraining periods, “the System will operate pursuant to a mathematical algorithm from which it cannot deviate—an algorithm that is programmed to achieve pre-defined and pre-disclosed objectives.” [79]                                          Nasdaq also will publish equity trader alerts when it anticipates that a model update may change mark-outs or fill rate by 10% or more in either direction.[80]                                          By including this set of information, Nasdaq has provided the Commission and public with information that allows them to understand how frequently the model will be retrained, the data used for retraining, and the criteria that will be used to determine whether to update the production model based on retraining. This information allows the Commission to understand when the proposed model may change and when it will remain constant, the circumstances under which a change would be implemented, and circumstances under which the public will receive notice of significant changes in the model's anticipated outcomes.                

                    In addition, a commenter stated their belief that the Exchange's proposal for Dynamic M-ELO would result in the exercise of discretion by a national securities exchange because the machine-learning model's decisions would vary over time based on the following: (1) varying parameter values; and (2) results of retraining cycles.[81]                                          The commenter stated that by making it possible for Dynamic M-ELO to behave differently when confronted by the same market conditions before and after the model is retrained, Nasdaq's model would be exercising discretion that is more akin to a broker than an exchange.[82] 

                    The commenter claimed that Dynamic M-ELO would operate outside of established non-discretionary methods, which require “fully disclosed procedures operating in a strictly linear, invariant, and deterministic fashion.” [83]                                          Additionally, the commenter stated that Nasdaq would be exercising discretion with Dynamic M-ELO to alter a participant's material order terms.[84]                                          The commenter claimed that Nasdaq would be using undisclosed data such as the                           ( print page 62860)                        buyer and seller counts and recent trade sizes.[85]                                          Further, the commenter stated:                

that (a) by determining the universe of data the system consumes, (b) by programming how the system thinks, (c) by controlling and supplying the information with which it thinks, and (d) by setting the goals and programming the nature and extent of its actions, and when it does all this to determine (e) when and in which prescribed intervals to set an ever variable time-in-force term for an order, a term which (f) dictates when to expose an order to the market to find contra-side interest, then without question Nasdaq is exercising control, judgment, and discretion over its customer orders.

                    In Amendment No. 1, Nasdaq added language to address these concerns.[86]                                          Among other things, Nasdaq stated that to the extent that the design of Dynamic M-ELO permits variation in the Holding Periods for such orders, it does so by design, and the “mere fact that the System may apply different weights over time to the factors it uses to determine whether and by how much to vary a Holding Period does not mean that the System will act with discretion in the same sense that a human being could be said to be exercise independent judgment when deciding whether and how to handle an order.” [87]                                          Additionally, Nasdaq stated the following in its second response to comments:                

                        It is also worth noting that presently, exchanges like Nasdaq already employ non-linear, non-deterministic functionalities, like the randomized timers it uses to resolve certain unavoidable race conditions that arise in the order handling process. Nasdaq employs these functionalities with the knowledge of the SEC, and without any suggestion that they somehow transform Nasdaq into a broker.[88] 

                    Furthermore, as noted above, the Exchange represents that outside of set retraining periods, “the System will operate pursuant to a mathematical algorithm from which it cannot deviate—an algorithm that is programmed to achieve pre-defined and pre-disclosed objectives.” [89]                                          The Exchange explains that outside of the set retraining periods “the System will behave predictably from day to day, such that its decisions when presented with given set of facts and circumstances in a given security on day 1 should be the same as they would be on day 2.” [90] 

                    Based on Nasdaq's representations described above, Dynamic M-ELO would operate pursuant to pre-determined, programmed procedures that would dictate order interaction and the terms for trading for each Dynamic M-ELO order entered on the Nasdaq trading facility. While the Exchange's procedures include conditions that, if satisfied under certain circumstances, might result in different outcomes for different M-ELO orders, such conditions and circumstances, if pre-determined, pre-defined, and programmed into the Exchange's trading facility, would be considered established and not discretionary. For example, according to the Exchange, Dynamic M-ELO may apply different pre-determined weights over time to pre-determined factors it uses to determine whether and by how much to vary a Holding Period.[91]                                          In such an event, Dynamic M-ELO will operate pursuant to pre-determined procedures and programmed mathematical algorithm from which it cannot deviate to “achieve pre-defined and pre-disclosed objectives.” [92]                                          Further, the procedures governing Dynamic M-ELO and use of M-ELO orders will be established before the beginning of each trading day. For example, Dynamic M-ELO will use preset methods to evaluate and weigh specific data elements to determine the dynamic holding periods. Such pre-set methods will be established during the prior retraining period, and outside regular trading hours, and will not vary intra-day until adjusted at the next retraining period.                

Given the pre-determined, programmed procedures and rules that Nasdaq has proposed to dictate trading for Dynamic M-ELO, the Commission does not believe that Dynamic M-ELO is designed to provide Nasdaq with judgement and flexibility, and therefore, discretion over the handling or execution of a M-ELO order entered on the Exchange.

### Unfair Discrimination

                    Section 6(b)(5) of the Act requires that the rules of a national securities exchange not be designed to permit unfair discrimination between customers, issuers, brokers, or dealers. In several letters, one commenter stated that Nasdaq inadequately explains how it will monitor and, if necessary, adjust Dynamic M-ELO to ensure no unfair discrimination.[93]                                          Initially, this commenter emphasized what they perceived to be silence on the part of Nasdaq with regard to whether Dynamic M-ELO will discriminate among categories of participant types.[94]                                          In its response to these comments, Nasdaq initially added a new representation to the filing in Amendment No. 1, stating that that Dynamic M-ELO is not designed to further the performance of any participant or any category of participant, but instead has twin objectives—the absolute values of mark-outs and fill rates. In Amendment No. 2, Nasdaq expanded on this representation by adding the following:                

                        Furthermore, Nasdaq performed internal tests of its AI model to detect indications of harmful bias in its performance results, and such tests concluded that no such indications exist. That is, the Exchange reviewed the impact on fill rates and mark-outs of Dynamic M-ELO, as compared to the “static” M-ELO, for those firms that accounted for more than 95% of M-ELO activity on the Exchange during Q1 2022 . . . . The Exchange analyzed results both in an absolute and a relative sense. Testing revealed that all participants experienced at least some improvements in fill rates and mark-outs when using Dynamic M-ELO versus static M-ELO, with the volume-weighted average improvement being aligned with the results expressed in the White Paper. We detected no material variations that might suggest that a particular participant or category of participant (                        *i.e.,*                         nature of firm; size of firm) benefitted from Dynamic M-ELO functionality to an extent that was unreasonably disproportionate to the benefits that other participants experienced. Thus, Nasdaq believes the model is objective, is designed to, and does avoid bias and discrimination.                    

                    In Amendment No. 2, Nasdaq also affirmed that it will periodically review its model to ensure that it continues to perform in accordance with the Exchange's rules and that it has not introduced any harmful bias in favor of or against any participant or class of participants.[95] 

                    In response to the above, the commenter submitted a fourth comment letter, in which they questioned the approach Nasdaq took to demonstrate that there is not bias against any one                           ( print page 62861)                        participant or class of participants.[96]                                          The commenter, among other things, expressed concern about Nasdaq conducting its analysis using data for firms that accounted for 95% of M-ELO activity during Q1 of 2022 rather than all M-ELO activity. The commenter states that Nasdaq did not describe how it determined the 5% of activity during that period to exclude from its analysis.[97]                                          For example, the commenter states that it is not clear whether Nasdaq excluded firms with large orders and trades, and the commenter opines that discarding any data could exclude activity that has qualitative or quantitative differences from the rest.[98] 

                    In response to this comment, Nasdaq represented that it conducted a supplemental analysis of the initially-excluded data—which were the activity of the least-active M-ELO firms from the control period of its initial analysis—to confirm whether its initial conclusions held for those participants.[99]                                          Nasdaq explains that the individual variations among the previously excluded participants was higher than that for the original batch of data, but that, based on simulated data, each of these participants would have experienced the same or better fill rates during the testing period if they had utilized Dynamic M-ELO.[100]                                          Based on this supplemental data analysis, Nasdaq concluded that there is no apparent biases for the Dynamic M-ELO, even among the least active M-ELO participants.[101] 

The Commission concludes that Nasdaq has adequately demonstrated that the proposal is not designed to permit unfair discrimination consistent with Section 6(b)(5) of the Act. Through the White Paper, amendments, and response letters, Nasdaq has demonstrated that it has analyzed the anticipated or simulated effects of the proposed change on all current M-ELO users, and that this work did not indicate that particular firms or classes of firms are anticipated to unfairly benefit from or be harmed by the proposed Dynamic M-ELO functionality.

### Prevention of Fraudulent and Manipulative Acts and Practices, Just and Equitable Principles of Trade, and the Protection of Investors and the Public Interest

Section 6(b)(5) of the Act also requires that the rules of a national securities exchange be designed to prevent fraudulent and manipulative acts and practices, to promote just and equitable principles of trade, and, in general, to protect investors and the public interest. In Amendment No. 2, Nasdaq addressed whether the Dynamic M-ELO is designed to prevent fraudulent and manipulative acts and practices. Nasdaq states that the design of Dynamic M-ELO does not lend itself to potential manipulation by a single participant or a small group of participants because the machine-learning model makes determinations regarding Holding Periods based upon prevailing market-wide conditions for a given symbol, rather than the behaviors of particular participants with respect to that symbol, or the activity of participants in M-ELOs involving that symbol. Nasdaq further states that manipulation of the machine-learning model would be difficult to accomplish given the large number of variables that factor into the machine-learning model's decisions to change Holding Periods during Change Events, as well as the different weights that apply to each such factor, which as described above, may vary over time. Furthermore, Nasdaq states that any benefits that a participant might derive from manipulating the duration of Holding Periods would likely be small and outweighed significantly by the difficulty and cost of effecting such manipulation.

                    The Exchange, in Amendment No. 2, also sets forth representations regarding how it will surveil its market after Dynamic M-ELO is implemented. First, Nasdaq represents that it will review the machine-learning functionality and operation periodically to affirm that it continues to perform in accordance with the Exchange's rules and has not introduced any harmful bias in favor of or against any participant or category of participants.[102]                                          Nasdaq also represents above that it will surveil for indications of manipulation and act accordingly if it detects such indications.                

                    The Commission finds that the proposed rule change is designed to prevent fraudulent and manipulative acts and practices consistent with Section 6(b)(5) of the Act. The Commission agrees that aspects of the Dynamic M-ELO design reduce opportunities for manipulation or are likely to make manipulation costly or difficult. The model's operation depends on 142 data elements, which are each likely to have effects on model outcomes of differing magnitudes and in different directions. Many of these data elements are also based on market-wide data, in some cases spanning periods of days,[103]                                          which are likely themselves difficult for market participants to manipulate. Given these design features, it appears likely that manipulating the duration of Dynamic M-ELO holding periods in any given symbol or group of symbols would be an extremely complex undertaking. In light of this complexity, and the size of M-ELO activity relative to the market for NMS stocks,[104]                                          Nasdaq's assertion that the potential benefits of manipulating the dynamic holding periods for these order types would be outweighed by the cost and complexity of manipulation also appears reasonable. Nasdaq has also represented that it intends to surveil the proposed order types for manipulation. This ongoing surveillance, to ensure the appropriate use of Dynamic M-ELO by Exchange Members and behavior by the machine-learning model, is important to the successful implementation of Dynamic M-ELO and appears appropriately tailored to the accomplish the intent of the M-ELO and M-ELO+CB order types.                

                    Furthermore, the Commission finds that overall structure of Dynamic M-ELO—particularly, the static numerical constraints set forth in the proposed rule text—is designed in general, to protect investors and the public interest and promote just and equitable principles of trade pursuant to Section 6(b)(5) of the Act. As described above, the model will continuously engage in dynamic analysis of current market conditions during trading hours, and outside of market hours, it will retrain with the goal of improving the overall performance of Dynamic M-ELO. These dynamic aspects of the proposal,                           ( print page 62862)                        however, are constrained by the static numerical thresholds set forth in the proposed rule text. For example, the initial Holding Periods for each trading day will be 1.25 milliseconds, the overall range for any Holding Period must be between 0.25 and 2.50 milliseconds during normal market conditions, and the Holding Period can only change by either 0.25 or 0.50 milliseconds at each Change Event during normal market conditions. Regardless of how the model analyzes the current market or changes the weighting of the data elements as a result of its retraining, Dynamic M-ELO cannot operate outside of the static numerical ranges and limitations or minimums set forth in the rule text. As such, the Commission finds that Nasdaq has designed Dynamic M-ELO to operate in a manner that in general protects investors and the public interest and promotes just and equitable principles of trade in accordance with Section 6(b)(5) of the Act.                

### Compliance With SRO Recordkeeping and Reporting Obligations

                    One commenter queried whether Nasdaq could maintain an adequate audit trail given the potential for frequently shifting Holding Periods for Dynamic M-ELO.[105]                                          In response, Nasdaq states that it will retain copies of each iteration of its system as part of its books and records and will disclose publicly statistics relating to Dynamic M-ELO performance.[106]                                          Nasdaq additionally represented that it will publish weekly and monthly Dynamic M-ELO performance statistics, which would include the weekly numbers of shares and trades in M-ELOs by symbol, weekly aggregated M-ELO share and trade data, and monthly aggregated block data, on Nasdaqtrader.com.[107]                                          Nasdaq also indicated it would add statistics to its existing M-ELO Monthly Report, which discloses quote stability by time horizon, about how frequently, on average, its system changes Holding Period durations for the top decile, median, and bottom decile of symbols, as measured by monthly M-ELO and M-ELO+CB trading volumes.[108] 

                    Nasdaq also added a representation to the filing, addressing how it would comply with its recordkeeping obligations.[109]                                          Nasdaq states that it will retain copies of each historical iteration of its models as part of its books and records, and make them available to the Commission upon request, should it wish to examine them to understand how the model changes over time.[110]                                          Nasdaq also states that it will publish an equity trader alert in advance of deploying a retrained version of Dynamic M-ELO when Nasdaq anticipates the retrained version will produce results that differ materially from the prior version.[111]                                          Based on these representations, the Commission finds that Nasdaq has met its burden to demonstrate that it will comply with all relevant exchange recordkeeping requirements and obligations when it implements Dynamic M-ELO. In addition, the Commission notes that Nasdaq must comply with its reporting obligations under Rule 613 of Regulation NMS [112]                                          and the National Market System Plan Governing the Consolidated Audited Trail (“CAT NMS Plan”) [113]                                          with respect to Dynamic M-ELO, which requires it to record and electronically report to the central repository the material terms of each order and each reportable event.[114] 

### Nasdaq's Obligation To File Proposed Rule Changes Relating to Dynamic M-ELO

                    Prior to the filing of Amendment No. 1, a commenter stated that it was unclear what types of changes to the model would lead Nasdaq to seek approval from the Commission via an SRO rule filing.[115]                                          As explained above,[116]                                          Nasdaq represents that it will not modify the underlying structure of Dynamic M-ELO without first obtaining the Commission's approval to do so, including modifications to the data elements the model considers in making decisions about Holding Period durations, the conditions under which the model may adjust the duration of Holding Periods, the frequency with which the model may adjust the Holding Periods, the range of Holding Period durations available to M-ELOs and M-ELO+CBs, the increments by which Holding Periods may change at any given Change Event, and the procedures for triggering, maintaining, and ending 12 millisecond Holding Periods during times of extraordinary instability. In contrast, the Exchange states that it will not seek Commission approval prior to retraining the model to adjust the weighting it applies to those data elements pursuant to the weekly retraining process.                

                    Section 19(b)(1) of the Act [117]                                          and Rule 19b-4 thereunder [118]                                          require an SRO to file a proposed rule change with the Commission whenever it seeks any proposed change in, addition to, or deletion from the rules governing the SRO and its members' activities on the SRO. As discussed above, the proposal sets forth the specific data elements that Dynamic M-ELO will use during the trading day. Furthermore, the proposed rule change sets forth when the machine-learning model will retrain and the extent to which the retraining can and cannot cause the machine-learning model to update Dynamic M-ELO's operation during subsequent trading days.[119]                                          In addition, the proposal sets forth the operation of Dynamic M-ELO, such as the potential range for a Holding Period, how often Dynamic M-ELO reevaluates market conditions for a given security to adjust a Holding Period, and the increment by which a Holding Period may be changed. Nasdaq represents that it will not change any of these aspects of the proposal or any other function of Dynamic M-ELO without first filing a proposed rule change.[120]                                          Nasdaq does, however, state that it would not file a proposed rule change in connection with the operation of the machine-learning model's weekly retraining and the results of that process.                

                    Based on the foregoing, the Commission believes that Nasdaq has adequately responded to the commenter's concern. Nasdaq will need to file a proposed rule to make any changes, additions, or deletions to the operation of Dynamic M-ELO as approved herein. Nasdaq has delineated when it would file a proposed rule change to alter the operation of Dynamic M-ELO, and when the machine-learning model would retrain and adjust the weighting it applies to the data elements without it filing a proposed rule change. Specifically, Nasdaq's proposed rule change and rule text reflect the 142 data elements Dynamic M-ELO will consider when determining the Holding Period for a security and the goals Nasdaq will consider when weighing those data elements (                    *i.e.,*                     reducing mark-outs and increasing fill rates) but does not set forth the relative weighting of each those individual data elements. Though                           ( print page 62863)                        the structure of the proposal does not disclose of the exact weighting for each of the 142 data elements, it does set forth the two goals Nasdaq will consider when weighing those data elements initially and during each weekly retraining, which provides information as to how those 142 factors will be used in determining the Holding Period for a security. Based on how the proposed rule sets forth the goals that will govern each retraining, the Commission believes that Nasdaq's delineation of when it would and would not file a proposed rule change to alter the operation of Dynamic M-ELO is consistent with Nasdaq's rule filing obligation. The Commission agrees that the weekly retraining to optimize the weighting of the 142 data elements considered by Dynamic M-ELO to best achieve those goals within the rule's parameters would not necessitate the filing of a proposed rule change with the Commission because those adjustments would be reasonably and fairly implied by the proposed rule. However, to the extent Nasdaq seeks to change, add to, or delete from the rule's construct in connection with the weekly retraining, it would first be required to file a proposed rule change with the Commission.                

## V. Solicitation of Comments on Amendment No. 2 to the Proposed Rule Change

Interested persons are invited to submit written data, views, and arguments concerning whether Amendment No. 2 is consistent with the Act. Comments may be submitted by any of the following methods:

### Electronic Comments

    - Use the Commission's internet comment form ( https://www.sec.gov/​rules/​sro.shtml ); or                
    - Send an email to rule-comments@sec.gov.                      Please include file number SR-NASDAQ-2022-079 on the subject line.                

### Paper Comments

    - Send paper comments in triplicate to Secretary, Securities and Exchange Commission, 100 F Street NE, Washington, DC 20549-1090.

                    All submissions should refer to file number SR-NASDAQ-2022-079. This file number should be included on the subject line if email is used. To help the Commission process and review your comments more efficiently, please use only one method. The Commission will post all comments on the Commission's internet website (                    *https://www.sec.gov/​rules/​sro.shtml*). Copies of the submission, all subsequent amendments, all written statements with respect to the proposed rule change that are filed with the Commission, and all written communications relating to the proposed rule change between the Commission and any person, other than those that may be withheld from the public in accordance with the provisions of 5 U.S.C. 552, will be available for website viewing and printing in the Commission's Public Reference Room, 100 F Street NE, Washington, DC 20549 on official business days between the hours of 10:00 a.m. and 3:00 p.m. Copies of the filing also will be available for inspection and copying at the principal office of the Exchange. Do not include personal identifiable information in submissions; you should submit only information that you wish to make available publicly. We may redact in part or withhold entirely from publication submitted material that is obscene or subject to copyright protection. All submissions should refer to File Number SR-NASDAQ-2022-079, and should be submitted on or before October 4, 2023.                

## VI. Accelerated Approval of Proposed Rule Change, as Modified by Amendment No. 2

                    The Commission finds good cause to approve the proposed rule change prior to the 30th day after the date of publication of Amendment No. 2 in the                     **Federal Register**                    . Amendment No. 2 does not include any material changes to the operation of the proposed Dynamic M-ELO and its machine-learning model. In Amendment No. 2, the Exchange: (1) adds the defined term “proprietary assessment of market conditions” to the proposed rule text, which consolidates certain details and explanations about how the machine-learning model would operate from prior versions into a single defined term; (2) revises the list of factors provided in Exhibit 3b to include expanded and “simplified” explanations of the terminology used therein; (3) adds a representation that the systems used to operate Dynamic M-ELO and machine-learning model are “SCI Systems” and thus subject to compliance with Regulation SCI; and (4) expands the legal analysis to address comments regarding unfair discrimination and the exercise of impermissible discretion by the Exchange.                

                    The Commission finds that Amendment No. 2 raises no novel regulatory issues that have not previously been subject to comment and is reasonably designed to prevent fraudulent and manipulative acts and practices, to promote just and equitable principles of trade, and, in general, to protect investors and the public interest, and not be unfairly discriminatory, or impose an unnecessary or inappropriate burden on competition. Amendment No. 2 does not alter the proposed operation or any material features of Dynamic M-ELO, which operation and features have been subject to two rounds of public comment. In response to public comment, the revisions to the proposal contained within Amendment No. 2 provide additional clarification and details regarding how Dynamic M-ELO and the machine-learning model will operate, as well as additional legal analysis to support the Exchange's position that the proposal is consistent with the Act. Accordingly, pursuant to Section 19(b)(2) of the Act,[121]                                          the Commission finds good cause to approve the proposed rule change on an accelerated basis prior to the 30th day after publication of notice of the filing of Amendment No. 2 in the                     **Federal Register**                    .                

## VII. Conclusion

                    It is therefore ordered, pursuant to Section 19(b)(2) of the Act,[122]                                          that the proposed rule change (SR-NASDAQ-2022-079), as modified by Amendment No. 2, be, and it hereby is, approved on an accelerated basis.                

                        For the Commission, by the Division of Trading and Markets, pursuant to delegated authority.[123] 

Sherry R. Haywood,

Assistant Secretary.

# Footnotes

1.                             15 U.S.C. 78s(b)(1).                    
        Back to Citation      

2.                             17 CFR 240.19b-4.                    
        Back to Citation      

3.                                                     *See*                         Securities Exchange Act Release No. 92844 (January 4, 2023), 88 FR 1438.                    
        Back to Citation      

4.                             15 U.S.C. 78s(b)(2).                    
        Back to Citation      

5.                                                     *See*                         Securities Exchange Act Release No. 96963, 88 FR 12710 (February 28, 2023).                    
        Back to Citation      

6.                                                     *See*                         Securities Exchange Act Release No. 97263, 88 FR 22498 (April 13, 2023).                    
        Back to Citation      

7.                             15 U.S.C. 78s(b)(2).                    
        Back to Citation      

8.                                                     *See*                         Securities Exchange Act Release No. 97844, 88 FR 44423 (July 12, 2023).                    
        Back to Citation      

9.                             All comments received by the Commission on the proposed rule change are available on the Commission's website at:                         *https://www.sec.gov/​comments/​sr-nasdaq-2022-079/​srnasdaq2022079.htm.*
        Back to Citation      

10.                                                     *See*                         SR-Nasdaq-2022-079 Amendment No. 1 (March 9, 2023), at                         *https://www.sec.gov/​comments/​sr-nasdaq2022-079/​srnasdaq2022079-20159016-327215.pdf.*
        Back to Citation      

11.                                                     *See*                         Securities Exchange Act Release No. 34-82825 (March 7, 2018), 83 FR 10937 (March 13, 2018) (SR-NASDAQ-2017-074) (“M-ELO Approval Order”).                    
        Back to Citation      

12.                                                     *See*                         Securities Exchange Act Release No. 34-86938 (September 11, 2019), 84 FR 48978 (September 17, 2019) (SR-NASDAQ-2019-048) (“M-ELO+CB Approval Order”).                    
        Back to Citation      

13.                                                     *See*                         Securities Exchange Act Release No. 34-88743 (April 24, 2020), 85 FR 24068 (April 30, 2020) (SR-NASDAQ-2020-011) (“M-ELO Timer Approval Order”).                    
        Back to Citation      

14.                             The Exchange examined each of its historical M-ELO executions to determine at what Midpoints of the NBBO the M-ELOs would have executed if their Holding Periods had been shorter than one-half second (500 milliseconds). After examining the historical effects of shorter Holding Periods of between 10 milliseconds and 400 milliseconds, the Exchange determined that a reduction of the M-ELO Holding Period to as short as 10 milliseconds would have caused an average impact on mark-outs of only 0.10 basis points (across all symbols). In other words, compared to the execution price of an average M-ELO with a one-half second Holding Period, the Exchange found that a M-ELO with a 10 millisecond Holding Period would have had an average post-execution impact that was only a tenth of a basis point per share—a difference in protective effect that is immaterial.                         *See*                         Nasdaq, “The Midpoint Extended Life Order (M-ELO); M-ELO Holding Period,” available at                         *https://www.nasdaq.com/​articles/​the-midpoint-extended-life-order-m-elo%3A-m-elo-holding-period-2020-02-13*                         (analyzing effects of shortened Holding Periods on M-ELO performance).                    
        Back to Citation      

15.                             M-ELO Timer Approval Order,                         *supra,*                         at 85 FR 24069.                    
        Back to Citation      

16.                                                     *See*                         Diana Kafkes et al., “Applying Artificial Intelligence & Reinforcement Learning Methods Towards Improving Execution Outcomes,” SSRN, October 19, 2022, available at                         *https://papers.ssrn.com/​sol3/​papers.cfm?​abstract_​id=​4243985*                         (attached hereto [sic] without modification from the prior version as Exhibit 3(a)) (the “White Paper”).                    
        Back to Citation      

17.                             Although the AI Core Development Group acknowledges that an optimal Holding Period would update with every incoming order, it determined that training a reinforcement learning model on every order would be too difficult to program and too difficult to implement given the nanosecond latency requirements of the Exchange. The Group then investigated more feasible update cadences and determined the point at which optimal outcomes were best balanced with the level of programming and implementation difficulty to be between 15 and 30 second updates. Ultimately, the Group chose a 30 second update cadence to give the model the greatest opportunity to learn between potential actions.                    
        Back to Citation      

18.                             As the White Paper explains, the Group developed a model to simulate activity on the Exchange involving M-ELOs and M-ELO+CBs during the Training Period.                         *See*                         White Paper,                         *supra,*                         at 10.                    
        Back to Citation      

19.                                                     *See id.*
        Back to Citation      

20.                             The AI Core Development Group experimented with a range of permissible Holding Period durations. Ultimately, it concluded that it could produce better outcomes for M-ELO and M-ELO+CB participants than the existing approach using Holding Periods as low as 0.25 milliseconds and as high as 2.5 milliseconds, under normal market conditions.                    
        Back to Citation      

21.                             Nasdaq attaches a full list of these data elements (attached hereto [sic] as “                        *Exhibit 3(b)*)”, along with an observation of the strength of the correlations that currently exist between changes to those data values and decisions the system makes to set the duration of Holding Periods at any given time. The Exchange notes that the version of this list attached to this Amendment No. 2 supersedes prior versions attached to prior versions of this filing. This version of the list includes expanded explanations of the terminology used therein.                         *See also*                         White Paper,                         *supra,*                         at 31, for a description of these features.                    
        Back to Citation      

22.                             The AI Core Development Group also applied to the model a paradigm called “retraining” to combat the degradation of model performance that can otherwise occur as the reference data it uses for initial comparison becomes stale. Finally, the AI Core Development group added a stability protection mechanism to the model to provide maximum production to participants in the event that the model observes extraordinary levels of instability in the National Best Bid and Offer during the prior three seconds as compared to reference data. When the model detects such instability, it is                                                 programmed to increase the length of the Holding Period to 12 milliseconds for a period of 750 milliseconds.                    
        Back to Citation      

23.                                                     *See*                         White Paper,                         *supra,*                         at 22.                    
        Back to Citation      

24.                                                     *See id.*
        Back to Citation      

25.                             As set forth in the proposed rule text, the phrase “proprietary assessment of market conditions” refers to the Exchange's evaluation of prevailing market conditions for a given symbol using an algorithm programmed to set a Holding Period duration which, at each Change Event, achieves an optimal blend of two objectives: maximization of M-ELO fill rates; and minimization of M-ELO mark-out rates. As the rule text states and as is discussed below, the algorithm ingests and analyzes 142 data points, which the Exchange identifies and describes in Exhibit 3b hereto. The Exchange derives these data from a combination of public data and M-ELO data feeds. Furthermore, the Exchange conducts weekly re-trainings of the algorithm, outside of Market Hours, to improve its performance relative to the immediately preceding period (in terms of the two aforementioned objectives). The Exchange deploys a retrained version of the algorithm only if it determines that doing so will, in fact, improve its performance relative to the immediately preceding period. The Exchange provides further information about the algorithm and the retraining process in a White Paper attached hereto [sic] as Exhibit 3a.                        **
        Back to Citation      

26.                             For purposes of this Rule, the System determines that “extraordinary instability” for a symbol exists through observations it makes following every change in the NBBO for that symbol that occurs during the trading day. When the NBBO changes, the System looks back at the prior three seconds of trading and measures the difference between the highest and the lowest NBBO midpoint values that occurred during that period, and then it compares that measurement to a threshold value for the symbol. The System concludes that extraordinary instability exists for a symbol if the measurement exceeds the threshold value. The threshold value for a symbol, in turn, is the difference between the highest and the lowest NBBO midpoint values for the symbol that, if applied to its trading activity during the prior trading day, would have caused the System to deem trading in the symbol to be extraordinarily unstable for as close to one percent of that day as possible.                    
        Back to Citation      

27.                             Prior to commencement of a new 12 millisecond Holding Period for a new or pending M-ELO or M-ELO+CB following a Halt, the System will first determine whether the M-ELO or M-ELO+CB is or remains eligible for execution. That is, the Holding Period will commence only if, upon commencement of trading following the Halt, the midpoint price for the Order is within the limit set by the participant. If not, the System will hold the Order until the midpoint falls within the limit set by the participant, at which time the 12 millisecond Holding Period will commence.                    
        Back to Citation      

28.                             Also as a safeguard, the System will apply a default Holding Period of 12 milliseconds to a M-ELO or M-ELO+CB if ever it fails to receive a signal during a Change Event as to whether the System should adjust or maintain the duration of the prevailing Holding Period. The System will continue to apply the default 12 millisecond Holding Period until the next Change Event where the signal is restored and the System is able to act dynamically again.                    
        Back to Citation      

29.                             During periods where the model is not undergoing retraining, the System will behave predictably from day to day, such that its decisions when presented with given set of facts and circumstances in a given security on day 1 should be the same as they would be on day 2.                    
        Back to Citation      

30.                             In addition to the proposed changes described above, the Exchange proposes to delete an extraneous reference in Rule 4702(b)(15) to M-ELO+CB being eligible to execute against a Midpoint Order on the Continuous Book if the Continuous Book order has the “Midpoint” Trade Now Attribute enabled. In a prior filing, the Exchange folded the concept of “Midpoint Trade Now” into the general “Trade Now” Attribute.                         *See*                         Securities Exchange Act Release No. 34-92180 (June 15, 2021), 86 FR 33420 (June 24, 2021)(SR-NASDAQ-2021-044).                    
        Back to Citation      

31.                                                     *See http://www.nasdaqtrader.com/​Trader.aspx?​id=​MELOSymbolData.*
        Back to Citation      

32.                                                     *See, e.g.,**https://www.nasdaq.com/​docs/​M-ELO-Monthly-Report.*                         Nasdaq understands that current users of M-ELO and M-ELO independently monitor the performance of these Order Types. Nasdaq often receives feedback from such users about M-ELO and M-ELO+CB performance, which Nasdaq then factors into decisions about improvements and enhancements. Nasdaq expects that this feedback loop will continue after implementation of Dynamic M-ELO.                    
        Back to Citation      

33.                             17 CFR 242.1000*et seq.*                         As set forth in Reg. SCI, the term “SCI Systems” means “means all computer, network, electronic, technical, automated, or similar systems of, or operated by or on behalf of, an SCI entity that, with respect to securities, directly support trading, clearance and settlement, order routing, market data, market regulation, or market surveillance.”                         *Id.*                         at 242.1000. An “SCI Entity” means “an SCI self-regulatory organization, SCI alternative trading system, plan processor, exempt clearing agency subject to ARP, or SCI competing consolidator.”                         *Id.*
        Back to Citation      

34.                             15 U.S.C. 78f(b).                    
        Back to Citation      

35.                             15 U.S.C. 78f(b)(5).                    
        Back to Citation      

36.                             M-ELO Approval Order,                         *supra*                         83 FR at 10938-39; M-ELO+CB Approval Order,                         *supra,*                         84 FR at 48980.                    
        Back to Citation      

37.                                                     *See*                         note 6,                         *supra.*
        Back to Citation      

38.                             To be clear, performance statistics for Dynamic M-ELO cited herein and in the White Paper are based upon data derived from weekly, not daily retrainings.                    
        Back to Citation      

39.                             As discussed above, Nasdaq will not seek Commission approval prior to allowing the model, as part of its re-training process, to vary the weighting of the data elements it ingests. Nasdaq believes this is appropriate because such variance will only occur to the extent that it will improve the model's performance with respect to pre-defined objectives. Nasdaq will alert traders if the retraining process would result in substantial performance changes, and it will also publish statistics to help participants to assess performance themselves. Moreover, Nasdaq will retain historical iterations of its models for the Commission's review, should it wish to examine how these models have changed over time.                    
        Back to Citation      

40.                             Beyond this grouping of participants, the activity levels of other individual M-ELO participants were so small as to be insignificant. In many cases, these participants entered only a handful of M-ELOs during the study period. As such, the Exchange believes it is reasonable to exclude such participants from its analysis to avoid their data distorting the results.                    
        Back to Citation      

41.                             The Exchange will review its AI model periodically to affirm that it continues to perform in accordance with the Exchange's rules and has not introduced any harmful bias in favor of or against any participant or category of participants.                    
        Back to Citation      

42.                                                     *See*17 CFR 240.3b-16(a)(2) (“(a) An organization, association, or group of persons shall be considered to constitute, maintain, or provide `a market place or facilities for bringing together purchasers and sellers of securities or for otherwise performing with respect to securities the functions commonly performed by a stock exchange,' as those terms are used in section 3(a)(1) of the Act, (15 U.S.C. 78c(a)(1)), if such organization, association, or group of persons: (1) Brings together the orders for securities of multiple buyers and sellers; and (2) Uses established, non-discretionary methods (whether by providing a trading facility or by setting rules) under which such orders interact with each other, and the buyers and sellers entering such orders agree to the terms of a trade.”).                    
        Back to Citation      

43.                                                             *See*                             Securities Exchange Act Release No. 40760 (December 8, 1998), 63 FR 70844, 70850 (December 22, 1998).                        
        Back to Citation      

44.                                                     *See id.*                         at 70900 (“an essential indication of the non-discretionary status of rules and procedures is that those rules and procedures are communicated to the systems users” and “[t]hus, participants have an expectation regarding the manner of execution—that is, if an order is entered, it will be executed in accordance with those procedures and not at the discretion of a counterparty or intermediary.”).                    
        Back to Citation      

45.                                                     *Cf. id.*                         at 70851 (explaining that a traditional block trading desk is an example of a system that does not use established, non-discretionary methods because the operators of such desks do not act according to fixed procedures known to their customers, but instead shop orders around for potential counterparties and make their own determinations as to whether and how to execute block orders, including by sometimes deciding to take a proprietary position in part of the block order).                    
        Back to Citation      

46.                                                     *See id.*                         at 80755 (describing an example of a system that would be non-discretionary in nature: “System I permits participants to enter a range of ranked contingent buy and sell orders at which they are willing to trade securities. These orders are matched based on a mathematical algorithm whose priorities are designed to achieve the participants' objectives. System I does not display orders to any participants. System I is included under Rule 3b-16.”);                         *see also*                         Securities Exchange Act Release No. 34-89686 (August 20, 2020), 85 FR 54438, at 54445, n.92 (September 1, 2020) (Order approving SR-IEX-2019-15) (rejecting argument that IEX's D-Limit order time is an exercise of discretion because “D-Limit orders will not allow IEX to exercise any discretion on any particular order by deviating from the CQI and D-Limit functionality, which is hardcoded in the IEX rulebook.”; Securities Exchange Act Release No. 34-78101 (June 17, 2016), 81 FR 41141, at 41153(June 17, 2016) (Order approving IEX Form 1 and D-Peg Order Type) (“the Commission does not believe that the hardcoded conditionality of the IEX proposed “discretionary” peg order type provides IEX with actual discretion or the ability to exercise individualized judgment when executing an order. Rather, if IEX's fixed formula determines the quote to be stable, the discretionary peg order can execute up to the midpoint; if it does not deem the quote to be stable, then it will hold the order to its pegged price. As such, IEX would not exercise discretion over the routing and execution of a resting order”). Nasdaq does not believe that it is necessary to codify its mathematical formula for Dynamic M-ELO in its Rules because Nasdaq has disclosed sufficient information in its Rules and in its filing to inform the public as to the possible and expected behaviors associated with Dynamic M-ELO, as well as a means for the Commission and/or investors to verify whether Dynamic M-ELO is performing appropriately. Much as the Commission does not                                                 require an exchange to codify the source code it uses to effectuate other behaviors or actions that it explains in its Rules, including the behaviors of other complex Order Types, there is no basis to require codification of the Dynamic M-ELO formula in this instance.                    
        Back to Citation      

47.                                                     *See*                         White Paper,                         *supra.*
        Back to Citation      

48.                             In approving this proposed rule change, the Commission has considered the proposed rule's impact on efficiency, competition, and capital formation.                         *See*15 U.S.C. 78c(f). One commenter questioned whether Nasdaq's pending patent applications for the systems it will use to operate Dynamic M-ELO imposes an unnecessary or inappropriate burden on competition.                         *See Letter from R.T. L*                        euchtkafer, dated January 21, 2023 (“Leuchtkafer Letter 1”), at 3. The Commission does not believe that the sole fact that Nasdaq has a pending patent application for the technology it has developed to operate the Dynamic M-ELO is indicative that the operation of Dynamic M-ELO on the Exchange would place an inappropriate burden on competition. As explained below, Nasdaq has provided sufficient public disclosure and analysis to explain how Dynamic M-ELO will operate.                    
        Back to Citation      

49.                             In addition to providing a statutory analysis in its filing, Nasdaq also acknowledges, above in Amendment No. 2, that the systems it will use to implement Dynamic M-ELO, including the Exchange's model development and retraining processes, are SCI systems under Regulation SCI,                         *see*17 CFR 242.1000*et seq.,*                         and thus, it will be responsible for compliance with Regulation SCI with respect to Dynamic M-ELO, including having appropriate policies and procedures.                         *See supra*                         note 33 and accompanying text.                    
        Back to Citation      

50.                             15 U.S.C. 78f(b)(5).                    
        Back to Citation      

51.                             15 U.S.C. 78f(b)(8).                    
        Back to Citation      

52.                                                     *See*17 CFR 201.700(b)(3).                    
        Back to Citation      

53.                             15 U.S.C 78s(b)(1).                    
        Back to Citation      

54.                                                     *See*                         Leuchtkafer Letter 1,                         *supra*                         note 48, at 1-2.                         *See also*                         Letter from Joseph Saluzzi, Partner, Themis LLC, dated January 25, 2023, at 3 (“Themis Letter”) (questioning whether the complexity of Dynamic M-ELO is necessary).                    
        Back to Citation      

55.                                                     *See*                         Themis Letter,                         *supra*                         note 54, at 2.                    
        Back to Citation      

56.                                                     *See*                         Leuchtkafer Letter 1,                         *supra*                         note 48, at 1.                    
        Back to Citation      

57.                                                     *See*                         Letter from Brett Kitt, Associate Vice President and Principal Associate General Counsel, Nasdaq, Inc., dated March 9, 2023, at Appendix A (“Nasdaq First Response to Comments”).                    
        Back to Citation      

58.                                                     *See*                         Letter from R.T. Leuchtkafer, dated May 2, 2023, at 8-9 (“Leuchtkafer Letter 2”).                    
        Back to Citation      

59.                                                     *See id.*
        Back to Citation      

60.                                                     *See*                         Letter from R.T. Leuchtkafer dated May 30, 2023, at 3-5; 8-9 (“Leuchtkafer Letter 3”). This commenter also cites to the rules governing the Crumbling Quote Indicator and D-Limit order type on the Investors Exchange (“IEX”), as well as language from the Commission's approval order for the D-Limit order type.                         *See id.*                         at 4. The commenter notes the level of detail with regard to how and when the D-Limit order type exercises its discretionary price-sliding that is set forth in the IEX Rulebook.                         *See id.*                         at 4;                         *see also*                         Themis Letter,                         *supra*                         note 54, at 2 (“Another exchange, IEX, operates a smart logic called CQI (Crumbling Quote Indicator) which aims to protect orders from being adversely selected. IEX has published detailed notes on how the CQI is calculated.”). Each proposal must be evaluated based on the specific facts and circumstances before the Commission. In this case, the Commission is only reviewing the proposed operation of Dynamic M-ELO and its machine-learning model. Accordingly, the level of detail provided in the IEX Rulebook for the D-Limit order type and Crumbling Quote Indicator—or the rulebooks for order types on other exchanges—does not determine whether Nasdaq has met its burden in this proposal.                    
        Back to Citation      

61.                                                     *See*                         Letter from Brett Kitt, Associate Vice President and Principal Associate General Counsel, Nasdaq, Inc., dated May 18, 2023 at Appendix A (“Nasdaq Second Response to Comments”).                    
        Back to Citation      

62.                                                     *See supra*                         note 25.                    
        Back to Citation      

63.                                                     *See*                         White Paper Section 3.1.                    
        Back to Citation      

64.                                                     *See*                         White Paper Section 4.1.                    
        Back to Citation      

65.                                                     *See*                         White Paper, Section 7.2.                    
        Back to Citation      

66.                                                     *See*                         Leuchtkafer Letter 2,                         *supra*                         note 58, at 3-4; Leuchtkafer Letter 3,                         *supra*                         note 60, at 4-5; Letter from R.T. Leuchtkafer, dated August 11, 2023, at 8-10 (“Leuchtkafer Letter 4”).                    
        Back to Citation      

67.                                                     *See*                         Nasdaq Second Response to Comments,                         *supra*                         note 61, at 3-5.                    
        Back to Citation      

68.                                                     *See infra*                         notes 91-92 and accompanying text.                    
        Back to Citation      

69.                                                     *See supra*                         notes 16-24 and accompanying text.                    
        Back to Citation      

70.                                                     *See, e.g.,*                         Form 19b-4 at 9, White Paper Section 5, and Nasdaq First Response to Comments at 2. In its White Paper, Nasdaq provides mathematical definitions of fill rate for a period of time and mark-out by trade (White Paper at 5, Equations 1 and 2), as well as of the assessment made by the agent in the model's reinforcement learning process (White Paper at 11, Equation 3).                    
        Back to Citation      

71.                                                     *See*                         White Paper Sections 3-5.                    
        Back to Citation      

72.                                                     *See*                         Exhibit 3B. As described below, these data elements are also those used in training and retraining the model.                    
        Back to Citation      

73.                                                     *See*                         Letter from Brett Kitt, Associate Vice President and Principal Associate General Counsel, Nasdaq, Inc., dated September 6, 2023 (“Nasdaq Third Response to Comments”).                    
        Back to Citation      

74.                             Nasdaq affirmed that, while this information was not included in all versions of the list of data elements, it remains accurate and valid.                         *See id.*                         at 3.                    
        Back to Citation      

75.                                                     *See*                         White Paper Section 5.3.                    
        Back to Citation      

76.                             For example, Nasdaq affirmatively states that if “a retrained model would be worse than the existing model in achieving its objectives, then the System will continue to use the existing model and discard the retrained model.”                         *See*                         Section III.A.1.,                         *supra.*
        Back to Citation      

77.                                                     *See*                         Nasdaq First Response to Comments,                         *supra*                         note 57, at 2-3.                         *See also**supra*                         note 30.                    
        Back to Citation      

78.                                                     *See*                         Nasdaq First Response to Comments,                         *supra*                         note 57, at 2-3. A commenter also noted that it was initially unclear when and how frequently the machine-learning model would retrain, stating that the White Paper set forth an analysis based on daily retraining, but the rule filing proposes weekly retraining.                         *See*                         Leuchtkafer Letter 2,                         *supra*                         note 58, at 4; Leuchtkafer Letter 3,                         *supra*                         note 60, at 2; Leuchtkafer Letter 4,                         *supra*                         note 66, at 9. In Amendment No. 2, the Exchange affirmatively represents that “the performance statistics for Dynamic M-ELO cited herein and in the White Paper are based upon data derived from weekly, not daily retrainings.”                         *See supra*                         note 38.                    
        Back to Citation      

79.                                                     *See supra*                         note 46 and accompanying text.                    
        Back to Citation      

80.                                                     *See, e.g.,*                         Amendment No. 2 at 19-20.                    
        Back to Citation      

81.                                                     *See*                         Leuchtkafer Letter 1,                         *supra*                         note 48, at 2; Leuchtkafer Letter 2,                         *supra*                         note 58, at 4-6; Leuchtkafer Letter 3,                         *supra*                         note 60, at 5-7; Leuchtkafer Letter 4,                         *supra*                         note 66, at 4-8.                    
        Back to Citation      

82.                                                     *See*                         Leuchtkafer Letter 1,                         *supra*                         note 48, at 2.                         *See also*                         Leuchtkafer Letter 3,                         *supra*                         note 60, at 5-7. For example, the commenter believes that Dynamic M-ELO will “exercise individualized judgment” such that it can set a different time-in-force for the very same order presented in the very same market conditions on, for example, August 21 than it set on May 15, depending on the system's undisclosed individualized judgments of market conditions and participant behavior from even days or weeks in the past.                         *See*                         Leuchtkafer Letter 4,                         *supra*                         note 66, at 6.                    
        Back to Citation      

83.                                                     *See*                         Leuchtkafer Letter 2,                         *supra*                         note 58, at 5-6.                         *See also*                         Leuchtkafer Letter 3,                         *supra*                         note 60, at 5-7.                         *See also*                         Leuchtkafer Letter 4,                         *supra*                         note 66, at 6-9 (” . . . . Nasdaq's rulebook won't set out the `totality of the discretionary feature' (I believe it can't, because the totality changes week-to-week and even minute-to-minute) and it won't define the `hardcoded conditionality' of its feature (again, I believe it can't), and a market participant won't be able to `recreate on its own' what Dynamic M-ELO has done (participants can't—it's not even clear anyone will be able to, as discussed below) . . . . Dynamic M-ELO departs from decades of this progress. Its behavior will not be deterministic or invariant over time, and purposefully so.”).                    
        Back to Citation      

84.                                                     *See*                         Leuchtkafer Letter 2,                         *supra*                         note 58, at 8.                         *See also*                         Leuchtkafer Letter 3,                         *supra*                         note 60, at 5-7.                    
        Back to Citation      

85.                                                     *See*                         Leuchtkafer Letter 2,                         *supra*                         note 58, at 8.                         *See also*                         Leuchtkafer Letter 3,                         *supra*                         note 60, at 5-7; Leuchtkafer Letter 4,                         *supra*                         note 66, at 5-7.                    
        Back to Citation      

86.                                                     *See supra*                         notes 42-46.                    
        Back to Citation      

87.                                                     *See supra*                         note 45 and accompanying text.                    
        Back to Citation      

88.                                                             *See*                             Nasdaq Second Response to Comments,                             *supra*                             note 61, at 5-9.                             *See also*                             Nasdaq First Response to Comments,                             *supra*                             note 57, at 5-7.                        
        Back to Citation      

89.                                                     *See supra*                         note 46 and accompanying text.                    
        Back to Citation      

90.                                                     *See supra*                         note 29.                    
        Back to Citation      

91.                                                     *See supra*                         note 45 and accompanying text.                    
        Back to Citation      

92.                                                     *See supra*                         note 46 and accompanying text.                    
        Back to Citation      

93.                                                     *See*                         Leuchtkafer Letter 3,                         *supra*                         note 60, at 7-8; Leuchtkafer Letter 2,                         *supra*                         note 58, at 5; Leuchtkafer Letter 1,                         *supra*                         note 48, at 3. Nasdaq's White Paper includes a “firm-level analysis” that “tried to identify patterns and trends that could potentially signify a systematic bias towards specific firms.” White Paper at 24. This analysis concluded that “Dynamic M-ELO will not result in systematic-biased execution towards any one firm.”                         *Id.*                         at 26.                    
        Back to Citation      

94.                                                     *See*                         Leuchtkafer Letter 3,                         *supra*                         note 60, at 7-8.                    
        Back to Citation      

95.                                                     *See*                         Amendment No. 2 at fn. 34.                    
        Back to Citation      

96.                                                     *See*                         Leuchtkafer Letter 4,                         *supra*                         note 66, at 1-4.                    
        Back to Citation      

97.                                                     *See id.*                         at 2-3 (“It seems Nasdaq trimmed its data before analyzing it for bias and constrained its analysis to `those firms that accounted for more than 95% of M-ELO activity on the Exchange during Q1 2022.' (I assume Nasdaq used the same data defined in the Filing as the `Training Period' for its analysis. Nasdaq doesn't say so, however.) Nasdaq doesn't describe the kind of M-ELO `activity' it filtered the data for, and specifically whether it filtered on order or trade counts or order or trade volume or some combination of two or more of these categories, or on some other factor, before removing firms from its analysis.”)                    
        Back to Citation      

98.                                                     *See id.*                         at 2-3.                    
        Back to Citation      

99.                                                     *See id.*                         at 5-6.                    
        Back to Citation      

100.                                                     *See id.*
        Back to Citation      

101.                                                     *See id.*
        Back to Citation      

102.                                                     *See supra*                         note 41.                    
        Back to Citation      

103.                                                     *See, e.g.,*                         Exhibit 3b.                    
        Back to Citation      

104.                             According to the “M-ELO Monthly Report” published by Nasdaq for July 2023 (                        *available at: https://www.nasdaq.com/​docs/​M-ELO-Monthly-Report*                         (accessed September 2, 2023)), the average daily notional volume executed in M-ELO was $624,556,748. The average daily notional volume executed in July 2023 across the market for NMS stocks was about $523,769,246,196.                         *See, e.g.,*                         Cboe, Historical Market Volume Data,                         *available at: https://www.cboe.com/​us/​equities/​market_​statistics/​historical_​market_​volume/​.*                         The average daily notional volume in M-ELO for that month was approximately 0.12% (just over one-tenth of one percent) of the average daily notional volume across the entire NMS stock market.                    
        Back to Citation      

105.                                                     *See*                         Leuchtkafer Letter 1,                         *supra*                         note 48, at 1.                    
        Back to Citation      

106.                                                     *See*                         Nasdaq First Response to Comments,                         *supra*                         note 57, at 3.                    
        Back to Citation      

107.                                                     *See*                         Section III.A.1.,                         *supra.*
        Back to Citation      

108.                                                     *See id.*
        Back to Citation      

109.                                                     *See id.*
        Back to Citation      

110.                                                     *See id.*
        Back to Citation      

111.                                                     *See id.*
        Back to Citation      

112.                                                     *See*17 CFR 242.613.                    
        Back to Citation      

113.                             The CAT NMS Plan was approved by the Commission, as modified, on November 15, 2016.                         *See*                         Securities Exchange Act Release No. 79318 (November 15, 2016), 81 FR 84696 (November 23, 2016).                    
        Back to Citation      

114.                                                     *See*17 CFR 242.613(c)(7).                    
        Back to Citation      

115.                                                     *See*                         Leuchtkafer Letter 1,                         *supra*                         note 48, at 2.                    
        Back to Citation      

116.                                                     *See*                         Section II.A.1.,                         *supra.*
        Back to Citation      

117.                             15 U.S.C. 78s(b)(1).                    
        Back to Citation      

118.                             17 CFR 240.19b-4.                    
        Back to Citation      

119.                                                     *See*                         Section III.A.1.,                         *supra.*
        Back to Citation      

120.                                                     *See supra*                         note 30 and accompanying text.                    
        Back to Citation      

121.                             15 U.S.C. 78s(b)(2).                    
        Back to Citation      

122.                             15 U.S.C. 78s(b)(2).                    
        Back to Citation      

123.                                 17 CFR 200.30-3(a)(12).                        
        Back to Citation      

[FR Doc. 2023-19728 Filed 9-12-23; 8:45 am]

BILLING CODE 8011-01-P