|  |                      |
|--|----------------------|
|  | Home | Previous Page |

|  |  | Special Study:Payment for Order Flow and Internalization in the Options Markets
*Office of Compliance Inspections and ExaminationsOffice of Economic Analysis**December  2000*Table of Contents
Executive Summary Introduction 

 Background:  Multiple-Listing and Trading Increases Competition       and Payment for Order Flow is Initiated       
 Overview of Options Order Routing Process 
Payment for Order Flow Arrangements and Other      Order Routing Inducements 

 Specialists' Payment for Order Flow Arrangements       
 Exchange-Facilitated Payment for Order Flow Plans 

 CBOE and PCX Specialists' Payments for Order Flow from Exchange-Collected Fees         
 Payments by CBOE and PCX Specialists from Exchange-Collected Fees         
 Potential for Misuse of Funds Collected by the Exchanges         
 Measures to Prevent Misuse of Funds from Exchange-Facilitated Payment for Order Flow Plans 
 Aggregated Payments to Order Routing Firms       
 Other Inducements to Route Options Order Flow:  Reciprocal Arrangements       
 Internalization and Affiliated Specialists 

 Internalization 

 Internalization of Retail Customer Options Orders           
 Exchange Crossing Rules 
 Routing to Affiliated Specialists 
Use of Payment for Order Flow by Order Routing Firms and Disclosure to Customers 

Use of Payments 	        
 Disclosure to Customers 
Information Used To Support Routing Decisions 

 Information Currently Available       
 Information Provided by Exchanges 

 CBOE Best Execution Assurance Program         
 Pacific Exchange Customer Execution Report         
PHLX Quality of Execution Report        
 AMEX Provides Execution Data to Independent Assessor         
 Difficulties Encountered by Broker-Dealers in Analyzing Exchange-Provided Execution Quality Reports 
Information Provided by Specialists 

 Specialist-Provided Reports        
 Difficulties Encountered by Broker-Dealers in Analyzing Specialist-Provided Execution Quality Reports 
 Information Provided by Independent Vendors      
 Standardized Execution Quality Information and Disclosure 
Analysis of 24 Firms' Market Quality Reviews 
Analysis of 24 Firms' Order Routing Practices 
Summary of OEA Findings and Methodology 

 Quoting Behavior and Costs 

Methodology         
 Quoted Spreads         
 Percentage of Quotes at Maximum Width         
 Effective Spreads         
Realized Spreads 
 Comparison of Market Quality Between Posts That Pay and Posts That do not Pay in the Same Class 
     Conclusion 
Appendix A:  List of 53 Options Classes Reviewed 
Appendix B:  Chronology of Specialists' Payment for Order Flow Arrangements 
Appendix C:  OEA Statistics Detailing Execution Information Executive Summary
Since 1973, United States exchanges have listed and traded standardized options on equity securities.  Until recently, most actively traded options classes were listed on only one exchange, giving brokers no choice of routing destinations for their options orders.  Over the past year and a half, the trading of listed options has undergone significant change.

In August 1999, the options exchanges began to multiply list many options that had previously traded exclusively on only one exchange, giving brokers a choice of where to send their customers' orders.  Accordingly, exchanges and their members are competing intensely for those orders.  This competition may take many forms, including providing faster or more reliable executions, lower transaction costs, and increased liquidity at the displayed quotes.  Competition also has been heightened by the entry of the first new options exchange in 27 years (the International Securities Exchange).  This new competition produced immediate benefits to investors, such as narrowed quotes and effective spreads.

With increased competition for options order flow, options market participants -- like participants in the equities markets -- have begun to offer direct and indirect economic inducements to brokers in return for brokers agreeing to route their customers' order flow to them.  These economic inducements principally take the form of direct cash payments to order routing firms ("payment for order flow").  In addition, other inducements also have arisen in the options markets, such as "internalization" of retail options orders, i.e. firms trading as counter-parties with their customer orders, or firms routing to affiliated specialists, and reciprocal order routing agreements.  In July 2000, the options exchanges began facilitating their members' payment for order flow strategies by instituting programs to collect transaction fees from specialists and market makers and to make these funds available to specialists to pay for order flow sent to the exchange.  

Payment for order flow is a method of transferring some of the trading profits from market making to the brokers that route customer orders to specialists for execution.  Internalization allows a firm to capture trading profits from trading against the firm's own customers' orders.  However, payment for order flow and internalization create conflicts of interest for brokers because of the tension between the firms' interests in maximizing payment for order flow or trading profits generated from internalizing their customers' orders, and their fiduciary obligation to route their customers' orders to the best markets.  The revenue generated from payment for order flow and internalization have the potential, as seen in the equity markets, to be partly passed on to investors in the form of reduced costs.  To date, however, few firms are passing along the benefits of payment for options order flow to their customers in the form of either reduced commissions or rebates.

Further, the Commission recently expressed its concern that payment for order flow and internalization contribute to an environment in which quote competition is not always rewarded, thereby discouraging the display of aggressively priced quotes.1  In the multiple trading environment, specialists on the competing options exchanges typically will promise to match the displayed prices of other exchanges.  If widespread, these passive "price matching" practices may weaken the incentive to display competitive quotes, because displaying a superior quote does not necessarily ensure attracting additional order flow.  Over time, therefore, the quotes being matched may become wider, increasing execution costs to investors.  By some measures, the improvements in quoted prices experienced after increased multiple-listing in August 1999 have been muted coincident with the increasing prevalence of payment for order flow and internalization.

In light of these changes in the options markets, on July 19, 2000 Chairman Levitt directed the Staff of the Office of Economic Analysis ("OEA") and the Office of Compliance Inspections and Examinations ("OCIE") to prepare a report describing current payment for order flow and internalization practices, and outlining how the practices of payment for order flow and internalization have affected order routing decisions and the execution quality of customer options orders.  The Staff expects that this report will be helpful to the Commission in determining whether regulatory action is needed to strengthen price competition and order interaction in the options markets.

In preparing this report, OCIE examined the various payment for order flow and internalization arrangements and analyzed the market quality reviews and order routing practices of 24 order routing firms over the fourteen-month period from August 1999 to October 2000.  OEA analyzed trade data to determine how the quality of the options markets changed over the same period.  OEA analyzed trade data from five one-week periods in August 1999, November 1999, and February, June, and October 2000.  A summary of the Staff's principal findings follows.2

Summary of OCIE Findings:
Payment for Order Flow
 The Staff found that the number of retail customer options orders paid for pursuant to payment for order flow arrangements has steadily increased.  As illustrated in the following charts, in August 2000, specialists paid order routing firms for over 75% of the retail options orders sent to them for execution.
**Percentage of Retail Customers' Orders Routed Pursuant to Payment Arrangements (53 Options Classes)**

 During the period of November 1999 to September 2000, options specialists paid over $33 million to order routing firms to induce these order routing firms to route their customer orders to them.  
 As of September 2000, 19 of the 24 broker-dealers examined by the Staff accepted payments for their customers' order flow.  Another firm had established reciprocal order routing arrangements with various specialists.  Four of the 24 firms reviewed maintained policies not to accept payment for order flow.
 One firm has received in excess of $6 million, and six firms have received in excess of $2 million in payments for their order flow.  As of November 30, 2000, these firms have not passed along to retail customers the benefits of payments received for order flow in the form of reduced retail commissions or direct rebates.  In fact, only one firm has significantly reduced retail customer commissions for executing listed options orders, and another firm maintains a policy to rebate payments received for order flow to customers.
 Several broker-dealers previously maintained stringent policies not to accept payment for order flow.  Some of these firms stated that the Commission's failure to abrogate the exchange-facilitated payment for order flow plans was tantamount to Commission approval of the practice, and therefore began accepting payment for order flow.
Market Quality Reviews

 All 24 firms reviewed by the Staff stated that they employ internal processes to regularly evaluate the quality of options market centers.  The Staff found that each firm reviewed by the Staff relied on virtually the same factors to assess market quality--liquidity, opportunities for price improvement, reliability and speed of each market center's systems.
 The Staff found that order routing firms that maintained policies not to accept payment for order flow almost never determined that market centers that paid for order flow were the highest quality markets.  Conversely, order routing firms with policies to accept payment for order flow contended to Staff that their market quality evaluations often demonstrated that market centers that paid for order flow were the highest quality markets.  
Order Routing Practices

 Payment for order flow has had an impact on order routing decisions.  In the options classes reviewed by the Staff, the firms that maintained policies to accept payment for order flow re-routed their customers' order to specialists that pay for order flow (and away from specialists that did not) much more often than did firms that maintain policies not to accept payment for order flow.
 While some firms that accepted payment for order flow acknowledged that such payments influenced their order routing decisions, most firms denied that it had any influence on their order routing decisions.  The Staff found that most of the firms that accept payment for order flow re-routed some of their customers' options orders to specialists that agreed to pay for order flow and away from specialists that did not pay.  In fact, the Staff found that four of the 19 firms reviewed that accept payment for order flow re-routed 75% or more of their customers' options orders in at least 8 of the 12 classes reviewed by the Staff to specialists that paid them for order flow.  Four other firms similarly re-routed 25% or more of their customers' options orders in at least 8 of the 12 classes to specialists that paid them for order flow.
 The Staff found that one of the 24 order routing firms re-routed its orders to its affiliated specialists, and another order routing firm re-routed some options classes to specialists with which the firm established reciprocal order routing arrangements.
 Order routing firms that do not accept payment for order flow generally routed their customer orders to the exchange that had the largest market share in a particular options class.  Overall, in the options classes reviewed by the Staff, the firms that maintained policies not to accept payment for order flow re-routed significantly fewer options classes to specialists that pay for order flow than did firms that maintain policies to accept payment for order flow.
Market Execution Quality Information

 Broker-dealers do not have adequate market execution quality information to reliably compare the quality of executions between specialist firms.
 The options exchanges are developing execution quality reports.  These reports, however, may not enable broker-dealers that route customer orders to adequately compare execution quality on different options exchanges because each report uses different measures and methodologies to calculate execution quality.
 A national best bid or offer ("NBBO") would facilitate the creation of uniform measures of execution quality.
 Independent execution quality vendors have been unable to develop reliable execution quality reports for order routing firms because the exchanges have not provided them with adequate execution data.
Internalization

 The Staff found that internalization of retail customer options orders is not yet a prevalent practice in the options industry.  Only two firms reviewed have developed systems to trade against retail customers' options orders.  
 At least one order routing firm routes its customers' options orders to affiliated specialists, thereby benefiting from the trading profits of its specialists.
 A number of firms are actively considering whether or not to establish internalization programs as a way to "monetize" order flow.  
Summary of OEA Findings:
OEA Staff analyzed pricing data from five one-week periods from August 1999 to October 2000 (August 1999, November 1999, and February, June and October 2000) to identify any changes in quoting behavior and execution costs during the fourteen-month period.  Summarized below are OEA's findings with respect to quoted and effective spreads.  Realized spreads are discussed in the body of the report.
Quoted Spreads

 The quoted spread is the difference between the displayed bid and offer price.  For example, if a specialist is bidding $10 and offering $12, the quoted spread on that market at that time would be $2.  The analysis of quoted spreads yields insights into quoting behavior, including how aggressively markets compete for order flow based on displayed prices.
 As reflected in the chart below, the trend in quoted spreads (along with other measurements of quoting behavior discussed in this report) suggest a decline in aggressive quoting that may be related to a concurrent growth of payment for order flow and other forms of internalization.  Because of the entry of the ISE into intermarket competition during the period, however, it is difficult to attribute changes in quoted spreads conclusively to any particular factor.  
Effective Spreads

 Effective spreads measure trading cost relative to the midpoint of the quoted spread at the time the trade occurred.  For example, assuming a specialist is bidding $10 and offering $12, if he actually purchased from customers at $10.5 and sold to customers at $11.5, the effective spread would be $1.
 As reflected in the chart below, effective spreads have remained generally constant after an initial decline following the advent of multiple-listing.

 Given that order flow payments are made by specialists and specialists are compensated based on effective spreads, the growth of payment for order flow intuitively could be expected, all other things being equal, to be accompanied by a widening of effective spreads.  The absence of such widening during the review period indicates that other market forces are at work.  
 For instance, it may be that the fluidity of order flow patterns that has followed multiple-listing is leading exchange markets to pursue objectives of market share preservation and expansion more vigorously than short-term profits.  Similarly, the competition for brand recognition between large specialists operating on each exchange and the exchanges themselves may be producing positive pressure on spreads.  OEA Staff intends to continue to monitor execution costs closely.
Future Developments in the Options Markets
Competition from multiple-listing is now reshaping the options markets.  Payment for order flow and internalization are early products of this competition and present familiar tensions inherent in the National Market System framework.  The Commission has sought to facilitate the development of competing transparent, connected markets in an effort to protect investors and ensure that they enjoy the benefits of markets driven to innovate, particularly in periods of intense and dynamic competition.  This requires considerable deference to market forces as they shape market structure.  At the same time, the Commission has sought to promote, consistent with this deference, the ability of investors to meet investors without undue structural obstacles-that is, order interaction.  Finally, the Commission has sought to protect intra-market competition by ensuring that those who compete to provide liquidity within exchange markets have meaningful incentives to engage in price competition.

The influence of payment for order flow and internalization on order routing practices and on quote competition for orders warrants ongoing attention.  At the same time, other changes in the market could result in increased competition in the markets both on the basis of published quotes, and on broader market quality factors.  For example, options markets are developing screen-based trading systems that may encourage their members to compete through publishing quotes that are displayed through the screen-based system.  The exchanges also have agreed to and are developing an options linkage system and trade-through rules that will enable the routing of trading interest on one exchange to a superior quote displayed on a competing market.

The conversion to decimal pricing also may result in narrower spreads in the options markets, and also may impact payment for order flow.  Currently, options markets plan to trade in 10¢ increments for options priced over $3.00 and 5¢ increments for options priced $3.00 and under.

Further, under a recent rule adopted by the Commission, brokers' options order routing choices and order routing inducements will be made known to investors and the public.  This rule requires brokers to disclose on a quarterly basis, starting in October 2001, the venues to which they route their options (and equities) order flow for execution, and any financial inducements the brokers are receiving for that order flow.  Thus, payment for order flow arrangements and other inducements will be more transparent to investors and, therefore, subject to greater competitive pressure than is currently the case.

Obviously, firms are required to seek best possible execution for their customers' orders, irrespective of payment for order flow and other routing inducements.  The Staff intends to continue to closely monitor execution quality and the order routing patterns of firms that accept payment for order flow.

Top   

I. Introduction
Since 1973, United States exchanges have listed and traded standardized options on equity securities.  Initially, pursuant to options allocation rules, most individual options classes were listed on only one exchange, giving market participants no choice of routing destinations for their options orders.  In 1989, the Commission adopted Exchange Act Rule 19c-5, which banned formal exchange limitations on the multiple-listing and trading of options classes.  The Commission, however, recently found that the options exchanges failed to comply with Rule 19c-5 by following a course of conduct under which they refrained from multiply-listing high volume options classes.

Over the past year, the trading of listed options has undergone significant change.  In August 1999, the options exchanges began to multiply list many options that had previously traded on only one exchange.  Further increasing the competitive landscape, in June 2000 the International Securities Exchange ("ISE"), an electronic options exchange, began operations.  In an effort to capture orders in multiply-listed options, many exchange specialists have begun to pay broker-dealers to send them their options orders.1  In addition, in July 2000, the options exchanges began collecting transaction fees from market makers and specialists to be used by their specialists to pay broker-dealers ("order routing firms") for retail customer options orders.  Also, throughout this period, certain broker-dealers developed methods of "internalizing" or trading against their own retail customer options orders.

In light of the increased payment for order flow and internalization in the nation's options markets, on July 19, 2000, Chairman Levitt directed the staff of the Office of Economic Analysis and the Office of Compliance Inspections and Examinations to prepare a report describing current payment for order flow and internalization practices, and outlining how the practices of payment for order flow and internalization have affected order routing decisions and the execution quality of customer options orders.2

In evaluating the effects of multiple-listings, payment for order flow, and internalization in the options markets, it is important to recognize that there have been a number of other Commission initiatives recently that could affect these practices, the full impact of which cannot be assessed until their implementation is complete.  First, to prevent instances of specialists trading through a better quote on another exchange, the Commission recently approved rules designed to provide incentives to exchanges to develop linkages to limit the possibility that customer orders will be executed at prices inferior to the best available price.3  In conjunction with this initiative, the Commission approved a linkage plan4 in which all five options exchanges are now participants.  The options exchanges have issued a "request-for-proposal" to vendors to develop the linkage for the markets.  The Commission expects that the exchanges will have made substantial progress towards implementation of the linkage by October 1, 2001.

The Commission also recently adopted a rule that requires brokers, beginning with the third quarter 2001, to make publicly available reports that identify the share of customer orders they routed to each options exchange, along with the existence of any internalization, payment for order flow, or profit-sharing arrangements in which they participated on each exchange.5   This disclosure is intended to alert customers to potential conflicts of interest that may influence a broker-dealer's order routing practices, and will provide more transparency to payment for order flow and other order routing inducements.  The Commission did not, however, require options market centers to make publicly available monthly electronic reports that include uniform statistical measures of execution quality, as is required in the equities markets.6  While the Commission explored applying the market center disclosure requirement to the options markets, it noted the potentially difficult issues that would need to be addressed before the uniform statistical measures of execution quality that will be available for equity securities can be made available for options.  In particular, because a consolidated best bid and offer is not currently calculated and disseminated for options, there is not a uniform basis for comparable statistics.

Finally, the conversion to decimal pricing may result in narrower spreads, and may impact payment for order flow.  Currently, options markets plan to trade in 10¢ increments for options priced over $3.00 and 5¢ increments for options priced $3.00 and under.  The rapidly growing amount of market information, particularly quotations, produced by the options markets has put pressure on the existing system for disseminating this information to the public in a timely manner.  The options markets are currently working to expand capacity and develop new ways to disseminate their market information, so that, among other things, penny increments are viable in the options markets in the same way as they will be in the equity markets.

In preparing this report, the Staff identified 24 broker-dealers that handled significant volumes of retail options orders and examined their market quality reviews and order routing practices from August 1999 to September 2000 ("review period").  The Staff also obtained information concerning payment for order flow arrangements during the review period from 188 specialists on the CBOE, Phlx, Amex, PCX, and ISE.  In addition, OEA analyzed options trade data provided by the options exchanges for five one-week periods. 
A. Background:  Multiple-Listing and Trading Increases Competition and Payment for Order Flow is Initiated
Prior to August 1999, an options class often was listed on only one options exchange, making that exchange the sole venue to trade a customer options order in that class.  In August 1999, inter-market competition for customer listed options orders significantly increased as the exchanges began multiply-listing classes that were previously listed on only one exchange.7  In August 1999, only 32% of equity options classes were traded on more than one exchange.  By the end of September 2000, the number of equity options classes that were multiply-listed had risen to 45%.  In addition, aggregate options volume traded only on a single exchange fell from 61% to 15% over this same period.8  As a result of these changes, specialists on multiple options exchanges began competing against each other for most customer options orders.

On September 11, 2000, the Commission brought and simultaneously settled an administrative proceeding against four options exchanges.  The Commission found that the options exchanges had significantly impaired the operations of the options market by: (a) following a course of conduct under which they refrained from multiply-listing a large number of options; and (b) inadequately discharging their obligations as self-regulatory organizations by failing to adequately enforce compliance with exchange rules governing the handling of customer orders and prohibiting anti-competitive conduct, such as harassment, intimidation, refusals to deal, and retaliation directed at market participants who sought to act competitively.  The exchanges, without admitting or denying the findings, consented to various remedial undertakings to reform and improve the options markets.9

Concurrently, the Department of Justice filed a lawsuit alleging that the exchanges' conduct in refraining from multiply-listing options violated antitrust laws.  The exchanges agreed to the entry of a consent decree designed to promote competition and lead to substantial reform and improvements in the options markets and in the rules of the options exchanges.10

As a result of increased competition for options orders, practices that are commonplace in the equities markets quickly developed in the options market.  The most controversial of these practices is payment for order flow.  Over the last year, specialists, using their own money or money collected through the assessment of a transaction fee by the options exchanges, began paying order routing firms to send their customer options orders to the exchange post where the specialist trades the options class.  Retail customer options orders are considered the most profitable because these orders are often "uninformed," and specialists and market makers can profit from the spread or gain valuable market trend information from aggregate customer options order flow.11

The Staff found that the number of retail customer options orders paid for pursuant to payment for order flow arrangements has steadily increased.  In fact, as illustrated below, in August 2000, in the most heavily traded options classes reviewed by the Staff, specialists paid for over 75% of the retail options orders.12

**Percentage of Retail Customers' Orders Routed Pursuant to Payment Arrangements (53 Options Classes)**

Generally, payment for order flow arrangements take the form of either a monthly flat fee or a fee per contract.  In a flat fee arrangement, a specialist pays a broker-dealer a fixed amount per month in exchange for that broker-dealer's options order flow.  In a fee per contract arrangement, a specialist pays a broker-dealer for each options contract that the order routing firm sends to the specialist.  Fee per contract arrangements may provide for payment based either on each contract sent or each contract executed.  

While cash payments are the most common form of payment for order flow in the options markets, the Staff found that other inducements also exist.  For instance, some firms routed customer options orders to affiliated specialists.  Firms that route order flow to an affiliated specialist are able to benefit through increased profits generated by that specialist, which does not have to pay cash for its affiliates' order flow.  Other firms have entered into reciprocal order flow arrangements, under which each agrees to route customer order flow to the other.  In addition, several firms have developed methods to internalize retail options order flow, which allow them potentially to profit from trading against their own retail customer order flow.  
B. Overview of Options Order Routing Process
As a result of increased multiple-listing and trading of listed options, most options orders can now be executed at more than one competing market center.  Order routing firms typically route their customers' options orders to a particular market with virtually no direct input from the individual investor.13  Order routing firms that handle retail customer options orders can be classified into three general groups:  introducing firms, consolidators, and full service firms.  Each type of firm, and the way they route retail options orders for execution, is described below.

"Introducing Firms" are broker-dealers that route their customer options orders to another broker-dealer that serves as a "consolidator" of options order flow.  Consolidators, in turn, route the introducing firm's customer orders to individual specialists on the options exchanges for execution.  Introducing firms generally are not members of the various options exchanges and include most discount brokerage firms and on-line brokerage firms.14  An introducing broker-dealer may receive payment for its customer options orders either from a specialist directly or indirectly from its consolidator, which may pass on the payments, or a portion of the payments, that it receives from specialists.

"Consolidators," also known as executing agents, are broker-dealers that route options orders to the various exchanges for execution and clear options orders for introducing broker-dealers.15  Consolidators do not typically have a direct relationship with the retail customer.  Consolidators are members of the options exchanges and, therefore, maintain electronic links to the various exchange floors and maintain a staff of floor brokers on each options exchange.  These firms utilize a variety of systems to route customer orders.  In some cases, the introducing broker-dealer has the ability to control the order routing destination.   In other instances, the consolidator determines the destination for customer orders, or the consolidator and the introducing broker-dealer share control over routing decisions.  Consolidators that receive payment for customer options orders are paid directly by the specialists and may pass payments along to introducing firms.

**Execution Path:  Introducing Firms and Consolidators**

"Full Service Broker-Dealers," unlike consolidators, have their own retail customers.16  They are most often members of the options exchanges to which they route their customer orders for execution.  These firms also maintain floor brokers on most exchanges.  Full service broker-dealers that receive payment for customer options order flow are paid directly by the specialists.

**Execution Path:  Full Service Firms**

Firms that have the capability to route customer orders electronically to an options exchange generally program their systems to route all retail-sized orders in the same options class to one designated exchange for execution.  Some firms' systems allow them to electronically route customer orders in the same options class to two or more exchanges.  Regardless of how an order is routed to the options exchange, the process for execution is similar.  Smaller orders are generally sent to an exchange's automatic execution system.17  Larger sized orders, typically those over 50 contracts, generally are sent to an exchange to be manually represented in the crowd by the firm's own floor broker or by a two-dollar floor broker.18  Many firms have personnel off the trading floor who monitor the quotations at each exchange and send larger sized orders to a broker on the floor of the exchange quoting the best price.  

Top   

II. Payment for Order Flow Arrangements and Other Order Routing Inducements
A. Specialists' Payment for Order Flow Arrangements
Since October 1999, many specialists have begun to pay order routing firms for their customer options order flow.  In October 1999, Susquehanna Investment Group ("Susquehanna"), a large multi-exchange options specialist, initiated payment for customer order flow in options orders and ultimately formed direct arrangements with 18 firms.  In December 1999, Spear, Leeds and Kellogg ("Spear Leeds"), another large multi-exchange options specialist, began paying for customer options orders.  From February 2000 to July 2000, at least 11 additional specialist firms entered into payment for order flow arrangements with order routing firms.19  (*See* Appendix B for a chronology of specialists' payment arrangements.)  
B. Exchange-Facilitated Payment for Order Flow Plans
On July 10, 2000, the CBOE filed a proposed rule change with the Commission that imposed a 40¢ per contract20 "marketing fee" on market makers and specialists when executing transactions with non-market makers.21  Pursuant to the rule change, the money collected as a result of this fee could be used by individual specialists to pay broker-dealers for options order flow.  The CBOE filed its rule change under Exchange Act § 19(b)(3)(A)(ii) and Rule 19b-4(f)(2) thereunder, which permit fee changes to become effective upon filing and do not require approval by the Commission.22  The Phlx, Amex, and PCX filed their own similar fees over the next two months.23  Subsequently, the ISE filed a similar rule change.24

Each exchange-facilitated plan is similar.  The CBOE, Amex and PCX impose a fee of 40¢ per contract on market makers and specialists when executing transactions with non-market makers.  The ISE imposes a 75¢ per contract fee on all transactions with customer orders.  The Phlx imposes a fee of $1.00 per contract, but only on transactions in the top 120 options classes as measured by overall volume.  Additionally, the Phlx excludes transactions between specialists and firm proprietary orders25 and transactions between specialists and broker-dealers from the fee.26  By excluding certain transactions from the fee, the exchanges are able to correlate closely the application of the transaction fee to retail customer orders -- the type of order flow the exchange members want to attract with payments to order routing firms.  

The exchanges collect the money and segregate it according to the options post where the option subject to the transaction fee is traded.  The money is then made available, on a monthly basis, to the specialist at each post.  The specialist can then use the money to pay broker-dealers for order flow in the classes of options that they trade at their station or post.  The exchanges all have deferred to the specialist the responsibility to determine which orders to pay for and the amount of any payment to be given to an order routing firm.
1. CBOE and PCX Specialists' Payments for Order Flow from Exchange-Collected Fees
As of November 30, 2000, only the CBOE and PCX had completed at least one full monthly cycle of collecting transaction fees, receiving payment instructions from specialists, and issuing payments to order routing firms according to those specialists' instructions.27  Not all specialists on the two exchanges agreed to pay out the proceeds of the transaction fees, and the specialists that have made distributions to order routing firms out of their exchange-maintained accounts have distributed only a portion of the available money.

The following table sets forth the aggregate fees collected by the CBOE and PCX, the payments made, and surpluses remaining from the exchange-facilitated payment for order flow plans at the CBOE and PCX:
 **CBOE****PCX**
**JULY 2000**
  
Collected

$7,230,454

N/A

Paid

$4,452,515

N/A

**AUGUST 2000**
  
Opening Balance

$2,777,939

N/A

Collected

$6,913,437

$2,549,067

Paid 

$5,259,131

$772,680

**SEPTEMBER 2000**
  
Opening Balance

$4,432,245

$1,776,387

Collected

$6,870,955

$2,830,412

Paid 

$5,500,530

$1,651,935

**OCTOBER 2000**
  
Opening Balance

$5,802,670

$2,954,864

**TOTAL PAYMENTS**

**$15,212,176**

**$2,424,615**

As the table illustrates, as of the end of the September 2000 payment cycle, both the CBOE and the PCX had collected a 40¢ per contract fee from their specialists and market makers and were holding money that specialists had not committed to distributing.  The CBOE is continuing to collect the fee at all posts pending the specialists' decisions whether to distribute the funds.  If a specialist decides not to distribute any funds, the CBOE plans to refund the money to those specialists and market makers who paid the fee at that post.  Surpluses may continue to exist at the CBOE in accounts maintained on behalf of specialists that have decided to distribute at least a portion of the money.  The PCX similarly is developing a plan that would refund all surpluses on a quarterly basis to the specialists and other market makers who paid the fee.

As of November 30, 2000, some order routing firms were not accepting payment for their order flow from specialists.  Most of these firms stated that they are continuing to evaluate the appropriateness of receiving payment for order flow.  As order routing firms decide to accept payment, the specialists may retroactively pay some of those firms from the surpluses.  In fact, one order routing firm that maintains a policy of not accepting payment for order flow informed the Staff that it had not yet decided whether to accept payment for order flow from the CBOE plan, but that monies from the CBOE plan were being held in escrow pending its decision.  According to the CBOE, no money is being held in escrow for any firm, but it is possible that the specialists have informed some firms that they will track the firms' order flow and maintain surpluses in their accounts that could be used to make retroactive payments to these firms.
2. Payments by CBOE and PCX Specialists from Exchange-Collected Fees
Specialists and order routing firms generally enter into agreements setting forth the amount to be paid by the specialist in exchange for a specified type or amount of order flow.  These contracts are freely negotiable and differ from specialist to specialist.  The same specialist also may enter into different arrangements with different order routing firms.  In addition, under the exchange-facilitated plans, specialists do not always negotiate specific payment contracts with order flow providers.  In such cases, the specialists decide unilaterally to send the order flow providers some amount of money from the transaction fees the exchanges make available for that purpose.

Accordingly, the amounts that specialists pay per contract to order flow providers pursuant to the PCX and CBOE plans vary greatly both within and between posts.  The CBOE and PCX impose a 40¢ per contract transaction fee on *all* customer orders executed at the post, but the specialists are generally using the monies collected from the fee to pay only for retail customer orders.28  Therefore, the per contract amount actually paid for order flow does not correlate to the per contract amount of transaction fees collected.  Most payments from the exchange-facilitated plans fall within a range of 40¢ to 80¢ per contract.  Some payments fall significantly outside that range, with several between $1.00 and $3.00 per contract and one payment of $13.00 per contract.  Several firms that had filled out a PCX form agreeing to accept payment for order flow from PCX specialists through the PCX-facilitated plan did not receive any payments from certain specialists.

The Staff was unable to ascertain the specifics of many payment for order flow arrangements between specialists and order routing firms.  Most order routing firms reviewed by the Staff were uncertain as to how payments that they have accepted were arrived at by the specialists.  Most payment for options order flow arrangements reviewed by the Staff between specialists and order routing firms were not in writing and the arrangements have few, if any, details other than the fact that the order routing firm will accept payments for order flow.29  Most order routing firms have accepted payments regardless of the uncertainty surrounding the process.  
3. Potential for Misuse of Funds Collected by the Exchanges
The Phlx, PCX, CBOE, and Amex have stated that specialists are expected to use the proceeds of the transaction fee to attract orders in the classes traded at the exchange post where the money was collected.30  The Staff notes, however, that the exchanges have no apparent means to monitor for, and, therefore, to enforce this requirement.  The exchange-facilitated plans generally require specialists to account to the exchange for the use they make of the funds but do not require the specialists to document to the exchange the specific options classes or contract volume for which the payments are made.  In addition, the exchange-facilitated payment for order flow plans do not permit the exchanges to exercise control over the specialists' payment decisions.31  By design, therefore, while they are assessing and collecting fees from market makers, the exchanges do not know the specifics of their specialists' payment for order flow arrangements, nor can they be certain that the money is utilized to attract order flow to individual specialists' posts.

As a result, the Staff believes the exchange plans could have the unintended consequence of enabling the few large, multi-exchange specialists to use these new pools of money to increase market share at their posts on other exchanges.  For example, for many actively-traded options classes, the majority of customer orders are routed to one options exchange ("primary exchange").32  If a multi-exchange specialist is the specialist in that options class at the primary exchange, it has access to a large pool of transaction fees collected at that post.  That specialist has little incentive, however, to use the money available to attract more order flow to the primary exchange in that class, because many of the customer orders in that class already are routed to the primary exchange without the need for payments.

Under the exchange-facilitated payment for order flow plans, the specialist has an incentive to use the money collected at one post on the primary exchange to attract order flow to another post trading a different option on the same exchange.  Or, if the specialist operates on multiple exchanges, it could use the money collected from transactions in the class on the primary exchange to attract customer orders on other exchanges for classes in which it is not the primary market.  A specialist's use of funds in such a manner would result in market makers on one exchange paying a fee that is used to attract order flow to crowds in which those market makers do not participate.

Under the exchange-facilitated payment for order flow plans, a specialist also may use funds collected through the plan to pay an affiliated broker-dealer for order flow.  However, the affiliated broker-dealer can then pass those payments through to correspondents to attract order flow in another options class traded by the affiliated specialist at a different post on the same exchange or on another exchange.  For example, a multi-exchange specialist that is affiliated with an order flow consolidator could legitimately pay the order flow consolidator through the exchange-facilitated plans for order flow received by the specialist at the post.  The consolidator, however, could pass those payments through to introducing broker-dealers for order flow the affiliated specialist received on other exchanges.  Even though such payments appear to be contrary to the purpose of the exchanges' collection of fees, no mechanism exists under the proposals filed with the Commission by the exchanges to monitor for or deter such payment practices.  
4. Measures to Prevent Misuse of Funds from Exchange-Facilitated         Payment for Order Flow Plans
As of November 30, 2000, the Phlx was planning to require its specialists to account to the exchange in greater detail for their use of funds made available through the Phlx's transaction fees.  Specifically, the Phlx is considering requiring its specialists to document the specific per contract or flat rate payments the specialists are paying each order routing firm.  Additionally, on its payment instruction forms, the Phlx is considering requiring the specialists to certify that any money they request the Phlx to pay out "has been used to purchase order flow delivered to the Phlx on each option at the corresponding amount identified above."  The Phlx also is considering requiring order routing firms to certify that any money they receive from the Phlx plan "will be used to reimburse the firm for money already expended to purchase order flow delivered to the Phlx."  By requiring the specialists and order routing firms to submit this documentation, specialists may be less likely to try to use funds generated from the Phlx plan to pay for order flow the specialist received at another post or on another exchange.  
C. Aggregated Payments to Order Routing Firms
The Staff found that payment arrangements for retail options orders have resulted in significant payments to some order routing firms.  As of November 30, 2000, specialists have paid in excess of $33 million to order routing firms for their retail options orders.  Aggregate amounts of payments to order routing firms vary according to a number of factors, including the length of time the firm has accepted payment for order flow, the number and type of arrangements the firm has in place, and the volume of customer options orders handled by the firm.  As of November 30, 2000, the Staff found that 19 broker-dealers received payment for customer options order flow.  One of these firms has received in excess of $6 million in payment for its customer options orders, another five firms have received in excess of $2 million in payments, and four other firms have received at least $1 million.  The remaining eight firms have received payments for their customer options order flow varying from under $100,000 to over $950,000. 
D. Other Inducements to Route Options Order Flow: Reciprocal Arrangements
In addition to making direct cash payments to broker-dealers in consideration for their retail options order flow, some specialists have entered into other non-cash arrangements with order routing firms.  These arrangements, typically referred to as reciprocal arrangements, take a variety of forms.  For example, several specialists have entered into reciprocal arrangements with firms, by which the specialists send the firm their equity order flow in exchange for that firm's sending its retail options order flow to the specialists for execution.  Reciprocal arrangements such as these provide broker-dealers with an inducement, which is similar in most respects to cash payments, to route options order flow.
E. Internalization and Affiliated Specialists
Other methods also exist for order routing firms to attain economic benefits similar to payment for order flow without receiving direct cash payments.  Specifically, these methods include arrangements designed to enable order routing firms to "internalize" customer options orders.  Internalization of retail options orders takes on various forms in the options markets.   First, some firms have developed methods to trade proprietarily off the exchange floor with retail customer orders, thereby generating trading profits.  Second, some firms maintain an ownership interest in an exchange specialist *and* in an order routing firm.  Internalization occurs when the order routing firm routes its customer orders to its affiliated specialist on the various options exchanges and therefore benefits from any trading profits generated by its specialist.  These internalization methods are described below.
1. Internalization 
Internalization is the practice whereby a dealer exclusively interacts as the counter-party with customer order flow without the ability for other market participants to compete in order to trade against those same customer orders.  The practice of internalization is prevalent in the Nasdaq market where Nasdaq market makers, who often pay for order flow or are sent order flow by an affiliate, trade proprietarily against incoming customer orders.  NASD rules do not require Nasdaq dealers to expose their internalized orders to competitors.  However, Nasdaq members are subject to rules that require them to use reasonable diligence to execute their incoming customer orders at the best price possible under prevailing market conditions.33  Typically, firms that internalize customer equity orders automatically execute customer orders at the national best bid or offer irrespective of their own quotations.  This practice is referred to as price matching.  In the options markets, internalization takes on many forms, none of which allow a dealer to exclusively interact with customer order flow without subjecting the customer orders to competition from other market participants. 

a. Internalization of Retail Customer Options Orders

In the options markets, customer orders must be executed on an options exchange.34  Pursuant to each exchange's rules, customer orders not executed by the exchange's automatic execution system are subject to competitive bids and offers via open out-cry as customer orders are presented to each trading crowd.35  Internalization of customer orders, as it is practiced in the equities markets, is not possible for trading listed options.  Therefore, some firms have developed internalization methods for use on options exchanges that expose retail-sized customer options orders to competitors in the trading crowd.  The most common method developed to date is where a firm's upstairs trading desk monitors its flow of retail-sized customer orders and, when it identifies desired trading opportunities, the trading desk sends, contemporaneously with the incoming customer order, a contra order to the exchange floor for a possible cross with the customer order.  Execution of the proposed cross is not guaranteed because exchange rules allow members in the trading crowds to "break-up" the cross and trade with the customer order at a better price, thereby reducing or eliminating participation in the execution of the customer order by the internalizing firm.

As of November 30, 2000, the Staff was aware of two broker-dealers that had developed methods of internalizing retail-sized options orders.  During the course of this study, the Staff was alerted to a concern among some options dealers that proposed crosses from internalizing firms may not receive the consistent benefits of competitive pricing.   Specialists and market makers in some trading crowds may feel reluctant to "break-up" proposed crosses out of concern that the order routing firms will re-route their other customer orders to other exchanges.  Pursuant to their obligations under the Exchange Act, the options exchanges must actively surveil for anti-competitive practices.36

b. Exchange Crossing Rules

The exchanges also have adopted rules to facilitate crossing, or internalizing, larger sized customer orders.37  Specifically, in May and June 2000, the Commission approved the Amex's, CBOE's, and PCX's rule proposals giving broker-dealers the right to trade as principal with up to 40% of each of their customers' orders above a certain size.  These rules allow the consolidator or full service broker-dealer to trade as principal against up to 40% of its own customer's order when the firm improves the best bid or offer made by the members of the trading crowd when the floor broker representing the order first asks them for their market.  The firm may receive a guaranteed percentage of its customer's order even when it merely matches the crowd's best price, but that percentage generally will be lower.38

In addition, the options exchanges are considering other methods to facilitate internalization.  For instance, in September 2000, the CBOE publicly announced that it had signed a letter of intent with Primex Trading N.A., LLC to license Primex's Auction System.  Pending final agreement, Commission approval, and implementation, broker-dealers sending customer options orders, including retail-sized options orders, to the CBOE would be able to forward their customer orders to the Primex system for possible internalization of some or all of its customers' orders.
2. Routing to Affiliated Specialists
As of November 30, 2000, the Staff identified at least one order routing firm that routes orders to an affiliated specialist.  Pursuant to this affiliation, whenever possible, the order routing firm routes its customer orders to its affiliated specialists and therefore benefits from any trading profits generated by its specialists.  Exchange trading rules guarantee only a certain participation rate to the specialist,39 accordingly, this form of internalization does not guarantee that all of the firm's customer orders are executed by its specialists.   A specialist faces potential competition from market makers in the trading crowd for these orders.  If there are few or no competing market makers, however, or these market makers are not aggressively narrowing their quotes, a specialist will be able to trade with an even greater proportion of order flow than that guaranteed by exchange rules. 

Top   

III. Use of Payment for Order Flow by Order Routing Firms and Disclosure to CustomersA. Use of Payments
The advent of payment for order flow, along with other changes in the options markets, has significantly lowered order routing firms' costs associated with executing customer options orders.  As discussed above, payment for order flow is a significant source of revenue for many order routing firms.  Additional changes in the options markets also have contributed to a reduction in order routing firms' transaction costs.  For instance, while order routing firms previously paid consolidating broker-dealers for routing their customer orders to the options exchanges for execution, most consolidators have eliminated all fees associated with the execution of retail options orders.  In addition, historically, order routing firms paid exchange fees to execute customer options orders; however, these fees have been reduced, eliminated, or absorbed by the exchange specialists.40  In sum, most order routing firms that were previously paying to have their customers' options orders executed are now generating revenue from their customers' options orders.

Despite the decrease in execution costs from the elimination of exchange fees and execution fees charged by consolidators, and the significant revenue generated from payment for order flow, order routing firms have not, as of November 30, 2000, directly passed these savings along to retail investors in the form of reduced commissions.41  In fact, as of November 2000, the Staff is aware of only two order routing firms that have either significantly reduced retail options commissions rates or rebated the payments for order flow to their customers since the advent of payment for order flow in the options markets.42
B. Disclosure to Customers
In 1994, the Commission required that broker-dealers disclose on customer confirmations whether the firm accepts payment for order flow and, upon written request from a customer, the source and nature of the payment for order flow.43  A typical disclosure on the back of a customer confirmation states: 

*Firm X does receive payment for order flow from the market center through which the transaction was executed.  The nature and source of cash or non-cash compensation, if any, received by us in connection with your transaction will be disclosed to you upon written request.*

The Commission recently adopted a rule that requires brokers, beginning with the third quarter 2001, to make publicly available reports that identify the share of customer orders they routed to each options market, along with the existence of any internalization, payment for order flow, or profit-sharing arrangements.44  This disclosure is intended to alert customers to potential conflicts of interest that may influence a broker-dealer's order routing practices.  The Commission did not, however, require options markets to make publicly available monthly electronic reports that include uniform statistical measures of execution quality, as is required in the equities markets.  While the Commission explored applying the market center disclosure requirement to the options markets, it noted the potentially difficult issues that would need to be addressed before the uniform statistical measures of execution quality that will be available for equity securities can be made available for options.  In particular, because a consolidated best bid and offer is not currently calculated and disseminated for options, there is not a uniform basis for comparable statistics. As noted, significant uncertainty currently exists surrounding most payment for order flow arrangements between specialists and order routing firms.  Most order routing firms reviewed by the Staff were uncertain as to how payments that they have accepted were arrived at by the specialists.  Given this uncertainty, many order routing firms would have difficulty complying with the requirement that they disclose, upon a customer's request, the source and nature of the compensation received in connection with a particular transaction.

Top   

IV.  Information Used to Support Routing Decisions
In accepting orders and routing them to a market center for execution, brokers act as agents for their customers and owe them a duty of best execution.  A broker-dealer's duty of best execution is derived from common law agency principles and fiduciary obligations.  It is incorporated both in self-regulatory organizations' rules and in the antifraud provisions of the federal securities laws through judicial and Commission decisions.45  This duty requires a broker-dealer to seek the most favorable terms reasonably available under the circumstances for a customer's transaction.  As a result, broker-dealers must periodically assess the quality of competing markets.46  Because of the increase in multiple-listing, particularly in actively-traded classes, order routing firms now have at least two choices in determining where to send their customers' options orders for each options class.  

The Commission has stated that the duty of best execution must evolve with changes in technology and market structure.47  As the options markets continue to change, order routing firms must regularly evaluate whether their customers' orders are receiving the best execution reasonably available.  To make this evaluation, order routing firms should obtain reliable execution quality information in order to compare the quality of the markets to which they route their customer orders with the quality of alternative markets.  If material execution quality differences exist between various market centers, an order routing firm must consider the quality differences when routing its customers' orders.48  Furthermore, an order routing firm must not allow an order routing inducement, such as payment for order flow or the opportunity to internalize customer orders, to interfere with its fiduciary obligations.49

Many order routing firms reviewed by the Staff are actively attempting to evaluate execution quality between options markets centers.  The Staff found, however, that reliable market center execution quality information that can be used to compare the quality of the various options market centers is limited.
A. Information Currently Available
Prior to the widespread multiple-listing of options, broker-dealers routed their customers options orders to whichever exchange traded the option.  With the advent of multiple-listing, however, broker-dealers routinely face a choice of where to send their orders to be executed, and they need a means to compare execution quality among possible destinations.  Until recently, however, execution quality information for listed equity options was not readily available.  Some order routing firms are pressuring the exchanges for execution quality information, while others, particularly some of the larger broker-dealers, use information from their own systems to evaluate execution quality.  A few broker-dealers continue to rely on customer complaints to measure the quality of executions received from a particular exchange, and do little, if any, analysis of execution quality.  In addition, specialist firms have requested that certain exchanges create execution quality information that they can provide to order routing firms.  As the options marketplace becomes more competitive, specialists and exchanges are seeking ways to differentiate themselves as a destination for order flow and view providing execution quality information as a valuable means of marketing themselves. 

Currently, order routing firms obtain execution quality information from a number of sources.  The options exchanges recently began to provide some execution quality information.  Over the last year, specialists also provided execution quality information, packaged primarily as a marketing tool, to attract order flow.  Finally, at least one independent information vendor offered to provide execution quality audits to broker-dealers for their listed equity options business.  While these are positive developments, firms stated to the Staff that the information currently available, and described below, is insufficient for broker-dealers to assess the quality of executions by specialists on the different options exchanges.  More detailed information is needed, and standardized information would facilitate comparison among market centers.
B. Information Provided by Exchanges
Since August 2000, the options exchanges have taken steps to formalize their reports on execution quality.  The Amex, CBOE, Phlx, and PCX have started providing execution quality reports to their member firms.  The ISE has indicated that it also intends to provide execution quality reports to its members.50  A summary of the various execution quality reports follows.
1. CBOE Best Execution Assurance Program
In August 2000, CBOE initiated its Best Execution Assurance Program ("Best Ex").51  CBOE stated that it developed this program "[i]n order to make its members aware of how the CBOE's systems, procedures, and rules help them satisfy their best execution obligations when they direct orders to the CBOE for execution...."52  As part of the Best Ex program, CBOE distributes a daily report to each order routing firm that identifies any orders sent by the firm to the CBOE that may have been executed outside the best bid or offer among the five exchanges as calculated by the CBOE53 ("trade-throughs").  The report indicates what action, if any, was taken to adjust the price of the execution.  In addition, the CBOE provides a monthly report that summarizes the information contained in the daily report and lists the number of potential trade-throughs by options class and by DPM post.  The monthly report also shows the number of orders that received price improvement and the average turnaround time for marketable orders at each post.  The CBOE classifies an order as price improved, if at the time the order was executed, it was executed at a price better than the prevailing best bid or offer as calculated by the CBOE.54
2. Pacific Exchange Customer Execution Report
In August 2000, the PCX initiated a program to provide execution quality information to order routing firms.55  The Pacific Exchange Customer Execution Reports, or "PACEX reports," are distributed daily to the PCX's top order routing firms and provide firm-specific information concerning price and speed of execution.  The reports break down the information on execution price into six categories showing the percentage of executions:  (1) where the PCX improved the best bid or offer as calculated by the PCX; (2) where the PCX set the best bid or offer; (3) where the PCX bid or offer matched the best bid or offer; (4) where the best bid or offer was set by another market and PCX honored the best bid or offer; (5) where the PCX was the only market listing the option; and (6) where the PCX failed to execute at the best bid or offer.  The reports also show how quickly orders were executed on the PCX, breaking down the turn-around time into 8 categories, ranging from 1 second to 30 minutes.  The PACEX report also categorizes the information by options class and by post.
3. Phlx Quality of Execution Report
In September 2000, the Phlx initiated its program to provide execution quality reports to Phlx specialists and order routing firms.56  The Phlx intends to distribute firm-specific daily and summary monthly reports capturing execution quality information related to the firm's market orders and marketable limit orders delivered to the Phlx through its Automated Options Market ("AUTOM") system.  The reports will list the best bid or offer, as calculated by the Phlx, that prevailed at the time AUTOM received the order and identify the execution price, execution size, and speed of execution for each order.57  The report also will identify trades that were executed outside the best bid or offer and show what action, if any, was taken to adjust the execution price.  As of mid-November 2000, the Phlx had not yet delivered its daily or monthly quality of execution report to its order routing firms and specialists.
4. Amex Provides Execution Data to Independent Assessor
Instead of producing its own reports, Amex provides its execution data to the Transaction Auditing Group ("TAG"), an independent company that provides execution quality analysis services.  Using the trade execution information provided by Amex, TAG generates a monthly report using a number of different measures of execution quality, including the percentage of executions within the spread, the percentage of trades executed at prices better than the best bid or offer as calculated by TAG, the percentage of contracts traded in excess of 20 contracts at the best quote, the liquidity premium,58 and the speed of execution.  The report analyzes overall execution quality at the Amex and does not, at this time, provide firm-specific, post-specific, or options class-specific information.  TAG provides its completed report to Amex, and Amex distributes the report to its order routing firms.  
5. Difficulties Encountered by Broker-Dealers in Analyzing Exchange-Provided Execution Quality Reports
The exchange-provided execution quality reports have several shortcomings that limit their effectiveness.  All firms interviewed by the Staff indicated that they need better and additional information to make better-informed order routing decisions.  Most importantly, the various reports do not contain consistent information, making it difficult for broker-dealers to compare execution quality among the exchanges.  The reports also lack certain information important to an analysis of execution quality.  Finally, many broker-dealers reported to the Staff that they did not view the reports as objective, because, in their view, the exchanges have a conflict of interest in providing their own execution quality reports.  Each shortcoming is described below. 

Based on the Staff's interviews with order routing firms, one of the difficulties broker-dealers encounter is determining how to evaluate and compare the different types of information provided by the exchanges.  For example, the CBOE and the PCX provide firm-specific reports to each order routing firm that analyze the treatment of that firm's orders, while the Amex provides an exchange-wide summary report.  In addition, only the CBOE and the PCX list their execution quality measures by class and by specialist.  Because execution quality varies among individual specialist posts, the absence of this information in other exchange reports makes it difficult for broker-dealers to assess differences in execution quality among the exchanges.  The exchange reports also include different measures of execution quality.  The CBOE, PCX, and Amex provide information on trade-throughs, price improvement, and speed of execution, while the Amex also includes additional execution quality measures, such as liquidity enhancement and liquidity premium.  Finally, the CBOE and the PCX provide daily and monthly reports, while the Amex provides only a monthly report.  

Also preventing broker-dealers from making accurate cross-market comparisons, each exchange has its own methodology for calculating execution quality statistics.59  The Amex and the PCX calculate the best bid or offer at the time the exchange receives an order and use that best bid or offer to compute execution quality statistics.  The CBOE, on the other hand, calculates the best bid or offer at the time the order is executed and uses that best bid or offer to compute execution quality statistics.  In addition, the CBOE, PCX, and Amex analyze market orders and marketable limit orders, but the CBOE also analyzes some non-marketable limit orders in its execution quality reports.  It did not appear to the Staff that order routing firms were aware of the various differences in each exchange's methodology used to calculate execution quality statistics.  Even firms aware of the differences in methodology, however, may find it difficult to make cross-market comparisons.

Moreover, the lack of a standardized national best bid or offer ("NBBO") limits the accuracy and comparability of the execution quality information provided by the exchanges.  Instead of a centrally-calculated consolidated NBBO, each of the exchanges individually calculates the best bid or offer among the options exchanges, relying on information provided by a data feed from OPRA that contains the "real-time" quotation updates of each options exchange.60  There is no uniformity in the best bid or offer calculated by the options exchanges because each of the exchanges, except the Amex, calculates its best bid or offer using its own internally-generated market information and the OPRA data feed for the information from the other four options exchanges.  Amex, on the other hand, relies on OPRA data for all market information, including its own.

Finally, some broker-dealers believe that the exchanges may be selective in the type of information they are willing to release, i.e. releasing only that information which appears favorable for them.  These firms may not find much utility in the reports because they doubt that they contain accurate and complete information.  Amex has tried to remedy this perception by providing its data to an independent third party for analysis.  
C. Information Provided by Specialists
Over the last year, specialists also have taken steps to provide execution quality information to order routing firms.  The types of execution quality reports offered by specialists range from a statistical analysis of spreads and execution prices to more informational reports describing the specialist's policies with respect to such items as automatic execution size guarantees and trade-throughs.  These reports are described below.
1. Specialist-Provided Reports
In early 2000, Susquehanna began distributing execution quality reports to order routing firms.  Susquehanna enlisted three professors of finance61 to conduct a study of spreads and execution prices in the top 15 option classes in which Susquehanna was the specialist.  Over the past few months, Susquehanna has expanded the number of option classes included in its monthly study from its top 15 classes to its top 50 classes, and most recently, to its top 75 classes.  The Susquehanna execution quality report consists of three basic measures:  (1) a comparison of the time-weighted spreads in the specific options classes on the exchange in which Susquehanna is the specialist (Susquehanna's "post") to the time-weighted spreads in those same classes on competing exchanges; (2) a comparison of the "print-throughs," or execution prices outside the best bid or offer, as calculated by Susquehanna, at Susquehanna's posts to the print-throughs at competing posts; and (3) a comparison of the "cost of liquidity," or how far away from the best bid or offer print-throughs occur, at Susquehanna's posts relative to competing posts.

Spear Leeds, upon request from an order routing firm, will provide a monthly execution quality report similar to the daily report provided by the CBOE.  The report identifies the number of trades executed outside the best bid or offer as calculated by Spear Leeds for a particular broker-dealer as well as the overall number of trades executed outside the best bid or offer by Spear Leeds.62  The report explains the reason the execution was outside the best bid or offer and indicates whether any adjustment was made by Spear Leeds to remedy the unfavorable original execution.  

Other specialist firms have taken less analytical approaches to providing information regarding execution quality.  Instead of providing execution data, these specialist firms have sent correspondence to order routing firms generally discussing their policies relating to certain factors they believe would be important to an order routing firm in deciding where to route orders.  The factors include:  market share, liquidity guarantees for automatic execution, liquidity guarantees for manual executions, trade-throughs, limit order protection, turn-around times, service and reliability, handling of large or complex orders, crowd evaluations, and cost.  The correspondence typically lists all option classes in which the firm acts as the specialist.
2. Difficulties Encountered by Broker-Dealers in Analyzing Specialist-Provided Execution Quality Reports 
The information provided by specialists poses the same types of problems for broker-dealers as the information provided by exchanges. The reports provided by specialists do not contain consistent information, making it difficult for order routing firms to compare execution quality information among specialists.  In addition, many broker-dealers reported to the Staff that they did not view the reports as objective.  These shortcomings are described below.

Several order routing firms interviewed by the Staff stated that they find it difficult to evaluate and compare the execution quality information provided by specialists.  Some specialists provide information regarding spreads and trade-throughs, while other specialists provide speed of execution and automatic execution size guarantees.  At least one specialist provides information regarding the number of orders executed at prices inferior to the best bid or offer available on the options exchanges along with the corresponding reason.  Some specialists do not provide any information to order routing firms regarding execution quality.  

Further complicating the comparability of specialist-provided reports is the fact that execution quality may vary within a specialist firm.  Some specialists operate on more than one exchange, and each exchange has its own automated system for handling order flow.  Execution quality statistics that are aggregated by specialist firm fail to identify differences between posts and among options classes that may be significant to order routing firms in making routing decisions.  

Finally, some broker-dealers believe that the information provided by specialists may be tailored to produce favorable execution quality statistics.  Because the execution quality information provided by specialists typically is accompanied by marketing materials, order routing firms stated that they are skeptical of the objectivity of the information provided in the reports.
D. Information Provided by Independent Vendors
Broker-dealers also may engage independent vendors to perform execution quality analyses.  At the time of this report, the only independent vendor known to the Staff that is providing execution quality analyses for options executions is TAG.  Under TAG's subscription agreement, a subscriber instructs firms executing its customer's orders to provide specific trade information to TAG.  Several TAG subscribers have requested that the options exchanges provide execution information to TAG, but, to date, Amex is the only exchange to provide complete data.  To mitigate the lack of exchange-specific information, a broker-dealer can provide TAG with detailed order routing and execution information from its own system so that TAG can provide approximate results.  However, exchange-specific information, such as the time the exchange received the order, as well as execution information, likely would be more accurate if the data came from the source of the execution.
E. Standardized Execution Quality Information and Disclosure
In connection with this Study, the Staff interviewed broker-dealers with significant customer options order flow and asked what types of execution quality information would assist them in making order routing decisions.  Nearly all of the broker-dealers agreed that standardized execution quality data would enable them to make more effective cross-market comparisons.  Some broker-dealers emphasized the importance of post-by-post comparisons to rule out the aggregation effect of analyzing information by exchange or by specialist unit.  Other broker-dealers insisted that overall execution quality information was important, but that they also would like to see an analysis of how orders routed from their firm were handled.

In the Commission's recent release adopting rules regarding the disclosure of order execution and routing practices, the Commission expressed its concern about the need for improved disclosure of execution quality in the options markets.63  The Commission did not extend the execution quality disclosure obligation to the options exchanges principally because a standardized NBBO is not, at this time, calculated and disseminated for options trading.  In the proposing release, the Commission requested comment on excluding listed options from the scope of the proposed rule and on whether there are other means to improve disclosure of execution quality for listed options.64  In addition, the Commission requested comment on whether it should require that a consolidated BBO be calculated and disseminated for the options markets, thereby facilitating the disclosure of order execution practices.  

The Staff believes that exchanges and specialists should take steps to provide more execution quality information to order routing firms, including standardized measures of execution quality.  This could, if utilized by order routing firms in making routing decisions, lead to enhanced quality of execution for customer options orders.

Top   

V. Analysis of 24 Firms' Market Quality Reviews
The Staff examined the internal market quality reviews of 24 firms with a significant retail options business.  Nearly every firm stated that obtaining executions at the best possible price and opportunities for price improvement were the most important factors in deciding where to route retail customer options orders.  The Staff found that 17 of the 24 firms stated that liquidity was an important factor in their routing decisions.  Some of the firms characterized liquidity as market share, while other firms characterized liquidity as depth of market or the size of the orders that market centers could efficiently execute.  In addition, the Staff found that most of the 24 firms stated that speed of execution, exchange systems, and customer service were factors the firm considered in determining where to route orders.  Other factors mentioned by the firms include the maximum allowable size for automatic execution systems, the reputation of the specialist, the firm's relationship with the specialist, the presence of the firm's personnel on the floor of the exchange, and the number of customer complaints received by the firm regarding a particular marketplace.

The Staff found that all firms maintained a policy to regularly conduct market quality reviews.  The Staff found that the quality of these reviews varied significantly, and that the firms employed various methods to evaluate market quality such as manual review of individual orders, review of aggregate market quality data, and monitoring customer complaints.  Approximately half of the 24 firms stated that to assess the quality of competing markets, the firm conducted a manual review of the executions obtained at various market centers.  Most of the firms examined a sample of executions to determine whether the customer order was executed at or within the best bid or offer.  At some firms, the sample was as small as three orders per day, while at other firms, the sample was as large as fifty orders per day.  One firm stated that it reviews every customer order to determine whether it was executed at, better, or worse than the best bid or offer at order receipt time.

All 24 firms reviewed by the Staff stated that they employ internal processes to regularly evaluate the quality of options market centers.  The Staff found that each firm relied on virtually the same factors to assess market quality--liquidity, opportunities for price improvement, reliability and speed of each market center's systems.  The Staff found that order routing firms that maintained policies not to accept payment for order flow almost never determined that market centers that paid for order flow were the highest quality markets.  Conversely, order routing firms with policies to accept payment for order flow contended to the Staff that that their market quality evaluations often demonstrated that market centers that paid for order flow were the highest quality markets.

Top   

VI. Analysis of 24 Firms' Order Routing Practices
The Staff reviewed the order routing patterns of 24 broker-dealers with a significant retail options business.  In conducting these reviews, the Staff requested customer order routing information in 53 options classes over a 13-month period.65  Of these 53 options classes, the Staff examined each firm's order routing practices in 12 high volume options classes to determine the extent to which firms re-routed customer orders to specialists that pay for order flow.66

At the time of the Staff's review, 19 of the 24 firms accepted payment for order flow.  Four of the 24 firms maintained policies not to accept payment for order flow or other order routing inducements.  These four firms provided several reasons for not accepting payment for order flow.  One firm stated that the regulatory environment was presently too uncertain for it to accept payment for order flow.  Two of the firms stated that they maintain a firm-wide policy not to accept payment for order flow because it conflicts with the firm's fiduciary obligation to its customers to seek the best possible execution of their orders.  Two of the firms stated that although payment for order flow is not necessarily inconsistent with providing best execution, they refrain from accepting payments to avoid even the appearance of a conflict of interest.****

The Staff found that two of the 24 firms reviewed by the Staff employ automated order-by-order routing systems for their retail customer options orders.  These routing systems, often referred to as "smart routers," route each customer order to the specialist that quotes the best price in a particular options class.  Other firms stated that they are evaluating smart-routing technology, but most believe that smart routing technology costs are prohibitive.  Some firms asserted that exchange linkages may lessen the desirability of implementing smart routing technology.

The Staff found that payment for order flow has had an impact on order routing decisions.  As described below, for the options classes reviewed by the Staff, the firms that accept payment for order flow re-routed their customers' order to specialists that pay for order flow (and away from specialists that did not) much more often than did firms that do not accept payment for order flow.67

The Staff found that, since September 1999, 17 of the 19 firms that accept payment for order flow changed the routing destination for some orders in high-volume options classes away from specialists that did not offer to pay for order flow to specialists that paid them for order flow.  In the options classes reviewed by the Staff, these firms re-routed options orders in varying degrees: (i) several firms re-routed the majority of their options orders to specialists that pay for order flow, thereby maximizing payment for order flow revenue; (ii) several firms re-routed orders to specialists that pay for order flow but only in classes in which the specialist had a "significant" market share; and (iii) other firms re-routed orders to multiple specialists in the same class according to the market share of each specialist. 

Specifically, in the 12 classes reviewed by the Staff, four firms re-routed most of their customers' orders to specialists that paid them for order flow (and away from specialists that did not pay).  These firms re-routed 75% or more of their customers' options orders in at least 8 of the 12 classes reviewed by Staff to specialists that paid them for order flow.  Four other firms similarly re-routed 25% or more of their customer order flow in at least 8 of the 12 classes reviewed by the Staff.  Each of the 19 firms that re-routed customer orders to specialists that pay for orders stated that all order re-routing decisions were based on a number of quality factors, although five firms noted that payment for order flow was a factor in their decision to re-route order flow.

In addition, one of the 24 firms reviewed by the Staff re-routed a significant amount of order flow in some classes to specialist firms with which it had reciprocal order flow arrangements; and another of the 24 firms re-routed all possible classes to its affiliated specialist.  Order routing firms that do not accept payment for order flow generally routed their customer orders to the exchange that had the largest market share in a particular options class.  Overall, the firms that did not accept payment for order flow re-routed significantly fewer options classes to specialists that pay for order flow.

The four firms that maintained policies not to accept payment for order flow re-routed orders far less frequently in the same 12 options classes reviewed by the Staff.  Two firms did not re-route any classes, one firm re-routed one class, and the fourth firm re-routed four classes.

Top   

VII. Summary of OEA Findings and MethodologyA. Quoting Behavior and Costs1. Methodology
OEA Staff analyzed trade and quote data to determine if there have been changes in options markets' quoting behavior over the fourteen-month period from August 1999 to October 2000.68  Quoting behavior is measured by the width of the quoted spread and the percentage of quotes with spreads that are at the exchange's maximum width.  Quoting behavior is important because it is one indication of the extent to which options specialists are competing on price for customer executions.  OEA Staff also examined execution costs.  Execution cost measures include effective spreads and realized spreads. These measures are important because they reflect the direct costs to investors of trading on a given options market.  Trade execution costs are measured both through time, for five different weeks from August 1999 to October 2000, and across different trade sizes.  

The analysis measures spreads in three forms: quoted spreads, effective spreads, and realized spreads.  The quoted spread is simply the difference between the quoted, or displayed, bid and ask.  The effective spread is a measure of the direct execution cost of the trade.69  The realized spread is a measure of trading costs taking into account the informational impact of the trade.70

There are, of course, other dimensions of execution quality, most notably, speed of execution.  However, OEA Staff was unable to measure execution speed because information regarding the time an order was entered was unavailable.  In preparing this report, the Staff obtained sample data from OPRA for five one-week periods, one each in August and November 1999 and February, June, and October 2000.  OEA Staff also obtained audit trail data from each of the five option exchanges for one week in June 2000.  OEA Staff analyzed data for all series in fifty of the most active equity option classes as of August 1999, representing approximately 70% of all equity options trades during the periods reviewed.
2. Quoted Spreads
OEA Staff measured quoted spreads for both individual exchanges and the consolidated BBO.71  Exchange-quoted spreads indicate how aggressively individual exchanges are setting their quotes.  The consolidated BBO indicates how the best prices available to investors are changing during the sample period because many customer trades, particularly automatically executed trades, are executed against the consolidated BBO.  As shown in the chart below, quoted spreads ("Average Exchange Quoted Spreads") measured in dollar terms fell sharply from August to November 1999, rose from November 1999 through June 2000 (to near their August 1999 levels), and then fell again in October 2000 (although not to the levels of November 1999).  The trade-weighted average exchange quoted spread for options under $20 (the most heavily traded options) was $0.35 in August 1999, $0.28 in November 1999, $0.34 in February 2000, $0.35 in June 2000 and $0.32 in October 2000.72  Put another way, quotes have narrowed only 8% since the expansion of multiple-listing in August 1999.  

The average exchange quoted spread increased 14% from November 1999, which is after multiple-listing but before growth of payment for order flow, to October 2000.  The average exchange quoted spread for June 2000, which is not significantly affected by the ISE's presence, was 25% wider than November 1999.  

As shown in the chart below, the trade-weighted consolidated BBO spreads for options under $20 fell sharply from $0.29 in August 1999 to $0.18 in November 1999, rose in June to $0.20, and returned to their November 1999 level of $0.18 in October 2000.  In June 2000, more exchanges were quoting each series and, therefore, the consolidated BBO more often represented the bid from one exchange and the offer from a different exchange than in August 1999.  Consequently, in June 2000, the consolidated BBO did not return to its August 1999 level, as was the case for the individual exchange's quotes.  The drop in October 2000 was apparently influenced by the presence of the ISE.  Assuming the quoting behavior of the other exchanges to be unchanged by the absence of the ISE, then the trade-weighted consolidated BBO for October would be slightly wider at $0.19.

In summary, quoted spreads in the options markets have declined since widespread multiple-listing of options classes in August 1999.  Quoted spreads widened concurrent with the initial growth of payment for order flow from November 1999 to June 2000, but declined somewhat from June 2000 to October 2000 after the advent of exchange-facilitated payment for order flow plans.  For the entire period, November 1999 to October 2000, quoted spreads have widened.  Interpretation of changes in quoted spreads during the review period is complicated by the entry of the ISE into intermarket competition, and particularly by its growth in trading volume after June 2000.  
3. Percentage of Quotes at Maximum Width73
Exchange rules govern the maximum allowable quote spreads.74  The frequency of individual exchanges quoting at the maximum allowable width fell sharply from August to November 1999, rose from November 1999 through June 2000 (to near their August 1999 levels), and then fell again in October 2000 (although not to the levels of November 1999).  The trade-weighted percentage of quotations at the maximum width for options below $20 in the same five time periods was 57%, 21%, 35%, 44%, and 32%, respectively.  The percentage of the quotes at the maximum dramatically declined 63% with the advent of multiple-listing from August 1999 to November 1999.  Between November 1999 and October 2000, however, during the advent and growth of payment for order flow, the percentage of quotes at the maximum increased 76%.

The results for quotes at the maximum width are consistent with those for quoted spreads.  Quoting was more aggressive after increased multiple-listing, became markedly less aggressive during the initial growth in payment for order flow, but became somewhat more aggressive following the advent of exchange-facilitated payment for order flow plans.
4. Effective Spreads
As shown in the chart below, effective spreads declined from August 1999 to November 1999 and have remained at approximately their November levels since.  Average effective spreads for options priced below $20 were $0.21 in August 1999, $0.17 in November 1999, $0.19 in February 2000, $0.17 in June 2000, and $0.17 in October 2000.75  There are very different patterns for the time series behavior of effective spreads depending on the size of the trade and execution system used.  For orders ineligible for automatic execution of 50 contracts or more handled by floor brokers or specialists, effective spreads in the five periods were $0.14, $0.14, $0.15, $0.15 and $0.15 respectively.  For trades automatically executed, the comparable numbers are $0.26, $0.18, $0.19, $0.18, and $0.17.76
5. Realized Spreads
The realized spread is a measure of trading costs taking into account the informational impact of the trade.  It compares execution prices to the bid/offer mid-point at a later point in time, which provides a hypothetical measure of a trade's profitability to the executing broker.  As shown in the chart below, realized spreads declined from August 1999 to November 1999, rose significantly from November 1999 to February 2000, and fell slightly from February 2000 to June and October 2000.  Average realized spreads for options priced below $20 were $0.18 in August 1999, $0.12 in November 1999, $0.18 in February 2000, $0.15 in June 2000 and $0.17 in October 2000.77  Time series behavior patterns of realized spreads varied depending on the size of the trade and execution system used.  For trades ineligible for automatic execution of 50 contracts or more handled by floor brokers or specialists, realized spreads in the five periods were $0.10, $0.10, $0.14, $0.15 and $0.16 respectively.  For automatically executed trades, the comparable numbers were $0.23, $0.12, $0.18, $0.15, and $0.16.78  The trend in realized spreads was up from November 1999 to June 2000, which tends to suggest that specialists were earning more per trade.  This has not, however, reduced effective spreads as would be expected in a competitive market.
B. Comparison of Market Quality Between Posts That Pay and Posts That Do Not Pay in the Same Class
In this section, OEA Staff compared execution quality measures for fourteen classes of the aforementioned 50 classes in which order flow was re-routed away from an exchange where the specialist did not pay for order flow and had the dominant market share as of August 1999 ("old exchange") to an exchange where the specialist paid for order flow ("new exchange").  OEA Staff compared execution quality measures between the old exchange and the new exchange in August and November, 1999 and February and June 2000. OEA Staff did not extend the comparison to October 2000 because of the widespread application of exchange-facilitated payment for order flow plans at that time.  In addition, if the specialist on the old exchange began to pay for order flow, OEA Staff eliminated both exchanges from the analysis for subsequent time periods. 

For the fourteen classes analyzed, OEA Staff found that the old exchanges quoted more aggressively than new exchanges, particularly in August and November 1999.  In August 1999, the average quoted spread was $0.46 for options under $20 bid on the old exchanges and $0.77 on the new exchanges.79  By November, the average spreads were $0.40 on the old exchanges and $0.49 on the new exchanges.  In June the average spreads were $0.42 on the old exchanges and $0.44 on the new exchanges.

As shown in the chart below, the percentage of quotations at the maximum width followed a similar pattern. In August 1999, there was a large difference in quotation aggressiveness between the old and new exchange.  Although narrowing over time, the difference continued to exist in June 2000.  In August 1999, the average percentage of quotes at the maximum width was 49% for options under $20 bid on the old exchanges and 97 % on the new exchanges.  By November 1999, the respective percentages were 21% and 46% and in June 2000 the respective percentages were 30% and 36% for the old and new exchanges.

In summary, specialists that began paying for order flow in November 1999 were not quoting as aggressively as those specialists not paying for order flow.  By June 2000, the difference in quote aggressiveness was smaller, but still present.  

As shown in the chart below, for effective spreads, both old and new exchanges have executed trades at roughly comparable effective spreads since November 1999.  Specialists paying for order flow typically execute trades at the BBO, so there are not significant differences in trade execution costs.  For options under $20, average effective spreads for old exchanges were estimated to be $0.20, $0.19, and $0.16 in August 1999, November 1999, and June 2000 respectively.  The comparable figures for the new exchanges offering specialist payment for order flow plans were $0.37, $0.16, and $0.16 respectively.  The figures for automatically executed trades follow a similar pattern although the difference in August 1999 is smaller, $0.32 vs. $0.25 for old and new exchanges respectively.

Top   

VIII. Conclusion
While the fierce competition brought on by increased multiple-listing produced immediate economic benefits to investors in the form of narrower quotes and effective spreads, by some measures these improvements have been muted with the spread of payment for order flow and internalization.  The Staff intends to closely monitor execution quality measures in the options markets.

The Staff found that payment for order flow has had an impact on order routing decisions.  The Staff also found inadequacies in the comparability of the information available that limits order routing firms' ability to measure the quality of competing markets.  Uniform qualitative measures of execution quality among the various market centers would assist order routing firms in making order routing decisions.  The Staff intends to closely monitor the order routing patterns of the firms that accept payment for order flow.  In addition, with greater transparency in order routing patterns and order routing inducements in October 2001, firms should anticipate greater scrutiny from customers and from the public.  The Staff expects firms to seek the best possible executions for their customers' orders, irrespective of payment for order flow or other order routing inducements.

There are additional recent or planned changes that may have a significant impact on the options markets.  The Commission's routing disclosure rules, options exchange linkage, screen-based trading systems, and the conversion to decimals are likely to further alter the competitive dynamic of the options markets and may impact the prevalence of payment for order flow and internalization.
Appendix A:List of 53 Options Classes Reviewed
1.

ADC Telecomm (ADCT)

28.

Kimberly Clark (KMB) 

2.

Advanced Micro Devices (AMD) 

29.

Knight Trading Group (NITE) 

3.

Aetna (AET) 

30.

Lucent Technologies (LU)

4.

Allstate (ALL) 

31.

McDonald's (MCD)  

5.

Alza (AZA)

32.

MedImmune (MEDI) 

6.

Amazon.com (AMZN) 

33.

Mellon Financial (MEL) 

7.

America Online (AOL)

34.

Micron Technology (MU)

8.

Amgen (AMGN) 

35.

Microsoft (MSFT) 

9.

Anheuser Busch (BUD) 

36.

Motorola (MOT) 

10.

Apple Computer (AAPL) 

37.

Oracle (ORCL)

11.

Applied Materials (AMAT)

38.

Palm (PALM)

12.

At Home (ATHM)

39.

Pfizer (PFE)

13.

AT&T (T)

40.

Phillip Morris (MO) 

14.

Biogen (BGEN)

41.

PMC- Sierra (PMCS)

15.

Cisco Systems (CSCO) 

42.

Qualcomm (QCOM)

16.

Citigroup (C) 

43.

Qwest Communications International (Q)

17.

CMGI (CMGI) 

44.

Safeguard Scientifics (SFE) 

18.

Dell Computer (DELL)

45.

Schering Plough (SGP) 

19.

E-Bay (EBAY) 

46.

Siebel Systems (SEBL) 

20.

EMC (EMC)

47.

Texas Instruments (TXN) 

21.

Exodus Communications (EXDS) 

48.

Time Warner (TWX) 

22.

First Union (FTU) 

49.

Tyco International (TYC) 

23.

Goldman Sachs Group (GS) 

50.

Unisys (UIS) 

24.

International Business Machines (IBM)

51.

World Com (WCOM)

25.

Intel (INTC) 

52.

Xerox (XRX) 

26.

J.P. Morgan and Co. (JPM) 

53.

Yahoo! (YHOO)

27.

JDS Uniphase (JDSU)
  Appendix B:Chronology of Specialists' Payment for Order Flow Arrangements
**October 1999:**

Susquehanna, a large multi-exchange specialist, initiates payment for customer order flow in options orders and ultimately forms direct arrangements with eighteen firms.  Susquehanna initially forms flat fee per month arrangements varying from $10,000 to $225,000 per month and fee per contract arrangements ranging from $.25 to $1.00 per contract.  

**December 1999:**

Spear Leeds, another large multi-exchange specialist, begins paying for customer options order flow, and ultimately forms direct flat fee or fee per contract arrangements with twelve firms.  Flat fee arrangements range from $80,000 to $120,000 per month and fee per contract arrangements range from $.40 to $.60 per contract.

**February 2000:**

Cole Roesler enters into fee per contract arrangements with several firms1 ranging from $.20 to $1.00 per contract.  

**April / May 2000:**

Letco, Gateway Partners, and Botta Capital Management ("Botta") begin to pay several broker-dealers for options order flow.  

**June / July 2000:**

Seven CBOE specialists enter into nearly identical fee per contract arrangements with one order routing firm.

**July 2000:**

Knight Financial Products enters into fee per contract arrangements with several firms for July 2000 order flow.  TFM Investment Group initiates fee per contract arrangements with several firms.
Appendix C:OEA Statistics Detailing Execution Information
This material is available in PDF Format.

**View Appendix C now**
Footnotes
**Executive Summary**

1  Securities Exchange Act Release No. 43590 (Nov. 17, 2000), 65 FR 75414 (Dec. 1, 2000) ("Execution Quality Disclosure Release").

2   This review was conducted by the Staff of the Commission.  The findings contained in this report are those of the Commission's Staff and do not represent the findings or views of the Commission.

**Main Body**

1   In this report, the Staff uses the generic term "specialist" when referring to American Stock Exchange ("Amex") and Philadelphia Stock Exchange ("Phlx") specialists, Chicago Board Options Exchange ("CBOE") Designated Primary Market Makers ("DPMs"), Pacific Exchange ("PCX") Lead Market Makers ("LMMs"), or ISE Primary Market Makers ("PMMs").  In addition, the Staff uses the term "market maker" to refer to all non-specialist securities dealers on the five options exchanges.

2   This report provides information and data as of November 30, 2000, but because the options markets are so dynamic, payment for order flow arrangements and practices are expected to change.

3   Securities Exchange Act Release No. 43591 (Nov. 17, 2000), 65 FR 75439 (Dec. 1, 2000).

4   Securities Exchange Act Release No. 43086 (July 28, 2000), 65 FR 48023 (Aug. 4, 2000).

5   Securities Exchange Act Release No. 43590 (Nov. 17, 2000), 65 FR 75414 (Dec. 1, 2000) ("Execution Quality Disclosure Release").

6*Id.*

7   The CBOE and the Amex listed options on Dell, which had previously been exclusively-listed on the Phlx and was the Phlx's most actively-traded issue.  The Phlx immediately responded by listing options on Coca-Cola, IBM, and Johnson & Johnson, which had been exclusively-listed on the CBOE, and Apple Computer, which had been exclusively-listed on the Amex.  The following week, the Phlx listed seven more of the CBOE's exclusively-listed options:  Hewlett Packard, EMC, Oracle, Merck, Eastman Kodak, Exxon, and Texas Instruments.  The CBOE simultaneously listed 15 options that had been exclusively-listed on the PCX, including Microsoft, Compaq Computer, Sun Microsystems, and Nike, and 14 more options that had been exclusively-listed on the Phlx.  In mid-September 1999, the CBOE listed 43 options that had been exclusively-listed on the Amex, including Intel Corp.  The Amex then listed 35 options from the CBOE, including AT&T.

8   Securities Exchange Act Release No. 43591 (Nov. 17, 2000), 65 FR 75439 (Dec. 1, 2000).

9   Securities Exchange Act Release No. 43268 (Sept. 11, 2000).****

10*United States v. American Stock Exchange, LLC et al.*, Case No. 1:00CV02174.2.

11   Informed orders are "submitted by persons with better information than is generally available in the market."  *See* Securities Exchange Act Release No. 43590 (Nov. 17, 2000), 65 FR 75414 (Dec. 1, 2000).  Conversely, uninformed orders are orders submitted by persons possessing, at most, information generally available in the market.

12   The Staff reviewed the order handling practices of 24 broker-dealers with significant retail options order flow.  The Staff requested and received order routing data in 53 options classes from each firm. The Staff chose classes that were multiply-listed on at least three options exchanges and traded by multi-exchange specialist firms.  *See* Appendix A for a list of the 53 options classes reviewed.  The Staff also collected information from broker-dealers, specialists, and exchanges detailing payment for order flow arrangements.  Based on this information, the pie charts illustrate the percentage of customer options orders in the 53 classes that were paid for by specialists versus the percentage of customer orders in those classes that were not paid for by specialists for the months of August 1999, December 1999, March 2000, and August 2000.

13   Most broker-dealers allow an individual investor to direct their orders to a particular market destination; however, individual investors rarely designate the market center to which they want their order routed.

14*E.g**.* Advanced Clearing Inc. (Ameritrade), Datek Online Brokerage Services LLC, E*Trade Securities Inc., TD Waterhouse Investor Services, Inc., and Wall Street Access.

15*E.g**.*ABN Amro Inc., DLJ Pershing , ED&F Man International Inc., Fiserv Securities Inc., Investec Ernst & Company, Mesirow Financial, National Financial Services LLC (Fidelity), and U.S. Clearing Corporation.

16*E.g**.*Merrill Lynch and Co., Salomon Smith Barney, Inc., Prudential Securities Incorporated, Paine Webber Incorporated, Dain Rauscher Incorporated, Charles Schwab and Co., Inc., Bear, Stearns and Co., Inc., Dreyfus Brokerage Services, Janney Montgomery Scott, LLC, Morgan Stanley Dean Witter, and Brown and Company.

17  Within the past several years, the exchanges have increased the maximum order size eligible for automatic execution.  Order routing firms typically route retail options market orders to each market center's automatic execution system because these systems provide quicker executions and reduced costs by eliminating manual handling of the orders.
Currently, public customer market and marketable limit orders for up to seventy-five contracts may be eligible for execution on the exchanges' automatic execution systems.  In addition, the Amex and the PCX recently submitted proposals to increase the maximum order size eligible for automatic execution from seventy-five contracts to one hundred contracts.  *See* Securities Exchange Act Release Nos. 43516 (Nov. 3, 2000), 65 FR 69079 (Nov. 15, 2000) (Amex-99-45) (approval order to increase from fifty contracts to seventy-five contracts); 43517 (Nov. 3, 2000), 65 FR 69082 (Nov. 15, 2000) (CBOE-99-51) (approval order to increase from fifty contracts to seventy-five contracts); 43518 (Nov. 3, 2000), 65 FR 69111 (Nov. 15, 2000) (PCX-00-32) (approval order to increase from fifty contracts to seventy-five contracts); 43515 (Nov. 3, 2000), 65 FR 69114 (Nov. 15, 2000) (Phlx-99-32) (approval order to increase from fifty contracts to seventy-five contracts); and 43519 (Nov. 3, 2000), 65 FR 69112 (Nov. 15, 2000) (PCX-00-18) (proposal to increase from seventy-five contracts to one hundred contracts); *See also* Amex-00-57 (proposal to increase from seventy-five contracts to one hundred contracts).

18   "Two-dollar floor brokers," also referred to as independent floor brokers, execute orders on the floor of exchanges on behalf of other firms in exchange for a negotiated commission, historically $2 per 100 shares in the equities markets.

19   In April 2000, CBOE specialists formed an association ("DPM Association") to market their execution quality to order routing firms.  Shortly after the DPM Association was established, several CBOE specialists entered into fee for contract arrangements with an order routing firm.  The DPM Association tracks its members' payments to order routing firms.

20   A "contract" represents 100 shares of the underlying security. 

21*See* Securities Exchange Act Release No. 43112 (Aug. 3, 2000), 65 FR 49040 (Aug. 10, 2000).

22  Securities Exchange Act § 19(b)(3)(A)(ii) provides that a rule change establishing or changing a due, fee, or other charge imposed by an exchange is immediately effective upon filing.  The Commission, however, may summarily abrogate such a rule change within 60 days of filing if it appears to the Commission that such action is necessary or appropriate in the public interest, for the protection of investors, or otherwise in furtherance of the purposes of the Exchange Act.  *See* Securities Exchange Act §19 (b)(3)(c).

23*See* Securities Exchange Act Release Nos. 43177 (Aug. 18, 2000), 65 FR 51889 (Aug. 25, 2000) (Phlx); 43228 (Aug. 30, 2000), 65 FR 54330 (Sept. 7, 2000) (Amex); and 43290 (Sept. 13, 2000), 65 FR 57213 (Sept. 21, 2000) (PCX).

24   The ISE filed a rule change under Securities Exchange Act § 19(b)(2) and Rule 19b-4(a), which requires a public comment period and approval or disapproval by Commission order.  The Commission published this proposal for comment.  *See* Securities Exchange Act Release No. 43462 (Oct. 19, 2000).  On December 1, 2000, the ISE filed rule change under Securities Exchange Act  § 19(b)(3), "proposing to establish a new marketing fee to be imposed on market makers to fund a payment for order flow program" on an interim basis pending the Commission's decision whether to approve ISE's § 19(b)(2) filing.   Securities Exchange Act Release No. 43688 (Dec. 7, 2000).

25*See* Securities Exchange Act Release No. 43480 (Oct. 25, 2000), 65 FR 70952 (Nov. 28, 2000).

26*See* Securities Exchange Act Release No. 43481 (Oct. 25, 2000), 65 FR 66277 (Nov. 3, 2000).  The Phlx defines a broker-dealer order as "an order, entered from other than the floor of the exchange, for any account: (i) in which the holder of a beneficial interest is a member or non-member broker-dealer; or (ii) in which the holder of a beneficial interest is a person associated with or employed by a member or non-member broker-dealer." *Id.*

27  As of November 30, 2000, the Amex had collected the transaction fee for trading in July and August 2000, but had not completed a full cycle of receiving payment instructions from specialists and issuing checks in accordance with those instructions.  The Phlx began collecting its transaction fee in August 2000, but had not completed a full cycle of receiving payment instructions or issuing checks.  The ISE began collecting its transaction fee in December 2000.

28  Small retail customer orders, which are almost always executed at the bid or offer, tend to be relatively more profitable for specialists than larger institutional orders, which tend to be represented by professional traders who negotiate with the dealers for better prices.

29   The Staff requested from all specialists and order routing firms reviewed by the Staff detailed information regarding any payment for order flow arrangements.

30*E.g**.* "The specialists will be able to use the funds collected with respect to a particular option to make payment to broker-dealers for order flow in that option."  Securities Exchange Act Release No. 43177 (Aug. 18, 2000), 65 FR 51889 (Aug. 25, 2000) (Phlx).  "The funds will be made available to the . . . LMM at the trading post where the funds were collected, for the LMMs use in attracting orders in the options traded at that post."  Securities Exchange Act Release No. 43290 (Sept. 13, 2000), 65 FR 57213 (Sept. 21, 2000) (PCX).  "The specific terms governing the orders that qualify for payment and the amount of any payment to be made will be determined by the DPMs in whatever manner they believe is most likely to be effective in attracting order flow to the Exchange in the options traded at the DPMs' assigned post."  Securities Exchange Act Release No. 43112 (Aug. 3, 2000), 65 FR 49040 (Aug. 10, 2000) (CBOE).  "Each specialist will use the funds to attract orders in the classes of options that the specialist trades."  Securities Exchange Act Release No. 43228 (Aug. 30, 2000), 65 FR 54330 (Sept. 7, 2000) (Amex).

31   Any rules that would establish precisely how a specialist paid order routing firms would have to be filed with and approved by the Commission.  The exchanges chose not to get involved in this aspect of their specialists' business.

32   For example, in September 2000, in 27 of 53 classes reviewed by the Staff, 50% or more of the order flow went to one primary options exchange.

33   NASD Rule 2320 requires Nasdaq members to "use reasonable diligence to ascertain the best inter-dealer market for the subject security so that the resultant price to the customer is as favorable as possible under prevailing market conditions."

34   No rule explicitly requires listed options to trade only on the floor of a national securities exchange.   A combination of Options Clearing Corporation bylaws and state laws, however, makes it difficult to trade listed options off an exchange floor.

35   The competitive pricing process is electronic on the ISE.

36   In settling the Commission's administrative proceeding, the Commission required the exchanges to improve their surveillance for anti-competitive practices.

37   Certain exchange rules guarantee participation by firms crossing options orders.  For example, CBOE Rule 6.74 (d) entitles a floor broker to cross at least 20% of the contracts in the order if the cross matches the trading crowd's market or 40% of the contracts in the order if the cross improves the trading crowd's market.  *See also* Amex Rule 950(d) and PCX Rule 6.47.

38*See* Securities Exchange Act Release Nos. 42894 (June 2, 2000), 65 FR 36850 (June 12, 2000) (File No. SR-Amex-99-36); 42835 (May 26, 2000), 65 FR 35683 (June 5, 2000) (File No. SR-CBOE-99-10); 42848 (May 26, 2000), 65 FR 36206 (June 7, 2000) (File No. SR-PCX-99-18).  *See also* Securities Exchange Act Release No. 42455 (Feb. 24, 2000), 65 FR 11388 (Mar. 2, 2000) (concerning the ISE's "facilitation mechanism"). 

39   The CBOE, Phlx, and ISE generally grant their specialists the right to trade with 30% of each order when three or more other crowd participants are present, although this percentage rises to 40% when only two others are present, and to 50% or 60% when only one is present.  On the Amex, the specialist receives 60% when only one crowd participant is on parity with the specialist, 40% when two to four are on parity, 30% when five to seven are on parity, 25% when eight to fifteen are on parity, and 20% when 16 or more are on parity.  The PCX generally grants the specialist 50% of each trade, although since multiple-listing, this has been reduced to 25% and 35% in some of the most actively-traded issues.  For a more detailed review, *see* Securities Exchange Act Release No. 43100 (July 31, 2000), 65 FR 48778 (Aug. 9, 2000).  

40*See e.g.*, rules reducing and eliminating options transaction fees:  Securities Exchange Act Release Nos. 43343 (Sept. 26, 2000), 65 FR 59243 (Oct. 4, 2000) (SR-Phlx-00-80); 43115 (Aug. 3, 2000), 65 FR 49280 (Aug. 11, 2000) (SR-PCX-00-16); 42850 (May 30, 2000), 65 FR 36187 (June 7, 2000) (SR-CBOE-00-06); 42675 (Apr. 13, 2000), 65 FR 21223 (Apr. 20, 2000) (SR-Amex-00-15); 41270 (Apr. 9, 1999), 64 FR 19395 (Apr. 20, 1999) (SR-CBOE-99-08); 41307 (Apr. 16, 1999), 64 FR 20349 (Apr. 26, 1999) (SR-PCX-99-09); 41317 (Apr. 24, 1999), 64 FR 23144 (Apr. 29, 1999) (SR-Phlx-99-09); and 41370 (May 5, 1999), 64 FR 25931 (May 13, 1999) (SR-Amex-99-12).

41   In the releases proposing and adopting disclosure of payment for order flow practices, the Commission noted that payment for order flow may result in several benefits for retail customers, such as lower execution costs, technological advances in retail customer order handling practices, and increased competition among broker-dealers and securities markets.  *See* Securities Exchange Act Releases Nos. 33026 (Oct. 6, 1993), 58 FR 52934 (Oct. 13, 1993) and 34902 (Oct. 27, 1994), 59 FR 55006 (Nov. 2, 1994).

42   In September 2000, one order routing firm reduced options commissions on market orders to $8.00 plus $1.75 per contract, from $25 plus $1.75 per contract.  Another firm passes on all payments for order flow to its customers.

43   Securities Exchange Act Rule 10b-10(a)(7)(iii) requires broker-dealers to state "whether payment for order flow is received by the broker or dealer for transactions in such securities and that the source and nature of the compensation received in connection with the particular transaction will be furnished upon written request of the customer."  In addition, Securities Exchange Act Rule 11Ac1-2 requires broker-dealers to inform customers, in writing upon opening a new account, the broker-dealer's policies regarding receipt of payment for order flow and a detailed description of the nature of the compensation received.  

44   Securities Exchange Act Release No. 43590 (Nov. 17, 2000), 65 FR 75414 (Dec. 1, 2000) ("Execution Quality Disclosure Release").  Firms will be required to identify the ten venues to which the largest number of total customer options orders were routed for execution and any venue to which five percent or more of customer options orders were routed for execution.  In addition, firms must identify the percentage of total non-directed options orders routed to the venue, and the percentages of total non-directed market orders, limit orders, and other orders that were routed to the venue.  The term "venue" is intended to be interpreted broadly to cover market centers as well as any other person or entity to which a broker routes customer orders for execution.  The term "market center" is defined for purposes of Securities Exchange Act Rule 11Ac1-6 as "any exchange market maker, OTC market maker, alternative trading system, national securities exchange, or national securities association."  

45*See* Restatement 2d Agency Sec. 387 (1958); *E.F. Hutton & Co.*, Securities Exchange Act Release No. 25887 (1988); *Opper v. Hancock*, 250 F. Supp. 668, 673-74 (S.D.N.Y.), *aff'd* 367 F.2d 157 (2d Cir. 1966); NASD Rule 2320; NYSE Rule 123.41.

46*See* Securities Exchange Act Release No. 37619A (Sept. 6, 1996), 61 FR 48290 (Sept. 12, 1996).

47*Id*. at 48323.

48*Id*. 

49*Id*.

50*See* Securities Exchange Act Release No. 43462 (Oct. 19, 2000), 65 FR 64466 (Oct. 27, 2000).  

51   Securities Exchange Act Release No. 43113 (Aug. 3, 2000), 65 FR 49038 (Aug. 10, 2000).

52*Id**.*

53  The best bid or offer ("BBO") is defined as the highest bid or lowest offer among the five exchanges.  Currently, there is no standardized options national best bid or offer.  Instead, each exchange calculates its own best bid or offer.

54   In the adopting release for the Order Handling Rules, Securities Exchange Act Release No. 37619A, (Sept. 6, 1996), 61 FR 48290 (Sept. 12, 1996), the Commission stated in part, "`[p]rice improvement' in this context is defined as the difference between execution price and the best quotes prevailing in the market at the time the order arrived at the market or market center."  *Id.* at footnote 357.  As noted above, the CBOE does not use this definition of price improvement in its execution quality reports.

55   Securities Exchange Act Release No. 43384 (Sept. 29, 2000), 65 FR 59486 (Oct. 5, 2000).

56   Securities Exchange Act Release No. 43436 (Oct. 11, 2000), 65 FR 63281 (Oct. 23, 2000).

57*Id**.*

58   TAG defines the liquidity premium as the dollar amount of variation between the execution price and the midpoint of the spread.

59   All of the exchanges appear to analyze only those orders received electronically.

60   The Options Price Reporting Authority ("OPRA") Plan for Reporting of Consolidated Options Last Sale Reports and Quotation Information ("OPRA Plan"), is a National Market System Plan approved by the Commission pursuant to Section 11A of the Securities Exchange Act of 1934 and Rule 11Aa3-2.  *See* Securities Exchange Act Release No. 17638 (Mar. 18, 1981).  The OPRA Plan provides for the collection and dissemination of last sale and quotation information on options that are traded on the participant exchanges.  The six exchanges that are participants in the Plan are the Amex, the CBOE, the ISE, the New York Stock Exchange, Inc., the PCX, and the Phlx.  

61   Robert Battalio, Assistant Professor, Department of Finance, University of Notre Dame; Brian Hatch, Assistant Professor, Department of Finance, University of Cincinnati; and Robert Jennings, Chairman and Jack R. Wentworth Professor of Business, Department of Finance, Indiana University.

62   To date, Spear Leeds has provided its report to several broker-dealers upon request.  Initially, these reports only included information regarding executions occurring on the Amex.  Spear Leeds stated that the reports will include information regarding executions on all exchanges.

63*See* Securities Exchange Act Release No. 43590 (Nov. 17, 2000), 65 FR  75414 (Dec. 1, 2000).

64   Securities Exchange Act Release No. 43084 (July 28, 2000), 65 FR 48406 (Aug. 8, 2000).

65   The Staff requested order routing information in 53 options classes from each firm reviewed. The Staff chose classes that were multiply-listed on at least three options exchanges and traded by at least one multi-exchange specialist firm.  *See* Appendix A for a list of the 53 options classes.

66  The Staff analyzed the following 12 classes: Amazon.com, Inc., America Online, Inc., Applied Materials, Inc., Cisco Systems, Inc., CMGI, Inc., International Business Machines, JDS Uniphase Corporation, Lucent Technologies, Inc., Microsoft Corporation, Oracle Corporation, Qualcomm Inc., and Yahoo! Inc.

67  In analyzing routing patterns, the Staff considered a particular options class to have been re-routed to another exchange destination when it was routed from one exchange to another for at least two consecutive months.

68   Results in this summary are provided for August 9-13 and November 8-12, 1999 and February 7-11, June 26-30, and October 23-27, 2000.  These dates were selected because August 1999 provides a benchmark for the period before multiple-listing; November 1999 was before payment plans by individual specialists had begun in earnest; February 2000 was a time when the plans were developing; June 2000 was before exchange-facilitated plans began but after individual specialist plans were in place; and October 2000 was after the introduction of exchange-facilitated plans.  There also was a significant increase in ISE's activity between June and October 2000.  The results for February 2000, as well as additional material, may be found in Appendix C.

69   The effective spread is twice the difference between the trade price and the midpoint of the bid-ask spread at the time the trade report was received by OPRA.  The lower the effective spread, the lower the cost to the investor.

70   The realized spread is twice the difference between the trade price and the midpoint of the bid-ask spread measured five minutes after the time the trade report was received by OPRA.   Realized spreads are a measure of the realized revenue for liquidity providers in executing a trade after allowing for the informational impact of the trade.  The lower the realized spread, the lower the revenue to liquidity providers.

71   For purposes of this section, the consolidated BBO refers to OEA Staff's calculation of the best bid or offer.

72   Options priced below $20 represent between 72% and 97% of all options traded during the five weeks of the sample. For options of all price ranges, the comparable quoted spreads for August, November, June, and October are $0.50, $0.38, $0.52, and $0.45, respectively. 

73   Measuring quoting behavior by examining the percentage of quotes at the exchanges' maximum allowable width is an alternative to dollar spreads for measuring quote aggressiveness.  Higher percentages of quoted spreads on the maximum are consistent with less aggressive quoting.  

74*See* Amex Rule 958(c), CBOE Rule 8.7, PCX Rule 6.37, and Phlx Rule 1014.

75   For all options the comparable effective spreads were $0.22, $0.21, $0.22, $0.18, and $0.17.

76   A more detailed analysis of execution costs by trade size and execution system for the week of June 26 is in Appendix C.

77   For all options the comparable realized spreads were $0.19, $0.16, $0.21 $0.16 and $0.17.

78   A more detailed analysis of executions costs by trade size and executions system for the week of June 26 is in Appendix C.

79   Many of the classes described as new were not multiply-listed in August 1999.  The figures for new exchanges in August apply only to the classes then multiply-listed. 

**APPENDIX B**

1  Cole Roesler made payments directly to the PCX on behalf of two firms to offset PCX fees incurred by these firms on the exchange.  
*http://www.sec.gov/news/studies/ordpay.htm*Home | Previous PageModified:12/19/2000 |
|--|--|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|