Quality Measures and Gap Analysis

CFTC Technology Advisory Committee

Subcommittee on Automated and 

High Frequency Trading

Working Group #2
Working Group #2 Team Members

WG2 Team Members

• Keith Fishe, Managing Partner, TradeForecaster Global Markets LLC, KHF@TradeForecaster.com

• Chris Isaacson, SVP, COO, BATS Global Markets, Inc., CIsaacson@BATSTrading.com

• Paul Kepes, Co-Founder, Chicago Trading Company, Paul@ChicagoTrading.com

• Chris Lorenzen, Founder and CEO, Eagle Seven, LLC, CLorenzen@EagleSeven.com

• Jim Northey, Partner and Co-Founder, LaSalle Technology Group, JimN@LasalleTech.com

Commission Staff

• Harold Hild, Economist, CFTC, HHild@CFTC.gov

• Jorge Herrada, Associate Director, CFTC, JHerrada@CFTC.gov

Additional Contributors

• Jeromee Johnson, VP Market Development, BATS Global Markets, Inc., JJohnson@BATSTrading.com

• Christina Sciotto, Executive Director, Chicago Trading Company, Christina.Sciotto@ChicagoTrading.com

2012-10-30 CFTC TAC HFT Subcommittee Working 

Group 2 2
Assignment

Working Group 2 was asked to comment on 

issues regarding Quality and Data

• Quality can be examined in the following 

areas:

– Marketplace

– Market Participant Activity

– Market Systems and Operations

• Gap Analysis 

2012-10-30 CFTC TAC HFT Subcommittee Working 

Group 2 3
Areas for Quality Measures

2012-10-30 CFTC TAC HFT Subcommittee Working Group 2 4

Marketplace

Market 

Participant 

Activity

Market 

Systems and 

Operations

“A measure for the 

efficiency of allocation 

and the fairness of 

dealing and 

pricing.” (Yano 2009)
MARKETPLACE QUALITY
Market 

Quality

Liquidity

Price 

Discovery

Volatility

Cost of 

Trading

2012-10-30 CFTC TAC HFT Subcommittee Working Group 2 6

Marketplace Quality

“Market quality 

measures price

discovery, liquidity, and 

volatility.” (Brogaard

2010)
MARKET PARTICIPANT QUALITY
Market Participant Quality

• How close are orders resting to the top of the 

book? 

• How much displayed liquidity (quantity) is 

provided? 

• How persistent are the resting bids and offers?

• How quickly are the orders adjusted given 

changing values?

2012-10-30 CFTC TAC HFT Subcommittee Working Group 2 8
Market Participant Quality

• Recent Study Hagströmer and Nordén (2012)

• Study categorized HFT activity by strategy type – HFT 

Market Making versus HFT Opportunistic

• Majority of HFT trading volume and more than 80% of 

limit order submissions were associated with Market 

Making

• Found that policies aimed at limiting the scope of HFT will 

primarily hit market makers. These policies include:

– Imposing a minimum limit order duration on orders

– Imposing reduced order-to-cancel trade ratios

– Transaction taxes

2012-10-30 CFTC TAC HFT Subcommittee Working Group 2 9
Quality Study

• Recent Study led by Professor Sir John Beddington (2012) 

(Over 150 leading academics from more than 20 countries have been involved in 

the work which has been informed by over 50 commissioned papers, which have 

been subject to independent peer review.)

• Study finds that CBT (Computer Based Trading) has:

– Improved Liquidity

– Improved Price Discovery

– Not increased Volatility

– Reduced Transactions Cost

• Study findings are similar to Hagströmer and Nordén

findings in that imposing minimum limit order duration on 

orders and reduced order-to-cancel trade ratios will be 

problematic to the overall quality of the market.

2012-10-30 CFTC TAC HFT Subcommittee Working Group 2 10
MARKET SYSTEM QUALITY
Market System Quality

• Market quality can be affected by issues of market system quality

• Industry has responded by publishing best practices and guidance 

covering systems and operational risk topics

– FIA Divisions ( IT, PTG and EPTA)

– FOA

• Regulators are responding

– ESMA

– ASIC

– SMA

• An independent effort emanating from Illinois Institute of Technology has 

been launched to create a Quality Management System Standard for 

automated trading (ANSI, X9, ISO)

– Interim AT9000 working group

– Draft standards created

– Formal working group to start in November

2012-10-30 CFTC TAC HFT Subcommittee Working Group 2 12
Market System Quality

W. 

Edwards 

Deming

If you can't describe what you are doing as a 

process, you don't know what you're doing.

Quality is everyone's responsibility.

Phil 

Crosby

Quality is the result of a carefully 

constructed cultural environment. It has to 

be the fabric of the organization, not part of 

the fabric.

Thoughts on 

Quality
Why consider a Quality Management 

System Standard?

Market Quality is 

Interdependent

Principles based

Are you willing to vouch for 

and guarantee your 

counterparties’ quality?

• All market participants 

are responsible

• Quality is only as good 

as the least 

performing participant

• Focus on the “WHAT”

• Not the “HOW”

• No one is in a position 

to do this

• This is the role of the 

QMSS

2012-10-30 CFTC TAC HFT Subcommittee Working Group 2 14
Lessons

Global Markets

Multiple 

regulatory 

regimes

One basic 

framework 

globally

Example

ISO 14000

Market Perception

All participants 

have an impact on 

market perceptions

Ensure all market 

participants conform 

to quality expectation

Example

Biofuel industry

Advantage of an 

industry led initiative

Quality is a full 

organizational 

commitment not 

a regulation

Example

Aerospace 

industry led 

quality – it works

Example

Medical industry 

impacted 

negatively by FDA 

regulation

2012-10-30 CFTC TAC HFT Subcommittee Working Group 2 15
AT 9000

• Other industries faced with automation and safety risks 

have turned to industry developed voluntary quality 

management systems.

• The Industry, especially the FIA, has responded by 

developing industry business best practice guidelines.

• Separately an initiative was started to create a quality 

management system standard for automated trading under 

the ISO 9000/9001 framework.

• The AT 9000 effort has cross referenced and used the FIA 

and FOA business practices as inputs.

• Standard is voluntary

– Self assessment

– Independent certification

2012-10-30 CFTC TAC HFT Subcommittee Working Group 2 16
ISO 9001

Registration 

ISO 90003

Guidance

AT 9000

Guidance

Firms desiring 

Independent Audit

Firms desiring self assessment

Self Assessment

QMSS Standards Are Voluntary

2012-10-30 CFTC TAC HFT Subcommittee Working 

Group 2 17

• The Supply Chain drives the adoption

• Requires industry commitment at key 

points in the supply chain
ISO 9001

Registration 

ISO 90003

Guidance ISO 9000

PTG/EPTA

Software Development 

and Change Control

Order Handling Access Risk 

Management

Existing ISO Standards

Quality Management System 

Standard

For Automated Trading

GUIDANCE ON SYSTEMS 

AND CONTROLS FOR 

ELECTRONIC TRADING 

ENVIRONMENTS

Guidelines

Systems and controls in an 

automated trading 

environment for trading...

ISO 12207

Risk Controls for 

Trading Firms

AT 9000 AT 9000

2012-10-30 CFTC TAC HFT Subcommittee Working Group 2 18
AT 9000 Timeline

Date Milestone

7/2012 Informal Working Group draft 

9/2012 New work item was approved by X9 

10/2012 Informal working group draft 

11/2012 X9 forms formal working group – need industry participants

12/2012 X9 completes review and revisions 

1/2013 X9 Ballot for approval 

1/2013 Education Process starts

2/2013 Propose ISO TC68 New work item

4/2013 ANSI Quality Management System for Automated Trading Issued

12/2013 ISO Quality Management System for Automated Trading Issued

2012-10-30 CFTC TAC HFT Subcommittee Working Group 2 19
DATA ISSUES
Market Behavior Data Gap Analysis

• The Futures Markets have the benefit of having SROs that 

provide the vast majority of data available now

• The CFTC should pursue industry standard data formats and 

definitions

• The FIX standard provides data item definitions and data 

model that are widely adopted by industry participants

– Simple

– Easy to access

– Industry accepted

– The FIA has provided quite a bit of enhancements to capture 

relevant data within FIX standard for automated trading already

2012-10-30 CFTC TAC HFT Subcommittee Working Group 2 21
Audit Data and Time Clocks

• Audit trail data should be collected in a way that enables meaningful 

aggregation, reconstruction, and analysis. 

• Critical tools to enhancing regulators’ ability to reconstruct and properly 

sequence activity include: uniform market participant identifiers, 

consistent approaches to the type of data to be collected, and the use of 

accurate time-stamp mechanisms. 

• Precise time-stamping is critical to reconstructing and sequencing market 

events. 

However, current state-of-the-art methods of clock synchronization still 

result in material differences versus actual time. Server clocks are 

inaccurate, and even after synchronization, they drift. This reality poses a 

significant hurdle in efforts to coalesce data that has been time-stamped by 

multiple clocks, across multiple trading venues. 

2012-10-30 CFTC TAC HFT Subcommittee Working Group 2 22
MARKET 

PARTICIPANT

BENEFICIARY

ORDER / QUOTING

CONNECTIVITY

TRADING 

PLATFORM

MARKET DATA 

CONNECTIVITY 

TYPE

Microsecond Time Stamp

Resolution

Colocated

Proximity

Remote

Hosted

MARKET DATA 

PROXIMITY

FIX

Proprietary

Terminal

ORDER / QUOTING

PROXIMITY

Derived Data

ENTITIES

Broker

Trading Firm

Trading Entity

LEI?

MARKET INFRASTRUCTURE

Colocated

Proximity

Remote

Hosted

TRADING METHOD

MARKET BEHAVIOR

Exchange 

API

Aggregator

ORDER BOOK ORDER BOOK EVENTS

CONSOLIDATED DATA

AUDIT TRAIL

Market Data

New Order

Cancel

Modify

Fill

Audit Trail Data Model

2012-10-30 CFTC TAC HFT Subcommittee Working Group 2 23

Common standard across 

global markets
Examples of Derived data

Statistic Aggregation Level Unit Description

Order Chain Duration Per Order Microseconds Length of time order remained in book

before being filled or cancelled

Order Duration Per Order Microseconds Length of time order remained in book

before being modified, filled, or cancelled

Average Order Duration Per Trader

Per Firm

Per Instrument

Microsecond Average order duration for all orders for a

specific instrument.

Order to Fill Ratio Per Trader SenderSubID(tag 

50)

Per Firm

Per Instrument

Count Number of orders entered to number of

orders filled.

Order Efficiency Per Trader

Per Firm

Per Instrument

Count Execution Quantity or Notional Value

(rather than execution count) to

Quantity/Value entered/modified

Order Aggressiveness Per Trader

Per Firm

Per Instrument

Count Orders that are on/near the Top of the

Book are more valuable/subject to more

risk than those that are away from the

market

Reject/Order Ratio Per Trader

Per Firm

Per Instrument

* by reject Reason

Count Can indicate systems issues or risk issues

2012-10-30 CFTC TAC HFT Subcommittee Working Group 2 24