Patent application EP19177515A (EP), filed 2019-05-30
Publication number: EP3576038A1
Title: TRADE PLATFORM WITH REINFORCEMENT LEARNING
Abstract: Systems are methods are provided for training an automated agent. The automated agent maintains a reinforcement learning neural network and generates, according to outputs of the reinforcement learning neural network, signals for communicating resource task requests. First and second task data are received. The task data are processed to compute a first performance metric reflective of performance of the automated agent relative to other entities in a first time interval, and a second performance metric reflective of performance of the automated agent relative to other entities in a second time interval. A reward for the reinforcement learning neural network that reflects a difference between the second performance metric and the first performance metric is computed and provided to the reinforcement learning neural network to train the automated agent.
IPC codes: G06N  20/    00            A I
G06N   3/    08            A I
G06Q  40/    04            A I
Claims: number: 1
text: A computer-implemented method of training an automated agent, the method comprising:
number: 2
text: The computer-implemented method of claim 1, wherein said computing said reward comprises:
number: 3
text: The computer-implemented method of claim 2, wherein said computing said reward comprises using an equation: Δ VWAP 2 , 1 − µ Δ VWAP σ Δ VWAP
number: 4
text: The computer-implemented method of claim 3, wherein said computing said reward comprises using an equation: reward = CLIP Δ VWAP 2 , 1 − µ Δ VWAP σ Δ VWAP − 1 , 1 10 .
number: 6
text: The computer-implemented method of claim 5, wherein
said processing said first task data comprises VWAP for trades completed in said first time interval in response to requests communicated by said automated agent and computing VWAP for trades completed in response to requests communicated by all entities.
number: 7
text: The computer-implemented method of any claim 6, wherein said normalizing comprises normalizing according to an equation: VWAP normalized = VWAP algo − VWAP market spread market average
number: 8
text: The computer-implemented method of any one of claims 1 to 7, further comprising:
number: 9
text: The computer-implemented method of any one of claims 1 to 8, further comprising:
number: 10
text: A computer-implemented system for training an automated agent, the system comprising:
number: 11
text: The computer-implemented system of claim 10, wherein said memory stores a master model including data for instantiating automated agents, and wherein said automated agent is instantiated according to said master model.
number: 12
text: The computer-implemented system of claim 11, wherein said automated agent is a first automated agent, and wherein said code when executed at said at least one processor further causes the system to instantiate a second automated agent according to said master model.
number: 13
text: The computer-implemented system of claim 12, wherein said code, when executed at said at least one processor, further causes said system to obtain update data from at least one of said first automated agent and said second automated agent, and to process said update data to modify the master model.
number: 14
text: The computer-implemented system of claim 13, wherein said code, when executed at said at least one processor, further causes said system to, upon modifying the master model in response to update data from at least one of the automated agents, apply a corresponding modification to at least the other one of the automated agents.
number: 15
text: A non-transitory computer-readable storage medium storing instructions which when executed adapt at least one computing device to:
Office action: search_report, dated 2019-12-04
Prior art cited: ref_id: US201916423082A
ref_type: patent
severity: background
ref_id: US2015100530A1
ref_type: patent
severity: background
ref_id: US2018040068A1
ref_type: patent
severity: background
ref_id: CN105894379A
ref_type: patent
severity: background
ref_id: CN105894379
ref_type: patent
severity: background
ref_id: US2018040068
ref_type: patent
severity: background
Final disposition: pending (2019-12-04)