Patent application EP18382188A (EP), filed 2018-03-21
Publication number: EP3544236A1
Title: METHOD AND SYSTEM FOR TRAINING AND VALIDATING MACHINE LEARNING ALGORITHMS IN DATA NETWORK ENVIRONMENTS
Abstract: A system and method for training and validating ML algorithms in real networks, comprising:- generating synthetic traffic (111, 113) and receiving (141) it along with real traffic (210, 310);- aggregating (142) the received traffic (121) into network flows by using metadata and transforming them to generate a first dataset readable by the ML algorithm, comprising features defined by the metadata;- labelling (400) the traffic and selecting (506) a subset of the features from the labelled dataset used in an iterative training (2041, 507) to generate (2010, 508) a trained model;- filtering out (512) a part of real traffic to obtain a second labelled dataset;- selecting (513) a subset of features from the second labelled dataset used for validating the trained model by comparing (516) predicted results (515) for the trained model and the labels; repeating the steps with a different subset of features to generate another trained model until results are positive in terms of precision or accuracy.
IPC codes: H04L  12/    26            A I
G06N  20/    00            A I
Claims: number: 1
text: A method for training and validating machine learning algorithms in real network environments wherein real traffic (210, 310) is moving between an internal network (200) and an external network (300), characterized by comprising:
number: 2
text: The method according to any preceeding claim, wherein the packets of the received traffic (121) are aggregated (142) by metadata extracted from the packets and selected from source IP address, destination IP address, source port, destination port, TCP sequence number, start date and end date or any other value of the network packet.
number: 3
text: The method according to any preceeding claim, wherein the packets of the received traffic (121) are aggregated (142) by metadata calculated (143) and selected from counters of packets per flow, list of protocol flags, average size of bytes and any other statistical traffic information derived from the aggregation of the packets belonging to the same network flow.
number: 4
text: The method according to any preceeding claim, further comprising storing (144) in a historic cache the aggregated network flow and including additional metadata from other previous related flows in a defined time window (145), the additional metadata being selected from number of identical flows, average number of packets per flow in the identical flows, time between identical flows, and any other features mathematically derived from the historic cache for a period of time in the past.
number: 5
text: The method according to any preceeding claim, further comprising:
number: 6
text: The method according to any preceeding claim, further comprising removing (505) the labels obtained from labelling (400) which are not needed in training process (2001) if the machine learning algorithm is unsupervised, before selecting (506) the features from the first dataset to be used in the iterative training (2041, 507).
number: 7
text: The method according to any preceeding claim, wherein the step of comparing (516) is performed in terms of precision, accuracy, or any other criteria defined by configurable parameters (510).
number: 8
text: A system for training and validating machine learning algorithms in real network environments comprising an internal network (200) and an external network (300) providing real traffic (210, 310) through a network device (121); the system characterized by comprising:
number: 9
text: The system according to claim 8, wherein the generator of synthetic traffic (111, 113) is selected from a synthetic user module (110) and a synthetic server (130).
number: 10
text: The system according to any of claims 8-9, wherein the probe (140) aggregates (142) packets of the received traffic (121) by metadata extracted from the packets and selected from source IP address, destination IP address, source port, destination port, TCP sequence number, start date and end date or any other value of the network packet.
number: 11
text: The system according to any of claims 8-10, wherein the probe (140) aggregates (142) packets of the received traffic (121) by metadata calculated (143) and selected from counters of packets per flow, list of protocol flags, average size of bytes and any other statistical traffic information derived from the aggregation of the packets belonging to the same network flow.
number: 12
text: The system according to any of claims 8-11, further comprising a historic cache for storing (144) the aggregated network flow and including additional metadata from other previous related flows in a defined time window (145), the additional metadata being selected from number of identical flows, average number of packets per flow in the identical flows, time between identical flows, and any other features mathematically derived from the historic cache for a period of time in the past.
number: 13
text: The system according to any of claims 8-12, wherein the train and validation module (150) is further configured for:
number: 14
text: The system according to any of claims 8-13, wherein the train and validation module (150) is further configured for removing (505) the labels obtained from labelling (400) which are not needed in training process (2001) if the machine learning algorithm is unsupervised, before selecting (506) the features from the first dataset to be used in the iterative training (2041, 507).
number: 15
text: The system according to any of claims 8-14, wherein the train and validation module (150) compares (516) in terms of precision, accuracy, or any other criteria defined by configurable parameters (510).
Office action: search_report, dated 2019-09-25
Prior art cited: ref_id: US8682812B1
ref_type: patent
severity: background
ref_id: US2016105364A1
ref_type: patent
severity: background
ref_id: US2014090058A1
ref_type: patent
severity: background
ref_id: US2016283859A1
ref_type: patent
severity: background
ref_id: US8682812B1
ref_type: patent
severity: background
ref_id: - THUY T T NGUYEN ET AL: "A survey of techniques for internet traffic classification using machine learning", IEEE COMMUNICATIONS SURVEYS AND TUTORIALS, INSTITUTE OF ELECTRICAL AND ELECTRONICS ENGINEERS, US, vol. 10, no. 4, 1 October 2008 (2008-10-01), pages 56 - 76, XP011249736, ISSN: 1553-877X
ref_type: npl
severity: background
ref_id: US8682812
ref_type: patent
severity: background
ref_id: - THUY T T NGUYEN ET AL, "A survey of techniques for internet traffic classification using machine learning", IEEE COMMUNICATIONS SURVEYS AND TUTORIALS, INSTITUTE OF ELECTRICAL AND ELECTRONICS ENGINEERS, US, (20081001), vol. 10, no. 4, ISSN 1553-877X, pages 56 - 76, XP011249736 [I] 1-15 * page  61  - page  62; figures 2-4 *
ref_type: npl
severity: background
Final disposition: pending (2019-09-25)