Measurementbased Online Available Bandwidth Estimation employing Reinforcement Learning. An accurate and fast estimation of the available bandwidth in a network with varying crosstraffic is a challenging task. The accepted probing tools, based on the fluidflow model of a bottleneck link with firstin, firstout multiplexing, estimate the available bandwidth by measuring packet dispersions. The estimation becomes more difficult if packet dispersions deviate from the assumptions of the fluidflow model in the presence of nonfluid bursty crosstraffic, multiple bottleneck links, and inaccurate timestamping. This motivates us to explore the use of machine learning tools for available bandwidth estimation. Hence, we consider reinforcement learning and implement the singlestate multiarmed bandit technique, which follows the epsilongreedy algorithm to find the available bandwidth. Our measurements and tests reveal that our proposed method identifies the available bandwidth with high precision. Furthermore, our method converges to the available bandwidth under a variety of notoriously difficult conditions, such as heavy traffic burstiness, different crosstraffic intensities, multiple bottleneck links, and in networks where the tight link and the bottleneck link are not same. Compared to the piecewise linear network a modelbased direct probing technique that employs a Kalman filter, our method shows more accurate estimates and faster convergence in certain network scenarios and does not require measurement noise statistics.

RLSketch: Scaling Reinforcement Learning for Adaptive and Automate Anomaly Detection in Network Data Streams When network is undergoing problems, such as DDoS attack, component failures, etc., the detection of heavy flows (e.g. heavy hitters and heavy changers) is much more critical. However, it has been increasing challenging to ensure the accurate detection of heavy flows while dealing with massive network traffic volume, diversified traffic distribution and the stringent memory requirement. Although recent research efforts like LDSketch are scalable for diverse network traffic, they depend on excessive memory to maintain high accuracy, such that they fail to work well when the memory is limited. We propose RLSketch, a adaptive sketch using reinforcement learning in detecting heavy flows. It predicts potential heavy flows based on the statistics of network traffic, to achieve both high accuracy and scalability with minor memory. Tracedriven evaluation shows that RLSketch achieves higher accuracy than stateoftheart sketchbased technologies with up to 17.79xc3x97 accuracy gain, while maintaining high robustness in extreme conditions.