An LSTM Framework For Modeling Network Traffic

Aggelos Lazaris, Viktor K. Prasanna

In Proc. of the 4th IEEE/IFIP International Workshop on Analytics for Network and Service Management (in conjunction with the IFIP/IEEE International Symposium on Integrated Network Management), Washington DC, USA, April 2019

PDF BibTeX

Overview

Builds a big-data pipeline for raw ISP packet captures and uses it to show that LSTMs can forecast backbone traffic at time scales of 30 seconds or less.

Key contributions

• End-to-end architecture that turns large PCAP logs from a Tier-1 ISP into model-ready time series.
• Comparison of vanilla, delta-based and cluster-based LSTM variants.
• Feature-based clustering so a single model serves each group of similar traffic sources.

Abstract

Forecasting fine-grained network traffic is crucial for many network management and optimization tasks such as traffic engineering, anomaly detection, network accounting, network analytics, load balancing, and traffic matrix estimation. However, building models that are able to predict a wide-variety of network traffic types is not a trivial task due to a) the diversity of network traffic, and b) the computational challenges in processing large datasets to train the prediction models. In this paper, we present a network traffic prediction framework that uses real network traces from a Tier-1 ISP to train a Long Short-Term Memory (LSTM) neural network and generate predictions at short time scales (≤ 30 seconds). In order to reduce the number of models needed to capture the very diverse dynamics of the various traffic sources, we develop a feature-based clustering framework that acts as a preprocessing step in order to group similar time-series together and train a single model for each group. Our extensive experimental evaluation study shows that LSTMs can indeed be used to predict network traffic with low prediction errors.

Cite this paper

@inproceedings{DBLP:conf/im/LazarisP19,
  author    = {Aggelos Lazaris and
               Viktor K. Prasanna},
  title     = {An {LSTM} Framework For Modeling Network Traffic},
  booktitle = {{IFIP/IEEE} International Symposium on Integrated Network Management,
               {IM} 2019, Washington, DC, USA, April 09-11, 2019},
  pages     = {19--24},
  year      = {2019},
  crossref  = {DBLP:conf/im/2019},
  url       = {http://ieeexplore.ieee.org/document/8717920},
  timestamp = {Wed, 16 Oct 2019 14:14:51 +0200},
  biburl    = {https://dblp.org/rec/bib/conf/im/LazarisP19},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}