Predicting Memory Accesses: The Road to Compact ML-driven Prefetcher

Ajitesh Srivastava, Aggelos Lazaris, Benjamin Brooks, Rajgopal Kannan, Viktor K. Prasanna

In Proc. of the ACM/IEEE MemSys 19, Washington DC, USA, September 2019

PDF DOI BibTeX

Overview

Takes the first step toward machine-learned hardware prefetchers by shrinking LSTM memory-access predictors by orders of magnitude while keeping their accuracy.

Key contributions

• Output-encoding compression that reduces LSTM parameters by a theoretical factor of n/log n.
• About 100x smaller models (up to 1000x in experiments) with only a small accuracy loss.
• Retrains in a few epochs from a short trace, making online learning practical; improves prefetch coverage over state-of-the-art prefetchers in simulation.

Abstract

With the advent of fast processors, TPUs, accelerators, and heterogeneous architectures, computation is no longer the only bottleneck. In fact for many applications, speed of execution is limited by memory performance. To address memory performance, more accurate prefetching is necessary. While sophisticated machine learning algorithms have shown to predict memory accesses with high accuracy, they suffer with several issues that prevent them from being practical solutions as hardware prefetchers. These issues are centered around size of the model that results in high memory requirement, high latency and difficulty in online retraining. As the first step towards building ML-based prefetchers, we propose a compressed-LSTM approach for accurate memory access prediction. With a novel compression technique based on output encoding, we show that for the problem of predicting one of n memory locations, our technique results in O(n/log n) compression factor over the traditional LSTM approach. We further demonstrate through experiments on several benchmarks that the prediction accuracy drop due to compression is small and the training is fast. The actual compression obtained is of the order of 100×.

Cite this paper

@inproceedings{DBLP:conf/memsys/SrivastavaLBKP19,
  author    = {Ajitesh Srivastava and
               Angelos Lazaris and
               Benjamin Brooks and
               Rajgopal Kannan and
               Viktor K. Prasanna},
  title     = {Predicting memory accesses: the road to compact ML-driven prefetcher},
  booktitle = {Proceedings of the International Symposium on Memory Systems, {MEMSYS}
               2019, Washington, DC, USA, September 30 - October 03, 2019},
  pages     = {461--470},
  year      = {2019},
  crossref  = {DBLP:conf/memsys/2019},
  url       = {https://doi.org/10.1145/3357526.3357549},
  doi       = {10.1145/3357526.3357549},
  timestamp = {Thu, 07 Nov 2019 11:15:09 +0100},
  biburl    = {https://dblp.org/rec/bib/conf/memsys/SrivastavaLBKP19},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}