Performance and power modeling on disparate computer systems using machine learning

Abstract

Performance and Power prediction is an active area of research due to its applications in the advancements of hardware-software co-development. We have performed experiments to evaluate the performance of several machine learning models. Our results for performance prediction show that the tree-based machine-learning models outperform all other models with median absolute percentage error (MedAPE) of less than 5% followed by bagging and boosting models that help to improve weak learners. We have collected performance data both from simulation-based hardware as well as from physical systems and observed that prediction accuracy is higher on simulation-based hardware due to its deterministic nature as compared to physical systems. Moreover, in physical systems, prediction accuracy of memory-bound applications is higher as compared to compute-bound algorithms due to manufacturer variability in processors. Furthermore, our result shows the prediction error of 15% in case of crosssystems prediction whereas in case of the cross-platform prediction error of 17% for simulation-based X86 to ARM prediction and 23% for physical Intel Core to Intel-Xeon system using best performing tree-based machine-learning model. We have employed several machine learning univariate or multivariate models for our experiments. Our result shows that runtime and power prediction accuracy of more than 80% and 90% respectively is achieved for multivariate deep neural network model in cross-platform prediction. Similarly, for cross-system prediction runtime accuracy of 90% and power accuracy of 75% is achieved for the multivariate deep neural network.