Investigation of the Possibility of Using Linear Regression for Predicting Memory Consumption in a Highload Information System
DOI:
https://doi.org/10.14529/ctcr180301Keywords:
machine learning, linear regression, operating system process, random access memoryAbstract
The article considers the actual problem of planning tasks in highloaded information systems at the moment. The purpose of this paper is to test the hypothesis that the congestion of highperformance information systems depends on the external parameters of the environment in which they operate. For verification, the system on which the corporate website of the company, the monitoring system and the application for the social network vk.com were collected and launched. As external parameters were chosen as natural phenomena, as well as statistical data of visiting popular sites, as well as exchange rates and shares. In our opinion, these parameters may to some extent influence the workload of the information system. The data was collected during the month of the system operation every ten minutes. At each collection of information for each running process in the system, the amount of memory it consumes is remembered. To identify the model, the linear regression method was chosen, as the most simple and often used option for verifying implicit dependencies between data. All the collected parameters were filtered out – checked for cross-matching and normalized. Using the constructed model, we predicted the value of memory consumed by processes. For each predicted value, the root-mean-square deviation was calculated. Analysis of the results showed that the model constructed has a number of problems. As recommendations for improving the results, the use of another method to build a model is indicated, as well as improvement of the quality and quantity of data collected. Further plans include exploring the possibility of predicting the CPU time of a highload information system using external parameters.
References
Окороков В.А. Операционные системы: курс лекций. Челябинск: Изд-во Челябинского гос. ун-та, 2011. 288 c. [Okorokov V.A. Operacionnye sistemy: kurs lektsiy [Operating Systems: Course of Lectures]. Cheliabinsk, ChSU Publ., 2011. 288 p.]
Peter S., Jialin Li, Zhang I., Dan R. K. Ports, Woos D., Krishnamurthy A., Anderson T., Roscoe T. Arrakis: The Operating System Is the Control Plane. ACM Transactions on Computer Systems, 2015, vol. 33, no. 4, article 11.
Yang R., Ouyang X., Chen Y., Townend P., Xu J. Intelligent Resource Scheduling at Scale: a Machine Learning Perspective. IEEE International Symposium on Service Oriented System Engineering, 2018, pp. 132–141. DOI: 10.1109/SOSE.2018.00025
Zheng H., Yuan J., Chen L. Short-Term Load Forecasting Using EMD-LSTM Neural Networks with a Xgboost Algorithm for Feature Importance Evaluation. Energies, 2017, vol 10, no. 8. Available at: http://www.mdpi.com/1996-1073/10/8/1168/htm (accessed 1 August 2017). DOI: 10.3390/en10081168
Divina F., Gilson A., Goméz-Vela F., García Torres M., & Torres J.F. Stacking Ensemble Learning for Short-Term Electricity Consumption Forecasting. Energies, 2018, vol. 11, no. 4. Available at: http://www.mdpi.com/1996-1073/11/4/949/htm (accessed 9 April 2018). DOI: 10.3390/en11040949
Dahua Gan, Yi Wang, Ning Zhang, Wenjun Zhu. Enhancing Short-Term Probabilistic Residential Load Forecasting with Quantile Long-Short-Term Memory. The Journal of Engineering, 2017, vol. 2017, iss. 14, pp. 2622–2627. DOI: 10.1049/joe.2017.0833
Назаров С.В., Широков А.И. Современные операционные системы. М., 2012. 367 c. [Nazarov S.V., Shirokov A.,I. Sovremennie operatsionnye systemi [Modern Operating Systems]. Moscow, 2013. 367 p.]
Fair scheduler (2018). Available at: http://hadoop.apache.org/docs/current/hadoop-yarn/hadoopyarn-site/FairScheduler.html (accessed 16 April 2018).
Capacity scheduler(2018). Available at: http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html (accessed 16 April 2018).
Wang F., Gao X., Chen G. Lowering the Volatility: a Practical Cache Allocation Prediction and Stability-Oriented Co-Runner Scheduling Algorithms. The Journal of Supercomputing, 2017, vol. 72, no 3, pp. 1126–1151. DOI: 10.1007/s11227-016-1645-7
Evans R., Gao J. DeepMind AI Reduces Google Data Centre Cooling Bill by 40%. DeepMind Blog (2016), vol. 20. Available at: https://deepmind.com/blog/deepmind-ai-reduces-google-data-centrecooling-bill-40/ (accessed 20 July 2016).
Bećirović E., Ćosović M. Machine Learning Techniques for Short-term load Forecasting. Environment Friendly Energies and Applications (EFEA), 4th International Symposium, 2016, pp. 1–4.
Kim T., Lee D., Choi J., Spurlock A., Sim A., Todd A., Wu K. Extracting Baseline Electricity Usage with Gradient Tree Boosting. Smart City/SocialCom/SustainCom (SmartCity), IEEE International Conference, 2015, pp. 734–741. DOI: 10.1109/SmartCity.2015.156
Freedman D.A. Statistical Models: Theory and Practice. Cambridge University Press, 2009. 456 p. DOI: 10.1017/CBO9780511815867
Neter J., Kutner M.H., Nachtsheim C.J., & Wasserman W. Applied Linear Statistical Models. Chicago: Irwin, 1996, vol. 4. 318 p.
Rao C.R., Toutenburg H. Linear Models. Linear Models: Least Squares and Alternatives. Springer, 1995, pp. 3–18. DOI: 10.1007/978-1-4899-0024-1, DOI: 10.1007/978-1-4899-0024-1_2
