Supercomputer Application Integral Characteristics Analysis for the Whole Queued Job Collection of Large-Scale HPC Systems
DOI:
https://doi.org/10.14529/cmse160403Keywords:
supercomputer, efficiency, system monitoring, job categories, integral job characteristics, queued job collection, job queue, resource utilization controlAbstract
Efficient use and high output of any supercomputer depends on a great number of factors. The problem of controlling granted resource utilization is one of those, and becomes especially noticeable in conditions of concurrent work of many user projects. It is important to provide users with detailed information on peculiarities of their executed jobs. At the same time it is important to provide project managers with detailed information on resource utilization by project members by giving access to the detailed job analysis. Unfortunately, such information is rarely available. This gap should be eliminated with our proposed approach to supercomputer application integral characteristics analysis for the whole queued job collection of large-scale HPC systems based on system monitoring data management and study, building integral job characteristics, revealing job categories and single job run peculiarities.
References
Top50 Supercomputers of Russia and CIS: [http://top50.supercomputers.ru/]. 15.02.2016.
Top500 Supercomputer sites: [http://top500.org/].15.02.2016.
Antonov A., Zhumatiy S., Nikitenko D., Stefanov K., Teplov A., Shvets P. Analysis of dynamic characteristics of job stream on supercomputer system Numerical Methods and Programming, 2013. Vol. 14, No. 2., P. 104-108.
Safonov A., Kostenetskiy P., Borodulin K., Melekhin F. A monitoring system for supercomputers of SUSU // Russian Supercomputing Days International Conference, Moscow, Russian Federation, 28-29 September, 2015, Proceedings. CEUR Workshop Proceedings, 2015. Vol. 1482, P. 662-666
Stefanov K. et al. Dynamically Reconfigurable Distributed Modular Monitoring System for Supercomputers (DiMMon) // Procedia Computer Science / Elsevier B.V., 2015. Vol. 66, P. 625–634.
Nikitenko D. Complex approach to performance analysis of supercomputer systems based on system monitoring data. Numerical Methods and Programming, 2014, Vol. 15, P. 85–97.
Voevodin V., Zhumatiy S., Nikitenko D. Octoshell: Large Supercomputer Complex Administration System // Russian Supercomputing Days International Conference, Moscow, Russian Federation, 28-29 September, 2015, Proceedings. CEUR Workshop Proceedings, 2015. Vol. 1482, P. 69-83.
Voevodin Vl., Antonov A., Bryzgalov P., Nikitenko D., Zhumatiy S., Sobolev S., Stefanov K., Voevodin Vad. Practice of "Lomonosov" Supercomputer // Open systems, 2012. No. 7, P. 36–39.
Zhumatiy S., Nikitenko D. Approach to flexible supercomputers management // International supercomputing conference Scientific Services & Internet: all parallelism edges, Novorossiysk, Russian Federation, 23-28 September, 2013, Proceedings. MSU, 2013. P. 296–300.
Voevodin Vl. Supercomputer situational screen // Open systems, 2014. No. 3, P. 36–39.
Shvets P. , Antonov A., Nikitenko D., Sobolev S., Stefanov K., Voevodin Vad., Voevodin V., Zhumatiy S. An Approach for Ensuring Reliable Functioning of a Supercomputer Based on a Formal Model. 11th Int. Conference on Parallel Processing and Applied Mathematics, Krakow, Poland, 6-9 September, 2015. Proceedings.
Voevodin V., Antonov A., Dongarra J. AlgoWiki: an Open Encyclopedia of Parallel Algorithmic Features // Supercomputing Frontiers and Innovations, 2015. Vol.2. c1. P. 4-18.
SLURM workload manager: [http://slurm.schedmd.com/]. 15.02.2016.
Cleo cluster batch system: [http://sourceforge.net/projects/cleo-bs/]. 15.02.2016.
Ganglia Monitoring System: [http://ganglia.sourceforge.net/]. 15.02.2016.
Collectd – The system statistics collection daemon: [https://collectd.org/]. 15.02.2016.
Clustrx: [http://www.t-platforms.ru/products/software/clustrxproductfamily/clustrxwatch.html]. 15.02.2016.
jQuery & jQuery UI: [http://jqueryui.com/]. 15.02.2016.
TagIt: [http://aehlke.github.io/tag-it/].15.02.2016.
