Modeling Influence of Monitoring System on Performance of MPI Collective Operations
DOI:
https://doi.org/10.14529/cmse210105Keywords:
supercomputer, performance monitoring, monitoring system noise, parallel job slowdown, modeling influence of monitoring systemAbstract
Studying parallel program with the means of monitoring systems is a common practice. To collect data about application, monitoring system agent activates periodically during the run of application, occupying resources and causing perturbation. Monitoring system developers often ignore studying the problem of monitoring tools interference into application performance, this problem remains poorly examined. This article discusses ways to study influence of supercomputer monitoring system on users’ applications. We suggest to use MPI collective operations as a tool to measure this influence. This method also allows to estimate influence of monitoring system noise on a synchronized application. MPI collective operations are measured in presence of injected noise generated by the program that imitates interference of monitoring tool. We estimate the noise level that each of the used collective operations is capable to sense in chosen configuration. All-to-All, All-Reduce and Barrier are used in the noise detection tool. We find parameters for All-to-All and Barrier operations to perform stably and detect low noise level.
References
Adhianto L., Banerjee S., Fagan M., et al. HPCTOOLKIT: tools for performance analysis of optimized parallel programs. Concurr. Comput. Pract. Exp. 2009. Vol. 22, no. 6. P. 685–701. DOI: 10.1002/cpe.1553.
Agelastos A., Allan B., Brandt J., et al. The Lightweight Distributed Metric Service: A Scalable Infrastructure for Continuous Monitoring of Large Scale Computing Systems and Applications. International Conference for High Performance Computing, Networking, Storage and Analysis, SC14 (New Orleans, LA, USA, Nov. 2014). IEEE, 2014. P. 154–165. DOI: 10.1109/SC.2014.18.
Beckman P., Iskra K., Yoshii K., et al. Benchmarking the effects of operating system interference on extreme-scale parallel machines. Cluster Comput. 2008. Vol. 11, no. 1. P. 3–16. DOI: 10.1007/s10586-007-0047-2.
Hoefler T., Belli R. Scientific benchmarking of parallel computing systems. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on, SC ’15 (New York, New York, USA, Nov. 2015). ACM Press, 2015. P. 1–12. DOI: 10.1145/2807591.2807644.
Laguna I., Marshall R., Mohror K., et al. A large-scale study of MPI usage in open-source HPC applications. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (New York, NY, USA, Nov. 2019). ACM, 2019. P. 1–14. DOI: 10.1145/3295500.3356176.
Mooney R., Schmidt K.P., Studham R.S. NWPerf: a system wide performance monitoring tool for large Linux clusters. 2004 IEEE International Conference on Cluster Computing (San Diego, CA, USA, Sept. 2004). IEEE, 2004. P. 379–389. DOI: 10.1109/CLUSTR.2004.1392637.
Petrini F., Kerbyson D.J., Pakin S. The Case of the Missing Supercomputer Performance. Proceedings of the 2003 ACM/IEEE conference on Supercomputing, SC ’03 (New York, New York, USA, Nov. 2003). ACM Press, 2003. P. 55. DOI: 10.1145/1048935.1050204.
Rohl T., Eitzinger J., Hager G., et al. LIKWID Monitoring Stack: A Flexible Framework Enabling Job Specific Performance monitoring for the masses. 2017 IEEE International Conference on Cluster Computing, CLUSTER (Honolulu, HI, USA, Sept. 2017). IEEE, 2017. P. 781–784. DOI: 10.1109/CLUSTER.2017.115.
Sottile M.J., Minnich R.G. Supermon: a high-speed cluster monitoring system. Proceedings. IEEE International Conference on Cluster Computing (Chicago, IL, USA, USA, Sept. 2002). IEEE Comput. Soc, 2002. P. 39–46 DOI: 10.1109/CLUSTR.2002.1137727.
Stefanov K., Voevodin V., Zhumatiy S., et al. Dynamically Reconfigurable Distributed Modular Monitoring System for Supercomputers (DiMMon). Procedia Computer Science. Elsevier B.V., 2015. P. 625–634 DOI: 10.1016/j.procs.2015.11.071.
Treibig J., Hager G., Wellein G. LIKWID: A Lightweight Performance-Oriented Tool Suite for x86 Multicore Environments. 2010 39th International Conference on Parallel Processing Workshops (San Diego, CA, USA, Sept. 2010). IEEE, 2010. P. 207–216. DOI: 10.1109/ICPPW.2010.38.
Voevodin Vl.V., Antonov A.A., Nikitenko D.A., et al. Supercomputer Lomonosov-2: Large Scale, Deep Monitoring and Fine Analytics for the User Community. Supercomput. Front. Innov. 2019. Vol. 6, no. 2. P. 4–11. DOI: 10.14529/jsfi190201.
Performance Co-Pilot. URL: https://pcp.io/ (accessed: 27.09.2020).
Open MPI: Open Source High Performance Computing. URL: https://www.open-mpi.org/ (accessed: 27.09.2020).
