Virtualization of Heterogeneous HPC-clusters Based on OpenStack Platform
Аннотация
We address to a problem of an integration of heterogeneous computing clusters to the united environment based on a virtualization technology. We select the software OpenStack as a platform for managing the virtual environment. The platform OpenStack provides a wide range of components and solutions for a functional interaction with different hypervisors. These include KVM, XEN, ESXi, QEMU and other systems. In addition to the platform OpenStack, we developed the specialized hypervisor shell. It provides a virtual machines run from queues of the traditional resource management systems, such as PBS, SLURM, LSF or SGE, that are used on clusters of a center of collective usage. The developed model of the resource allocation for virtual machines allows us jointly to use the knowledge about the job requests, resource characteristics and current state of the environment, and the expertise of it administrators. The realized tools provide the capability for the "painless" integration of heterogeneous clusters with the preinstalled local resource managers to the virtual cluster with the required configuration. Extensive modeling show that the hypervisor shell can improve an efficiency of integrated environment nodes through reallocating virtual machines to queues of the traditional resource management systems.
Ключевые слова
Полный текст:
PDF (English)Литература
Gergel V., Senin A. Metacluster System for Managing the HPC Integrated Environment. Methods and Tools of Parallel Programming Multicomputers. Second Russia-Taiwan Symposium, MTPP 2010 (Vladivostok, Russia, May 16-19). LNCS, vol. 6083. pp. 86-94. DOI: 10.1007/978-3-642-14822-4
Mladen A., Eric S., Patrick D. Integration of High-Performance Computing into Cloud Computing Services. Handbook of Cloud Computing. 2010. pp. 255-276. DOI: 10.1007/978-1-4419-6524-0_11
Bychkov I.V., Oparin G.A., Novopashin A.P., Feoktistov A.G., Korsukov A.S., Sidorov I.A. High-performance computing resources of ISDCT SB RAS: State-of-the-art, prospects and future trends. Comput. Tech. 2010. vol. 15. pp. 69-81. (in Russian).
Bogdanova V.G., Bychkov I.V., Korsukov A.S., Oparin G.A., Feoktistov A.G. Multia-gent Approach to Controlling Distributed Computing in a Cluster Grid System. J. Comput. Syst. Sci. Int. 2014. vol. 53. pp. 713–722. DOI:10.1134/S1064230714040030
Bychkov I.V., Oparin G.A., Feoktistov A.G., Bogdanova V.G., Pashinin A.A. Service-oriented multiagent control of distributed computations. Automat. Rem. Contr. 2015. vol. 76. pp. 2000–2010. DOI: 10.1134/S0005117915110090
Bychkov I.V., Oparin G.A., Feoktistov A.G., Sidorov I.A., Bogdanova V.G., Gorsky, S.A. Multiagent control of computational systems on the basis of meta-monitoring and imitational simulation. Optoelectron., Instr. and Data Process. 2016. vol. 52, pp. 107–112. DOI: 10.3103/S8756699016020011
Irkutsk Supercomputer Center of SB RAS. Available at: http://hpc.icc.ru (accessed: 16.02.2017).
Buyya R., Broberg J., Goscinski A.M. Cloud Computing: Principles and Paradigms. Wiley, 2011. 637 p. DOI: 10.1002/9780470940105
Sridharan S. A Performance Comparison of Hypervisors for Cloud Computing. University of North Florida, 2012. 269 p.
Docker. Available at: http://docker.com (accessed: 16.02.2017).
QEMU. Available at: http://qemu.org (accessed: 16.02.2017).
KVM. Available at: http://www.linux-kvm.org (accessed: 16.02.2017).
Xen. Available at: http://cam.ac.uk/research/srg/netos/projects/archive/xen (accessed: 16.02.2017).
vSphere ESXi. Available at: https://vmware.com/support/vsphere-hypervisor.html (accessed: 16.02.2017).
Bumgardner V.K. OpenStack in Action. Manning Publications, 2016. 358 p.
Apache CloudStack. Available at: https://cloudstack.apache.org/ (accessed: 16.02.2017).
Euacalyptus. Available at: http://www.eucalyptus.com/ (accessed: 16.02.2017).
OpenNebula. Available at: https://opennebula.org (accessed: 16.02.2017).
Bichkov I.V., Oparin G.A., Novopashin A.P., Sidorov I.A. Agent-Based Approach to Monitoring and Control of Distributed Computing Environment. Parallel Computing Technologies: 13th International Conference, PaCT 2015 (Petrozavodsk, Russia, August 31-September 4). LNCS, vol. 9251. pp. 253–257. DOI: 10.1007/978-3-319-21909-7_24
Sidorov I.A. Methods and tools to increase fault tolerance of high-performance compu-ting systems. In proc. of the 39th International Convention on information and communication technology, electronics and microelectronics, MIPRO-2016 (Opatija, Croatia, 30 May-3 June 2016). Riejka: Croatian Society for Information and Communication Technology, Electronics and Microelectronics 2016. pp. 242–246. DOI: 10.1109/MIPRO.2016.7522142
Feoktistov A.G, Sidorov I.A. Logical-Probabilistic Analysis of Distributed Computing. In proc. of the 39th International Convention on information and communication technology, electronics and microelectronics, MIPRO-2016 (Opatija, Croatia, 30 May-3 June 2016). Riejka: Croatian Society for Information and Communication Technology, Electronics and Microelectronics 2016. pp. 247–252. DOI: 10.1109/MIPRO.2016.7522142
Hastie T., Tibshirani R., Friedman J. The elements of statistical learning: Data Mining, Inference, and Prediction. 2001, 533 p.
Sholomov L.A. Logical research methods of discrete choice models. Moscow: Nauka, 1989, 288 p. (in Russian)
GPSS World Tutorial Manual. Available at: http://www.minutemansoftware.com (ac-cessed: 16.02.2017).
DOI: http://dx.doi.org/10.14529/cmse170203