USE OF THE LINGUISTICALLY ORIENTED PYTHON LANGUAGE MODULES FOR HANDLING LARGE TEXTS IN THE EASTERN LANGUAGES IN ORDER TO MINE THE ORIENTALISTICS DATA (WITH NLTK MODULE TAKEN AS AN EXAMPLE)

Authors

  • Bulat G. Fatkulin South Ural State University

Keywords:

NLTK, eastern languages, modules, Python, natural language processing, code, encoding utf-8, big data, UNIX

Abstract

This article analyzes the contemporary linguistically oriented software created on the basis of
the programming language Python. The Natural Language Toolkit (NLTK) is selected as an example.
The research considers not only the general principles of the NLTK but also the principles especially
applied to the eastern languages: Farsi, Arabic and Chinese. The author shows certain solutions
for work with texts in Unicode as input-output for Python text processing modules.

Author Biography

Bulat G. Fatkulin, South Ural State University

associate professor, department of General Linguistics, South Ural State University, Chelyabinsk, Russian Federation, bfatkulin@gmail.com

References

Курбатов, С.С. Программное обеспечение

для лингвистически-ориентированного пополнения

онтологии: докл. / С.С. Курбатов, А.П. Лобзин,

Г.К. Хахалин // Четырнадцатая конференция по

искусственному интеллекту с международным

участием. – Казань, 2014. – Т. 3. – С. 164–172.

Маслов, А.В. Системы автоматической об-

работки текстов на естественном языке: лин-

гвистические аспекты и перспективы развития /

А.В. Маслов // Вестник Московского государст-

венного лингвистического университета. – 2013. –

№ 13 (699). – С. 167–170.

Фаткулин, Б.Г. Прикладная лингвистика и

обработка текстов на восточных языках: совре-

менные перспективы / Б.Г. Фаткулин // Вестник

ЮУрГУ. Серия «Лингвистика». – 2014. – Т. 11,

№ 3. – С. 15–18.

Bird, S. Natural language processing with

Python. / S. Bird, E. Klein, and E. Loper. – Beijing;

Cambridge; Mass: O’Reilly, 2009. Print.

Garrette, D. An extensible toolkit for computational

semantics / D. Garrette, E. Klein // Proceedings

of the Eighth International Conference on Computational

Semantics (IWCS-8 ’09) / H. Bunt, V. Petukhova,

S. Wubben (Eds.). Association for

Computational Linguistics, Stroudsburg, PA, USA,

– P. 116–127.

Perkins, J. Python text processing with NTLK 1.0

Cookbook over 80 practical recipes for using Python’s

NLTK suite of libraries to maximize your natural language

processing capabilities / J. Perkins. – Birmingham

Mumbai: PACKT Publishing, 2010. Print.

Downloads

Issue

Vol. 12 No. 1 (2015)

Section

Articles