[This article and all those to come (Under this specific category) are all extractions from my university BA research paper. Changes, updates and improvements are to be made when necessary. Parts I think won't contribute much or are becoming obsolete shall also be ignored. New suggestions shall be presented eventually as an update. All rights reserved for the owner.][Part 1] Note: Compared to the original paper, various parts are ignored for multiple reasons, most of which is due to major updates have taken place and the issues found with the older methods. (For example, the interface was made with Python Tkinter GUI framework. And How the results were preprocessed, proceeded and presented. Bugs fixed and imporvements in processing speed) We shall, however, discuss the updates in the upcoming articles. Introduction There is a place where linguistics, computer science (Mathematics, algorithms and data structures) and logic all meet and melt to construct what is known as Computational Linguistics (CL). CL in simple words, is the use of computers for linguistic research and application. According to Prat (1992), the main goal of CL is to make computer systems that understand and speak any human language (also called natural language).The field of CL, in hand with its engineering domain of Natural Language Processing (NLP), has exploded in recent years. Both NLP and CS have shifted towards robust learning applied to larger corpora. (Clark, Fox, and Lappin.2010) Natural Language Processing is the automatic processing of a human language, it may include text classification, tokenization, stemming, lemmatizing, part-of-speech tagging (POS-tagging), and parsing, etc. NLP has many advantages and is applied in various domains including but not limited to Machine Translation (MT), Search Engines, Texts Processing, Sentiment Analyses, Named-Entity-Recognition, Text-to-Speech (TTS) and in many more.With the growth of powerful computers and resources, NLP has become the central of a new world of possibilities. Bird, Klein, and Loper (2014) stated that " NLP is important for economic, social , and cultural reasons. [...] For this reason it is important for a wide range of people to have a working knowledge of NLP". Stemming is one of the very first phases of what is known as text normalizing. Inside a stemmer, we define patterns and rules that will eventually recognize some affixes and strip them off words. It is not a well-defined process as Bird, Klein, and Loper stated (2004:107). It is, however, if worked-precisely, a very helpful tool that could prove to be rather very useful in many tasks such as translation and information retrieval, or Chatterbots. 1. Statement of the research problem In their work "A Grammar of Moroccan Arabic", Moha Ennaji et al, state that MA is a root-based language that combines a root with a pattern to construct a stem. Stems are usually word-class forms relating to grammatical classes. We attach affixes to roots to serve a lexical or a grammatical meaning. The produced stems, for instance a V-stem may express a sentential meaning. Meaning that it could tell information on the subject, verb and the object.This allows us to have V-stems such as mʌmʃItIʃ meaning (you have not gone) Because of such properties of the Moroccan Arabic language, we could construct some very complex words combining these affixes meaningfully. 2. Rationale for the research In the context of machine learning for instance, stemming proves to be rather useful because it would make out data more dense and less variable. The number of words saved in a dictionary would be less since stemming reduces a given word to its root or base word.Languages such as French or Arabic and its dialects, for example, can generate dozens of words in the case of verbs from one single stem. Another usage of stemming would be in information retrieval systems where you might want to not only search for results corresponding to the given words, e.g. studying, studies, or studied, but also search for its basic form, study, which the former three are inflected from. In the case of Moroccan Arabic,no prior works regarding stemming has been done before or on the time of writing this research. What this research proposes could help us easily process and retrieve information. Let us assume for example we had an electronic dictionary, Moroccan Arabic - English. With words such as "‫"كتب‬ and "‫"شرب‬ as the corresponding entries to the English equivalents "write" and "drink". If we searched for the exact two MA words, we would get a match. Until now it is fair enough. But imagine we were foreigners who were just learning MA and we found the two following words on an online article or forum: "‫"مكتبناهمش‬ and "‫"شربواهم‬. The latter two words are just one possible generated (inflected) output of the two basic roots "‫"كتب‬ and "‫"شرب‬ found in our dictionary. Of course no matches would be found when we look the two words up on the dictionary, but as long as we have a proper stemmer implemented,looking up words such as "‫"مكتبناهمش‬ and " ‫م‬ ‫ه‬ ‫ا‬ ‫شربو‬ " would be stemmed to "‫"كتب‬ and "‫"شرب‬ and then translated to "write" and "drink". Moreover, additional useful information would be supplemented based on the recognized patters and affixes. 3. Research objectives This study shall seek to reach the following objectives: <ol><li>Build a model system that could identify the stem and affixes of a given MA string or word, and that could also provide useful information on the properties or the functions of these affixes.</li><li>Integrate the stemmer algorithm with a simple dictionary to demonstrate the usefulness of stemmers.</li><li>Convince the audience that mapping such a phenomenon as that of constructing words in MA (and in any language), requires some precise grammar and a generally agreed-upon orthographic standards.</li></ol> 4. Research questions Throughout this work, we shall attempt to answer the following questions: <ol><li>Is it possible for us to build a model system that could identify and stem affixes off words and return the stem word and additional related information?</li><li>Does the absence of orthographic standards for writing in Moroccan Arabic introduces any challenges or issues when attempting to map such phenomenon as that of which patterns occupy which slots?</li></ol> This research paper is organized into sections, mainly four sections apart from the general introduction and the conclusion. The general introduction proposes that statement of the problem, the objectives and questions. The first section, Literature Review, focuses on some background knowledge and the literature guiding this research. The second section, Methodology, shades light on how this research is done, or must be done to meet theexpectations raised at the beginning of this work. It draws mainly on the computational part of this work and the requirements. It demonstrates a snippet of the algorithm and the flowchart for it. Section three of this research presents the results both gathered from the questionnaire and generated and processed by our stemmer. Chapter four discusses the results and highlights both the limitations and the delimitations of this work. It also suggest future works and how they must be conducted and with what techniques or approaches. Lastly, the conclusion, sums up what has been stated and found earlier in this work.

Darija NLP: Text Processing And Morphological Stemming - When It All Started

Majdi Mahfoud