[This article and all those to come are all extractions from my university BA research paper. Changes, updates and improvements are to be made when necessary. All rights reserved for the owner.][Part 5] Note: Compared to the original paper, various parts are ignored for multiple reasons, most of which is due to major updates have taken place and the issues found with the older methods. (For example, the interface was made with Python Tkinter GUI framework. And How the results were preprocessed, proceeded and presented. Bugs fixed and imporvements in processing speed) We shall, however, discuss the updates in the upcoming articles. 1. Restatement of the research objectives &amp; questions 	We have already identified some patterns such as that of forming the negation form of aword in Moroccan Arabic. There are of course more patterns that we shall illustrate some ofwhich. We hereby know MA abides to some rules and patterns, and therefore it is possible tomodel the phenomenon. By means of this, we propose that we could build a system that notonly identifies some of these patterns and the stem or root, but also based on that, identify thePart of speech and the inflectional properties of the word and its affixes if any found. We shallalso attempt to convince the audience that to properly and efficiently process the MoroccanArabic, we would need a precisely defined grammar and a standard orthography that is agreedupon.	This study shall seek to investigate the following questions: <ol><li>Does Moroccan Arabic abide any morphological rules when constructing words?</li><li>If the former question is affirmative, could we build a model system that could identify and stem affixes off words and return the stem word?</li><li>Does the absence of a standard orthography for written Moroccan Arabic pose any challenges or issues?</li></ol> 2. Research methodology [removed]3. Requirements [removed]4. Transliteration 	To explain what transliteration is, we adopt Beesley's definition: "the term transliterationdenotes an orthography using carefully substituted orthographical symbols in a one-to-one,26fully reversible mapping with that language’s customary orthography." But, why do we needtransliteration? Because as pointed by (N. Habash, A. Soudi and T. Buckwalter, 2007) : "The basicArabic Alphabet has 28 letters and eight diacritical marks. However, there are eight additionalsymbols that can be treated as separate letters and/or special combinations of letter andadditional diacritics". This proves to be rather challenging, especially that, as pointed earlier,Arabic have multiple shapes (allographs) for one single character depending on its position inthe word. In this research, as we have mentioned previously, we adopt The Buckwalter Arabictransliteration for three reasons: <ul><li>Because it is written in ASCCI</li><li>It is a strict transliteration, a one-to-one character mapping</li><li>It has been used in many NLP publications including resources developed at the Linguistic Data Consortium [LDC] (N. Habash, A. Soudi and T. Buckwalter, 2007)</li></ul> Transliteration is one of the main preprocessing steps of our algorithm. 5. Algorithm Side note: I am only presenting this so people with little background in coding have a simple idea of what's going on. Although many steps and approaches are still preserved, this algorithm, however is outdated and a little poor. This is only a brief segment of the algorithm which takes care of the preprocessing andstemming. <pre class="ql-syntax" spellcheck="false">✓ Initiation and Preprocessing:

1.1. Initiation

Define variables:
# each instance as such example(argument) is a method with attribute data

input_x  input from the user
tokens = []  holds the tokenized input
stemmed_tokens = []  holds the stemmed tokens
stemmed_affixes = []  holds the affixes stemmed
...

1.2. Preprocess:

normalize(token)  cleans Arabic punctuation (removes)
transliterate(input_x)  Maps input with the BuckWalter scheme
tokenize(input_x)
stem(token)  for each token in our list of tokens[]

✓ Stemming Algorithm:

# define variables

affixes = []
stem_and_affixes ={}
is_verb = False
is_noun = False
is_object = False
...

# we define MA affixes such as that of prepositions using the correspondent characters of
the BuckWalter scheme , accordingly:

__step1_preposition_prefixes = ('f', 'l','b')

# ('‫'ف‬ , '‫'ل‬ , '‫)'ب‬ respectively.
# when we have specified all the known affixes to this research, we define in which
patterns or word positions they occur.
# example: prepositions prefixes patterns

if prefix in __step1_preposition_prefixes and pattern is pattern:
 if prefix == "some_prefix"#thenreturn word &amp; remove the prefix(es)
 
→ continue analyzing, if completed, then:
save_data  we save the found stem alongside its affixes and their descriptions.
</pre>

Darija NLP: Text Processing And Morphological Stemming - Methodology

Majdi Mahfoud