[This article and all those to come are all extractions from my university BA research paper. Changes, updates and improvements are to be made when necessary. All rights reserved for the owner.]

[Part 7]

Note: Compared to the original paper, various parts are ignored for multiple reasons, most of which is due to major updates have taken place and the issues found with the older methods. (For example, the interface was made with Python Tkinter GUI framework. And How the results were preprocessed, proceeded and presented. Bugs fixed and imporvements in processing speed) We shall, however, discuss the updates in the upcoming articles.

4. Absence of orthographic standards for writing MA

Moroccan Arabic is far from being standardized. We do not have, for example, any strict

rules to how words should be written. This is totally fine when writing a text message to a

friend or writing a short story, but when it comes to mapping patterns, it gets messy and hard.

To illustrate this point, a questionnaire was set up. This questionnaire explored the knowledge

of eight users of Moroccan Arabic at the university of Qadi Ayyad. The participants were

asked to write some English words, phrases and sentences in Moroccan Arabic using the

Arabic alphabets. The results are as follows: Words = W; Orthography = O

Chart 3.3. The absence of orthographic standards

in Moroccan Arabic

Chart 3.3. - Continue

The chart above clearly illustrates how the absence of orthographic standards pose

uncertainty and discrepancy between how some

thing should be written and how it is written. The most noted instance is that of whether to

place an "‫"ا‬, where and when. Let's take for example the word [mɑmʃiːtʃ] (I haven't gone):

50% of the participants wrote the word as ‫مامشيتش‬, with an "‫"ا‬ after the negation prefix "‫"م‬, and

the other 50% wrote the same word as ‫ممشيتش‬, without an "‫"ا‬. Another example is that of a

preposition in relation to a noun. 75% of the users wrote the word [fəd'ɑːr] (At home) as

‫فالدار‬, whereas 12.5% wrote it as ‫الدار‬ ‫في‬ where both the preposition and the noun are separate;

the other remaining 12.5% wrote it as ‫فدار‬. There are many other instances both in chart 3.3.

and in the questionnaires provided with this research.

Since there are no strict rules that everybody agrees upon to how a word or a class of

words must be written, we have devised and implemented most of the reappearing, repetitive

orthographic patterns. Meaning our model could recognize both ‫مامشاش‬ and ‫ممشاش‬ and other

similar instances as the same word. The two instances are then stemmed to their basic root

‫مشى‬. When our stemmer recognizes patterns (affixes) such as the negation circumfix, it may

not recognize what is being negatited, since in MA we could negate a verb as in ‫ممشاش‬ ([he]

hasn't gone), an adjective as in ‫مزوينش‬ ([he/it is] not beautiful), and sometimes a noun as in

‫مكتابش‬ ([it is] not a book).

For our stemmer to decide what the root word or the most basic stem is, it needs to

search for more patterns within the given string. The more patterns there is, the more

information our stemmer could recognize and provide. We are talking about Part of speech

tagging.

5. Part of speech tagging

In this research, we have focused mainly on inflectional morphology. Inflectional

morphology is helpful in the sense that it could help us tag the affixes which may stand for

individual lexical items in other languages. When our stemmer successfully stems a string

such as "‫"فالدر‬, the parsed results we get are as the following:

[prep-f-In/At + DET-Al-The + NOUN-dAr].

If we detransliterate the affixes and the stem back to the Arabic alphabet:

[prep-‫ف‬-In/At + DET-‫ال‬-The + NOUN-‫[دار‬

If we go back to output 3.1 on this section, and look at the transliterated results of the input

string ‫ومغيكتبواهاش‬, we could see that all the affixes are labeled with some tags either regarding

the function or the form of the affixes. Here is the affixes de-transliterated into Arabic

characters:

[conjuction-‫و‬-And + Negation_prefix-‫م‬-Not + Negation_suffix-‫ش‬-Not +

future_action_prefix-‫غي‬-Will + Noun_object_suffix-‫ها‬ + Pronoun_suffix-‫وا‬ +

VERB_past_stem-‫[كتب‬

The affixes are tokenized and most of their features (including part of speech tags) are labeled based on the patterns found.

Having part of speech tags of strings in their functional affixes could help us do even more

processing, such as parsing and building syntactic structures that could be used for other

purposes including but not limited to semantic analysis.

Darija NLP: Text Processing And Morphological Stemming - Results: Continue