Hello, there. :) In today's article we shall continue with our topic category 10 Useful Python Functions. If you haven't checked out the previous article under the same category, here it is 10 Useful Python Functions . So let us get started already.

3. Transliteration

The naming can be a little vague for some, so first here is a little definition. "the term transliteration denotes an orthography using carefully substituted orthographical symbols in a one-to-one, 26 fully reversible mapping with that language’s customary orthography." Beesley. Or simply: A transcription from one alphabet to another. One of the best known transliteration schemes for Arabic is the BuckWalter scheme. BuckWalter Arabic scheme source code found on github . You may not be interested in Arabic, but following the scheme, you could build your own transliteration scheme. Now, why would you need that? Honestly if you are dealing with a language such as English, you wouldn't need to because the English alphabet is already all part of the ASCCI (128 characters), but most other languages are not, and processing those languages without a proper transliteration or at least encoding could be really hard and pron to errors. If you are not interested in natural language processing, we could still show you some useful uses of such methods, some of which are used in building some functinalities throughtout our our website. But first here is a snippet of the BuckWalter Arabic scheme:

Code snippet 1.0

# begin scheme

BW_TO_UNICODE = {
    "'": u"\u0621", # hamza-on-the-line
    "|": u"\u0622", # madda
    ">": u"\u0623", # hamza-on-'alif
    "&": u"\u0624", # hamza-on-waaw
    "<": u"\u0625", # hamza-under-'alif
    "}": u"\u0626", # hamza-on-yaa'
    "A": u"\u0627", # bare 'alif
    "b": u"\u0628", # baa'
    "p": u"\u0629", # taa' marbuuTa
    "t": u"\u062A", # taa'
}

Here is the sheme, now how do we make use of it?

Code snippet 1.1

# (de-)transliteration methods

def uni2bw(inputx):
    # Lets loop and replace each char (v) with its equavalent char (k)
    for k, v in BW_TO_UNICODE.items():
        inputx = inputx.replace(v, k)
    # Return our input after it has been transliterated
    return inputx

def bw2uni(inputx):
    # Lets loop and replace each char (k) with its equavalent char (v)
    for k, v in BW_TO_UNICODE.items():
        inputx = inputx.replace(k, v)
    # Return our input after it has been de-transliterated
    return inputx

The two methods above are crucial to our transliteration and de-transliteration. The uni2bw (unicode to BuckWalter) method takes in a input of Arabic data (could be another language for you), and then for each Arabic character in the input and the defined scheme, we replace the value with the key. An example would be great:

Code snippet 1.2

# Transliterate
inputx = "العلم هو واحد الحاجة اللي زوينة بزاف" # Translation:" Science is something so beautiful"
transliterated_data = uni2bw(inputx)
print(transliterated_data)
# Outputs "AlElm hw wAHd AlHajp zwynp bzAf"


# de-transliterate
de_transliterated_data = uni2bw(transliterated_data)
print(de_transliterated_data)
# Outputs "العلم هو واحد الحاجة اللي زوينة بزاف"

Now as we know how to go back and forth with transliteration, let us show one more example that you may be using in your day-to-day coding:

Code snippet 1.3

# TAGS scheme

TAGS = {
    "Adjective": "ADJ",
    "Verb": "VERB",
    "Common Noun": "NNC",
    "Proper Noun": "NNP",
    "Pronoun": "PRON",
    "Adverb": "ADV",
    "Preposition": "ADP",
    "Conjuction": "CONJ",
    "Determiner":"DET"
}

def shorten_tags(tag):
    # Lets loop and replace each char (v) with its equavalent char (k)
    for k, v in TAGS.items():
        tag = tag.replace(k, v)
    return(tag)

def expand_tags(tag):
    # Lets loop and replace each char (k) with its equavalent char (v)
    for k, v in TAGS.items():
        tag = tag.replace(v, k)
    return(tag)

Example:

Code snippet 1.4

# Let's shorten a tag:
tag = "Common Noun"
short_tag = shorten_tags(tag)
print(short_tag)
# Outputs "NNC"


# Let's expand the same tag
expanded_tag = expand_tags(short_tag)
print(expanded_tag)
# Outputs "Common Noun"

I hope you found this method useful. Now we shall proceed to method number four.

4. Most Frequent

Code snippet 1.5

# most frequent
def most_frequent(lst):
    return max(set(lst), key = lst.count)

The method above simply receives a list or tuple of values and returns the most frequent one of all:

Code snippet 1.6

# some list of values
some_list = [1, 2, 1, 2, "c", 3, 4, 5, 4, 1, "a", "B", 1]

# we pass our list to our method
most_frequent(some_list)

# outputs 1

5. To-Dictionary

Code snippet 1.7

# to-dictionary
def to_dictionary(keys, values):
    return dict(zip(keys, values))

The above method takes in a list or tuple of values as keys and another list or tuple of values as values and creates a dictionary out of them

Code snippet 1.8

# lists of keys and values
x = [1,2,4,54,545,56]
y = ['y', 'y', 'n', 'n', 'y', 'n']
to_dictionary(x, y)
# Outputs dictionay {1: 'y', 2: 'y', 4: 'n', 54: 'n', 545: 'y', 56: 'n'}


# tuples of keys and values
x = (1,2,4,54,545,56)
y = ('y', 'y', 'n', 'n')
to_dictionary(x, y)
# Outputs the same


# MIXTURE OF A LIST AND A TUPLE
x = [1,2,4,54,545,56]
y = ('y', 'y', 'n', 'n', 'y', 'n')
# Outputs the same

What if you had a list or tuple of keys of a different length than that of values or vice-versa?

Code snippet 1.9

# keys and values of different sizes

x = (1,2,4,54)
y = ('y', 'y', 'n', 'n', 'y', 'n')
# Outputs {1: 'y', 2: 'y', 4: 'n', 54: 'n'}


x = (1,2,4,54,545,56)
y = ('y', 'y', 'n', 'n')
# Outputs the same

So Python does not run into some error such as an Out of index, it maps only as many keys and values as there in the smallest list or tuple provided. If they're equal in length, of course it will map all the keys to all the values.

I hope this was helpful to someone out there. We shall meet in the next and final article of this category, and until then keep learning!

10 Useful Python Functions: Part 2

Majdi Mahfoud