Keep records with Spanish accents in Database using Python

I need to purge a DataBase that is in Spanish but the requirement is that I must keep accent marks.

For instance, if the DB contains "Administración" and "Administracion" I have to identify them as equals but keep the one with the accent mark. After some research every solution, like turn Unicode to ASCII or use PyEnchant, keeps the one without the accent mark.

Is there any Library (For Python 3.5) or way to determine the correct one and keep it?

2 Answers
2

Caveats

Depending on the content of the database, this may well be a nontrivial task, since though there may be misspellings:

administracion

administración

There are also many pairs of words in Spanish which differ only by an accent, but which both are valid words:

ejército

ejercito

ejercitó

tu

tú

If you are only considering nouns, this number decreases a lot, mostly to foreign loanwords with different stresses:

beisbol

béisbol

and a few native words multiple spellings:

período

periodo

reúma

reuma

Query

If you are unlikely to encounter such cases, you could use a sql query along the lines of:

SELECT a.word AS "Good word", b.word AS "Bad word"
FROM spanish_db AS a
JOIN spanish_db AS b

--Spanish words have at most one accent so can safely nest REPLACE
ON REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(a.word, "á", "a"),
"é", "e"),
"í", "i"),
"ó", "o"),
"u", "u") = b.word

--So as not to match identical words
AND a.word != b.word


This will return all pairs of words where an accented and unaccented form appear. You can adapt this to edit/delete/cleanse the entries as is required.

Example

Good word Bad word
"acedía" "acedia"
"aeróbic" "aerobic"
"aeróstato" "aerostato"
"afrodisíaco" "afrodisiaco"
"alcalá" "alcala"
"alvéolo" "alveolo"
"alérgeno" "alergeno"
"amoníaco" "amoniaco"
"anémona" "anemona"
"arcén" "arcen"


What do you think about indexing your database queries in a dict, by their ascii equivilent? Assumeing there is only one forms of an ascii key:

def ascii_word(word):
accents=(("á","a"), ("é","e"), ("í","i"), ("ó","o"), ("ú","u"), ("ü","u"), ("ñ","n"))
for acc in accents: word=word.replace(acc[0],acc[1])
return word

query_result = ["Administración", "Administracion", "si", "sí", "hola"]
filtered_dict={}

for s in query_result:
ascii=ascii_word(s)
if ascii in filtered_dict.keys():
if s!=ascii: filtered_dict[ascii] = s
else:
filtered_dict[ascii] = s

result=list(filtered_dict.values())
print(result)


This prints to ['Administración', 'sí', 'hola']


['Administración', 'sí', 'hola']


By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.