Search and Replace multiple words or characters with Python

A most frequently question is how to replace all occurrences of a word or a character inside a string or a file.
If you just want to replace a simple character or word, then all you have to do is to use the replace() method that Python provides for that purpose. The python replace() method takes 2 arguments and a third optional one. It’s definition is:

replace(old, new[, count])

And as the documents tell, it returns a copy of the string with all occurrences of substring ‘old’ replaced by ‘new‘. If the optional argument ‘count‘ is given, only the first count occurrences are replaced. If you leave that optional argument empty, then all occurrences will be replaced.
An example of this method might be:

my_text = 'Hello everyone. Say "Hello" to me!'
my_text = my_text.replace('Hello', 'Goodbye')
print my_text
#prints 'Goodbye everyone. Say "Goodbye" to me!'

or :

my_text = 'Hello everyone. Say "Hello" to me!'
print my_text.replace('Hello', 'Goodbye')

or:

print 'Hello everyone. Say "Hello" to me!'.replace('Hello', 'Goodbye')

It’s pretty the same. Use what you think is best for the situation or your style.
It’s a very simple and straigth forward method that you will not find difficult to include it in your code for simple replacements.

But what about if we want to replace multiple characters or words inside a string or file?
My implementation is a simple one. With it you can replace all occurences of a single character or word as the python replace() method mentioned above, but also multiple characters and words inside a string or a whole file.
Let’s see it:

def replace_all(text, dic):
    for i, j in dic.iteritems():
        text = text.replace(i, j)
    return text

Our method, replace_all(), takes 2 arguments. The first one, text, is the string or file (it’s text) that the replacement will take place. The second one, dic, is a dictionary with our word or character(s) to be replaced as the key, and the replacement word or character(s) as the value of that key. This dictionary can have just one key:value pair if you want to replace just one word or character, or multiple key:values if you want to replace multiple words or characters at once.
A sample dictionary is like that one:

reps = {'a':'@', 'e':'3', 's':'5'}

With this dictionary we define that we want to replace ‘a’ with ‘@’, ‘e’ with ’3′ and ‘s’ with ’5′. Of course you’ll make your own dictionary with your custom key:values.

So, let’s make a working example and see if it works so far, before we see how our method works.

# define our method
def replace_all(text, dic):
    for i, j in dic.iteritems():
        text = text.replace(i, j)
    return text

# our text the replacement will take place
my_text = 'Hello everybody.'

# our dictionary with our key:values.
# we want to replace 'H' with '|-|'
# 'e' with '3' and 'o' with '0'
reps = {'H':'|-|', 'e':'3', 'o':'0'}

# bind the returned text of the method
# to a variable and print it
txt = replace_all(my_text, reps)
print txt    # it prints '|-|3ll0 3v3ryb0dy'

# of course we can print the result
# at once with:
# print replace_all(my_text, reps)

Save it and run it from the console and see what you got.
Pretty simple so far, isn’t it? So let’s get inside our replace_all() method and see how it works.
First we start iterating in our dictionary using the iteritems() method that Python provides for a dictionary:

for i, j in dic.iteritems():

With the iteritems() method you can retrieve the key and corresponding value at the same time, so that’s why we use ‘for i, j’ and not a simple iterator. As we iterate, we bind the current key we are to ‘i’ and it’s corresponding value to ‘j’.

Next is the replacement method. Here we use the replace() method that we mentioned in the beginning.
We simply tell that we want to replace the ‘i’ key with it’s corresponding ‘j’ value in our text and then bind the returned copy of the python replace() method to our text again, so it will always be updated with the replacements that took place so far:

text = text.replace(i, j)

And lastly we return our text so that we can use it.

Here I must warn you about something that you may face using the dictionary with your custom key:values.
If you use this method for simple character or word replace, single or multiple, then it will work as expected.
For example both of the following dictionaries work as expected:

# for search & replace of whole words
dic = {'hello':'goodbye', 'bad':'good', 'yes':'no'}
# for search & replace of characters
dic = {'a':'@', 'e':'3', 'o':'0', '8':'eight'}

We fill our dictionaries with as many words we want to be searched and replaced in the first case, and as many characters we want to be searched and replaced in the second one, and everything works fine.

But what about if in rare occasions we mix up characters and words together and our dictionary looks like the following one?

# assuming that our string the replacement
# will take place is 'hello everybody'
dic = {'hel':'HEL', 'e':'3', 'o':'0'}

It will work, replacements will take place, but not as you expected to. You’d expect it to return something like ‘H3Ll0 3v3ryb0dy‘ or ‘HELl0 3v3ryb0dy‘ I guess, eh?
No. Most possibly it will return ‘h3ll0 3v3ryb0dy‘, at least that’s what I get when I run it on my machine. But why?
This happens because our ‘e‘ key is overlapped with the ‘hel‘ key and because when python iterates through a dictionary, the ordering of the keys and values retrieved from that dictionary cannot be defined. That means that you can’t be sure in which order it searches the keys. And most of the times, definitely not in the order that you define them in your dictionary. The algorithm that Python uses to search through a dictionary is a complex one to discuss it here. That means that our ‘e‘ key may come before our ‘hel‘ key or in other occasions the opposite. In our example ‘e‘ comes first, so first it finds the ‘e‘ in our text and replaces it with ‘3‘. Then it searches for the ‘hel‘ key. But now there is no ‘hel‘ in our text, we modified it with the previous replacement. Now we have ‘h3l‘, so no ‘hel’:'HEL‘ replacement can take place. Understood?
So don’t mix characters along with words in your replacement. In rare occasions that you wanna do it, it’s better to define two dictionaries, one for the words and one for the characters, and then use the replace_all() method two times. So for our previous example, you can do something like this:

text = 'hello everybody'
w_dic = {'hel':'HEL'}
c_dic = {'E':'3', 'e':'3', 'o':'0'}
text = replace_all(text, w_dic)
text = replace_all(text, c_dic)
print text  # prints 'H3Ll0 3v3ryb0dy'

And of course we must always remember that the python replace() method is case sensitive, don’t forget that ever. That’s the reason we included both ‘E‘ and ‘e‘ in our dictionary.

You can implement this method in whatever way you want inside your code, just find the one that suits you.
And of course if all you want is to replace just a single character or word, you can use the python replace() method mentioned in the beginning, so you don’t need a dictionary and my method. Just use the method I provided for multiple word or character replacements instead.
And both methods of course can be used to replace occurrences inside a simple string or a file’s text. That is up to you and very easy to implement.

Hope you found that post interesting. For any questions or recomendations or whatever, just leave a comment.

[PS: And once again I must apologize for my bad use of the english language.]

About these ads

« Convert a file in UTF-8 or any encoding with Python

Change the extension of multiple files in a chosen directory and it’s sub-directories with Python »