How to Combine Map And Reduce in Python

This is just about functional programming, not about distributed systems MapReduce

The aim of this page📝is to explain the combination of Python’s map() and reduce() functions, typical for functional programming. Pedagogically, I am grateful to the authors of PEP 484 – Type Hints as it makes the points made in code examples more understandable.

There are 4 ingredients in this technique, each of them having its section.

1. CUSTOM MAPPING FUNCTION

• As noted in How to Use Map Function in Python, the specific function doing the mapping contains as many arguments as there are collections passed into the common map function
• It follows, that the custom mapping function below will allow only for a single collection to be passed into map()
• The exemplary function counts words in a document that is passed as a string and returns dict of frequencies

RUBBERDUCK

2. COMMON MAP

• now, we’ll map the custom count_words() function over a list of documents using the generic map() function
• map is lazy and returns a single-use map object

• let’s evaluate the map object and have it pretty printed — what we are getting is a list of dict

OUTPUT

• As you can see, the result is a sequence of dictionaries with word frequencies for the individual documents

3. CUSTOM REDUCING FUNCTION

• As written in How To Use Reduce Function In Python, a custom reducing function by definition needs 2 arguments and the custom reducing function is combining them into a single “product”
• The custom reducing function for our use case combines the frequencies of 2 documents

RUBBERDUCK

4. COMMON REDUCE CONTAINING COMMON MAP: MAP-REDUCE

• The final step is the combination of all of the components which can look like this

FINAL OUTPUT