This is spartan_ideas1.py #Generates random strings in processes and sends them

Question

This is spartan_ideas1.py

#Generates random strings in processes and sends them to main() using a queue
import multiprocessing as mp
import random
import string

#Define an output queue
output = mp.Queue()

#Example function
def rand_string(length, output):
"""Generates a random string of numbers, lower/upper case characters"""
rand_str = ''.join(random.choice(string.ascii_lowercase +
string.ascii_uppercase +
string.digits) for i in range(length))
output.put(rand_str)

def main():
#set up a list of processes
processes = [mp.Process(target=rand_string, args=[5, output]) for x in range(4)]
#Run processes
for p in processes:
p.start()
#Exit the completed processes
for p in processes:
p.join()
#Get process results from the output queue
results = [output.get() for p in processes]
print(results)

if __name__ == "__main__":
main()

And the code is mp01.py

#Spawn a process
import multiprocessing as mp

def foo(i):
print("Called function in process:", i, __name__)
return

def main():
Process_jobs = []
print(__name__)
for i in range(4):
p = mp.Process(target = foo, args = [i])
Process_jobs.append(p)
p.start()

if __name__ == "__main__":
main()

Use the command line to execute programs in the following steps.

Start a python shell from the command line. Import multiprocessing as mp and use mp.cpu_count() to find out how many cores your system has.

Copy mp01.py from trace and execute it from the command line. Note the name that’s printed by the generated processes. As you can see, there is a lot of similarity between the way the programmer uses multiple processes and multiple threads.

Remove the if statement (if __name__) and see what happens when you execute the program (be sure to unindent the call to main()).

Put the if statement back (be sure to indent the call to main()). Inside the for loop, add p.join() immediately following p.start(). Run the program several times. What do you notice about the order in which the processes print their messages? Now take p.join() out and run the program several times. Does the order change? What does that tell you about p.join()?

Add a second function, foo2(). foo2() should take the same argument as foo(). foo2() should print something to distinguish it from foo() but should also print the argument and __name__. Modify main() so that it creates a process to call one of the functions when the loop index (i) is odd and the other when the loop index is even (recall that i % 2 can tell you whether i is even or odd). Note the name of the processes. Can you find a way to distinguish them? Hint: You can give a process a name, just as you can a thread. There doesn’t seem to be a getName() method but name is an instance variable. Import multiprocessing into a Python shell and use the help function to find a method that might be useful in accessing the current process. Of course, in this case, the argument i serves to disambiguate the processes.

Copy the program spartan_ideas1.py (from a blog at Michigan State University) from trace and execute it. Look at the code and note that multiple processes can also communicate using a Queue—but this time it’s a queue defined in the multiprocessing module. This is not shared memory—there’s some message passing going on in the background to enable the processes to all put their output in the queue.

Note the way spartan_ideas1.py uses p.join()—it makes a list of processes then starts them, one by one. It joins the processes in a separate loop so they will all be started without waiting for the joins. Take out the calls to join(). Does it make any difference in the way the program works? You can see that the join() calls are not needed in this program—the main process will wait until all processes have finished before it exits (we can set processes to daemon if we want the main process to be able to exit while some are still executing). But sometimes we have to let all processes finish before going on to some other operation—then the joins are necessary.

It’s impossible to predict what order the strings will be in when they go into (and come out of) the queue. Order probably isn’t very important when you’re looking at random strings but sometimes order is important. Modify spartan_ideas1.py to print the strings in the order they were created.

Add the loop variable (x) as an argument to rand_string().

Instead of just putting the string into the queue, put a tuple that has, as its first element, the new loop variable argument and, as its second element, the rand_str.

In main, sort the results (results is a list; results.sort() will sort on the first element in each tuple).

There may be some Python elements you aren’t used to seeing in this program. In particular, list comprehensions are widely used by people who program a lot in Python (processes and results are created using list comprehensions and rand_str seems to be using a list comprehension as well, although it looks a bit like magic to me). Dictionaries, tuples, and sets can also be created using comprehensions.

Explanation / Answer

import multiprocessing as mp
import random
import string

output = mp.Queue()

def rand_string(length, output):
rand_str = ''.join(random.choice(string.ascii_lowercase +
string.ascii_uppercase +
string.digits) for i in range(length))
output.put(rand_str)

def main():
processes = [mp.Process(target=rand_string, args=[5, output]) for x in range(4)]
for p in processes:
p.start()
for p in processes:
p.join()
results = [output.get() for p in processes]
print(results)

if __name__ == "__main__":
main()

import multiprocessing as mp

def foo(i):
print("Called function in process:", i, __name__)
return

def main():
Process_jobs = []
print(__name__)
for i in range(4):
p = mp.Process(target = foo, args = [i])
Process_jobs.append(p)
p.start()

if __name__ == "__main__":
main()

Related Questions

Navigate

• Browse (All)
• Browse T
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.