Difference between parallel processing and pipeline processing? Solution Pipelin

Question

Explanation / Answer

Pipeline parallel processing Another type of parallelism is pipeline processing. Here a number of tasks do partial processing of data and then forward the partially processed result to another processing client down the pipeline for further processing. A task familiar to radio astronomers is the processing of spectra from correlator lags to spectra. Three steps in the processing are 1) multiply the observed lags by a window function 2) fft the lags 3) subtract off a baseline to obtain the computed spectrum. const demopipeline := function() { # initialize necessary variables private := [=] public := [=] private.parallelclient := [=] private.numparallelclients := 0 private.pipelineconstructed := F # open 'hosts' file; then read in and start clients hosts := ["slave1", "slave2", "slave3"] numhosts := len(hosts) tasks := ["pipe1", "pipe2", "pipe3"] taskpreface := "/home/beowulf1/twillis/aips++/code/hia/apps/pipeline/" for (i in 1:numhosts) { # use split function to delete endline hostname := hosts[i] taskend := tasks[i] taskname := spaste(taskpreface,taskend) private.numparallelclients := private.numparallelclients + 1 private.parallelclient[private.numparallelclients] := client(taskname, host=hostname) await private.parallelclient[private.numparallelclients]->initialized } # handle event associated with final output by printing it. whenever private.parallelclient[3]->finalspectrum do { outputspectrum := $value.reducedspectrum print 'outputspectrum ',outputspectrum } # instead of using these 'whenevers' to pass events, use the 'link' command # below # whenever private.parallelclient[1]->tostep2 do # private.parallelclient[2]->[$name]($value) # whenever private.parallelclient[2]->tostep3 do # private.parallelclient[3]->[$name]($value) # set up pipeline links link private.parallelclient[1]->tostep2 to private.parallelclient[2]->* link private.parallelclient[2]->tostep3 to private.parallelclient[3]->* # pause here to make sure links get set before events start flowing through them d := client("timer", "-oneshot 2.0") await d->ready const public.dopipeline := function(speclength = 1024) { wider private # create a block spectrum spectrumarray := array(0.0,speclength) specbegin := speclength / 8 specend := speclength - speclength / 8 spectrumarray[specbegin:specend] := 1.0 # send 5 identical block spectra through the pipeline specrecord := [=] specrecord.spectrum := spectrumarray for (i in 1:5) { print 'sending event ',i private.parallelclient[1]->inputdata(specrecord) } return T } return public } a := demopipeline() a.dopipeline(8)

Related Questions

Navigate

• Browse (All)
• Browse D
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.