Problem 6. Basics for Spark Streaming and GraphX (Spark Streaming) Suppose we cr

Question

Problem 6. Basics for Spark Streaming and GraphX (Spark Streaming) Suppose we create a Spark Streaming Context "streamingContext" in Python Spark. streamingContext- StreamingContext(sc, 3) where "c" is the SparkContext instance. What does the second parameter "3" represent here? Answer: We want to stream the data from TCP source with IP Address "123.45.67.89" and port number "4321". Please provide the function for streaming data from that TCP source. The return variable is a (Spark GraphX) Suppose we have a graph in the following. 4 The left figure shows the graph, and the right column shows the edge list file. Suppose the full directory of the file is "/home/rob/data/ToyGrapho1.txt". "SC is the Spark Context. Please provide the command to load the graph edge list file in Scala 1: If we want to get the number of vertices, we should use the following command in Scala 1:

Explanation / Answer

Basically for answer 1 streamingcontext constructor defined here has two parameters and streamingcontext have various constructors for defining varios streamcontext here we have the following constructor:

StreamingContext(sc,3);

where sc is is the spark context instance and the generalized constructor is

StreamingContext(spark context Instance,batch duration)

hence 3 here defines -BATCH Duration

2) function for streaming data from TCP source

ssocketTextStream(localhost,port)

3)we use the following function load edges if sc is the spark context

sc.edges.map()

for vertices we can use

sc.vertices ()

Related Questions

Navigate

• Browse (All)
• Browse P
• Subjects

---

---

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.