We are using the following dataset(there are total of 4.4M tweets in this file,

Question

We are using the following dataset(there are total of 4.4M tweets in this file, but you only need to read 1M): http://rasinsrv07.cstcis.cti.depaul.edu/CSC455/OneDayOfTweets.txt

Write a python coded that will download the adove link's data and save to a local text file. ONLY needs to read 1,000,000 lines worth of tweets.

We cant use: read() or readlines() without any parameters at any point. That command will attempt to read the entire file which is too much data.

This is want I have so far:

import time
import urllib.request

Start = time.time()

webFD=urllib.request.urlopen("http://rasinsrv07.cstcis.cti.depaul.edu/CSC455/OneDayOfTweets.txt")

New_file = open('downloadedfile.txt','w')

Line = webFD.readline() - I can use that so what do I need to do instead

tweetsdata= (Line.decode('utf8')).split('EndOfTweet')
  
for L in range(100000):
   New_file.write(L +' ')

New_file.close()

End=time.time()

print("difference is", (End-Start), "seconds")
print("performance:", 100000/(End-Start), "operation per second")
print("new file for tweetdata is on the desk top")

Explanation / Answer

main.py

========================================================

Related Questions

Navigate

• Browse (All)
• Browse W
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.