We are using the following dataset(there are total of 4.4M tweets in this file,

Question

We are using the following dataset(there are total of 4.4M tweets in this file, but you only need to read 1M): http://rasinsrv07.cstcis.cti.depaul.edu/CSC455/OneDayOfTweets.txt

Need to use python to download from the web and save to a local text file at least 1,000,000 lines worth of tweets. Test your code with fewer rows first and only time it when you know it works. Report how long did it take.

We cant use: read() or readlines() without any parameters at any point. That command will attempt to read the entire file which is too much data.

Explanation / Answer

data = urllib.request.urlopen("http://rasinsrv07.cstcis.cti.depaul.edu/CSC455/OneDayOfTweets.txt").read(20000)#reads 20k characters
str_data = str(data)
lines = str_data.split(' ') # then split it into lines
fx = open('downloadedfile.txt','w')
for line in lines:
    fx.write(line+' ')
fx.close()

Related Questions

Navigate

• Browse (All)
• Browse W
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.