Write an algorithm(Not Visual basic or c++ code) that describes how to implement

Question

Write an algorithm(Not Visual basic or c++ code) that describes how to implement a search engine

This should be a description of the steps that can
be taken to generate the output. These steps need to be in a form of a numbered list. Clearly
identify what the inputs and outputs are. You should describe any necessary steps that the
program should take before a user begins the program as well. This defines what computations
need to be processed by the program before it is ready to accept user input.

Search Engine Links:
http://computer.howstuffworks.com/internet/basics/search-engine.htm/printable
http://en.wikipedia.org/wiki/Search_engine

Explanation / Answer

I list only the basic steps (not going too much into technical part. Understanding how each step works is important). Each step is described clearly. If you wish to learn making a web search engine from scratch take a look here: https://www.udacity.com/course/cs101

So here we go:

First of all we need to build a web crawler (Wait!!! WTF is that? Read here : http://en.wikipedia.org/wiki/Web_crawler) which will extract all the words from a page and if user queries any of that words we can provide the link to the page where the word was found.

So we need a data structure of some kind (dictionary/hash table) to store our web crawler output data and to match user queries.
We can make a dictionary of words and each word will store page links of where it was found.

ALGORITHM :

Step 1: Provide a base page link
Step 2: Scrape (extract) all the words from the page and store this page link with all the words scraped here. Use a dictionary or a hash table.

Step 3: While scraping if we come across a link we go to that link and perform Step 2. To stop the looping use some base condition like to store only 10 page links per word and max words to store is 10000. (BTW google web crawler is always running.)

Step 4: Now while doing the above steps you need to think of an amazing page ranking algorithm. Like how are you going to rank all the results. Different search engines use different ranking algorithms. Google initially used PageRank algorithm. (Check here if intereseted : http://en.wikipedia.org/wiki/PageRank)

Step 5: Now build a user interface (web page for example) and whenever user searches for a particular word you look up for that word in our dictionary (we made in Step 2). If the word found you can show all of the links stored there.

Related Questions

Navigate

• Browse (All)
• Browse W
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.