How do Search Engines Work?
Many web pages are excluded from most search engines by policy including the searchable databases mounted on the web, such as library catalogs and article databases. All this material is referred to as the "Invisible Web"
Search engines (SE) searches a database of web pages that built by computer robot programs called spiders and indexed by another computer program. If a web page is never linked from any other page, search engine spiders cannot find it. The only way a brand new page can get into a search engine is for other pages to link to it, or for a human to submit its URL for inclusion.
What Makes a Search Engine Good?" - a table (PDF file) summarizing useful factors for evaluating search engines.
Search Engine Features -- Search features and functions of leading search engines.
Crawler-based Search engines Google is one of search engines which works on the method of crawling the data on web, using an automatic software or a bot and all the data was indexed in an Index file , helping in fast retrieval of results.
Meta search engines attempt to search several different Web search tools (both search engines and subject directories) with one search. A metasearch engine is a search tool that sends user requests to several other search engines and/or databases and aggregates the results into a single list or displays them according to their source. Metasearch engines enable users to enter search criteria once and access several search engines simultaneously. Popular meta search engines include DogPile (which searches more of the large search engines than any other meta search engine) and Ixquick. (i.e.:
Vertical search engines searches a specific industry, topic, type of content (e.g., travel, movies, images, blogs, live events), piece of data, geographical location, and so on. It may help to think of vertical search as a search within a particular niche. (i.e. IMDB - The Internet Movie Database)
Google Knowledge Graph is based on more than search engine rankings.
Google PageRank is a graphical representation of relevancy ranking.
Relevancy ranking is the process of sorting the document results so that those documents which are most likely to be relevant to your query are shown at the top. The simplest method for this is to show highlighted teasers or dynamic page summaries. This technique displays snippets of the source document in the search results with query terms (or expansion terms) highlighted. If the user can quickly see exactly why the document is retrieved (because, for example, both of the user's terms occur close together within a sentence of the document), this makes it more apparent how the engine is working and provides the user with a greater level of comfort that the engine is behaving rationally.
Second, the user query terms can be highlighted within the document itself. Naturally this is a tricky proposition in the wild and wooly world of the World Wide Web, but it can be done and can help the user to determine exactly why a document was retrieved.
Other search engines will provide explanations such as:
All of these techniques, if they can be made readily apparent to the end-user, provide elbow-room for search engines to expand what it means to provide "User Understandable Relevancy."