Search Tech Blog

How-To develop a real search engine from scratch

Dogfooding

D

Dogfooding a Quick Guide to Internal Testing Eating your own dog food, also called dogfooding, occurs when an organization uses its own product. 1970 Alpo’s Dog Food Advertising Alpo, with the help of Lorne Greene, convinced consumers to buy their products because they themselves used them. 1980 Memo from the Apple CEO „We believe the typewriter is obsolete. Let’s prove it inside before we try...

PHP dynamic Favicon Script

P

What is a dynamic Counter Favicon you will ask? No, you already know! We all have already seen the small icon in the Browser Tab called Favicon. But if you want to read more about this tiny little thing, take a look at the Favicon-Cheat-Sheet on GitHub Page from Audrey Roy Greenfeld. But today i want to direct your attention to an other topic. Dynamic content in such an favicon. This can be used...

User-Agents of the Top 10 Web-Crawler

U

There are thousends of bots and web crawlers working the internet but below is my list of the 10 popular search engines user-agents. If you browse the logfiles of your website, you will always see the access to a file called “robots.txt”. These are usually calls from search engines. Their web crawlers with there user-agents that read the robots.txt file (hopefully you have one). They...

Build a search engine in just 3 days

B

Data is a fairly new resource, which gets more important every day. BigData is a common and important keyword for every company that develops and is trying to success over its competitors. The company that can use its data best may win this competition. For this you need software – to collect, to store, to provide and to analyse data. A search engine is a key tool to get the needed data...

Linklist you need if you want to build a search engine

L

Linklist Here are some background informations about how a search engine exactly work. We light ub what is difficult to crack if we try to build our own web crawler search engine from scratch: Giga Blast This page is a bit outdated (2004). But here you can read from the developer Matt Wells personally: All steps the search engine GigaBlast went through during the development process: After that...

Stop-Word List

S

What is a stop-word list and what advantage does it have to remove them? Stop words are extremely common words A Stopword is a word without essential information content, such as “and”, “the”, or “www”, etc. In English, the terms “stopword” or “stopwords” are used for this purpose. They are used very often, but do not really provide any...

Writing Your Own Search Engine is Hard

W

Why is it so hard? Anna Patterson, Software Engineer. As well as contributed to search engines and artificial intelligence at Google, and co-founded Cuil. Makes following quotation about developing search engines: “There must be 4,000 programmers typing away in their basements trying to build the next “world’s most scalable” search engine. It has been done only a few times. It has never...

Search Tech Blog

Latest posts

Latest comments

Categories

Tag Cloud