What is Robot.txt in website SEO?

Posted by Rishi Kumar
6
Dec 15, 2014
6 Views
18 people like it
avatar avatar avatar avatar avatar
Comments (13)
avatar
Rishi Kumar
6

SEO Webmaster

#Hi Gaurav, Its a bad practice to hide information using robot.txt file, actually its used for how to instruct the search engine (web crawlers/spiders) to crawl and index web pages.

Apr 7, 2014 Like it
avatar
Gaurav Kumar
12

Sr. SEO Executive

robots.txt file used to hide web pages from search engine crawlers.

Apr 7, 2014 Like it
avatar
Emily Marie
6

marketer

The Robots.txt file tells the search engine spiders what not to index. Also make sure the file doesn’t accidentally exclude important files, directories or the entire site (this has been known to happen!).

Mar 19, 2014 Like it
avatar
Askir.bobker@gmail.c...
2

couture 100% en main

Help me for citybank trensfering my foud to my account bank in moroco

Mar 18, 2014 Like it
avatar
Miguel G.
7

Marketing,sales,advertising consult

The robots.txt file is one of the basic parts of a web hosting settings and has a great importance in the basic settings at the level of SEO.
As we have already mentioned the robots.txt is a file that says its extension is a txt file or text file, this is known as the robots exclusion protocol or robots.txt protocol.

Mar 18, 2014 Like it
avatar
Natilia James
6

Part Time Writer

when you do not want to show any type web page or content from search engine boats. you apply robat.txt for that. according to me.

Mar 18, 2014 1 Like Like it
avatar
Rajeev Sharma
3

Travel Services

ya I Agree with preet sharma

Mar 18, 2014 Like it
avatar
Preet Sharma
10

Content Writer

Great answer by Sheryl and Vishwajeet cinnhotels.com

Mar 15, 2014 1 Like Like it
avatar
Vishwajeet Kumar
14

Blogger

robot.txt file helps search Engines to determine which page of your site to crawl or not.

Mar 15, 2014 Like it
avatar
Rishi Kumar
6

SEO Webmaster

thanks to all of you.

Actually, It is the Robots Exclusion Protocol to tells the browsers to visiting to the all parts of the websites.

Its a bad practice to hide information using Robot.txt

Mar 14, 2014 Like it
avatar
Sheryl K. Malone
10

Custom Logo Design by Award Winners

Web site owners use the /robots.txt file to give instructions about their site to web crawlers!

Mar 14, 2014 3 Likes Like it
avatar
Radha Chauhan
8

Search Engine Optimizer

Robot.txt tell the crawler which page you dont want to visit them

Mar 13, 2014 2 Likes Like it
avatar
HansRaj Deshwal
5

Web Developer and SEO Expert

robots.txt file used to hide web pages from search engine crawlers for protection of duplicate content pages or security pages!!

Mar 13, 2014 2 Likes Like it
avatar
Please sign in to add comment.