SIMPLE STEPS FOR ROBOT.TXT FILE
Today’s article cover the most simple steps for generating a robot.txt file using Yoast SEO. Let’s start discussing importance of robot.txt file before we even start generating a robot.txt file. A website owner and/or admin must be familiar with such tech hacks for authenticity of their website.
WHAT IS A ROBOT.TXT FILE?
Robot.txt is a plain text file in root directory of your website. The main function of this file is to tell search engines where they should crawl and what areas of your website to avoid. If there is a blank robot.txt within your website public_html that means you consent to show all content of your website. I must say that a a website admin must be aware of such facts and importance of robot.txt file.
HOW A ROBOT.TXT FILE LOOKS LIKE?
Sequel to above mentioned content, it is a plain text file. All website comes with a standard robot.txt format that looks like below. This format allow search engine to crawl your website freely and show the results they are crawling or collecting from your website without intimating you about this.
HOW TO OPTIMIZE ROBOT.TXT IN WORDPRESS
You can upload or modify the existing robot.txt file in root directory but if you are using Yoast SEO plugin that means you can do it within an eye jerk. So let’s start setting up robot.txt file using Yoast SEO plugin with these simple steps:
- Hover over Yoast icon in the left menu of WordPress admin and select Tools.
- On the next screen select File editor from below options
- Import and Export
Import settings from other SEO plugins and export your settings for re-use on (another) blog.
- File editor
This tool allows you to quickly change important files for your SEO, like your robots.txt and, if you have one, your .htaccess file.
- Bulk editor
This tool allows you to quickly change titles and descriptions of your posts and pages without having to go into the editor for each page.
- Import and Export
- Next you will see Create Robot.txt file as the first option or it could be a text block having standard robot.txt permissions in it as mentioned above.
- If you are a WordPress user then i strongly recommend using following format for setting up robot.txt file for your website but you can also format the one that already exists.
Remember don't use figures in the start..
- Change sitemap URLs to the one for your website sitemap and Save Changes.
Don’t forget to check out:
Do you want a Free Instant SEO Analysis and SEO Site Checkup for your Website? Get in Touch