Sitewide HTTPS:// and SEO

admin Search Engine Optimisation Comments

In the coming weeks I`m planning on making the whole Sitebee website secure through the SSL protocol, now the only drawback is that the search engines will see two different versions of the Sitebee website. One on the standard http:// and the other on https:// protocols.

In the eyes of search engine crawlers that equates to duplicated content. We need to tell the crawler to which version of the website to crawl, you can do this using .htaccess and robots.txt

1. Create a robots file and name it robots_ssl.txt  in the root directory (public_html).

2. Add the following code on the robots_ssl.txt file:-

User-agent: * 
Disallow: /

3. Add the following code to your .htaccess file in your root directory.

RewriteEngine on 
Options +FollowSymlinks 
RewriteCond %{SERVER_PORT} ^443$ 
RewriteRule ^robots.txt$ robots_ssl.txt 

That`s it, you have now told the search engine crawler to index only your https: pages, thus removing the chance of receiving a duplicate content penalty. The .htaccess file will now redirect the search engine crawlers to the robots_ssl.txt which tells the crawler to use port 443. (If you have a custom port for ssl just replace 443 with your port number)

There`s a little more going on here, but without trying blow your brain, it`s the simplest way I can explain it.