How to Kill Duplicate Content
The discussion of Duplicate Content just will not go away. So instead of hiding from it we are going to face it head on and completely destroy it. We all want to impress our friends at Google hoping to gain some favor with a great ranking so we will do what they tell us to concerning duplicate content. Google says to use an xml sitemap with a robots.txt file so that they may crawl your site better. It just so happens that we will use these exact tools to destroy duplicate content on your site.
An xml sitemap will list your sites urls for the search engine spiders to follow. They will then follow those links to the pages and begin crawling them for unique content. It’s at this stage that the spiders are starting the process of deciding to index the page or not. If they find that the content is extremely similar to content they already have indexed they may not include it in the search results. By controlling the links on your xml sitemap you can begin the process of weeding out any duplicate content on your site.
Creating a sitemap.xml file is as easy as pie. For the serious webmaster with a little working knowledge of Python scripts you can download Googles sitemap generator and install it on your server. You can get it here. For the less tech savvy (me) I found a site that will let you build your sitemap for free (under 500 pages). They also offer a standalone version in PHP for $19.99 or $29.99 installed on your server. The site is XML Sitemaps.com and I think you will like their service. Here is another site with some good resources http://xmlsitemap.com/
The second part of our solution to duplicate content is a robots.txt file. While the sitemap.xml file will tell the spiders where to go, the robots.txt file will tell them where not to go. You can restrict or even deny search spiders with this file. You can also disallow urls with this function and Google recognizes and follows the directions of a robots.txt file. So if you have pages that contains content you know is duplicated throughout your site or the net you can put those urls in your robots.txt file.
Their is a ton of information on the internet concerning robots.txt files but you may as well start at the beginning and visit http://www.robotstxt.org/. You will learn everything you ever need to know about these files on that site. By directing the movements of the search spiders you can help to ensure the long term health of your website.
For WordPress users we have a very simple and extremely effective XML Sitemap plugin. I created this video to show you the settings I use to create my XML sitemap. Use the link above to grab the newest release.
| This video requires JavaScript to be enabled. If you are you using a browser with JavaScript disabled please enable it now. Otherwise, please update your version of the free Flash Player by downloading here |
Thanx and remember…Always strive to improve and success will follow
David Knops Jr
DKWebsites LLC
Related Websites
- The Worst Mistakes by Search Engine Optimizers Flash If you're planning to make a new website using Flash making a new website and thinking of using Flash - stop. Though Adobe has...
- Googlebot, Robots.txt Sitemaps and Wordpress Plugins It's a bit of a mouthful for a title I know, but I hope this post will be useful for a number of reasons. Recently,...
- 3 Long Term Methods To Get More Blog Traffic - Part I Traffic is the mainly Assert for a blog. Driving more traffic to your new blog is one of the toughest part of blogging. In actual,...
Related posts:
[...] unknown wrote an interesting post today onInternet Marketing Techniques-Website Marketing Tips, Tools, and IdeasHere’s a quick excerptDuplicate content runs rampant on the internet and the search engines hate it.Discover how to kill duplicate content on any site.Video for WordPress users. [...]
[...] unknown wrote an interesting post today onInternet Marketing Techniques-Website Marketing Tips, Tools, and IdeasHere’s a quick excerptDuplicate content runs rampant on the internet and the search engines hate it.Discover how to kill duplicate content on any site.Video for WordPress users. [...]
[...] unknown wrote an interesting post today onInternet Marketing Techniques-Website Marketing Tips, Tools, and IdeasHere’s a quick excerptDuplicate content runs rampant on the internet and the search engines hate it.Discover how to kill duplicate content on any site.Video for WordPress users. [...]
[...] unknown wrote an interesting post today onInternet Marketing Techniques-Website Marketing Tips, Tools, and IdeasHere’s a quick excerptFeatured · Internet Marketing Tips · Affiliate Marketing Tips · Article Marketing Tips · Email Marketing Tips · Network Marketing Tips · Niche Marketing Tips · S.E.O. Tips · Social Networking Tips … [...]
[...] Another fellow blogger placed an interesting blog post on Internet Marketing Techniques-Website Marketing Tips, Tools, and IdeasHere’s a brief overviewFeatured · Internet Marketing Tips · Affiliate Marketing Tips · Article Marketing Tips · Email Marketing Tips · Network Marketing Tips · Niche Marketing Tips · S.E.O. Tips · Social Networking Tips … [...]
[...] Internet Marketing Techniques-Website Marketing Tips, Tools, and Ideas [...]
Hi Dave,
No worries on the link – you have a nice site going here. You can actually thank another WordPress plugin (pingcrawl.php) for finding you in the first place! It can be a bit trigger happy at times so I review the links and your site is clearly seo related and useful :0)
Welcome to my bookmarks
Adam Arnold
ceo.smartergroup.biz
Adam Arnolds last blog post..WordPress Plugins