Catching Unwanted Spiders And Content Scraping Bots In ASP.NET

by Colin Cochrane 12/30/2007 7:20:00 PM
If you have a blog that is even moderately popular then you have likely fallen victim to some form of content scraping. Ever since it became possible to earn money through ads on a website there have been people trying to find ways to cheat the system. The most widespread example of this comes in the form of splogs and similar spam-based websites, which consist only of ads from Google AdSense and duplicated content from other sites. In this post I will share a method you can use to identify "evil" spiders and bots that are wasting your website's resources. [More]

Tags: , ,

ASP.NET | SEO | Web Development

Is There Added Value In XHTML To Search Engine Spiders?

by Colin Cochrane 12/8/2007 11:10:00 AM
The use of XHTML in the context of SEO is a matter of debate. The consensus tends to be that using XHTML falls into the category of optimization efforts that provide certain benefits for the site as a whole (extensibility, ability to use XSL transforms) but offers little or no added value in the eyes of the search engines. That being said, as the number of pages that search engine spiders have to crawl continues to increase every day, the limits to how much the spiders can crawl are being tested. This has been recognized by SEOs and is reflected in efforts to trim page sizes down to make a site more appealing to the spiders. Now it is time to start considering the significant benefits that a well-formed XHTML document can potentially offer to search engine spiders. [More]

Tags: , ,


Powered by BlogEngine.NET

All Content and Intellectual Property is under Copyright Protection | Colin Cochrane ©2007

About the author

Colin Cochrane

Colin Cochrane

SEO and ASP.NET Developer.

Recent comments

Recent posts




This is a personal weblog. The opinions expressed here represent my own and not those of my employer. © Copyright Colin Cochrane 2014

Sign in