Search Results for:
*
Robots.txt Testing In The SEO Spider
View URLs blocked by robots.txt, the disallow lines & use the custom robots.txt to check & validate a site's robots.txt thoroughly, and at scale.
Crawling Password Protected Websites
Crawl websites that require a login, using web forms authentication using our inbuilt Chrome browser.
How Accurate Are Website Traffic Estimators?
If you’ve worked at an agency for any significant amount of time, and particularly if you’ve been involved in forecasting, proposals or client pitches, you’ve likely been asked at least one of (or a combination or amalgamation of) the following...
Web Scraping & Custom Extraction
Scrape any data from the HTML of a page using CSS Path, XPath and regex to enhance a crawl, such as author name, comments, shares or more.
Screaming Frog SEO Spider Update – Version 4.0
I’m really pleased to announce version 4.0 of the Screaming Frog SEO Spider, codenamed internally as ‘Ella’. We have been busy in development working on some significant improvements to the SEO Spider due for release later this year, which include...
Screaming Frog Changes For VAT In 2015
I’m writing this post on the eve of 2015, as I prepare to update 28 VAT rates for each country in the EU as they enter the New Year. We first found out about the EU VAT changes on the...
Screaming Frog SEO Spider Update – Version 2.30
I am really pleased to announce version 2.30 of the Screaming Frog SEO spider code-named ‘Weckl’. This update includes new features that provide greater control of crawls, as well as how the data is analysed. Thanks again to everyone for...
Negative SEO - Should You Monitor Your Backlinks?
Historically links were often either counted or ignored by Google in their scoring. However, as their algorithms evolved over the past 5 years and more recently with the introduction of Penguin in April 2012 and increase in manual actions, links...
The 11 Most Infamous Google Bombs In History
Google bombing is a practise whereby a specific web page is targeted to rank in 1st position in the SERPs for a particular search phrase, so that when that phrase is typed in Google it brings often humorous or controversial...
Screaming Frog at LinkLove London
Screaming Frog at LinkLove London 2012 Last Friday the Screaming Frog SEO team ventured out of our comfortable setting of sunny Henley-on-Thames, to head into the Big Smoke to attend Distilled’s much lauded LinkLove conference. Distilled have been running successful...
