Search Results for:
*
Advanced Screaming Frog Crawling Tips and Use Cases for Ecommerce Auditing
Auditing ecommerce websites with seemingly infinite pages can be a real challenge. Luckily, the Screaming Frog SEO Spider tool can be a powerful ally for SEO professionals seeking to dig into all the nitty-gritty details of their websites. However, if...
Robots.txt
The robots.txt is a file that sits on the root of a domain, for example: https://www.screamingfrog.co.uk/robots.txt This provides crawling instructions to bots visiting the site, which they voluntarily follow. In this guide, we’ll explore why you should have a robots.txt,...
How to Approach Awkward Link-building Questions: An SEO Guide for PRs
Get ready for the most caveat-ed article you’ve ever read, ever. Picture the scene… you’re working away, head down, when *ping* a client email drops into your inbox. And unfortunately, they’ve posed a somewhat tricky question that you’re not 100%...
Leveraging Paid Data for SEO Insights
PlayStation or Xbox? Apple or Android? These debates have divided the friendship groups of millions around the world, but within the digital marketing world, it’s SEO and PPC that is sometimes a cause of contention. However, in a world seemingly...
What I Learnt from Analysing 7 Million Log File Events
It’s been almost three years since I started to work intensely on log file analysis for SEO purposes, and to make the analysis simple, I decided on the Screaming Frog Log File Analyser as my tool of choice. Because I...
How To Manage Your Inner Chimp In The PR Jungle
If you find yourself feeling gut-wrenchingly anxious, bubbling with anger or out of your depth in the daily PR grind, it could be because your inner chimp is taking over. Everyone has an inner chimp, whether they like to admit...
Screaming Frog SEO Spider Update – Version 16.0
We’re excited to announce Screaming Frog SEO Spider version 16.0, codenamed internally as ‘marshmallow’. Since the launch of crawl comparison in version 15, we’ve been busy working on the next round of prioritised features and enhancements. Here’s what’s new in...
How To Crawl A Staging Website
Find out how to crawl a staging or development website, considering robots.txt, authentication, and the SEO Spider configuration.
How To Use List Mode
Upload a list of URLs using 'list mode', and control what other elements are crawled, such as external links, or images for laser-focused crawling.
Page Titles
Writing a good page title is an essential skillset for anyone in SEO, as they help both users and search engines understand the purpose of a page. In this guide we take you through the fundamentals, as well as more...
