Directives: Unavailable_After

URLs containing a ‘unavailable_after’ directive in either a robots meta tag or X-Robots-Tag in the HTTP header.

This directive allows an exact time and date to be specified for Google to stop showing the page in the search results.

How to Analyse in the SEO Spider

View URLs with this issue in the ‘Directives’ tab and ‘Unavailable_After’ filter, and export all URLs using the ‘Export’ button.

What Triggers This Issue

This issue is triggered when URLs contain a ‘unavailable_after’ directive, either in a robots meta tag within the HTML or in an X-Robots-Tag in the HTTP header.

For example:

https://www.screamingfrog.co.uk/

Has the meta tag:

<meta name="robots" content="unavailable_after: 2023-05-19">

How To Fix

URLs with a ‘unavailable_after’ should be reviewed to ensure the date/time specified is correct for the pages to not be shown in the SERPs.