For Google AdSense publishers experiencing recent ad fulfillment issues, crawler access might be the problem

A small segment of Google AdSense publishers have noticed changes in ad delivery on their sites over the past month or so. Publishers say ads are not being served and the ad slots on their pages appear as blank spaces, causing drops in revenue.

There has been speculation on the AdSense forum that the issue might be caused by policy changes, or by dynamic pages or URL parameters, or when a new page has not yet been crawled by the AdSense bot. It turns out that last guess is closest to being correct.

Google is making some changes to buying processes on the Google Display Network. As a result, Google will not automatically monetize sites or pages that the AdSense bot can’t crawl, Marketing Land learned Friday. The AdSense crawlers categorize, classify and label content for monetization.

Therefore, the issues some publishers are experiencing may be the result of not properly granting AdSense crawler access to their sites. Google says the problems aren’t widespread, but the changes — which are still in early stages — are affecting some publishers more than others. (Blank ad spaces can also appear if the ad impression simply isn’t fulfilled.)

If you’re experiencing problems with ads not being served properly — and there are no policy violation notices in your AdSense account — check the crawler access error tab located under “Status” in AdSense for notice of crawler implementation errors. It’s possible the crawler is being blocked in the site’s robots.txt file (see details in this Help Center article) or getting tripped up by when content is served behind a login (see how to give access to login-protected pages here).

Google says it is working on identifying publishers that are impacted to help them resolve issues and will be issuing further guidance in the short term. The team will be evaluating results of the early-stage changes over the coming weeks.