Welcome to Crawl or No Crawl - this is Carolyn Holzman and today is October 31 - 2022 -
Today is a test day
New content is published 3 times a week - Monday Wednesday and Friday which for this week is today so All other days are observation.
You’re looking at the Indexation Research Log - it’s where dates, times and updates are notated - this is a free resource - you can find it on the American Way Media site - search for google index detector - top result
New page was published, submitted via the Google Indexing API - and the first bot arrived one minute later -
Big news - for the first time since October 10 - new content last week’s content in fact - is now findable by its keywords - Monday and Friday test pages - Wed test had no love - but the big news is new content processed through the indexing and serving system - yay!
Search console page coverage update happened on Saturday night
Have two things to share
I went back to look at what the desktop bots were crawling Friday and I did notice a few more referral crawls meaning - in the record you can see what if anything lead to the crawl - for instance the page URL of your backlink sources can be shown - so google discovered a page where a link back to you was placed and then follows that link and it’s recorded in your server logs -
And remember I said that 27 pages that were previously findable by their keywords appeared in the crawled but not indexed bucket -
That’s good to know - that content can go from findable to not findable - so I decided to do a wholesale resubmission of those 27 pages to the Indexing API
I want to test to see if I can increase the number of items that show in the site search group - that’s where you limit to only pages from your site by searching site colon domain name - and it looks like you can. Went from 17 to 35 to 40 in the site search operator - so 24 of those 27 are now back to being findable
October 10 is the pivotal day - its been an entirely different ball game since then - it was a wholesale site shift where almost 100 pages were/are no longer served by their keywords and today being the first time new content tested on by regular html rendering and b javascript rendering
Those pages are not in the site search operator group - so now we’re back to it doesn’t only contain pages that are being served.
Remember how I told you to get those pages indexed that show up as not indexed and the reason is Google SYstems.
What we have learned so far is
That indexed while it is a requirement to be served - being indexed in search console DOES NOT MEAN the same as finding your content ON Google by it’s keywords.
Let’s look at the bots.
This is the least amount of crawling I’ve observed. Make me wonder what they are up to and I and others are sensing that this behavior - an excess of crawling and a minimal of crawling might mean something - we’ll see. I’m still considering we’re in a change phase.
Very little recrawl with the desktop bots - so if you’re making changes to your content and not submitting via the Google Indexing API - it will be a while
Very little recrawl with the mobile bots - if you’re publishing and using search console to let Google know about your content - it’s gonna take a really long time
You may see traffic shifts IF you are looking at individual pages and terms - if you’re looking at the overall data in the graphs in search console - that may not be telling you the whole story -
Stay tuned.
Please Subscribe - it costs you nothing and it tells me this is of interest to you - - feel free to post any comments or questions about what you’re seeing currently -
I’m will be offering the Forensic SEO training again in early 2023 - starting in late January most likely so we’re all past the holidays - if you’re interested in getting a heads up - go to forensicseotraining.com and use the form to ask to get on the early bird list.