The crawler sends GET requests to the server. The server responds with headers and the contents of the file, which then gets saved.
The request is likely to come from a mobile user-agent since Google is mostly on mobile-first indexing now. You can check to see how Google is crawling your site with the URL Inspection Tool inside Search Console. When you run this for a URL, check the Coverage information for “Crawled as,” and it should tell you whether you’re still on desktop indexing or mobile-first indexing.
The requests mostly come from Mountain View, CA, USA, but they also do some crawling for locale-adaptive pages outside of the United States. I mention this because some sites will block or treat visitors from a specific country or using a particular IP in different ways, which could cause your content not to be seen by Googlebot.
The workload like this whatsapp number list allows both the vendor and the affiliate to focus on. Clicks are the number of clicks coming to your website’s URL from organic search results.
Some sites may also use user-agent detection to show content to a specific crawler. Especially with JavaScript sites, Google may be seeing something different than a user. This is why Google tools such as the URL Inspection Tool inside Google Search Console, the Mobile-Friendly Test, and the Rich Results Test are important for troubleshooting JavaScript SEO issues. They show you what Google sees and are useful for checking if Google may be blocked and if they can see the content on the page. I’ll cover how to test this in the section about the Renderer because there are some key differences between the downloaded GET request, the rendered page, and even the testing tools.
It’s also important to note that while Google states the output of the crawling process as “HTML” on the image above, in reality, they’re crawling and storing all resources needed to build the page. HTML pages, Javascript files, CSS, XHR requests, API endpoints, and more.
2. Processing
There are a lot of systems obfuscated by the term “Processing” in the image. I’m going to cover a few of these that are relevant to JavaScript.
Resources and Links
Google does not navigate from page to page as a user would. Part of Processing is to check the page for links to other pages and files needed to build the page. These links are pulled out and added to the crawl queue, which is what Google is using to prioritize and schedule crawling.