Wait for Google to visit the new page
The most natural way to find out about new subpages of a website is through internal and external links leading to it. Google robots, upon discovering them, drag them onto the list of pages to crawl and index. If it enjoys authority and is popular, you won't have to wait long for the page to be added to Google. Otherwise, the process may drag on. In that case, it's best to try the methods below.
Indexing a page using Google Search Console
Just a year ago, adding a page via Google Search Console was one of the most effective methods. The URL would appear in search results up to half an hour after submitting a report. Now, the indexing process after reporting can take a very long time. Despite this, I recommend reporting the page via this tool first.
To do this, in the upper bar in the search field, enter the address of the subpage and press Enter. After downloading the data about the page, click the REQUEST INDEXING button. After a few minutes, the request will be processed and all that remains is to wait for the web robots to visit and index the website.
request indexing button
If the page is not in the index, we can send a request to index it
Creating a sitemap and image map
An XML sitemap is a great source of URLs for crawlers. Unless it's in the standard sitemap.xml address, it's going to be hard for a crawler to find. That's why I recommend verifying it in Google Search Console (Sitemaps tab) and adding a directive to the end of your robots.txt file:
Sitemap - Google Search Console Verification
Verify your sitemap in Google Search Console to control your site's indexing status
Adding links on popular sites
It is also worth making sure that links to the new subpage overseas chinese in australia are placed on popular pages of our website. Modules with recently added articles, recommended or related entries, which we can place, for example, on the main page or under older articles, are helpful in this.
Placing a link on another website also sends a signal to the search engine.
Reporting Multiple Addresses to Google
Finally, a method for advanced users who are familiar with programming and API use. Using one of them - Google Indexing API - we are able to send a request to add or remove up to 200 URLs per day. Although this method is mainly intended for indexing new pages containing JobPosting or BroadcastEvent structured data placed in the VideoObject object, I tested it on several ordinary pages and the effects were satisfactory for me.
If you want to learn more about Google Indexing API and how to configure queries, visit this post on my blog.
Why is my site not in search results? - most common causes
There are several reasons why a page has not yet been added to the Google index. The most common reason is human error - the page is blocked during its creation, and after it is made public, the developer simply forgets to make it available to robots as well.
Blocking in robots.txt file
Check your robots.txt file to see if there is a directive blocking crawling of subpages. It will look something like this:
User-agent: *
Disallow: /
Dislallow: /blog/
If you want to be sure that Google is not blocking your site, use the robots.txt Validator and Testing Tool or the one provided by Google ( Robots Testing Tool ). The latter allows you to check what version of the file Googlebot uses to determine whether a given address can be crawled or not.
Sometimes, despite the blocking in robots.txt, the URL can be indexed, but it will not contain information about the page, and much less will it be displayed for the desired keywords. The information placed in Title Link Google will take from, for example, the anchor text of the links .
title link in google
Title link in Google organic results
Robots meta tag block
Placing <head>a robots tag with the content="noindex", content="none"or attribute in your meta section content="unavailable_after: [miniona data ]"tells robots not to index the page's content.
You can check if a page contains such a meta tag manually (Dev Tools, page source), via browser plugins e.g. META SEO Inspector or popular crawlers (e.g. Screaming Frog).
HTTP Header Blocking
Having the header returned by the server in response to a Googlebot request HTTP: X-Robots-Tag: noindex, X-Robots-Tag: none, X-Robots-Tag: unavailable_after: [miniona data]will have the same effect as including a robots meta tag with the same attributes on the page.
You can check HTTP headers manually (Dev Tools), using crawlers or using online tools (e.g. redirect-checker.org )
The canonical address points to a different URL
If <link>we provide a different canonical address in the tag, HTTP header, or sitemap, Google may not index that address. This is not a binding directive and the page can be indexed without any problems.
To check if the canonical addresses are set correctly, you can use crawlers, plugins (e.g. the previously mentioned META SEO Inspector) or do it manually.
Google chose a different canonical address than the user
Even if you don't set a canonical address for a subpage, Google can do it for you. If the search engine finds that this page is confusingly similar to another, it won't add it to the index.
To check if this is the case for you, go to Google Search Console and download the address information.
