F
or many webmasters and site owners, the issue of indexing their sites is very scrupulous. And there is something to worry about, because most of the Internet traffic falls on search engines, then it’s logical that the faster the pages of the site go to the index of search robots (and do not fall out of it!), The greater the probability of transitions.
When there are problems with indexing, the first to fall under the hot hand of hosting providers. Is it really? Does the indexing of the site depend on hosting? And what factors affect the speed
What determines the rate of indexation?
This factor is influenced by a large number of factors, among which I would like to highlight: the choice of the domain zone, the age of the domain, the choice of cmc, the frequency of content filling. Further about each of them is more detailed.
Does hosting affect indexing speed?
The direct speed of indexing from the provider does not depend, so I did not assign hosting to the main factors that affect indexing. But if you drip deeper, hosting determines the availability of the site for users and search engines. The provider must ensure maximum uptime of servers and internal network. If you visit a robot resource – it will be unavailable several times in a row, it is possible to drop first several pages from the search index, and then the entire site. Also affects the speed of loading pages, if the site is loaded for a very long time, the robot will also assume that the site is unavailable. In the end, we can say that not hosting itself, but its quality, reliability and speed still affect the indexing of the site.
How does the domain affect the indexing of the site?
It would seem that it depends on the domain, but there are two nuances. Firstly, it is the choice of the domain zone, it is best to choose second-level domains ie com, net, etc. There is an opinion that is slowly indexed .info.
Secondly, this is the age of the domain. The newly registered ones do not enter the index immediately, they will have to wait several weeks. If you buy a domain in the secondary market where the site was previously hosted and the domain was indexed by search engines, then the robot will go to a new site much earlier. I would like to clarify that the factor of choosing a domain influences, mainly, the primary indexation. How often your site will be visited by a search robot is affected by the following factors:
The impact of CMS on site indexing
All popular CMS are already optimized for web standards: pure HTML code, easy management of CSS files (ideally one), and using Schema (schema.org). The validity of the code is validator.w3.org.
Therefore, problems with indexing should not arise. But to speed it up, use the additional advantages that modern site management systems provide: creating Sitemap.xml files, Robots.txt, canonical tags that solve the problem of duplicate content at different addresses, 301 redirects, code optimization and page loading speed.
It is also advised to avoid frames, Javascript, Flash, to display content.
The underlying factor: the frequency of content updates
How often do you update the site? Once a day, a week, a year? Search engines go to the popular news resources several times a day, so that the news immediately fell into the index. If they analyze that the content on the site is updated rarely, then they will rarely look at such a site.
If you blog, then regularly update it with new content, if you have a static site, then create at least a news feed. There is nothing to write? Congratulations at least to your visitors on holidays – show the search engines, and users that the site is live, that its content is periodically updated! You have a news resource – well then “Google” itself ordered to write daily (not to the detriment, the uniqueness and usefulness of information is over).
What can you do to make the site index faster?
Search engines have special services that allow webmasters to “tell” about new sites, here are the most popular ones:
- Include your URL in Google
- Submit your site to Bing
Social networks will help to speed up the indexing of new pages
Do not overlook such a powerful tool as social buttons, telling social network users about a new record, you will speed up its entry into the index. Search robots scan social networks at least as often as news resources, as the content there is updated every second, and therefore a new link to your site will also fall into the index.
What should I do if the site is not indexed?
There may be several reasons for this. Recheck the robots.txt file, perhaps indexing the site is forbidden for all search engines. The entry looks like this:
User-agent: *
Disallow: /
To allow all robots to index the entire site, change the entry to
User-agent: *
Disallow:
Maybe something is wrong in the code of the site itself or the pages that are not visible to the search engines. For example, the text is enclosed in <noindex> </ noindex> tags. And one of the most unpleasant situations can be a violation of one of the conditions of search engines. Then your site can be filtered and the site is excluded from the index.
Image credit: Geralt