-
High server load after Sitemap upload
Posted by on December 3, 2020 at 8:14 pmWe have a website with over 3 million URLs. We recently uploaded a complete sitemap for the 3 million pages (1 index linking to over 60 sitemaps with 50.000 URLs each). Since we uploaded the sitemap to GSC we´ve been dealing with a sudden increase in server load. We tried everything and removing the sitemap was the only thing that made the issue stop.
Is our theory correct that Googlebot’s are triggering high server load due to the sitemap?
How can we prevent this from happening?Thanks for your help,
-
3 Replies
-
That’s a better question for the r/bigseo group. Not people here have 3 million urls
-
Check the server logs but most likely the Bots are crawling due to new document and they do sometimes spike CPU usage.
-
That could be it. You can reduce Googlebot’s crawl rate in search console to prevent them from overwhelming your site. You can also test your theory out by removing your sitemap and monitoring any impact that has on your server load. Server logs would also be a good place to look.
Log in to reply.