# Crawling

> The process by which search engine bots discover and read web pages.

Source: https://anotherseoguru.com/en/glossary/crawling

Crawling is the process where search engine bots (like Googlebot) systematically browse the web to discover new and updated content. Bots follow links, read content, and send data back to be indexed.

## Example

Googlebot discovers your new blog post by following an internal link from your homepage, then reads and stores the content.

## How to apply it

Ensure a clear site structure, submit an XML sitemap, use internal linking, avoid crawl traps (infinite calendars, faceted navigation), and monitor crawl stats in GSC.

> **Pro tip:** Check Server Logs to see exactly what Googlebot is crawling. GSC's crawl stats only show aggregates.

## Related terms

- [Indexing](https://anotherseoguru.com/en/glossary/indexing)
- [Robots.txt](https://anotherseoguru.com/en/glossary/robots-txt)
- [XML Sitemap](https://anotherseoguru.com/en/glossary/xml-sitemap)
- [Crawl Budget](https://anotherseoguru.com/en/glossary/crawl-budget)