The TYPO3 Crawler is an extension which provides possibilities, from both the TYPO3 backend and from CLI that helps you with your cache and e.g. search index.
Having a warm cache is essential for a performant website, this is a task that the Crawler takes over for you. The Crawler takes care of filling the TYPO3 Cache, on page changes. If using the ext:seo it will even crawl pages with respect to SEO priorities. Using indexed_search? no problem the Crawler will fill your search index for you. Cache Warmup and full search index