If you’re not making use of Google search console data you should be. Of course you only get backlink data for your site but there’s tons of tons of data about what happens up to and including the click on Google search results. There’s some great resources about how to access that data at scale via the API on the web.
Unfortunately there’s no API for crawl or indexation data but still the interface does a good job of helping you understand some of that.