Robots.txt
Overview
Section titled “Overview”GitHub will manage access to search engines and bots through the use of a Robots.txt file.
This file should be placed into the public folder of your local machine Repo, then eventually Git pushed into your online GitHub repository in the usual way.
For this site, traditional search engines are allowed (GoogleBot, AppleBot etc) , AI and LLM bots are not.