TokenPost·
OpenAI Will Honor Shared Robots.txt Signal for AI Training

The file communicates crawler preferences, but its rules do not block access. OpenAI separates its training and search crawlers, and changes do not establish that older material was removed.