Miha
I didn't know The Wayback Machine blocks access to website snapshots if robots.txt blocks web crawlers. So this old website got shut down, the domain name gets parked by a domain squatter that set the robots.txt to ban all web crawlers. Upon archive request, Wayback Machine checks the domain, reads robots.txt, then effectively blocks you from accessing the snapshots.
8y, 39w
1 reply
¬