# All robots will spider the domain User-agent: Mediapartners-Google* Disallow: User-agent: ia_archiver Disallow: