User-agent: * Disallow: /wp-admin/ Disallow: /academic-interview/ Disallow: /thank-you/ Disallow: /thank-you-application/ Disallow: /readme.html # Blocking Acunetix User-agent: Acunetix Web Vulnerability Scanner Disallow: / User-agent: Acunetix Security Scanner Disallow: / #Crawl delay on swiftype User-agent: Swiftbot Crawl-delay: 5