just update user-agent: * Disallow: /images Disallow: /docs Disallow: /js Disallow: /includes Disallow: /modcentre Disallow: /old_site_archive Disallow: /WS_Admin