User-agent: * Allow: / # Disallowed Sub-Directories Disallow: /assets/ Disallow: /images/ Disallow: /public/ Disallow: /themes/ Disallow: /.well-known/ Disallow: /data/ Disallow: /externals/ Disallow: /media/ Disallow: /public/