User-agent: * 
Disallow: /core/
Disallow: /data/
Disallow: /inc/
Disallow: /js/
Disallow: /thm/