Sitenizin anasayfasını Google gibi getir derseniz bu süre kısalabilir.
User-agent: *
Allow: /
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content
Disallow: /comments/
Disallow: /show-error-*
Disallow: /*?
Disallow: /xmlrpc.php
Disallow: /trackback/
Disallow: /comment-page-
User-agent: Mediapartners-Google
Allow: /
User-agent: Adsbot-Google
Allow: /
User-agent: Googlebot-Image
Allow: /
User-agent: Googlebot-Mobile
Allow: /
# Site haritalari
Sitemap: http://www.siteadi.com/sitemap.xml