Birkaç örnek:
http://sorbize.com/robots.txt
http://sorbize.com/robots.txt
User-agent: *
Disallow:
Sitemap: http://www.sorbize.com/sitemap.xml.gz
http://nonloser.com/robots.txtUser-agent: *
Disallow:
Sitemap: http://www.nonloser.com/sitemap.xml.gz
http://yakuter.com/robots.txtDisallow: /wp-*
http://bidusun.com/robots.txt User-agent: Googlebot
Disallow: /wp-admin
User-agent: *
Disallow:
Sitemap: http://bidusun.com/sitemap.xml.gz
http://aorhan.com/robots.txt User-agent: *
Disallow:
Sitemap: http://www.aorhan.com/sitemap.xml.gz
http://arslania.com/robots.txt User-agent: *
Disallow: /search/
Disallow: /goto/
sorbize ile aynısını yaptım. Teşekkürler Süleyman hocam (: