ملف robots.txt الخاص بمحرك البحث الشهير Google.com

---

إذا كنت تود معرفة ملف robots.txt الخاص بمحرك البحث الشهير Google.com


كود:


:User-agent: *Allow: /searchhistory/Disallow: /news?output=xhtml&Allow: /news?output=xhtmlDisallow: /searchDisallow: /groupsDisallow: /imagesDisallow: /catalogsDisallow: /cataloguesDisallow: /newsDisallow: /nwshpDisallow: /?Disallow: /addurl/image?Disallow: /pagead/Disallow: /relpage/Disallow: /rel*******Disallow: /sorry/Disallow: /imgresDisallow: /keyword/Disallow: /u/Disallow: /univ/Disallow: /cobrandDisallow: /customDisallow: /advanced_group_searchDisallow: /advanced_searchDisallow: /googlesiteDisallow: /preferencesDisallow: /setprefsDisallow: /swrDisallow: /urlDisallow: /defaultDisallow: /m?Disallow: /m/search?Disallow: /wml?Disallow: /wml/search?Disallow: /xhtml?Disallow: /xhtml/search?Disallow: /xml?Disallow: /imode?Disallow: /imode/search?Disallow: /jsky?Disallow: /jsky/search?Disallow: /pda?Disallow: /pda/search?Disallow: /sprint_xhtmlDisallow: /sprint_wmlDisallow: /pqaDisallow: /palmDisallow: /gwt/Disallow: /purchasesDisallow: /hwsDisallow: /bsd?Disallow: /linux?Disallow: /mac?Disallow: /microsoft?Disallow: /unclesam?Disallow: /answers/search?q=Disallow: /local?Disallow: /local_urlDisallow: /froogle?Disallow: /products?Disallow: /froogle_Disallow: /product_Disallow: /products_Disallow: /printDisallow: /booksDisallow: /patents?Disallow: /scholar?Disallow: /completeDisallow: /sponsoredlinksDisallow: /videosearch?Disallow: /videopreview?Disallow: /videoprograminfo?Disallow: /maps?Disallow: /mapstt?Disallow: /mapslt?Disallow: /translate?Disallow: /ie?Disallow: /sms/demo?Disallow: /katrina?Disallow: /blogsearch?Disallow: /blogsearch/Disallow: /blogsearch_feedsDisallow: /advanced_blog_searchDisallow: /reader/Disallow: /uds/Disallow: /chart?Disallow: /transit?Disallow: /mbd?Disallow: /extern_js/Disallow: /calendar/feeds/Disallow: /calendar/ical/Disallow: /cl2/feeds/Disallow: /cl2/ical/Disallow: /coop/directoryDisallow: /coop/manageDisallow: /trends?Disallow: /trends/music?Disallow: /notebook/search?Disallow: /musicDisallow: /browsersyncDisallow: /callDisallow: /archivesearch?Disallow: /archivesearch/urlDisallow: /archivesearch/advanced_searchDisallow: /****/search?Disallow: /****/reportbadofferDisallow: /****/s2Disallow: /urchin_test/Disallow: /movies?Disallow: /codesearch?Disallow: /codesearch/feeds/search?Disallow: /wapsearch?Disallow: /safebrowsingDisallow: /reviews/search?Disallow: /orkut/albumsDisallow: /jsapiDisallow: /views?Disallow: /c/Disallow: /cbkDisallow: /recharge/dashboard/carDisallow: /recharge/dashboard/static/Disallow: /translate_c?Disallow: /translate_s?Disallow: /translate_t?Disallow: /s2Disallow: /transconsole/portal/Disallow: /gcc/Disallow: /aclkDisallow: /cse?Disallow: /tbproxy/


و يمكنك مشاهدته من هذا الرابط :
http://www.google.com/robots.txt

منقول