The country's new robots.txt file

this is a fascinating if somewhat geek centric insight into one of the first differences of the Obama whitehouse. As the page reads “the robots.txt file tells search engines what to include and not include in their indexes” In other words it tells google and other what they can and can’t index or cache from the page, clearly there’s a different approach here.