Added robots.txt

raw_icon
John Crepezzi 14 years ago
parent 5b66695ca6
commit 2299819781
  1. 1
      TODO
  2. 2
      static/robots.txt

@ -6,7 +6,6 @@ add link to about page
allow power users to force highlight lang - by supplying an extension!!!!!! allow power users to force highlight lang - by supplying an extension!!!!!!
maybe start serving highlighting on the other end to save transfer on highlight.js maybe start serving highlighting on the other end to save transfer on highlight.js
start using CDNs for most assets start using CDNs for most assets
robots.txt nofollow
kick expiration back by increment on each view kick expiration back by increment on each view
# shared version only # shared version only

@ -0,0 +1,2 @@
User-agent: *
Disallow: /
Loading…
Cancel
Save