Added robots.txt

14 years ago · 2299819781
parent 5b66695ca6
commit 2299819781
2 changed files with 2 additions and 1 deletions
--- a/1
+++ b/1
@ -6,7 +6,6 @@ add link to about page
 allow power users to force highlight lang - by supplying an extension!!!!!!
 maybe start serving highlighting on the other end to save transfer on highlight.js
 start using CDNs for most assets
-robots.txt nofollow
 kick expiration back by increment on each view

 # shared version only
--- a/static/robots.txt
+++ b/static/robots.txt
@ -0,0 +1,2 @@
+User-agent: *
+Disallow: /