Add noindex pages, robots.txt and operator footer

- Page.noindex: marked pages get a robots meta tag and a Disallow line in the
  generated robots.txt (single source of truth in src/pages.py)
- templates/robots.txt: public pages indexable, noindex pages and data files not
- Footer shows an optional operator line from contents/snippets/footer_operator.md
- Tests for robots.txt, noindex meta and footer

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
This commit is contained in:
2026-10-10 15:33:37 +02:00
co-authored by Claude Sonnet 5.5
parent 874838f525
commit 81edf08d8e
5 changed files with 89 additions and 6 deletions
+11
View File
@@ -0,0 +1,11 @@
# robots.txt – generated by the site generator, do not edit by hand.
# The course pages may be crawled and indexed. Pages with contact details may not.
User-agent: *
Allow: /
{% for page in noindex_pages -%}
Disallow: /{{ page }}
{% endfor -%}
# Data files (search indexes etc.) are not pages
Disallow: /*.json$
Disallow: /*.zip$