Commit Graph

2 Commits

Author SHA1 Message Date
da4215dbf2 Update robotstxt file
git-svn-id: https://svn.fournier38.fr/svn/ProgSVN/trunk@5340 bf3deb0d-5f1a-0410-827f-c0cc1f45334c
2019-06-10 14:38:23 +00:00
1f466ffa47 robotstxt : Add support to understand the robots.txt files
git-svn-id: https://svn.fournier38.fr/svn/ProgSVN/trunk@5339 bf3deb0d-5f1a-0410-827f-c0cc1f45334c
2019-06-10 14:35:49 +00:00