|
|
033b817627
|
Add license BSD on each file header
git-svn-id: https://svn.fournier38.fr/svn/ProgSVN/trunk@6096 bf3deb0d-5f1a-0410-827f-c0cc1f45334c
|
2020-09-07 14:13:56 +00:00 |
|
|
|
e1f1ddaa24
|
robotstxt : catch the errors with the associated lines.
robotstxt : more unit tests
git-svn-id: https://svn.fournier38.fr/svn/ProgSVN/trunk@5986 bf3deb0d-5f1a-0410-827f-c0cc1f45334c
|
2020-05-11 18:26:01 +00:00 |
|
|
|
330dfc3993
|
robotstxt : the analyze must be done at start. Use the constructor
git-svn-id: https://svn.fournier38.fr/svn/ProgSVN/trunk@5344 bf3deb0d-5f1a-0410-827f-c0cc1f45334c
|
2019-06-12 07:03:44 +00:00 |
|
|
|
f574476427
|
robotsTxt : update Tests
git-svn-id: https://svn.fournier38.fr/svn/ProgSVN/trunk@5343 bf3deb0d-5f1a-0410-827f-c0cc1f45334c
|
2019-06-11 18:59:44 +00:00 |
|
|
|
ded1628c1f
|
robotstxt : rewrite all the parser with the logic defined in RFC
git-svn-id: https://svn.fournier38.fr/svn/ProgSVN/trunk@5342 bf3deb0d-5f1a-0410-827f-c0cc1f45334c
|
2019-06-11 09:08:53 +00:00 |
|
|
|
da4215dbf2
|
Update robotstxt file
git-svn-id: https://svn.fournier38.fr/svn/ProgSVN/trunk@5340 bf3deb0d-5f1a-0410-827f-c0cc1f45334c
|
2019-06-10 14:38:23 +00:00 |
|
|
|
1f466ffa47
|
robotstxt : Add support to understand the robots.txt files
git-svn-id: https://svn.fournier38.fr/svn/ProgSVN/trunk@5339 bf3deb0d-5f1a-0410-827f-c0cc1f45334c
|
2019-06-10 14:35:49 +00:00 |
|
