|
|
330dfc3993
|
robotstxt : the analyze must be done at start. Use the constructor
git-svn-id: https://svn.fournier38.fr/svn/ProgSVN/trunk@5344 bf3deb0d-5f1a-0410-827f-c0cc1f45334c
|
2019-06-12 07:03:44 +00:00 |
|
|
|
f574476427
|
robotsTxt : update Tests
git-svn-id: https://svn.fournier38.fr/svn/ProgSVN/trunk@5343 bf3deb0d-5f1a-0410-827f-c0cc1f45334c
|
2019-06-11 18:59:44 +00:00 |
|
|
|
ded1628c1f
|
robotstxt : rewrite all the parser with the logic defined in RFC
git-svn-id: https://svn.fournier38.fr/svn/ProgSVN/trunk@5342 bf3deb0d-5f1a-0410-827f-c0cc1f45334c
|
2019-06-11 09:08:53 +00:00 |
|
|
|
2cfe3f4d17
|
Add RobotsTXT test
git-svn-id: https://svn.fournier38.fr/svn/ProgSVN/trunk@5341 bf3deb0d-5f1a-0410-827f-c0cc1f45334c
|
2019-06-10 16:02:27 +00:00 |
|
