Dominique.Fournier/DomFramework
1
0
Fork 0
Code Issues Pull Requests Projects Releases Activity

Update robotstxt file

Browse Source 
git-svn-id: https://svn.fournier38.fr/svn/ProgSVN/trunk@5340 bf3deb0d-5f1a-0410-827f-c0cc1f45334c
This commit is contained in:
Dominique Fournier
2019-06-10 14:38:23 +00:00
parent 1f466ffa47
commit da4215dbf2
1 changed files with 3 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
robotstxt.php
View File

@@ -1,17 +1,15 @@
<?php <?php
/** DomSearch /** DomFramework
* @package domsearch * @package domframework
* @author Dominique Fournier <dominique@fournier38.fr> * @author Dominique Fournier <dominique@fournier38.fr>
*/ */
namespace models;
/** This class analyze the provided robots.txt file content and allow to /** This class analyze the provided robots.txt file content and allow to
* get the configured data for DomSearch. * get the configured data for DomSearch.
* It allow to examine an URL against the robots.txt file and return if the URL * It allow to examine an URL against the robots.txt file and return if the URL
* is allowed to be used or not * is allowed to be used or not
*/ */
class robots class robotstxt
{ {
// PROPERTIES // PROPERTIES
// {{{ // {{{