Archive for October, 2007

check for robots.txt

Tuesday, October 23rd, 2007

Some times it is useful to check if a given HTTP server has a robots.txt file in it. If it exist it may disclose interesting information, useful for a pentest :)
(more…)

Posted in Networking, Shell Script | 1 Comment »