Web Scraping with Firefox and PHP, using XPath

Last year, I faced two projects which required automated Web scrapping – to aggregate content from web pages. I evaluated different methods for Web scraping with varied level of success. Thanks to the changing structure of Web pages, non well-formed pages and URL redirects.

Amongst using regular expressions and DOM (Document Object Model) parsing, I used XPath too. XPath works great for well-formed Web pages. Read the rest of this entry »

Simple script to alert about inaccessible servers

Following is a simple script, written in PHP, which will send an E-mail alert to you when one or more of your Web servers is inaccessible.

<?php
/* things to edit/change */

$fileToFetch = "isalive.txt";
$mailTo = "shekhar@it4enterprise.com";
$mailFrom  = "alert@it4enterprise.com";

$websites = array(
"Website 1" => "http://www.website1.com",
"Website 2" => "http://www.website2.com",
"Website 3" => "http://www.website3.com"
);

/* end of things to edit/change */

foreach($websites as $name=>$value)
  if(!($fp = @fopen($value."/".$fileToFetch,"r")))
    mail($mailTo,"WEBSITE DOWN: ".$name,"WEBSITE DOWN: ".$name,"From: ".$mailFrom);

?>

Read the rest of this entry »