Reading the documentation and Googling would result in many methods – some are confusing, some are not so neat and others apply to earlier versions of Drupal. Following is my favourite method to setup different page templates, depending on the Content Type, in Drupal 6.

What is a Page Template
This is the template which contains a Node. This must not be mistaken with the node template, for example, a Story node which displays the title, teaser, images, comment form and et al. In Drupal, the default page template is written in the file named page.tpl.php (and the node template is in node.tpl.php).

It is the page template with which you can display three columns for certain content types and two columns for others. It is the page template using which you can dictate the column which will show a Story, a Blog entry, a Forum topic or a Webform. This template is where you insert the Regions for Blocks and Views.

A Neat Solution
Append or write the following code in a file named template.php in your theme’s directory:

<?php
function <theme-name>_preprocess_page(& $vars) {
	if (isset ($vars['node']) && $vars['node']->type == "blog") {
		$vars['template_files'] = array();
		$vars['template_files'][] = 'page-blog';
	}

	if (isset ($vars['node']) && $vars['node']->type == "forum") {
		$vars['template_files'] = array();
		$vars['template_files'][] = 'page-forum';
	}

	if (isset ($vars['node']) && $vars['node']->type == "webform") {
		$vars['template_files'] = array();
		$vars['template_files'][] = 'page-webform';
	}
}
?>

In the above code replace <theme-name>, in the name of the function, with the name of your Theme’s directory. For example, suppose the name of your Theme’s directory is “blue”, then:

function blue_preprocess_page(& $vars)

Now all you need to do is what you are most creative at. Write the templates for each content type. Drop files in your theme’s directory named page-blog.tpl.php, page-forum.tpl.php and page-webform.tpl.php for the Blog entry, Form topic and Webform  respectively. Feel free to set different page layouts and Regions in these files. Note that any Content Type which is not specified in the above code will use page.tpl.php – the default page template. For example, in this case, the Story content type will use page.tpl.php.

Important: Log into Drupal as admin and goto Administrator>Site configuration>Performance and click on the button “Clear cached data” (at the bottom).

I prefer this solution because the template selection logic goes separated into a different file, leaving the Page templates friendly for a designer. Plus, template.php is the logical place to add overrides or specific behaviour. Most likely a seasoned Drupal developer will first look into this file for customizations.

Created or added a new content type? All you need is to add another IF block in the above code and drop a new page template. Before concluding that it is not working for you, do remember to clear the cache.

To enable E-mail notifications, login into Google Wave. Move your mouse over Inbox (on the left). Click on the Arrow (pointing downwards) and select Notifications.

Google Wave: E-mail Notification Feature

You can opt to get notified about new and updated Waves immediately (as it happens), hourly or weekly. The notifications are sent to the E-mail address associated with your Google Wave. This will be the GMail or Google Apps (in my case) E-mail address at which you got the Google Wave invite.

I tested it with “immediately” option and found that Google Wave sends an E-mail when

• A new Wave is created and you are added as a participant
• A new Blip (message) is created in a Wave in which you are a participant
• When an existing Blip is updated in the Wave in which you are participating

Google Wave will not send a notification when:

• It has already sent you notification for a new or updated Wave. It will not send any further notification for any update in that particular Wave, till you log into Google Wave and click on the new or updated Wave.

A new Wave E-mail Notification sent by Google Wave

Issues/Bugs:

• The notifications are sent even when you are using Google Wave – logged into it and seeing the Waves. Why a notification when I can see a new Wave(s) being created and updated in front of me ? So what if I choose to click on them after sometime. Ideally the notifications should be sent after I have closed the Google Wave window on my Web Browser. I said “closed” and not logged out because logging out of Google Wave may mean to log out from other Google apps too.
• The “Immediately” notification is not so immediate. Few notifications were delayed by as long as 14 minutes! Though at times I have received the notification within a minute. This does not fit well with a real time app like Google Wave. Immediate notifications must be “immediate”, as it happens and within seconds.
• The notification settings link in the E-mail (notification) just opens Google Wave and does not show the Window or popup to change the settings. Note that this bug is a known issue as per the Google Wave blog.

Good to have in the next version:

1. There should be a provision to send notifications to any E-mail address and not only to the associated Gmail or Google Apps E-mail address.
2. The notification settings, besides being system wide and applicable to all Waves, should be present for individual Waves too. The notification settings for individual Waves should override the system wide settings. This way one can subscribe to notifications only for the Wave in which he/she is interested.
3. If point 2 is implemented, there should be an exclusive option to get notified about New Waves created by others – where you are added as a participant.

I was wanting this feature so much that last year I wrote a Robot for my own use (with a hard coded E-mail address) to send me notifications for updates in Waves. I add this Robot to the Waves of my interest. I even wrote an article in PCQuest magazine about this Robot. For those who are interested in the code, refer to this link at the PCQuest forums to download the article with the code intact. If Google Wave could bring the Notification settings to individual Waves, I would no longer need to use my Robot.

I own an Android Dev Phone 1 (a.k.a. ADP1). It was just today that Microsoft Tag Reader app for Android was announced. The Tag Reader app allows to scan, extract and store information in Microsoft Tags using the phone’s camera.

I wasted no time in firing up Android Market on my ADP1 and searched for “microsoft”. And voila Microsoft Tag Reader showed up on the top, in the list of apps. Installed it. Alternatively, one can browse to the URL  http://gettag.mobi using the Android’s web browser. This will automatically launch the Android Markert app, prompting to install Microsoft Tag Reader.

To see the app in action, you will need to have a Microsoft Tag handy. One of the quickest way is to create your own at http://www.microsoft.com/tag/. The different types of information that one can store in a Tag are URL, Free Text, vCard and Dialer. The Microsoft Tag Reader is well integrated with Android and invokes the appropriate actions for each of these types like - launching the web browser with the URL, displaying the text, storing the vCard into Android’s Contacts and launching the Dialer with the number.

Pros

• Instant and fast scanning
• Integrated with Android apps like Web browser, Contacts and Dialer

Whilst there are some cons too

Cons

• VCard: Does not store IM Address in Contact’s Chat addresses.
• VCard: This seems to be a bug. In case of commas (,) in the address or any other fields, the Tag Reader prefixes a backslash (\) to each comma. For example: an address like “124-A, New Delhi” is stored as “124-A\, New Delhi” in the Android’s Contacts 
• The app displays the text for Free Text but does not allow to copy (for copy-n-paste) – a good to have feature.

Cons aside (its likely that these will be fixed in the next version), the noteworthy point is Micrsoft Tag Reader is the first ever Android app from Microsoft!

I supply my Google Apps credentials each time I wipe off (hard reset) the phone. Though optional, I wipe off the phone’s data whenever I update my phone with a new ROM. Following the official and non official upgrades, I’m currently running CyanogenMod 4.2.14.1 on my ADP1.

Then came a day when I did not want to wipe off the data but only wanted to change the Google Account associated with the phone.This was the day when I wanted to play with Google Buzz which requires a GMail or non Google Apps account. I googled around and found solutions suggesting:

• Clear data for Gmail and Gmail storage under Manage Applications
• Clear data for Google Apps and you will see a message in the Android’s notification bar to setup a new Google account.

None of these worked for me. What worked for me is as follows:

• Goto Home (by clicking on the Home hardware button).
• Click on Menu (hardware button)
• Click/touch on Settings>Applications>Manage Applications.
• Click on Google Apps.
• Click on the button labelled “Clear data”
• When prompted with “All information you’ve saved in this application will be deleted permanently”, click on OK
• Subsequently, click on back button thrice and then click on “Data synchronization”. Alternatively, click on Home>Menu>Settings>Data synchronizaion.
• This will start the wizard which will prompt for a new Google account. This wizard is the same that you would have seen when you first (after you bought it) powered on the phone or whenever you wipe off the phone’s data.

This is what helped me in testing out Google Buzz on Google Maps on my Android phone. In your case, this may simply help you in changing the Google Account without hard resetting your Android.

Update (February 14, 2009): After following the above procedure, I noticed that though my E-mails were syncing, I could not search my E-mails. Whenever I typed in a search keyword in the GMail app and clicked on the search button, I saw a message saying something as “E-mails will sync shortly”. This can be fixed as follows. Goto Settings>Applications>Manage Applications>Gmail. Click on “Clear data” button.

I guessed that it would require a Google Maps upgrade on the phone. Instantly fired Android Market on my Android Dev Phone 1 (a.k.a. ADP1), clicked on the Downloads tab and was greeted by an available upgrade to Google Maps – Google Maps version 4.0.0. Opted to install the upgrade hoping to spot the Buzz on the maps but alas – it wasn’t a smooth sail. After the upgrade, I fired the Google Maps app on my ADP1. clicked on Menu>Layers>Buzz and spotted a new icon  on the maps. Clicking on the icon showed me a screen where I could post a new Buzz (a message).

Post a Google Buzz using Maps on Android

Along with typing a text message, this screen also allowed me to upload a new photo that I could click via the phone’s camera or select an existing picture from the Android phone’s Gallery. Once I posted a Buzz, I could see a new speech bubble on the map at my current location – determined via GPS or Cell location (without GPS).  Over time, I realized, I could see only my Buzzes. Was no one else buzzing?

Buzz is not for Google Apps
Later during the day, I realized that my friends, using their GMail account, could see other’s Buzzes on their Nokia phones. Also they could not see my Buzzes.

I instantly suspected that it could be all because of using a Google Apps (not GMail) credentials on my Android phone. My Android phone is tied up with a Google Apps Standard Edition account which is not a regular GMail/Google account. That’s when I got onto the Net and  this blog post confirmed my suspicion. But the confusion remained for a while. How could I post a Buzz (as said above) but not see other’s Buzzes?

So I spent the remaining day (of 10th February) and 11th February ensuring that Google Buzz indeed does not work with an Android using Google Apps account. I changed my ADP1’s Google account from Apps to a regular GMail account as explained in this blog post – Android: How to change the Google Account without a hard reset. Subsequently,  I changed it back from GMail to Apps. Kept flipping the accounts several times. Witnessed a few erratic and confusing behaviour – call it caching. Eventually, I concluded the following:

• Google Buzz on maps in Android works with a regular GMail account and not with a Google Apps account.
• If you are switching from a Google Apps account to a GMail account on Android after installing Google Maps 4.0, refer to the following:
  1. Launch Android Market
  2. Click on the Downloads tab
  3. Select Google Maps
  4. Click on the button “Uninstall updates”. This will downgrade Google Maps.
  5. Subsequently, reinstall/update to Google Maps 4.0.0
  6. Now, launch Google Maps and click on Menu>Layers>Buzz
  7. And voila, you should be able to see what others are Buzzing on the map.
     

     Google Buzz on Maps in Android on February 10

• Buzz should work, if you have switched from a Google Apps account to a GMail account before installing Google Maps 4.0.0

Clicking on a bubble shows the Buzz with Photo (if any)

A few other observations and notes:

• Buzz on Google Maps for Mobile has no privacy settings. All your Buzzes along with photos will be visible to everyone.
• As per my friends, Google Buzz seem to work equally well on Google Maps in Nokia and Windows Mobile. Nokia and Windows mobile users can goto http://m.google.com/maps to install/upgrade Google Maps to version 4.0.0.
• As reported by friends using Nokia, the Buzzes may disappear at times. In this case, removing and adding the Buzz layer on Google Maps will show the Buzzes again.
• Seems that the “speech bubbles” for Buzzes, older than a day, disappear from the map.
  

  A list of all the Buzz at a location, along with comments

While I have been playing with Buzz on my Android for the last two days, I may soon switch back to my Google Apps account. I can’t sacrifice my official E-mails on Android for the Buzz. Hope, Google Buzz for Google Apps is released soon. In the meanwhile I will play with Google Buzz on my Gmail account

A blog post using Google Wave for content and comments

This became possible, recently, with the introduction of the read-only access feature. I created a blog post with a Wave talking about the Frequently Asked Questions (FAQs) about Google Wave and a blank (to start with) Wave for comments. Click here to see this blog post. This was achieved as follows:

• Logged into Google Wave and created a new Wave about Frequently Asked Questions about Google Wave, as showed in the screenshot.
  

  A Wave about frequently asked questions about Google Wave

• Added the Public participant to this Wave. This can be done as follows:
  • Click on the + button on the bottom right pane titled Contacts to add a new contact
  • For Address type in public@a.gwave.com
  • Add this new contact by clicking on the Submit button.
  • Next add this contact to the Wave, which you want to make public. For this click on the + button on the top of the Wave.
  • On the Add Participants pop-up, search/browse, and click to add the contact named Public.
  • You will see a notification on the top of the Wave saying “You gave everyone access”.
• I gave Read-Only access to the Public contact/participant. Refer to this blog post on how to assign read-only access.
• Next I created another Wave, for comments, with just one blip or message which says: Wave your comments about the FAQs on Google Wave.
• Added the Public contact to this Wave but did not setup read-only access this time. That means, anyone (with a Google Wave account) can add/edit/delete blips/messages on this Wave. That’s what commenting is all about, right?
• To embed Google Waves in this blog post, I installed and activated the Wavr plugin for WordPress. You can download this plugin from here
• With Wavr plugin you can embed any Google Wave in a blog post using the following syntax:[ wave id = "<Google-Wave-ID> ]
  Please note that, unlike shown above, there should be no space between the opening/left square bracket and the text “wave id”.
• To find out the Wave ID of a Google Wave, Click on the Wave. Copy the URL (shown in your web browser) from googlewave.com upto the . (dot) or upto the end. Exclude the dot. Replace %252B with+ (plus). For example, in my case, the Wave IDs for the content and comments Waves are googlewave.com!w+3413MFFlD and googlewave.com!w+53bOqvqzB
• Next I created a new blog post in WordPress and embedded the Waves as follows:
  [ wave id = "googlewave.com!w+3413MFFlD" ]
  [ wave id = "googlewave.com!w+53bOqvqzB" ]
• You can change the default colors of the embedded Waves from the WordPress admin panel – Settings>Wavr. I changed the default background color and default color to #FFFFFF (white) and #000000 (black) respectively. I also set the height to 700px. Note that , you can set these options on a per Wave basis too. You can find more about it, at the Wavr’s download page.
• Last, since I wanted to encourage comments via the Google Wave only, I disabled comments to the blog post. This can be done while writing or editing the post. Click on the Discussions panel (at the bottom of the editor) and untick “Allow Comments”.

The output of the above efforts can be seen here. Be ready to catch me, in real time, when and while I update the FAQ wave. Though it is more likely that I may catch you while you are writing a comment and could reply instantaneously. After all I pledged to keep Google Wave always open in my web browser.

Refer to this post on how to use Google Wave for blog posts and comments on your own WordPress powered blog.

I fired Android Market on my phone, searched for “wordpress” and the official app was right at the top, proudly named as WordPress, just WordPress. Yea, that’s indeed the official app!After installation, I launched the app. It prompted me for Blog URL, for which I supplied http://www.shekhargovindarajan.com. For the username and password, I typed in the admin’s username and password, since I blog using the admin account. Subsequently, it allows me to manage comments, the blog posts and the pages on my blog. After playing with it for a couple of hours, I noticed the following pros and cons in the WordPress app for Android:

Pros:

• New comment(s) notification on the phone’s notification bar. You can set the app to check for comments every 5 mins to every 24 hours.
• Ability to reply to comments
• Comments moderation – approve, unapprove, mark spam
• Nice tabbed interface with readable font. Big, touch friendly buttons
• Ability to add images to the blog post from the phone’s gallery
• Ability to compose a blog post on the phone, set it’s status to publish but publish it later on the blog site via “Upload Draft to Blog”.  Useful feature, when at times, we don’t have an active data connection on the phone.
• Categories, once downloaded from the blog, are cached.
• Ability to manage more than one WordPress based blogs.

Cons

• No audio alert with the new comment alert
• Limited editor options - bold, underline, italics, link and blockquote
• The editor is not a rich text editor – the HTML tags show in the post’s text, which looks messy
• Limited options to place the images – either at the top or at the bottom of a post
• No option to delete a post once it is published, if by mistake
• No option to insert a “read more” tag

I found the WordPress app limiting when it comes to composing a blog post. This is because I like to use HTML lists (<ul>, <ol> and <li>) and I like to place images in between the post’s text. I may use the comment features – notification, moderation and reply. Someday, with upgrades and more features, the WordPress app may make it to my list of must-have Android apps.

You cannot browse the Web using Kindle (Global Wireless) in India. Image source: Wikipedia.org

Subsequently, I placed an order on behalf of my friend and the Kindle was shipped to New Delhi, India to my residence in about 2 days. I got to setup, configure and play with this nifty device, before I could hand it over to my friend.

I thought it would be a good idea to share my experience of the ordering, shipping, and how well Kindle behaves in India. Instead of writing the entire experience chronologically, I jot it down as FAQs (Frequently Asked Questions).

1. Where did you order the Kindle from?
   Amazon.com
2. Which version of Kindle did you order? Kindle 1, Kindle 2, Kindle DX or Kindle Global Wireless … ?
   The officially available and latest models are called Kindle (Global Wireless) and Kindle DX (Global Wireless). I bought Kindle (Global Wireless).
3. What does Global Wireless mean?
   With Global Wireless versions, you can browse/search through the Amazon Book Store and Wikipedia through a free Internet connection. This connection (which uses the mobile phone network) is pre-paid by Amazon and you do not have to pay any bills for the aforementioned browsing/searching.
4. Why did you buy Kindle and not Kindle DX ?
   Kindle DX is almost double the price of Kindle
   
5. What is the total cost of Kindle?
   $378.57 (USD)
6. Why do I have to pay so much ?
   This cost includes the shipping and the duty. Following is the bifurcation :
   Cost of the device: $259.00
   Import Duty: $98.59
   Shipping and Handling: $20.98
7. Can I reduce the cost by opting for an alternate shipping option, even if it takes longer to ship ?
   No, alternate shipping option is offered for shipping to India.
8. Is there any hidden cost ?
   No.
9. What is the mode of payment ?
   Credit/Debit Card
10. Can I pay through Paypal ?
    No.
11. How much time does it take to ship the Kindle to India ?
    2 days
    Note: I ordered the Kindle on 18th January at 21:00 IST  and got it on 21st January, around 11:00 IST. It was a Monday evening when I ordered and got the Kindle on Thursday morning. Note that the device was shipping during business days. If holidays or weekends fall in between, the shipping may be delayed.
12. Will  I be notified about the order and the shipping status ?
    You will get notified by two  E-mails sent from auto-confirm at amazon dot com and Amazon.com (ship-confirm at amazon dot com). The first E-mail is sent after a successful order with the order details. The second E-mail is sent as soon as the Kindle is dispatched for shipping and includes the tracking number to track the device in transit.
13. How much time does it take to dispatch the Kindle for shipping after a successful order ?
    Less than 24 hours.
14. How can I track my shipping ?
    You can track the device by logging into your Amazon account – which you used to order the Kindle.
15. What are the contents of the box that gets delivered
    The Kindle device, a getting started manual, a USB cable and a power adapter – U.S. or United States style, two pin plug.
16. How do I charge the Kindle?
    The USB cable can be plugged into the power socket for charging through AC power. For this you will need a U.S. to India power plug converter. Else you can charge the Kindle using your laptop/PC, using the USB cable.
17. The wireless connection works without any configuration?
    Yes. But for the very first time, when I tried searching through the Amazon Store, the Kindle complained about “no connection” and suggested to reboot. After a reboot the connection was up. Ever since. the connection has always been on and working.
18. Can I browse the Web using Kindle?
    No. Kindle does have a built-in Web browser which is suppose to work well for text-only or low-graphics Websites. But when I try to browse to a Website, the browser shows an error saying “Due to local restrictions, web browsing is not available for all countries”.   
19. Can I Google using Kindle?
    No. The search fails with the message saying “Due to local restrictions, web browsing is not available for all countries”. Only Wikipedia and Amazon Store search works, over the wireless network (refer to FAQ 3)
20. Does Kindle support PDFs?
    This question is not specific to India but to the device. Yet the answer is yes, Kindle supports PDF. But you cannot zoom or enlarge the text/font size of a PDF document. This makes some PDFs unreadable on Kindle.
21. Can I transfer my own documents or books to the Kindle?
    This question is not specific to India but to the device. Yes, you can transfer any Kindle supported documents on the device. I have myself transferred and read .pdf, .txt, and .mobi formats on the Kindle.
22. Can I transfer and read pirated E-Books, downloaded via Torrents, on the Kindle?
    Sorry, this blog is not a place to ask this question.
23. I have heard that not all books are available for Indian users
    I have heard that too. Will keep this post updated when I comes across such titles.
24. Instead of Kindle, why you did not recommend an Apple iPad to your friend?
    The launch of iPad was uncertain. As of this writing, its launch and pricing in India is not known. Also iPad is a multi-purpose and bulkier device.

Hope this FAQ helps my countrymen in going ahead and bringing home an Amazing Kindle!

Disclaimer: The above FAQ is strictly based on my personal buying and usage experience. Any or all information may be different in your case.

Amongst using regular expressions and DOM (Document Object Model) parsing, I used XPath too. XPath works great for well-formed Web pages. A HTML Web page is called well-formed when all the opening tags have corresponding closing tags and the tags are nested properly (refer to this link for more). A well-formed HTML page is also called an XHTML page.

XPath is a query language to access content on a well formed page – XHTML or XML. All the content in a Web page lie within HTML elements or tag pairs. The following is needed to extract out the content, of interest, using XPath from a Web page:

1. A well-formed Web page
2. An XPath expression for the HTML element of interest
3. XPath query using the expression

An XPath expression looks as cryptic as  .//*[@id='home_featured']/div. Here is where Firefox web browser with a plugin called FireXPath comes to help (as explained below). The second and third requirements are met by PHP (in my case) which is used for automated Web scrapping using the XPath expression.

Click and build XPath expressions
Building the cryptic XPath expression is easy and intuitive with Firefox and a couple of its plugins – namely Firebug and FireXPath. Install Firebug from here, followed by FireXPath from here and restart Firefox. As of this writing, I’m using the following versions of Firefox and the plugins:

• Firefox 3.5.6
• Firbug 1.4.5
• FireXPath 0.9.1

After, the Firefox restart, browse to the Web page of your interest. Next:

• Launch Firebug by clicking on the bug like icon on the right corner of the status bar.
• Click on the tab named XPath (in FireBug)
• Click on the arrow (blue arrow) and move your mouse over the content, of interest, on the web page
• Once the required chunk is highlighted (with a blue border), note down the XPath expression in the textfield.

For example, the screenshot below, shows the XPath expression (.//*[@id='home_featured']/div  ) for the “Featured Post” chunk, on the home page, of this blog:

With FireXPath, point your mouse on the chunk/element of interest, to build an XPath expression

A few examples of XPath expressions:

• .//*[@id='latest_post']/span[1] : The posted and modified dates of the latest post on this blog – http://www.shekhargovindarajan.com
• .//*[@id='top-stories']/div[1]/h2/a : The headline of the top story on Google News – http://news.google.com
• .//*[@id='mp-tfa']/p[1] : Contents of featured article on Wikipedia – http://en.wikipedia.org/wiki/Main_Page

Automated scarping using PHP
With the XPath expression in hand, a PHP script as follows, can extract out the required content from the Web page:

<?php

class WebScrap
	{
	private $url;
	private $xpath;

	public function WebScrap($url,$xpath)
		{
		$this->url = $url;
		$this->xpath = $xpath;
		}

	public function GetScrap()
		{
		// use Tidy to try to make the page well formed
		$page = $this->TidyIt($this->url);

		// create a document out of the well formed content
		$domDocument=new DOMDocument();
		$domDocument->loadHTML($page);

		// create an XPath object out of the document and query it for the supplied xpath
		$domXPath = new DOMXPath($domDocument);
		$domNodeList = $domXPath->query($this->xpath);

		// Get the content (HTML) out of the NodeList returned by the DOMXPath::query
		$content = $this->GetHTMLFromNodeList($domNodeList);

		return $content;
		}

	private function TidyIt($url)
		{
		$tidy = new tidy();
		$tidy->parseFile($url);
		$tidy->cleanRepair();
		return $tidy;
		}

	private function GetHTMLFromNodeList($domNodeList)
		{
		$domDocument = new DOMDocument();

		$node = $domNodeList->item(0);   

		foreach($node->childNodes as $childNode)
			$domDocument->appendChild($domDocument->importNode($childNode, true));

		return $domDocument->saveHTML();
		}

	}

?>

Note that, the function/method called GetScrap( ) of the WebScrap PHP class first calls the TidyIt( ) method. This function uses the Tidy library to fix (if required) the HTML for well-formedness. The PHP class for Tidy is used to fetch the web page via the given URL and repair it. Henceforth the DomXPath object is used to query the well-formed Web page content for the XPath expression.

To use the above code, you will need to install PHP modules for XML and Tidy. On a RedHat/CentOS/Fedora Linux machine, these modules can be installed using the following command:

yum install php-xml php-tidy

Save the above code in a file named class.WebScrap.php (say). Subsequently, the WebScrap class can be used as:

<?php

include("class.WebScrap.php");
$scrap = new WebScrap("http://news.google.com",".//*[@id='top-stories']/div[1]/h2/a");
print($scrap->GetScrap());

?>

The code should be self explanatory for a seasoned PHP programmer. If not, shoot your questions via comments to this post. For repeated and automated Web scarping, a scheduler like Cron can be used to execute the above PHP script at regular intervals and fetch the latest content.

The Leftovers
The code above is readable, crisp and focusses on the subject. For this reason, it has deliberate exclusions.  In a real world application, you should:

• Use Curl library in PHP  or external tools like Wget to fetch the URL. Then pass on the fetched content to Tidy. Hint: use the  parseString( ) of Tidy instead of parseFile( )
• Handle errors in case of errors in Tidy and XPath.
• Fallback to other means (say regular expressions) in case of errors in Tidy or XPath querying

Note that XPath may not always give you the structured content that you desire. For example, using the expression .//*[@id='latest_post']/span[1] (for posted and modified dates of the latest post on this blog) will result into something as follows:

<strong>Posted on:</strong>
January 8, 2010
<span class="dot">⋅</span>
<strong>Last modified:</strong>
January 8, 2010 @ 4:49 pm

So there are tags and text (like “Posted on:” and “Last modified:”) to be stripped out, to get the posted date (January 8, 2010) and the modified date (January 8, 2010 @ 4:49 pm). For this, you may still have to use regular expressions and/or string manipulation functions like split( ).