However, for me, installing the Google Plugin didn’t go as smooth as expected. I have been using Fedora Linux on my laptop since the last 5+ years. Fedora ships with the Eclipse IDE. When I tried installing the Google Plugin in Fedora Eclipse, I faced the following error. This error surfaced with Eclipse 3.5 on Fedora 12 and Fedora 13.

Cannot complete the install because one or more required items could not be found.

  Software being installed: Google Plugin for Eclipse 3.5 1.3.2.v201003242055
                                        (com.google.gdt.eclipse.suite.e35.feature.feature.group 1.3.2.v201003242055)

  Missing requirement: Google App Engine Plugin 1.3.2.v201003242055
                                   (com.google.appengine.eclipse.core 1.3.2.v201003242055)
                                   requires 'bundle org.eclipse.wst.xml.core 0.0.0' but it could not be found

  Cannot satisfy dependency:
              From: Google Plugin for Eclipse 3.5 1.3.2.v201003242055
              (com.google.gdt.eclipse.suite.e35.feature.feature.group 1.3.2.v201003242055)
              To: com.google.appengine.eclipse.core [1.3.2.v201003242055]

Googling around gives numerous solutions – from using a third party repository (with a broken link ) to installing “Eclipse IDE for Java EE Developers” from www.eclipse.org. But the solution that worked for me was the simplest. Please note that this issue and the following solution could be specific to Eclipse on Fedora and Google Plugin.

The Solution
Launch Eclipse. Click on Help>Install New Software. On the subsequent window titled “Available Software”, click on blue link ‘Available Software Sites’. Look for the URL http://download.eclipse.org/releases/galileo in the list of Software Sites. Select and click on Enable (button on the right). Click on OK.

Enable the Galileo software site before installing the Google Plugin

Back on the “Available Software” window, type the URL http://dl.google.com/eclipse/plugin/3.5 in the field labeled “Work with:” . Click on Add. This will show Plugin and SDKs in the bottom pane. Check the boxes against them and click on Next. Follow the onscreen instructions. This will install the Google plugin and SDK. When prompted, restart Eclipse.

A few Troubleshooting Tips
On a freshly installed Eclipse, all software sites are disabled by default. This may not be true, if you have been updating or installing other plugins in Eclipse. I recall that, on Fedora 12, I faced the issue of numerous conflicts till I disabled all but the http://download.eclipse.org/releases/galileo site. Bottom line: before installing the Google Plugin, make sure that you only have the Galileo site enabled.

Second, this happened to me once that, even after following the above steps I got the same error. It seemed that Eclipse had problem in downloading the required software and ended up throwing the same error, instead of reporting a failed download. In this case, restart Eclipse, else you may keep getting the same error on retries. After restart, follow the steps mentioned in The Solution again.

Yesterday, I had setup Paradox database using DOSEMU on Linux, for a client. On Putty, the Paradox interface showed garbled making it unusable for the client. My instant suspect was DOSEMU. A quick check revealed that it’s happening with other apps too - those with a ncurses or text based user interface.For example: The garbled text, around the borders of UI elements, showed in ntsysv (as shown in the screenshots). ntsysv is a nifty console app, found in RedHat/CentOS/Fedora Linux, to start services during boot up.

The Solution
Launch Putty. On the Category pane (on the left), click on Windows>Translation. For the dropdown labeled “Received data assumed to be in which character set”, select UTF-8. To save this setting, click on Sessions (in the Category pane). On the right pane, select “Default Settings” and click on the button labeled Save. This is what worked for me with Putty version 0.60.

Amongst using regular expressions and DOM (Document Object Model) parsing, I used XPath too. XPath works great for well-formed Web pages. A HTML Web page is called well-formed when all the opening tags have corresponding closing tags and the tags are nested properly (refer to this link for more). A well-formed HTML page is also called an XHTML page.

XPath is a query language to access content on a well formed page – XHTML or XML. All the content in a Web page lie within HTML elements or tag pairs. The following is needed to extract out the content, of interest, using XPath from a Web page:

1. A well-formed Web page
2. An XPath expression for the HTML element of interest
3. XPath query using the expression

An XPath expression looks as cryptic as  .//*[@id='home_featured']/div. Here is where Firefox web browser with a plugin called FireXPath comes to help (as explained below). The second and third requirements are met by PHP (in my case) which is used for automated Web scrapping using the XPath expression.

Click and build XPath expressions
Building the cryptic XPath expression is easy and intuitive with Firefox and a couple of its plugins – namely Firebug and FireXPath. Install Firebug from here, followed by FireXPath from here and restart Firefox. As of this writing, I’m using the following versions of Firefox and the plugins:

• Firefox 3.5.6
• Firbug 1.4.5
• FireXPath 0.9.1

After, the Firefox restart, browse to the Web page of your interest. Next:

• Launch Firebug by clicking on the bug like icon on the right corner of the status bar.
• Click on the tab named XPath (in FireBug)
• Click on the arrow (blue arrow) and move your mouse over the content, of interest, on the web page
• Once the required chunk is highlighted (with a blue border), note down the XPath expression in the textfield.

For example, the screenshot below, shows the XPath expression (.//*[@id='home_featured']/div  ) for the “Featured Post” chunk, on the home page, of this blog:

With FireXPath, point your mouse on the chunk/element of interest, to build an XPath expression

A few examples of XPath expressions:

• .//*[@id='latest_post']/span[1] : The posted and modified dates of the latest post on this blog – http://www.shekhargovindarajan.com
• .//*[@id='top-stories']/div[1]/h2/a : The headline of the top story on Google News – http://news.google.com
• .//*[@id='mp-tfa']/p[1] : Contents of featured article on Wikipedia – http://en.wikipedia.org/wiki/Main_Page

Automated scarping using PHP
With the XPath expression in hand, a PHP script as follows, can extract out the required content from the Web page:

<?php

class WebScrap
	{
	private $url;
	private $xpath;

	public function WebScrap($url,$xpath)
		{
		$this->url = $url;
		$this->xpath = $xpath;
		}

	public function GetScrap()
		{
		// use Tidy to try to make the page well formed
		$page = $this->TidyIt($this->url);

		// create a document out of the well formed content
		$domDocument=new DOMDocument();
		$domDocument->loadHTML($page);

		// create an XPath object out of the document and query it for the supplied xpath
		$domXPath = new DOMXPath($domDocument);
		$domNodeList = $domXPath->query($this->xpath);

		// Get the content (HTML) out of the NodeList returned by the DOMXPath::query
		$content = $this->GetHTMLFromNodeList($domNodeList);

		return $content;
		}

	private function TidyIt($url)
		{
		$tidy = new tidy();
		$tidy->parseFile($url);
		$tidy->cleanRepair();
		return $tidy;
		}

	private function GetHTMLFromNodeList($domNodeList)
		{
		$domDocument = new DOMDocument();

		$node = $domNodeList->item(0);   

		foreach($node->childNodes as $childNode)
			$domDocument->appendChild($domDocument->importNode($childNode, true));

		return $domDocument->saveHTML();
		}

	}

?>

Note that, the function/method called GetScrap( ) of the WebScrap PHP class first calls the TidyIt( ) method. This function uses the Tidy library to fix (if required) the HTML for well-formedness. The PHP class for Tidy is used to fetch the web page via the given URL and repair it. Henceforth the DomXPath object is used to query the well-formed Web page content for the XPath expression.

To use the above code, you will need to install PHP modules for XML and Tidy. On a RedHat/CentOS/Fedora Linux machine, these modules can be installed using the following command:

yum install php-xml php-tidy

Save the above code in a file named class.WebScrap.php (say). Subsequently, the WebScrap class can be used as:

<?php

include("class.WebScrap.php");
$scrap = new WebScrap("http://news.google.com",".//*[@id='top-stories']/div[1]/h2/a");
print($scrap->GetScrap());

?>

The code should be self explanatory for a seasoned PHP programmer. If not, shoot your questions via comments to this post. For repeated and automated Web scarping, a scheduler like Cron can be used to execute the above PHP script at regular intervals and fetch the latest content.

The Leftovers
The code above is readable, crisp and focusses on the subject. For this reason, it has deliberate exclusions.  In a real world application, you should:

• Use Curl library in PHP  or external tools like Wget to fetch the URL. Then pass on the fetched content to Tidy. Hint: use the  parseString( ) of Tidy instead of parseFile( )
• Handle errors in case of errors in Tidy and XPath.
• Fallback to other means (say regular expressions) in case of errors in Tidy or XPath querying

Note that XPath may not always give you the structured content that you desire. For example, using the expression .//*[@id='latest_post']/span[1] (for posted and modified dates of the latest post on this blog) will result into something as follows:

<strong>Posted on:</strong>
January 8, 2010
<span class="dot">⋅</span>
<strong>Last modified:</strong>
January 8, 2010 @ 4:49 pm

So there are tags and text (like “Posted on:” and “Last modified:”) to be stripped out, to get the posted date (January 8, 2010) and the modified date (January 8, 2010 @ 4:49 pm). For this, you may still have to use regular expressions and/or string manipulation functions like split( ).

Each time, I upgrade my phone, I wipe off its data. This is because I took the upgrades as an opportunity to wipe off applications which I install, but do not use or use occasionally. But, following is the list of Android applications which I crave to install after each wipe off. These applications are my must-haves on my Android phone.

1. android-wifi-tether
A computer or PC/Laptop can be connected to a phone via USB, Bluetooth or Wi-Fi to use the phone’s Internet connection – typically GPRS/EDGE/3G. This kind of connection sharing (with the phone) is called tethering. ADP1 does not ship with tethering capabilities. But thanks to the android-wifi-tether app which can be found here. Don’t go by the “Wifi” in the app’s name. It supports tethering through Bluetooth too. I have used tethering via Wifi, till I switched to CyanogenMod (see below). Once installed, it turns your phone into a Wi-Fi ad-hoc point! Think about it as a Wireless Access Point. Therefore, no hassle of downloading drivers to connect to it from a laptop.

android-wifi-tether app for phone's Internet connection sharing via Wifi/Bluetooth

Features:

• Tethering through Wi-Fi and Bluetooth
• Access control based on MAC address. Notification, about connecting clients, on the phone’s notification bar – with sound.
• 128-bit WEP (Wired Equivalent Privacy)

Pros:

• Behaves as a Wi-Fi ad-hoc point. This means, typically, no configuration/driver hassles on a PC/Laptop.
• MAC address based access control.
• Lots of configuration options for Wi-Fi, like SSID, channel, IP range etc.

Cons:

• Requires ROM update (a kernel update)

Go to Settings>Wireless controls and check Internet tethering. Works with CyanogenMod Android ROMs

I no longer use android-wifi-tether – after flashing my phone with  CyanogenMod. With Cyanogen ROMs, the capability to tether via USB is built-in. USB tethering works both on Linux and Windows – I have it working on Fedora 12 Linux and Windows 7. android-wifi-tether works with the CyanogenMod (version 4.2.13, as of this wiring) without requiring the kernel update. But you will need to download and install the app from here. With android-wifi-tether and CyanogenMod, you may like to enjoy the flexibility of tethering through USB, Wi-Fi and Bluetooth! Though, I’m my happy self with USB tethering.

2. BlueRSS
Once I discovered this free RSS reader named BlueRSS, from the Android Market, I never felt the need to look for an alternative. Lots of features, yet fast and responsive. This is what sets this app apart, for my daily routine, to check out the tech world through RSS feeds. I’m using version 3.0.1 of BlueRSS, as of this writing.

BlueRSS: A fast and feature rich RSS reader for Android phones

Features:

• Discover RSS feeds on Websites or using keywords
• Exports feeds to OPML, opens GMail (or the default mail client) with the OPML file attached, to send.
• Automatic background update
• Ability to refresh all feeds as well as individual feeds
• Add feed items to Favourites
• Full screen mode

Pros:

• Fast and responsive. Don’t remember if I ever had to “Force Close” it
• Import from Google reader

Cons:

• No feed categories
• The feed item’s description/summary does not open in landscape mode.

3. Twidroid (free version)
This app does not require an introduction to Android users. It is undoubtedly the best Twitter client for Android. I started with using Twidroid as the Twitter client on my Android phone. I switched to numerous other Twitter apps to check out the alternatives. I tried Swift, Seesmic, Twidgit and TwitterRide (now called TweetsRide). But I kept coming back to Twidroid and have settled down to it. Twidroid is available from the Android Market and I’m using version 3.0.9.

Twidroid: The feature-rich Twitter client for Andorid

Features

• Supports Twitter lists and the new Retweet feature (or API) of Twitter
• Ability to upload photos
• Twitter Trends, Search and Favorites
• Built it browser to open URLs
• Options to choose a preferred URL shortening service and image upload service
• Built-in profile browser with option to follow/unfollow
• Autocomplete of Twitter usernames while Reply-ing or Direct Message
• Share Tweets via Email, SMS and Bluetooth
• Stores data on the SD card

Pros

• Fast and responsive UI
• One touch access to most functions like Reply, Direct Message and Retweet
• New tweets notification, with the Tweets’ text, in the Notification bar of the phone.
• Automatically shortens the URLs
• Remembers the scroll position with an option to jump to the latest Tweet
• Frequent new releases.

Cons

• Features like widgets, video and multiple accounts available only in the Pro version

4. Nav4All
I have been living in New Delhi, India for 31 years. But Connaught Place a.k.a. CP is a place where I always lose my way to the destination and back home. Thanks to the large number of roundabouts. I recall driving for more than a hour in CP to figure out my way. That’s when I desperately started looking for a navigation app for my GPS capable Android phone. Nav4All is what I got and what I use. It works great for the Delhi roads. Nav4All is available in the Android Market and I’m using version 9.0.25. To be true, I haven’t used all its features. Yet following is the jot down of the features that I see on the application’s screen:

Nav4All: A simple and free app that works for turn-by-turn voice based navigation

Features

• A huge list of countries
• Option of large number of voices (for turn-by-turn navigation)
• Can pick addresses from the phone’s contacts
• View route to destination with turn by turn textual instructions and on the map
• Details like distance, driving speed and journey time
• “Start follow me” feature to trace your current route
• Alternate routes

Pros

• Free
• Precise, numbered, exit points on roundabouts
• Works in New Delhi, India
• Narrows down to the building name!

Cons

• Bad user interface, looks like wireframes
• Buggy with the hardware keyboard of ADP1
• Address search asks for too many details
• Not so good looking map

I haven’t been able to use the address search for Nav4All. It asks for too many details – upto the house number. The best way I use Nav4All is to record the destination coordinates – when I somehow reach the destination for the first time. On the contrary, Nav4All amazed me by telling the name of a client’s office building – located in the midest of a large number of houses.

Note that the Nav4All website hints that the app will remain free until 1st Jan 2010. But I have been using it for free, on my Android phone, till now. Hope it continues to remain free.

5. Car Mode
How many times it has happened, that, while driving you get a phone call. You, then, struggle to put the phone on the speaker phone mode. Car Mode is a nifty app which when clicked, places itself on the phone’s notification bar. Subsequently, it automatically enables the speaker phone for subsequent calls. Car Mode can be disabled by clicking on its icon on the notification bar. It can be downloaded from the Android Market. I’m using version 1.2.0 of this app.

Pros

• A simple and free app which works!

Cons

• It’s encourages to attend phone calls which driving

That’s all folks. Five must have Android apps that I live with. Note that this list does not include the built-in apps – like GMail, Calendar, Contacts, etc – else GMail will Gmail Push (instant push E-mail) is my most used app. Will keep this blog post updated with any future must-have apps which I get used to.

Update (February 2, 2010): I said above that Nav4All may become paid, but the latest update is worse. Got an E-mail announcing that  Nav4All will be discontinued.  Following is the exact copy of the E-mail as sent by CEO of Nav4All:

Subject: Nav4All navigation shut down by Navteq

Amsterdam, The Netherlands, 28-01-2010 16:30 GMT+1

Subject: Nav4All navigation shut down by Navteq

Letter to 27,625,631 Nav4All navigation customers

Dear Customers,

It is with the deepest regret that we hereby notify you that the
global navigation of Nav4All and the Tracking & Tracing will go
offline in 3 days. The reason for the same is that the data licence
agreement with Navteq (a 100% Nokia subsidiary) was not extended, in
a totally unexpected manner. It is not possible to implement data
from another supplier in our Nav4All systems within the short term.
The Nav4All navigation system was developed for Navteq data. Nav4All
has therefore been constrained to stop.

We greatly regret the fact that we have to suspend the operation of
our service. With your help, we have developed Nav4All into a global
product with 27.5 million users in 56 languages, in 5 years. This has
made Nav4All the largest navigation supplier. This large number of
users also has to do with the fact that Nav4All works on hundreds of
different mobile telephones of many makes such as Blackberry, Sony
Ericsson, Samsung, Motorola, Android, HTC, Nokia, LG, Iphone, Ipod
etc.

After 5 years of testing and market development, we witnessed rapid -
in fact, exponential - growth during the last two years. That growth
was reported in the licence reports to Navteq. In mid-December 2009,
the global coverage was extended to include the Philippines, Morocco
and Kenya.

Please contact the Nav4All support desk in case you have any
questions: www.nav4all.com/support. If there is any further
information from Nav4All concerning the subject of this letter, the
same will be published on our website: www.nav4all.com. For reasons
of privacy, Nav4All does not have the email addresses of all its
customers, and we therefore request you to forward this email to the
maximum extent possible, in order to ensure that everyone is
informed.

Kind Regards,

Hennie J.M. Groot Koerkamp (CEO)

Nav4All BV

Keizersgracht 62-64

1015 CS Amsterdam NL