Overview

In order to install kind of new operating system I needed to have root access on the device. This meant I had to root it. Since Android 1.6 (Release R10) on the G1 doesn’t have any useful exploits I had to downgrade the OS to Release R7 first. Once it is downgraded it you can use the rooting exploit in R7 to get root access and install a recovery image that will allow the installation of an updated radio software and the new operating system.

Preliminaries

Firstly read all of this blog post and all of the pages linked to from this blog post. There is at least one point in this process where you can “brick your phone”. I.e. you can put it in a state where by a regular user cannot make it usable.

Secondly you will be wiping your phone and replacing the software on it. Back anything up you want to keep. This includes purchased applications, data, contacts, whatever.

Make sure the phone is fully charged. You don’t want it running out of power mid-flash!

Finally I provide no warranty express or implied. You are doing this to your phone. If it goes wrong its your fault! You have been warned!

Downgrading and rooting

Downgrading, rooting the phone, and installing a recovery image is described in the link below. However since there are specific requirements for upgrading to Android 2.X I’ll explain each stage here.

http://forum.xda-developers.com/showthread.php?t=442480

Downgrading

Before you do anything take note of the radio version string in the boot-loader. Its the fourth line down. You get the boot-loader screen up by holding down the camera button when powering on the phone. With the original loader you should have multi-coloured bars with yellow text in the top left hand corner. On my phone the radio version string read “RADIO-2.22.19.26I”. Write down the string and store it somewhere safe. You’ll need it later.

Now download the appropriate older image for your phone. The links to the image downloads are on the page above however here’s the link for RC7 for those with UK phones. Download the image, unzip it, and save the DREAMIMG.nbh file in the root of your SD card. While you’re at it also download recovery-RA-dream-v1.5.2.img which is a recovery image and will be used later. Save this in the root of your SD card also.

Turn off your phone, insert the SD card, and turn it back on again holding down the camera key as before. It should now re-flash your phone. There should be a progress bar. When its all complete you can restart your phone normally (without the camera key held down). This will result in some old school Android graphics and you wondering how anyone ever coped with that software. Once the phone has booted find the phone details in settings and check the software build version number. It should appear as something like TC5-RC7 112931.

Rooting the phone

Now you have exploitable image installed it is time to root the phone. This can be done using a back door. Go to the home screen then type…

<return>
<return>
telnetd
<return>

…on the slide out keyboard. This will look like a search in contacts for the name of telnetd. In fact telnetd is the telnet daemon which allows you to connect to shell on the phone via telnet. So now you need a telnet client. Use the Android Market to download a telnet client. Once installed you should be able to telnet to localhost and get a telnet prompt.

If all is well you have shell access as root (a ‘#’ prompt). Yay!

Installing a recovery image

Now we are going to install the recovery image we downloaded earlier. This will also give us permanent root access. In the shell type…

mount -o rw,remount -t yaffs2 /dev/block/mtdblock3 /system
cd sdcard
flash_image recovery recovery-RA-dream-v1.5.2.img
cat recovery-RA-dream-v1.5.2.img > /system/recovery.img

For those who are interested the lines do the following:

1. Remounts the flash memory in read/write mode. This allows you to save to the flash in the phone.
2. Navigate to the sdcard (which is mounted at /sdcard)
3. Flashes the recovery-RA-dream-v1.5.2.img image file in to the special recovery partition in the flash.
4. Copies the recovery-RA-dream-v1.5.2.img as /system/recovery.img (The Android busybox doesn’t have the copy command but cat with a pipe works just as well!).

Once you’ve done this and have a prompt again you can reboot the phone. This time if you hold down the home and power buttons when you power on the phone it will boot in to the recovery image. You can use the short cuts or the track ball to select a menu item.

Now we are ready to install the new software.

New radio software and DangerSPL

Now we have to repartition the flash to give enough room for the new platform. This is the scary bit. As I said before…

Finally I provide no warranty express or implied. You are doing this to your phone. If it goes wrong its your fault! You have been warned!

Read this page thoroughly http://wiki.cyanogenmod.com/index.php?title=Dream:DangerSPL. If you’ve gotten this far you fulfil the prerequisites to install Danger/SPL. You now need to download the appropriate radio image, place it in the root of your SD card and flash it. Remember that number we noted down from the boot-loader screen at the start (“RADIO-2.22.19.26I” or similar). You will need it now.

Boot in to recovery by holding down the home and power buttons. Select “Flash zip from sdcard”. Select update.zip (if that’s the name of the file) from the file menu. The image should be flashed and will then reboot the phone. Reboot the phone again.

Take out the SD card and boot back in to the boot loader. It should now appear as below.

Upgrading to FROYO and installing Google Apps

All the hard work is now done. You have a rooted phone with a nice recovery menu which allows you to flash anything that will fit.

To install the latest Android read this link: http://wiki.cyanogenmod.com/index.php?title=Dream:Installing_CyanogenMod_5%2B.

Clear out your SD card and download Froyo for the G1/Dream and the Google Tiny Apps (The G1 is a bit short on flash space so wont fit the all singing all dancing versions of the Google apps). Boot in to recovery as before (home and power buttons) and again select “Flash zip from sdcard”. Flash first Froyo and then the apps.

Finally reboot your phone. You should now have Android 2.2 installed!

Issues

The only issue I had was an exception from the keyboard software the first time I used Froyo. This can be solved using the “Wipe” menu in the recovery image. I wiped data, cache and delvik cache (The latter generated an error I ignored) and then re-installed Froyo.

I really wanted to run ‘proper’ Eclipse as released by Eclipse so it was time to Google for the problem and I found this link. It turns out that there is a bug in Eclipse to do with GTK which will be fixed in the next release. However you can work around it. Put the following shell script in to your Eclipse directory and when you run Eclipse using that and not the binary directly its fixed.

#!/bin/bash
export GDK_NATIVE_WINDOWS=1
/path/to/eclipse/eclipse

Anyway that’s a bit of an aside. So now the post proper, how do you install Eclipse for multiple languages? This is what I did.

Eclipse for C/C++

Go to http://www.eclipse.org/downloads, select packages and grab Eclipse IDE for C/C++. Untar it somewhere suitable like /usr/local. You will have to be superuser to untar there so use sudo.

Java Development Tools

These come from http://download.eclipse.org/eclipse/downloads. Scroll down the page until you find ‘JDT Runtime Binary’ and select the latest version. This will download a ZIP file containing all the additional Java classes which provide the Java Development Tools.

Eclipse ZIP files are typically one of two kinds. Either a collection of files you unzip in the directory where you eclipse directory is or one which that’s an ‘All in 1′ update site. The latter is a zip file that you can install via the Eclipse IDE itself since it contains the appropriate site.xml file used by the Eclipse installer.

The Eclipse JDT ZIP is not an update site so you will need to unzip in it in the directory you untarred the original files. I.e. /usr/local. If there are any clashes (because the file already exists) just select ‘N’ meaning don’t overwrite.

Once you’ve unpacked start Eclipse (sudo /path/to/eclipse/eclipse.sh) to ensure it looks sane. You run it as root until you’re completely installed so that the extensions are installed for all users. Hopefully, if all is well, you should see C/C++, CVS and Java projects available in the Projects menu. Leave Eclipse running.

DLTK, EMF-XSD and GEF

DLTK is the Dynamic Languages Toolkit extension; EMF is the Eclipse Modelling Framework extension and GEF is the Graphical Editing Framework. These form dependencies on the Web Tools and the PHP Development Tools. Download the ‘All in 1′ update sites for these as follows.

DLTK

Go to http://download.eclipse.org/technology/dltk/downloads/. Select ‘Stable’ and then the ‘All in 1 Update Site’ bundle. Save the ZIP somewhere sensible.

EMF-XSD

Go to http://www.eclipse.org/modeling/emf/downloads/?project=emf. Select the ‘All in 1 Update’ bundle and again save it.

GEF

Go to http://www.eclipse.org/gef/downloads/ and select the ‘All in 1 Update’ and save it.

Installing

In the help menu select ‘Install new software’. The top right of the dialog that appears is an ‘Add’ button which adds a new install site, i.e. a URI from which new software can be installed. Click on it.

A new dialog appears with two fields. Type ‘DLTK Local’ in the first (‘local’) field. This is the short name the install site will be known as locally. Click on the ‘Archive’ button and navigate to the DLTK ZIP file you downloaded. Select it. Then click the ‘OK’ button in the new site dialog.

Now Eclipse will display what software is available from your install site (i.e. the DLTK ZIP file). If you expand the tree you will see a whole selection of different components. Some will have ‘SDK’ as part of the name. These components are ones which allow you to develop the Eclipse components themselves. You don’t need these if you are using Eclipse to develop regular code so ignore those. For DLTK I selected the core component and support for Python and Ruby (since I want to learn them at some point).

Click ‘Next’. All the dependencies should be there. Eclipse may install additional packages which are dependencies on the one’s you’ve selected. It may also show a licence agreement you need to agree to. Click through until ‘Finish’ is shown. Click ‘Finish’ to install the software. Eclipse may want to restart at this point, everyone apart from Ubuntu 9.10 users can click ‘Restart’ here. Because of the bug Ubuntu 9.10 users should click ‘No’ and exit and restart Eclipse manually using the shell script as described above.

Do the same for the EMF-XSD package and select ‘XSD-XL Schema Model, XSD Documentation and XSD Edit’. (Probably not all needed but useful). Install and restart as before. Add the GEF ZIP to the install sites so that it can be found later but don’t bother installing anything if you don’t want to. This seems to be needed by the Web Tools but I didn’t want anything from it myself.

You should now have Ruby and Python projects as well as DTD, XSD, etc. projects available in the projects menu. Quit Eclipse.

Web Tools Platform

WTP provides support for developing HTML, JavaScript etc. and is needed by the PHP Developer Tools. The download site is http://download.eclipse.org/webtools/downloads/. Select the latest release and then download the web tools platform, wtd, ZIP file.

Like the JDT ZIP this is not an install site so needs to be unpacked in the same directory as you unpacked Eclipse and the JDT ZIP. You will need to be superuser as before. Again there may be files that already exist so you have to stop them from being over written.

Restart Eclipse as superuser so that it can update its state. Now you should have web tools projects available like JavaScript, HTML etc. in the new project menu.

PHP Developer Tools

Finally we can install PDT. You can get the all in one update site from http://www.eclipse.org/pdt/downloads/ as a ZIP. In Eclipse using the ‘Install New Software’ dialog add it as an install site, select the PDT tools and install them. Eclipse will install anything it needs from the other install sites you used earlier. Finally restart Eclipse one last time.

Yay! You should have Eclipse installed for a whole bucket load of languages. Because you installed the plug-ins as superuser they are available for everyone on your system. Finally, because you don’t want to have to do this all over again should something go wrong, tar up the entire Eclipse directory and put it somewhere safe.

PHP session handling

PHP session handling is a way of having persistent or stateful data across HTTP requests. The simplest snippet of PHP to achieve this is shown below.

// Start the session
session_start();
 
// Create a new session object if its a new session
if (!isset($_SESSION['session'])) {
	$_SESSION['session'] = new MySessionObject();
}

I tend to use a PHP class to encapsulate the session data. Using the code above a new instance of MySessionObject() is created if a session could not be restored. Otherwise the instance is restored from the previous time the code was called.

PHP actually manages this using a session ID stored in a cookie. (There is an alternative method of managing the session ID if cookies are disable in a browser but that’s beyond the scope of this post.) The session_start() call will parse the value of the Cookie: HTTP request header field looking for a value similar to PHPSESSID=b330c9b604d371e5c88cd0919990f41c. It will use the session ID value to retrieve the appropriate session data. It will deserialise any stored values and store them in the $_SESSION global array variable.

If the PHPSESSID does not exist session_start() will create a new empty session (and $_SESSION) and use Set-Cookie: in the HTTP response to pass the session ID cookie to the browser so it can be returned on the next request. The session cookie is valid for the lifetime of the browser.

The Java client

However this post is about using PHP sessions from Java. To make an HTTP request from Java use the HttpURLConnection class. This class does have a few gotchas which I will explain below but it is sufficient for our needs. Typically I encapsulate the class in my own class to manage extra data. The bones of such a class is as follows.

public class MyConnection {
 
	private final static String HTTP_HEADER_SET_COOKIE = "set-cookie";
 
	private URL url = null;
 
	private String cookies = null;
 
	public MyConnection(URL url) {
		this.url = url;
	}
 
	protected String readCookies(HttpURLConnection con) {
		// ...
	}
 
	public String sendRequest(String requestString) 
		throws IOException, SocketTimeoutException {
		// ...
	}
}

You create an instance of the class when you want to start communicating with the remote URL and it will store cookies between each communication session with the remote server. The main work is carried out by the sendRequest() method which is shown below.

HttpURLConnection is a little tricky to use due to the time at which the request is actually sent to the server. If you set header fields (using setRequestProperty()) too late then they wont be included. It is similar for other properties too.

By default the class expects to have an empty body to the request sent to the server and returns the body of the response to the user by way of the input stream obtained using getInputStream(). (Note that input and output here refers to data being input to the connection and output from the connection. I.e. input in to the server and output from the server which may be the reverse of what you’re expecting.) However you can tell the connection you intend providing a body to the request by calling setDoOutput() with a value of true. In this case the output data is buffered and sent with the headers when getInputStream() is called.

public String sendRequest(String requestString) 
	throws IOException, SocketTimeoutException {
 
	StringBuilder buffer = new StringBuilder();
	String line;
 
	// Get the connection.  Java will create a new one each time.
	HttpURLConnection con = (HttpURLConnection)url.openConnection();
 
	// Set the Cookie header
	if (cookies != null) {
		con.setRequestProperty("Cookie", cookies);
	}
 
	// Set the content-size header
	con.setRequestProperty("Content-size", 
		Integer.toString(requestString.length()));
 
	// Set the request method.
	con.setRequestMethod("POST");
 
	// Enable both reading and writing to and from the connection.
	con.setDoInput(true);
	con.setDoOutput(true);
 
	// Get the output stream
	OutputStreamWriter out = new OutputStreamWriter(con.getOutputStream());
 
	// Send the request
	out.write(requestString.toString());
	out.flush();
 
	// Get the input stream.  This is the point at which data is sent to 
	// and read from the server.
	BufferedReader in = new BufferedReader(
		new InputStreamReader(con.getInputStream()));
 
	// If the response from the server is OK handle the content of the response
	String responseCode  = con.getResponseCode();
 
	if (responseCode == HttpURLConnection.HTTP_OK) {
		// Read the cookies
		String cookies = readCookies(con);
		if (cookies != null) {
			this.cookies = cookies;
		}
 
		// Read the content
		while ((line = in.readLine()) != null) {
			buffer.append(line);
		}
 
		return buffer.toString();
	}
 
	// Error from server
	return null;
}

Basically the method obtains a connection and sets up important headers, the request type, sends the request string as the body of the request returning the response string to the caller. This is a fairly typical use of the HttpURLConnection class. The important bit with respect to this particular post is the cookie handling.

In the code above, if the cookies variable is set then it is passed to the server as the value of a Cookie: header otherwise that header is omitted. When an instance of the MyConnection class is created the cookies variable is initialised to null and is only set when a Set-Cookie: header is received from the server. This is handled by the readCookies() method.

protected String readCookies(HttpURLConnection con) {
	StringBuilder cookieBuffer = null;
	String        cookieField  = null;
	String        headerName   = null;
 
	for (int i = 1; (headerName = con.getHeaderFieldKey(i)) != null; i++) {
		if (headerName.toLowerCase().equals(HTTP_HEADER_SET_COOKIE)) {
			cookieField = con.getHeaderField(i);
			cookieField = cookieField.substring(0, 
				cookieField.indexOf(";"));
 
			if (cookieBuffer != null) {
				cookieBuffer.append("; ");
			} else {
				cookieBuffer = new StringBuilder();
			}
			cookieBuffer.append(cookieField);
		}
	}
 
	if (cookieBuffer != null) {
		return cookieBuffer.toString();
	} else {
		return null;
	}
}

Annoyingly there isn’t a getter equivalent to setRequestProperty(). You can only obtain headers provided in the response by index. This means that you have to iterate through all the headers looking for the ones your interested in. This is slightly simplified by the getHeaderFieldKey() which returns the header name, for example, Set-Cookie for the specified index. So you iterate through the header names obtaining the header data for each of the names you are interested. The value of the cookie is everything up to the first semi-colon. The rest is meta-data.

However nothing is simple in this world. It appears that the Android version of HttpURLConnection converts the header field name to lower case but the Sun version (in the JDK) does not. So above I convert the header name to lowercase before comparison to ensure I’m always comparing like with like.

Yet another gotcha is that you could have more than one Set-Cookie header but you can only call setRequestProperty() once for each header. If you call it more than once you overwrite the previous value. So I concatenate all the cookies received in to one cookie string separating them appropriately.

So combining everything the Java client creates an instance of MyConnection with the appropriate URL. When sendRequest() is first called no cookies are set. The PHP on the server calls session_start() that creates a new session. The session ID is returned to the Java client, obtained by readCookies() and stored in the instance of MyConnection.

Subsequent calls to sendRequest() will return the session ID back to the PHP running on the server for as long as the instance of MyConnection exists. This way creating a new instance of MyConnection creating a new session on the server automatically.

So that’s how you use PHP session from a Java client. I hope this post is useful and welcome any comments about content or style.