Archive for the 'Snowball' Category

Extracting APK files from your Android “ADP1″

Tuesday, March 10th, 2009

The ADP1 is the Unlocked/Routed version of the HTC G1 (running Android) being sold separately by Google, intended for developer use only.

Prerequisites:
- Android SDK
- Android USB driver

To extract APK files:
(1) Plug your “Routed” Android phone into your computer via the included USB cable.
(2) From your computers command line type “adb shell“.
(3) Switch to root user “su“.
(4) “cd data/app“.
(5) List all the installed apk files with “ls“.
(6) In another command window, pull one of the files on to your computer:
Format:
adb pull /data/app/.apk .apk
Example:
adb pull data/app/com.zedray.snowball.apk snowball.apk

That wasn’t so hard was it?

The Android OS has effectively no protection for downloaded executables, so applications developers should seek alternative mechanisms for securing their applications against piracy.  Now (don’t get me wrong), I’m coming at this not as a user wishing to pirate software, but as a developer looking to sell the fruits of my labour over the Android Market.

Google have implmented their own reginal/device based locking with limited success (e.g. developers can’t see their own applications on the store).  However, I think a real solution would be to allow matching devices/users accounts to actual purchases, so applicaiton developers can monitor legitimately purchased applications on the server side.

HTC G1 Application storage limitation:
Both the HTC G1 and the forthcoming HTC Magic suffer from having extremely limited on-device Application storage.  This amounts to about 70MB after all the default OS applications (email, browser, etc.) have been installed.  This is a serious drawback for the device because Android Market applications can’t be installed off the SD card.  Meaning that a user will be able to typically download ~10 applications before their phone complains of lack of space (so much for trying out the thousands of free applications on the store).  I believe this is an issue which Google have no intention of fixing because of the security problems described above.

This is a real deal breaker when anyone asks me if they should get themselves a G1, and should also be considered by anyone trying to make money by selling applications on the Android market.

This problem will be solved over time as other manufactures start putting out devices with 8GB or 16GB+ of on-device storage.

Overcoming this limitation:
I am told that executing an APK file from an SD card currently requires a Linux file system partition (which I haven’t attempted).  My aim is to remap the SD card to the “data/app/” directory, so I can install Android Market executables directly to my 8GB SD card.  Then I could download and use hundreds of applications at the same time.

Android Bug/Architectural Issue: How do I handle multiple versions of my own Content Provider?

Thursday, January 8th, 2009

I am writing a component that makes certain functionality available to any application running on an Android device (e.g. an advertising service, stock ticker cache, Snowball server, etc).

cp1.gif
This functionality is useful for application developers, but not interesting to actual users i.e. my component should be included as part of a new application, but not require the end user to explicitly install the additional component themselves.

cp2.gif
I expect that over time multiple applications installed on the phone will want to communicate with my component.  As each new application will have a different certificate, I want to communicate between applications using an Android Content Provider.  To save resources on the device (networking, caching, etc) only one instance of my component should be appointed to handle all queries.

cp3.gif
This works well as Android only registers the first Content Provider for a given URI and then ignores the rest (throwing an “WARN/PackageManager: Skipping provider name xxxx name already used” error each time a new one is installed).

cp4.gif
However if the registered Content Provider is uninstalled, it will immediately break all the other applications that rely on it, even though other instances of the component still exist.

Questions:

  • Does anyone have any suggestions on how to better handle this situation?
  • If I could reregister Content Providers I could handle situations like this, and upgrade components when newer versions are installed.  Perhaps the Android OS could also handle this situation better, by tracking Content Provider naming collisions?
  • Should I be looking at other communication methods to solve this issue?

Note: This has been posted on the following discussion boards:

  • Android Developers (click)
  • Android-platform (click)

Edit 20090108
Just to clarify, the attribute “android:multiprocess” in the “provider” tag seems to have a behaviour that is not relevant to this problem, i.e. it prevents multiple instances of the same Content Provider from existing, rather than affecting multiple Content Providers trying to use the same namespace.

Edit 20090109
Thanks everyone for the replies.  Just to clarify: I am writing an advertising server component at the moment, but other projects that I have in the pipeline essentially have the same requirement.

>>Dianne Hackborn - As far as naming issues, this is why you should use an authority in a namespace/domain you own, so that there will not be naming collisions.<<
My application sits in its own namespace.  I am only having conflicts with different copies of the same Content Provider Component being installed with different applications.

>>Dianne Hackborn - …any client app that is relying on a component that is not part of the base platform needs to deal with the error cases if that component does not exist.<<
If the component is not there, then the application would show an error on start and normally refuse to run.  Result: Bad user experience.

>>Peli - Hm, I think you end up with unnecessary code duplication if you include your component / content provider in every application.<<
This isn’t an issue, as my component is delivered as a small Java class included with the main application.  When activated, the component inflates in size as it downloads content over the internet.  My goal is to manage this process by having only one component activated regardless of how many applications are installed on the device.

>>Peli - I think it would be better if every application that wants to use your component shows a small dialog “You need to install the following component…” with a button that brings you to the download page of the Market for your component…<<
Catching a missing component is easy enough.  It’s just that what you describe adds up to a bad user experience:

  • No user will download a 3rd party advertising server (or any non-core functionality) just to run a game.
  • Installation is already a convoluted process and should only take place once.  Additional APK installations require more downloading, application permission settings, button pressing, confusion, etc.
  • No client of mine will accept a solution that asks the user to jump through any additional hoops (e.g. even the Android Developer Challenge only permitted a single executable).

>>Peli - At OpenIntents we try to resolve dependencies as late as possible - i.e. not at installation time, but the first time the user actually wants to use a particular feature (they may never use it in which case there is no need to install it).<<
This doesn’t work for me as ad funded Applications generally require my component to be available on startup.

>>Peli - I think in the long run Market should have better tools for resolving compulsory dependencies automatically, but for the moment I think this is a good compromise for developers and end users.<<
I agree that this needs to be improved, as there is currently no support for components (downloading, versioning, upgrading, etc) in Android or on the Android Market.  Hence I am trying to get around this issue by “managing” a Content Provider.

>>Dianne Hackborn - Oh wait if what is being talked about is having the same content provider in multiple applications, that is completely the wrong approach.  For a given authority there must be one and only one content provider implementation, living in a single specific .apk.<<
This is how it’s done at the moment, i.e. Android will log a warning (and do nothing) every time an application tries to register a Content Provider URI that’s already in use.  This means that subsequent applications will be broken (and need to be reinstalled) if the original Content Provider is ever uninstalled.  I don’t seem to be able to reregister a missing Content Provider URI at runtime.

>>Muthu Ramadoss - AFAIK, a Content Provider is tied up to a unique Authority.  If you want to support multiple URI’s you do so accordingly by providing different URI paths using the same authority.<<
I explored this approach initially, but it still has the following problem:

Over time, I release a number of versions of my component that other developers subsequently integrate into their own applications.  My component uses the following URI conventions:
content://mynamespace.mycomponentnamespace.version1/
content://mynamespace.mycomponentnamespace.version2/
content://mynamespace.mycomponentnamespace.version3/

Android allows me to search and interrogate each URI to check the status of each Content Provider (allowing me to version my component).  This is fine until v3 of my component becomes really popular and a user installs five games on their phone, each with the same component trying to register the same URI.

The user then gets bored of the first game and uninstalls it, thereby breaking all the other games that rely on the Content Provider that it contains.  As URIs are set at install time, none of the other games will work until the user reinstalls an existing game or downloads a new one - This is a terrible user experience.

Edit 20090109 (#2)
>>Peli - Indeed, for an advertising server, it makes more sense to include that in every application separately, for the reasons you mentioned. I could suggest the following…and all applications can access that one content provider. Would this sound like a suitable solution?<<
I didn’t think of just doing the synchronisation using intents (as they support a one-to-many relationship between components).  I’ll do a proof of concept on Monday to see if this works.  It’ll be a bit of a hassle for the guy who has to incorporate this into their application, but it will do until Android is improved to support components. ;-)

Thank you kindly!!

>>Mark Murphy - Since the content in question is going on /sdcard (right? right?!?), solve the problem by managing the space, not managing the components. Since you already require each application to have your own code anyway, just use a regular Java class to access the /sdcard data store and dump the ContentProvider interface.<<
>>Peli - Ad images and banners should be placed on the SD card, but private data for accounting (number of times an ad had been shown, click- through-rate, last ad shown, etc.) could be stored in a content provider, and it makes sense to share this content provider among several applications installed (e.g. so that the end user is not presented the same ad in every application they launch).<<
While Mark Murphy’s solution could be made to work, communicating outside of the OS would be quite ugly in terms of the security and the hardware assumptions which would need to be made.  I also expect the security regime and file permissions to morph somewhat over the next couple of Android releases, potentially breaking my OS bypass hack.  I’ll put this idea down as a plan B, thanks anyway!!

I think I have enough ideas to get this to work now, and will post my results to the thread and blog early next week.

Edit 20090115 - Solution
The simplest solution (not using intents) is to instruct each user of my component to define a new content provider:

content://myComponentNamespace.theirApplicaitonNamespace

Android lets me search through all available content providers, which can be string matched to my own component namespace.

context.getPackageManager().queryContentProviders(null, 0, 0);

If I find more than one valid Content Provider I can then pass around Synchronisation information (validity, compatibility, version, status, etc) using any of the ContentProvider class methods (I used getType()).

The only problem with this approach is that the application developer who imports my component (as a JAR) will have to define their own custom Content Provider.  This is done by extending the MyComponentContentProvider class (from my JAR) and defining a CONTENT_URI variable.

package test;
import com.whitemice.MyComponentContentProvider;
public class TheirContentProvider extends MyComponentContentProvider {
public static final Uri CONTENT_URI =
Uri.parse( "content:// myComponentNamespace.theirApplicaitonNamespace“);
}

As well as adding the custom provider to the Android manifest.

--provider
android:name="test.TheirContentProvider"
android:authorities=" myComponentNamespace.theirApplicaitonNamespace"

As this is a work time project, I will have to check if I can release the synchronisation code as a working example.

Edit 20090116
>>Peli  - Probably you should not mis-use getType()…<<
I originally choose getType() for simplicity (URI in -> String out), but I will now use query() instead.  However, as my application is not compatible with any other software (i.e. my data exchange has no MIME type), I would be interested if anyone could provide a concrete example of where an inappropriate getType() response might cause problems.

Also, for simplicity I am not using intents.

>>Dianne Hackborn - …Along those same lines, looking for your own special text in the authority to identify content providers you are going to work with is also kind-of screwy.  Typically how we do this is attach meta-data to the component that whoever is interested in identifying specifically tagged components looks for.  Look at the way input method service components are discovered in Cupcake as an example. <<

Here is a small component discovery rewrite, now using meta-data to mark valid Content Providers:

####AndroidManifest.xml####

–provider–
android:name=”test.TheirContentProvider”
android:authorities=”myComponentNamespace.theirApplicaitonNamespace”

–meta-data android:name=”type” android:value=”myComponentNamespace”–
–/provider–

####Java Code####

List–ProviderInfo– pis = androidContext.getPackageManager().queryContentProviders(null, 0, PackageManager.GET_META_DATA);


for(ProviderInfo pi:pis) {
Bundle md = pi.metaData;
if (md != null &&
md.getString(”type”) != null &&
md.getString(”type”).equals(”myComponentNamespace”)) {

    //Congratulations you have now found a valid Content Provider
//Note: keep looking as you might find more

    }
}

The upside of this is that the application developer can now deploy my Content Provider using any URI namespace they choose.  However, I will still recommend the previous naming scheme to prevent damaging naming conflicts.

>>Dianne Hackborn - …Also, have you thought about security at all?  What kind of information is stored on the content provider?  Who can access it?<<
As I understand it, any cross application Content Provider on Android needs to implement its own (i.e. roll-your-own) security solution when handling incoming requests.

My current application (an advertising server) treats this as an open/free to use API, with risk scenarios being managed mainly on the server side.

My next application (component for peer-to-peer data sharing) will require a custom Android Permission for sending data.  This is similar to the “android.permission.INTERNET” permission whereby the user still gets a say as to whether the application can send data off the phone.

I am not yet confident enough in the security capabilities of Android and/or my knowledge of them, to implement an application that handles confidential data.

Google Developer Day London 2008

Tuesday, September 16th, 2008

Google Day London 2008 Keynote
This event turned out to be a lot of fun, despite a registration glitch that sent me a “this event if full, try again next year” email.  A big thanks to Silvia for encouraging me to follow up on that email and get myself re-registered (you’re a star) and to someone at Google call Liz for forwarding all the final event details.

What made the event was the free food, friendly presenters and the chance to make some quality Android contacts.  Highlights being Mike “I’m-not-an-American-I’m-Canadian-with-a-British-passport” Jennings’s Android keynote/device demo/Q&A, meeting some up and coming Android consultants from novoda.com and a rather solid discussion on application traceability with alsutton.com.

A note also goes to Reto Meier who is writing (or frantically updating for 0.9) his book titled: “Professional Android Applications”.  By the sounds of things, the book will definitely be worth a read and made for a good excuse for the speaker to pick on the author while fielding some of the more technical questions.

Unfortunately I only got to *watch* the lightning talks, and not demo Snowball as I had already upgraded my laptop to Android SDK 0.9 and hadn’t had time to bring all my code up to spec.  This was a problem shared by many (to varying degrees), and is a reminder to inventors to always carry around their elevator pitch/product demo on PowerPoint.

According to an unnamed Google employee, constant SDK updating is just one of many Android problems that will be “solved by the launch”.

Android 0.9 SDK Beta (r1) a first look

Tuesday, August 19th, 2008

After several months of black out, it’s suddenly new Android SDK day!
This is important as it gives us ‘fan boys’ a chance to take a look at how the new mobile phone operating system from Google is progressing.

Android SDK v0.9

Download here:
http://code.google.com/android/

Existing developers should follow the upgrade guide:
http://code.google.com/android/intro/upgrading.html

A few observations:

  • Updated user interface/home screen (looks pretty as ever), messaging and picture applications now present. Hopefully this will mean less peering into grainy YouTube videos to see the user interface in action.
  • App store still missing (see comment about grainy YouTube videos).
  • Applications must be cryptographically signed, so as to identify the developer. This is a new and important feature, but lacks any mention of how this fits into the Android ecosystem. Historically the Java ME implementation of application signing was appallingly bad, so I will be keeping a close eye on how this develops.
  • Bluetooth support is conspicuous by its absence. Bluetooth is hard to get right (just look at all those buggy phones over the years), so this was somewhat expected. We are told to wait until version 1.0 for this feature, but I could imagine this sort of thing holding up the whole show.
  • Network access now requires explicit permission at install time. Perhaps all those developers with unlimited 3G data plans forgot about the little guys.
  • GTalk is now officially *gone* from this release (citing security reasons), which will kill a lot of developers who were taking this shortcut to peer-to-peer.
  • There are unspecified changes to the installer, so I will have to do some experimentation with the “adb” to see how flexible this will be.
  • They have *fixed* the long startup times bug. Although I didn’t seem much difference with startup time, this always seems to happen when I am giving a presentation (see Murphy’s law).

I will follow up with more detail once I start developing.

Snowball:
I promised myself that I wouldn’t touch Snowball until Google released an improved emulator. Now here it is, I will have to schedule some development time after my vacation next week.

For this SDK release, my Snowball “to do” list looks like this:

  • Implement the Wi-Fi Channel, so handsets can share content with Wi-Fi base stations and other users in a peer-to-peer fashion.
  • Create a Snowball development kit, including the “Snowball Engine” client application and example code so other developers can try it out.
  • Bluetooth support will have to wait for 1.0.

JAX08: Where am I? Location Based Services

Wednesday, March 26th, 2008

Its official, I will now be presenting at the JAX08 Conference with a presentation entitled:

Where am I? Location Based Services
Location Based Services stellen einen der großen Benefits mobiler Devices dar, der weit über die Bestimmung der eigenen Position hinausgeht. Im Java-ME-Umfeld werden derartige Services u.a. mittels Location API (JSR 179) realisierbar. Die Session zeigt die Möglichkeiten und Grenzen Java-ME-basierter Location Based Services und führt sowohl in die theoretischen Grundlagen als auch in die praktische Verwendung der Location API ein. Darüber hinaus wird anhand einer auf der Androids-Plattform basierenden Beispielanwendung eine Alternative zum klassischen LBS-Ansatz aufgezeigt. (click)

I will be co-presenting with my CEO - Lars Roewekamp of OpenKnowledge GMBH.

Mark Brady Openknowledge
A very exciting time to be in mobile development and location based services in general.
I will also slip in something on Snowball and demo a few Android proofs of concept applications.

Mobile Portfolio

Saturday, February 2nd, 2008

A new year and a good opportunity to look at all the mobile related stuff in my portfolio since:

2004

  • EdiMap (MSc Edinburgh University)
    Tourist map of Edinburgh, with downloadable POI.
  • NotPacMan (Self training)
    The familiar PacMan game.

2006 (Starting at OpenKnowledge)

  • original4.gif
    Barometer, Horoskop and Lovecheck (AirMotion)
    Fun and games, and a lot of device testing.
  • MiniMove (BMW via BBDO)
    Rotate the BMW mini on your mobile.

2007

  • original3.gif
    MobileAct (Sony Ericsson via AirMotion)
    Weekly magazine content to go with the Channel 4 series.
  • original2.gif
    MiniMove China (BMW via BBDO China)
    Rotate the BMW mini on your mobile in Chinese.
  • original1.gif
    original.gif
    Mitch&Co Blackbox (Tchibo via MINICK)
    Explore the latest Mitch&Co fashon range on either male of female models.
  • NowHere (Burda Wireless)
    Photograph where you are and upload as a NowHere POI.

2008…

  • Ads by a [big mobile network operator]*
    Important stuff
  • Google Android
    Self training on the new mobile OS from Google
  • Snowball - The Mobile Location Based Search API
    My world beating business idea.

* To comply with my company’s code of conduct for blogs, I have removed all references to customers and projects currently in progress.


Geo Visitors Map