Wednesday, September 28, 2011

How To Do Android C2DM Push Notifications Part 2

On the server side, in order to send a message, you need the following:
  • clientLogin id
  • role email account authorized to send c2dm messages (email and password)

If your application has a lot of traffic, you will want to write a windows service or a cron job to send out the messages in batch instead of sending them real time. The source code in this post will be based on a windows service written in C# (C# is very similar to Java).

In my database, I have the following tables:
  • c2dm_user - stores the user id and c2dm token pairs
  • c2dm_message - stores the messages for the windows service to send out in batch, say each minute 1000 messages

When user A sends a message to user B, a message alert will be inserted into the c2dm_message table if both users A and B exist on c2dm_user table. C2DM is not meant for heavy weighted messages. The intent is to alert the users that there are new messages. What I did is I use the unread message count as the payload. Users will see a notification saying "You got x messages!" in their mobile devices.


Getting the ClientLogin Id

public static String getC2dmAuthToken()
        {
            String responseFromServer = null;

            try
            {
                HttpWebRequest request = (HttpWebRequest)WebRequest.Create(CLIENT_LOGIN_URL);
                request.ContentType = "application/x-www-form-urlencoded";

                Stream dataStream = null;
                request.Method = "POST";

                dataStream = request.GetRequestStream();

                StringBuilder postData = new StringBuilder();

                postData.Append("Email=" + EMAIL);
                postData.Append("&Passwd=" + PASSWORD);
                postData.Append("&accountType=" + ACCOUNT_TYPE);
                postData.Append("&source=" + SOURCE);
                postData.Append("&service=" + SERVICE);

                byte[] byteArray = Encoding.UTF8.GetBytes(postData.ToString());

                dataStream.Write(byteArray, 0, byteArray.Length);
                dataStream.Close();


                HttpWebResponse response = (HttpWebResponse)request.GetResponse();

                dataStream = response.GetResponseStream();
                byte[] bytesToRead = new byte[response.ContentLength];

                int actuallyRead = 0;
                for (; actuallyRead < bytesToRead.Length; )
                    actuallyRead += dataStream.Read(bytesToRead, actuallyRead, bytesToRead.Length - actuallyRead);

                responseFromServer = Encoding.UTF8.GetString(bytesToRead);

                dataStream.Close();
                response.Close();
            }
            catch (Exception e)
            {
                // log exception
            }

            if (responseFromServer != null && !responseFromServer.Equals(""))
            {
                if (responseFromServer.Contains("Auth="))
                {
                    responseFromServer = responseFromServer.Substring(responseFromServer.IndexOf("Auth=") + 5);
                    responseFromServer = responseFromServer.Replace("\n", "");
                }
                else
                    responseFromServer = null;
            }

            return responseFromServer;
        }

If your email is a gmail mail, then ACCOUNT_TYPE is "GOOGLE"; service is "ac2dm"; source is a log of what is your app.  If you get an error, register here again - http://code.google.com/android/c2dm/signup.html, and make sure it's a gmail account.


Sending a message

public static String SendMessage(string userId, string registrationId, string msg)
        {
            if (authToken == null || authToken.Equals(""))
            {
                authToken = getC2dmServerAuthKey();
                if (authToken == null || authToken.Equals(""))
                {
                    authToken = getC2dmAuthToken();
                }
            }

            ServicePointManager.ServerCertificateValidationCallback += delegate(
                                            object sender,
                                            X509Certificate certificate,
                                            X509Chain chain,
                                            SslPolicyErrors sslPolicyErrors)
            {
                return true;
            };

            String responseFromServer = null;

            try
            {
                HttpWebRequest request = (HttpWebRequest)WebRequest.Create(SEND_MESSAGE_URL);
                request.Timeout = 2000;
                request.KeepAlive = true;
                request.ContentType = "application/x-www-form-urlencoded";

                request.Headers.Add("Authorization", "GoogleLogin auth=" + authToken);

                Stream dataStream = null;

                request.Method = "POST";

                StringBuilder postData = new StringBuilder();
                postData.Append("registration_id=" + registrationId);
                postData.Append("&collapse_key=0");
                postData.Append("&data.payload=" + msg);
                postData.Append("&data.userid=" + userId);

                byte[] byteArray = Encoding.UTF8.GetBytes(postData.ToString());
                request.ContentLength = byteArray.Length;


                dataStream = request.GetRequestStream();
                dataStream.Write(byteArray, 0, byteArray.Length);
                dataStream.Close();

                HttpWebResponse response = (HttpWebResponse)request.GetResponse();

                if (response != null)
                {
                    if (response.StatusCode == HttpStatusCode.Unauthorized || response.StatusCode == HttpStatusCode.Forbidden)
                    {
                        // 401 - unauthorized, 403 = forbidden
                        throw new Exception("401");
                    }
                    else if (response.StatusCode == HttpStatusCode.ServiceUnavailable)
                    {
                        // 503 - implement exponential backoff
                        throw new Exception("501");
                    }
                    else if (response.StatusCode == HttpStatusCode.OK)
                    {
                        string updateClientAuth = response.GetResponseHeader("Update-Client-Auth");
                        if (!string.IsNullOrEmpty(updateClientAuth))
                        {
                            authToken = updateClientAuth;
                        }

                        responseFromServer = (new StreamReader(response.GetResponseStream())).ReadToEnd();

                    }
                    dataStream.Close();
                }
                response.Close();
            }
            catch (Exception e)
            {
                if (e.Message != null)
                {
                    if (e.Message.IndexOf("401") > 0)
                    {
                        throw new Exception("401");
                    }
                    else if (e.Message.IndexOf("503") > 0)
                    {
                        throw new Exception("503");
                    }
                }
                throw new Exception(e.Message);
            }

            if (responseFromServer != null && !responseFromServer.Equals(""))
            {
                processSendMessageResponse(responseFromServer, userId, registrationId);
            }

            return responseFromServer;
        }

Note that we are sending the ClientLogin Id in the header for authentication. If you get a 503, implement exponential backoff to avoid getting banned by Google.


Error Processing

public static void processSendMessageResponse(String response, String userId, String registrationId)
        {
            if (response == null || response.Equals(""))
                return;

            if (response.StartsWith("Error="))
            {
                string error = response.Substring(response.IndexOf("Error=") + 6);
                switch (error.ToLower().Trim())
                {
                    case "quotaexceeded":
                        break;
                    case "devicequotaexceeded":
                        break;
                    case "invalidregistration":
                        break;
                    case "notregistered":
                        int s1 = DAL.logServiceStartTime(9999, 0, Thread.CurrentThread.Name + " deleting token");
                        DAL.deleteToken(registrationId);
                        break;
                    case "messagetoobig":
                        break;
                    case "missingcollapsekey":
                        break;
                    default:
                        break;
                }

            }

        }

You may want to record the errors.


10 threaded C2DM Windows Service

using System;
using System.Collections.Generic;
using System.Configuration;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.ServiceProcess;
using System.Text;
using System.Threading;

namespace C2dmService
{
    public partial class C2dmService : ServiceBase
    {

        private static int BACKOFF_FACTOR = 2;
        private static int INITIAL_BACKOFF = 1; // 1 min
        private static int DEBUG_TIMEOUT = 30000;

        public static bool isRunning = false;
        public static bool stopRequested = false;
        private static Object queuelock = new Object();
        private static int NUM_MSGS_PER_THREAD = 500;

        private Queue<ScheduledAlert> mQueue;

        private List<Thread> c2dmThreads = new List<Thread>();

        public C2dmService()
        {
            InitializeComponent();
        }

        protected override void OnStart(string[] args)
        {
            Thread.Sleep(DEBUG_TIMEOUT);

            stopRequested = false;

            while (c2dmThreads.Count < 10)
            {
                c2dmThreads.Add(null);
            }

            if (c2dmThreads != null && c2dmThreads.Count > 0)
            {
                for (int i = 0; i < c2dmThreads.Count; i++)
                {
                    if (c2dmThreads[i] != null)
                    {
                        if (c2dmThreads[i].IsAlive)
                            c2dmThreads[i].Abort();

                        c2dmThreads[i] = null;
                    }

                    c2dmThreads[i] = new Thread(Run);
                    if (c2dmThreads[i].Name == null)
                        c2dmThreads[i].Name = "c2dm Thread " + i;
                    c2dmThreads[i].Start();
                }
            }

            isRunning = true;
        }

        protected override void OnStop()
        {
            stopRequested = true;

            int counter = 0;

            Thread.Sleep(500);

            while (isRunning && counter++ < 60)
            {
                if (c2dmThreads != null && c2dmThreads.Count > 0)
                {
                    for (int i = 0; i < c2dmThreads.Count; i++)
                    {
                        if (c2dmThreads[i] != null)
                        {
                            c2dmThreads[i].Join(500);
                        }
                    }
                }
            }
        }

        public void Run()
        {
            DateTime currentTime = DateTime.Now;

            while (true)
            {
                int counter = 0;
                while (counter++ < 60)
                {
                    if (C2dmService.stopRequested)
                    {
                        C2dmService.isRunning = false;
                        return;
                    }

                    Thread.Sleep(1000);
                }

                executeMsgSchedulerReporting();
            }
        }

        private bool executeMsgSchedulerReporting()
        {
            int backoff = INITIAL_BACKOFF;
            int recordId = -1;

            try
            {
                List<ScheduledAlert> alerts = new List<ScheduledAlert>();

              
                lock (queuelock)
                {
                    if (mQueue == null || (mQueue != null && mQueue.Count <= 0))
                    {
                        mQueue = DAL.getScheduledAlerts();
                    }

                    int numMsgsToFetch = (mQueue.Count > NUM_MSGS_PER_THREAD) ? NUM_MSGS_PER_THREAD : mQueue.Count;

                    for (int i = 0; i < numMsgsToFetch; i++)
                        alerts.Add(mQueue.Dequeue());
                }


                if (alerts.Count > 0)
                {
                    foreach (ScheduledAlert a in alerts)
                    {
                        try
                        {
                            C2dmMessaging.SendMessage(a.Receiver_id.ToString(), a.Token, a.Count.ToString());
                        }
                        catch (Exception e2)
                        {
                            if (e2.Message != null)
                            {
                                if(e2.Message.Equals("401"))
                                {
                                    // renew authkey
                                    C2dmMessaging.renewC2dmAuthToken();
                                }
                                else if (e2.Message.Equals("503"))
                                {
                                    // google c2dm service is unavailable - exponential backoff
                                    Thread.Sleep(backoff * 60);
                                    backoff *= BACKOFF_FACTOR;
                                }
                            }
                        }
                    }
                }

            }
            catch (Exception e)
            {
                EventLog.WriteEntry(C2DM_SERVICE, C2DM_SERVICE + " an error - " + e.ToString(), EventLogEntryType.Error);
            }

            EventLog.WriteEntry(C2DM_SERVICE, C2DM_SERVICE + " has finished.", EventLogEntryType.SuccessAudit);

            return false;
        }

       
    }
}

Tuesday, September 27, 2011

How To Do Android C2DM Push Notifications Part 1

There are two ways to do message notification: a local pull notification service, or a server side push notification service.

This post is about doing push notifications (C2DM).

You need the following:
  • An email account that is authorized to send and recieve C2DM messages. C2DM is currently is beta; You can request an account here - http://code.google.com/android/c2dm/signup.html. (It is best to use a gmail account. I tried to use my own hosted account and it doesn't seem to work.)
  • An application that's already on the market.
  • A server that you can push messages to the C2DM servers.
We will use the following case to develop the C2DM notification.

We have a mobile membership app where users constantly send messages to each other. Users want to be notified on their phones whenever they get a message.

Each application on the phone can request C2DM tokens. Our server will send messages using these tokens. So each user id in our membership site will have a corresponding token (or multiple tokens when a user uses more than one device to access the app).

High Level Overview of How Everything Works together:

Token Registration:
  1. Device requests registration from a C2DM server.
  2. C2DM server sends back a token (or errors) to the device.
  3. Device sends the user id, and the token to our server.
  4. Our server saves the user id, token pair for future use.

Sending a messge:
  1. User A sends a message to user B in our application.
  2. Server pushes this message to the C2DM server. (Ex. Windows Service or a Cron job)
  3. The C2DM server will send the message to the corresponding device. 

Technical Implementation

Token Registration:

In your Android application, you will want to request a token and store it in a persistent storage, (preference object, or sql lite). If the token exists, you don't need to request it. The C2DM server can also renew or deactivate the token anytime. You will also need to have a broadcast Reciever to listen to these events. Note that if you recieve a 503, or other status that indicate server busy, you will need to implement exponential backoff to prevent yourself from getting banned by Google.

Android Manifest


<uses-permission android:name="android.permission.WAKE_LOCK" />
<permission android:name="com.demo.android.permission.C2D_MESSAGE" android:protectionLevel="signature" />
<uses-permission android:name="com.demo.android.permission.C2D_MESSAGE" />
<uses-permission android:name="com.google.android.c2dm.permission.RECEIVE" />

<service android:name=".C2DMReceiver" />

<receiver android:name=".C2DMBroadcastReceiver" android:permission="com.google.android.c2dm.permission.SEND">
    <!-- Receive the actual message -->
          <intent-filter>
              <action android:name="com.google.android.c2dm.intent.RECEIVE" />
              <category android:name="com.demo.android" />
          </intent-filter>
          <!-- Receive the registration id -->
          <intent-filter>
              <action android:name="com.google.android.c2dm.intent.REGISTRATION" />
              <category android:name="com.demo.android" />
          </intent-filter>
</receiver>

Note that C2DM requires Android SDK 2.2 or above. You will want to replace com.demo.android to your app's package name.

Next we will define four classes:
  • C2DMBaseReceiver.java - handles token registration, receiving c2dm messages
  • C2DMBroadcastReceiver.java - listens to C2DM events and starts the C2DMReceiver intentService
  • C2DMmessaging.java - for storing registration id and related data in the shared preference object
  • C2DMreceiver.java - handles token registration, receiving c2dm messages on a higher level

C2DMBaseReceiver.java

package com.demo.android;

import java.io.IOException;

import android.app.AlarmManager;
import android.app.IntentService;
import android.app.PendingIntent;
import android.content.Context;
import android.content.Intent;
import android.os.PowerManager;
import android.util.Log;

/**
 * Base class for C2D message receiver. Includes constants for the strings used
 * in the protocol.
 */
public abstract class C2DMBaseReceiver extends IntentService {
    private static final String C2DM_RETRY = "com.google.android.c2dm.intent.RETRY";

    public static final String REGISTRATION_CALLBACK_INTENT = "com.google.android.c2dm.intent.REGISTRATION";
    private static final String C2DM_INTENT = "com.google.android.c2dm.intent.RECEIVE";

    // Logging tag
    private static final String TAG = "C2DM";

    // Extras in the registration callback intents.
    public static final String EXTRA_UNREGISTERED = "unregistered";

    public static final String EXTRA_ERROR = "error";

    public static final String EXTRA_REGISTRATION_ID = "registration_id";

    public static final String ERR_SERVICE_NOT_AVAILABLE = "SERVICE_NOT_AVAILABLE";
    public static final String ERR_ACCOUNT_MISSING = "ACCOUNT_MISSING";
    public static final String ERR_AUTHENTICATION_FAILED = "AUTHENTICATION_FAILED";
    public static final String ERR_TOO_MANY_REGISTRATIONS = "TOO_MANY_REGISTRATIONS";
    public static final String ERR_INVALID_PARAMETERS = "INVALID_PARAMETERS";
    public static final String ERR_INVALID_SENDER = "INVALID_SENDER";
    public static final String ERR_PHONE_REGISTRATION_ERROR = "PHONE_REGISTRATION_ERROR";

    // wakelock
    private static final String WAKELOCK_KEY = "C2DM_LIB";

    private static PowerManager.WakeLock mWakeLock;
    private final String senderId;


    public C2DMBaseReceiver(String senderId) {
        // senderId is used as base name for threads, etc.
        super(senderId);
        this.senderId = senderId;
    }

    /**
     * Called when a cloud message has been received.
     */
    protected abstract void onMessage(Context context, Intent intent);

    /**
     * Called on registration error. Override to provide better error messages.
     *
     * This is called in the context of a Service - no dialog or UI.
     */
    public abstract void onError(Context context, String errorId);

    /**
     * Called when a registration token has been received.
     */
    public void onRegistered(Context context, String registrationId)
            throws IOException {
        // registrationId will also be saved
    }

    /**
     * Called when the device has been unregistered.
     */
    public void onUnregistered(Context context) {
    }

    @Override
    public final void onHandleIntent(Intent intent) {
        try {
            Context context = getApplicationContext();
            if (intent.getAction().equals(REGISTRATION_CALLBACK_INTENT)) {
                handleRegistration(context, intent);
            } else if (intent.getAction().equals(C2DM_INTENT)) {
                onMessage(context, intent);
            } else if (intent.getAction().equals(C2DM_RETRY)) {
                C2DMMessaging.register(context, senderId);
            }
        } finally {
            // Release the power lock, so phone can get back to sleep.
            // The lock is reference counted by default, so multiple
            // messages are ok.

            // If the onMessage() needs to spawn a thread or do something else,
            // it should use it's own lock.
            mWakeLock.release();
        }
    }

    static void runIntentInService(Context context, Intent intent) {
        if (mWakeLock == null) {
            PowerManager pm = (PowerManager) context
                    .getSystemService(Context.POWER_SERVICE);
            mWakeLock = pm.newWakeLock(PowerManager.PARTIAL_WAKE_LOCK,
                    WAKELOCK_KEY);
        }
        mWakeLock.acquire();

        String receiver = context.getPackageName() + ".C2DMReceiver";
        intent.setClassName(context, receiver);

        context.startService(intent);
    }

    private void handleRegistration(final Context context, Intent intent) {
        final String registrationId = intent
                .getStringExtra(EXTRA_REGISTRATION_ID);
        String error = intent.getStringExtra(EXTRA_ERROR);
        String removed = intent.getStringExtra(EXTRA_UNREGISTERED);

        if (removed != null) {
            // Remember we are unregistered
            C2DMMessaging.clearRegistrationId(context);
            onUnregistered(context);
            return;
        } else if (error != null) {
            // we are not registered, can try again
            C2DMMessaging.clearRegistrationId(context);
            // Registration failed
            //Log.e(TAG, "Registration error " + error);
            onError(context, error);
            if ("SERVICE_NOT_AVAILABLE".equals(error)) {
                long backoffTimeMs = C2DMMessaging.getBackoff(context);

                //Log.d(TAG, "Scheduling registration retry, backoff = " + backoffTimeMs);
                Intent retryIntent = new Intent(C2DM_RETRY);
                PendingIntent retryPIntent = PendingIntent
                        .getBroadcast(context, 0 /* requestCode */, retryIntent,
                                0 /* flags */);

                AlarmManager am = (AlarmManager) context
                        .getSystemService(Context.ALARM_SERVICE);
                am.set(AlarmManager.ELAPSED_REALTIME, backoffTimeMs,
                        retryPIntent);

                // Next retry should wait longer.
                backoffTimeMs *= 2;
                C2DMMessaging.setBackoff(context, backoffTimeMs);
            }
        } else {
            try {
                onRegistered(context, registrationId);
                C2DMMessaging.setRegistrationId(context, registrationId);
            } catch (IOException ex) {
                Log.e(TAG, "Registration error " + ex.getMessage());
            }
        }
    }
}



C2DMBroadcastReceiver.java

package com.demo.android;

import android.app.Activity;
import android.content.BroadcastReceiver;
import android.content.Context;
import android.content.Intent;

public class C2DMBroadcastReceiver extends BroadcastReceiver {
   
    @Override
    public final void onReceive(Context context, Intent intent) {
        C2DMBaseReceiver.runIntentInService(context, intent);
        setResult(Activity.RESULT_OK, null, null);       
    }
}


C2DMmessaging.java

package com.demo.android;

import android.app.PendingIntent;
import android.content.Context;
import android.content.Intent;
import android.content.SharedPreferences;
import android.content.SharedPreferences.Editor;

public class C2DMMessaging {
    public static final String EXTRA_SENDER = "sender";
    public static final String EXTRA_APPLICATION_PENDING_INTENT = "app";
    public static final String REQUEST_UNREGISTRATION_INTENT = "com.google.android.c2dm.intent.UNREGISTER";
    public static final String REQUEST_REGISTRATION_INTENT = "com.google.android.c2dm.intent.REGISTER";
    public static final String LAST_REGISTRATION_CHANGE = "last_registration_change";
    public static final String BACKOFF = "backoff";

    // package
    static final String PREFERENCE = "com.google.android.c2dm";
   
    private static final long DEFAULT_BACKOFF = 30000;

    public static void register(Context context,
            String senderId) {
        Intent registrationIntent = new Intent(REQUEST_REGISTRATION_INTENT);
        registrationIntent.putExtra(EXTRA_APPLICATION_PENDING_INTENT,
                PendingIntent.getBroadcast(context, 0, new Intent(), 0));
        registrationIntent.putExtra(EXTRA_SENDER, senderId);
        context.startService(registrationIntent);
    }

    public static void unregister(Context context) {
        Intent regIntent = new Intent(REQUEST_UNREGISTRATION_INTENT);
        regIntent.putExtra(EXTRA_APPLICATION_PENDING_INTENT, PendingIntent.getBroadcast(context,
                0, new Intent(), 0));
        context.startService(regIntent);
    }

    public static String getRegistrationId(Context context) {
        final SharedPreferences prefs = context.getSharedPreferences(
                PREFERENCE,
                Context.MODE_PRIVATE);
        String registrationId = prefs.getString("dm_registration", "");
        return registrationId;
    }

    public static long getLastRegistrationChange(Context context) {
        final SharedPreferences prefs = context.getSharedPreferences(
                PREFERENCE,
                Context.MODE_PRIVATE);
        return prefs.getLong(LAST_REGISTRATION_CHANGE, 0);
    }
   
    static long getBackoff(Context context) {
        final SharedPreferences prefs = context.getSharedPreferences(
                PREFERENCE,
                Context.MODE_PRIVATE);
        return prefs.getLong(BACKOFF, DEFAULT_BACKOFF);
    }
   
    static void setBackoff(Context context, long backoff) {
        final SharedPreferences prefs = context.getSharedPreferences(
                PREFERENCE,
                Context.MODE_PRIVATE);
        Editor editor = prefs.edit();
        editor.putLong(BACKOFF, backoff);
        editor.commit();
    }

    static void clearRegistrationId(Context context) {
        final SharedPreferences prefs = context.getSharedPreferences(
                PREFERENCE,
                Context.MODE_PRIVATE);
        Editor editor = prefs.edit();
        editor.putString("dm_registration", "");
        editor.putLong(LAST_REGISTRATION_CHANGE, System.currentTimeMillis());
        editor.commit();
    }

    static void setRegistrationId(Context context, String registrationId) {
        final SharedPreferences prefs = context.getSharedPreferences(
                PREFERENCE,
                Context.MODE_PRIVATE);
        Editor editor = prefs.edit();
        editor.putString("dm_registration", registrationId);
        editor.commit();
    }
}


C2DMreceiver.java

package com.demo.android;

import android.content.Context;
import android.content.Intent;
import android.os.AsyncTask;
import android.os.Bundle;
import android.util.Log;

public class C2DMReceiver extends C2DMBaseReceiver {
    public C2DMReceiver() {
        // Email address currently not used by the C2DM Messaging framework
        super(YOUR_ROLE_ACCOUNT_EMAIL);
    }

    @Override
    public void onRegistered(Context context, String registrationId)
            throws java.io.IOException {
        Log.e("C2DM", "Registration ID arrived.");
        Log.e("C2DM", registrationId);

        if (registrationId != null) {
            // send user id and registration id (token) to server
        }
    }
   
    public void onUnregistered(Context context) {
        // unregister user id from server
    }

    @Override
    protected void onMessage(Context context, Intent intent) {
        // Extract the payload from the message
        Bundle extras = intent.getExtras();
        if (extras != null) {
            // this can be the number of unread messages
            String payload = (String) extras.get("payload");           
            NotificationUtil.showPushNotificationAlerts(context, payload);
        }
    }

    @Override
    public void onError(Context context, String errorId) {
        // error processing
    }
   
}

Note that YOUR_ROLE_ACCOUNT_EMAIL is the email you registered for C2DM.

For more information about error handling, you can visit http://code.google.com/android/c2dm/index.html.

This concludes the Android cilent side code. In Part 2 of this series, I will cover the server side implementation where you send out a message to the C2DM server.