An Introduction To IMAP IDLE: Why Yahoo!’s iPhone Push System Isn’t Working So Well

Another thing to consider is the 29 minute window for the connection to be IDLE. I have used both VGSMail and Flexmail 2007 for PocketPC and neither worked properly with default settings. For my carrier and IMAP provider I found through trial and error changing the default from 29 to 3 minutes fixed the problem. Both programs send the IDLE heart beat back to the IMAP sever after a certain amount of time and if the connection has been dropped you will see an authentication attempt in the logs. I kept reducing the IDLE time until I was consistently able to keep the connection open. 3 minutes was the sweet spot. It still uses less power than doing POP3 every 10 minutes, which shows the true resource savings of IMAP IDLE. Keep an eye out for the P-IMAP standard which is designed for mobile devices.

Err, no. The iPhone doesn't support IDLE, only P-IMAP, which Yahoo (and just about no one else) implements.

The 3-minute timeout on IDLE is probably due to a more broken than usual NAT system on your mobile network. A good network shouldn't be timing out connections, and even a poor one should manage more than three minutes.

And P-IMAP is a dead I-D, pushed by Oracle, which ended up being cherry-picked for value for Lemonade, which does use IDLE, and is deployed, as well as being an IETF Proposed Standard.

Now. Your article suggests that IDLE is inherently un-robust - not true. The issue is that with a NAT in the way, a TCP connection may be severed for no good reason. TCP survives the lower layer dropping out just fine. From a technical standpoint, then, a TCP connection running dormant is the ideal solution. There's nothing magical about using TCP and IDLE to provision push notifications, as compared to some other method.

Microsoft's solution relies on repetitive HTTP queries, which cost battery power, and, for many users, actual real money. It's still client provisioned, it's just much worse.

an anonymous poster over at http://blog.twenty08.com/2007/07/01/iphone-push...
claims that the yahoo push solution isn't p-imap or idle.

Has anyone tried to capture traffic between an iphone and the yahoo servers to verify this?

Interesting. I'm going to be following this more. As soon as there's a solution for implementing this on Linux I'm going to implement it and publish the instructions.

There exists a true push mail mechanism, complementary to IDLE, called EMN based on WAP push (over SMS and HTTP, I believe - not sure).

After receiving a notification from the mail server, the WAP server sends an XML message by SMS to the mobile device which then triggers the mail client to eventually FETCH the new message headers or BODY part (according to predefined configuration settings).

EMN is used as an outband notification mechanism in the Lemonade Notification draft.

If you know how to use Kannel, there might be a way to receive messages from an SMTP server, extract the relevant part, construct an EMN XML message and turn it into an HTTP POST - assuming the phone is compatible, of course.