An Introduction To IMAP IDLE: Why Yahoo!’s iPhone Push System Isn’t Working So Well

By Daniel Miessler on July 2nd, 2007: Tagged as Protocols | Sysadmin | Yahoo | iPhone

I just finished a short write-up on IMAP IDLE — a system that’s often called an email “push” technology. Here are a couple of excerpts:

IDLE is not — according to my definition above — a true push technology. IDLE actually requires an active connection in order to work, and that connection cannot be initiated by the server.
[Update]: Yahoo! uses P-IMAP, not IDLE to implement “push”…[ Link: An Introduction to IMAP IDLE ]

  • Digg
  • Reddit
  • StumbleUpon
  • description
  • Google
  • del.icio.us
  • TwitThis
  • Facebook
  • MySpace
  • E-mail this story to a friend!
  • Print this article!

Viewing 6 Comments

    • ^
    • v
    Another thing to consider is the 29 minute window for the connection to be IDLE. I have used both VGSMail and Flexmail 2007 for PocketPC and neither worked properly with default settings. For my carrier and IMAP provider I found through trial and error changing the default from 29 to 3 minutes fixed the problem. Both programs send the IDLE heart beat back to the IMAP sever after a certain amount of time and if the connection has been dropped you will see an authentication attempt in the logs. I kept reducing the IDLE time until I was consistently able to keep the connection open. 3 minutes was the sweet spot. It still uses less power than doing POP3 every 10 minutes, which shows the true resource savings of IMAP IDLE. Keep an eye out for the P-IMAP standard which is designed for mobile devices.
    • ^
    • v
    Err, no. The iPhone doesn't support IDLE, only P-IMAP, which Yahoo (and just about no one else) implements.
    • ^
    • v
    The 3-minute timeout on IDLE is probably due to a more broken than usual NAT system on your mobile network. A good network shouldn't be timing out connections, and even a poor one should manage more than three minutes.

    And P-IMAP is a dead I-D, pushed by Oracle, which ended up being cherry-picked for value for Lemonade, which does use IDLE, and is deployed, as well as being an IETF Proposed Standard.

    Now. Your article suggests that IDLE is inherently un-robust - not true. The issue is that with a NAT in the way, a TCP connection may be severed for no good reason. TCP survives the lower layer dropping out just fine. From a technical standpoint, then, a TCP connection running dormant is the ideal solution. There's nothing magical about using TCP and IDLE to provision push notifications, as compared to some other method.

    Microsoft's solution relies on repetitive HTTP queries, which cost battery power, and, for many users, actual real money. It's still client provisioned, it's just much worse.
    • ^
    • v
    an anonymous poster over at http://blog.twenty08.com/2007/07/01/iphone-push...
    claims that the yahoo push solution isn't p-imap or idle.

    Has anyone tried to capture traffic between an iphone and the yahoo servers to verify this?
    • ^
    • v
    Interesting. I'm going to be following this more. As soon as there's a solution for implementing this on Linux I'm going to implement it and publish the instructions.
    • ^
    • v
    There exists a true push mail mechanism, complementary to IDLE, called EMN based on WAP push (over SMS and HTTP, I believe - not sure).

    After receiving a notification from the mail server, the WAP server sends an XML message by SMS to the mobile device which then triggers the mail client to eventually FETCH the new message headers or BODY part (according to predefined configuration settings).

    EMN is used as an outband notification mechanism in the Lemonade Notification draft.

    If you know how to use Kannel, there might be a way to receive messages from an SMTP server, extract the relevant part, construct an EMN XML message and turn it into an HTTP POST - assuming the phone is compatible, of course.
 

Trackbacks

(Trackback URL)

close Reblog this comment
Powered by Disqus · Learn more
blog comments powered by Disqus

My Twitter Microblog

    Discovered Content


    About dmiessler.com

    This site is an avatar for my own self-assigned life purpose--an attempt to model the world in the most accurate way possible, and to do so without bias or fear of unpleasant truth. I desire to develop, articulate, and perpetually improve models of how things work, and then to use that understanding to increase happiness and reduce suffering. I seek those on similar paths and thrive on sharing an appreciation of the interesting and beautiful with others.



    Blog
    Study
    Writing
    Infosec
    Technology
    Politics

            

    Top Original Content


    Security & Technology

    Philosophy & Science

    Culture & Society

    Blog Archives