ENTRIES
Welcome to Eric Cheng's online journal! You are not logged in. [ Log in ]
«  :: index ::  »

My email workflow (evolved)

:: Thursday, February 10th, 2011 @ 12:11:26 pm

:: Tags: , ,

I’ve been struggling with email workflow since selling my MacBook Pro. My notebook computer used to be communication central — the (somewhat) mobile device that held my master email archives and the most current version of projects in process. My more powerful Mac Pro was used for heavier projects (media-related), and occasionally received a sync of working files from the MacBook Pro.

I’m more “local” now, but strangely, it has also made me more mobile. At the same time, I’ve lost access to my main email archive when I’m away from my Mac Pro (which has become the new central repository).

In 2005, I started redirecting a copy of all of my email to Gmail. I didn’t really use Gmail at that point, but I liked that I could go online and search my email archives (from 2005 onward) from anywhere. Shortly after, I started using Gmail as an email aggregator for mobile devices: all of my email accounts redirected mail to Gmail, which then redirected a copy to whatever mobile device I happened to be using at the time. It was fantastic back then because Gmail’s collaborative anti-spam filters were so good compared to what others had to offer.

During all this time, I didn’t actually use my gmail account actively. If someone emailed my gmail address, it would get redirected to my mobile device, but my computer mail apps would never see it. This was clearly not ideal because I missed a lot of mail and had to expend effort to try to prevent people from emailing me at Gmail. I’ve since added my gmail address to my list of active email addresses and use it as my main email service.

At work, we use Google Apps for Domains. I use Gmail’s web client, Mail.app on Mac OS X (via IMAP), and Mail on iOS, and religiously maintain a zero-message inbox. This is easy because I am able to subscribe to the “All Mail” label from IMAP clients; because Google favors archive over delete, I can just delete messages from any client Inbox and know that they will remain archived and searchable in “All Mail” both in Gmail and on local clients.

I’d love to implement a similar workflow at home, but there are a couple problems with enabling IMAP access to “All Mail” on my normal account:

  1. My “All Mail” label currently has over 100,000 messages in it and takes up 9GB. Most mail clients do not play well with these numbers.
  2. I also have an organized local mail archive containing over 100,000 messages (pre-2005 email), and searches would bring up duplicate messages.

Also, I hate the idea that my mail archives would be split between Gmail and a local message store. I want the local message store to be complete, but I also require that my local Mail applications stay stable (which they don’t when you have so much email).

So my email workflow goals are:

  1. Have a full archive of email on both Gmail and in Mail.app (local)
  2. Be able to archive messages properly from any e-mail client
  3. Retain good performance in local Mail apps

I think I’ve come up with a solution:

  1. Sync full archives by copying unarchived email into Gmail. The dumb way to do this is to enable access to “All Mail” and copy all of the messages that aren’t already on Gmail into it. IMAP will synchronize the folder and push all the messages to Gmail. However, this is clearly not a good solution since enabling IMAP access to “All Mail” in the first place breaks my mail app because it immediately tries to download 100,000 messages. Instead, I created a new Gmail label called “old” and enabled IMAP access to it. The new label appears as an IMAP folder in Mail, and when I copy messages to it, they are uploaded to Gmail. After I copy all 100,000+ old messages into Gmail, I will remove the “old” label from the system. This will give me full archives in both Gmail and Mail.app.
  2. Automatically label new messages coming into Gmail. I created a new IMAP-enabled label (I called it “~”) and a filter that automatically tags all incoming Gmail. This means that a new message will appear in an IMAP mail client twice: once in the inbox, and once in the “~” IMAP folder. When I’m done with the message, I just delete it from the inbox, and it remains archived in “~”. However, my mail client only retains the message in an IMAP cache folder, so I will have to periodically flush the cache out to a local message store by moving the messages in “~” to a local folder (which automatically removes the “~” label from those messages in Gmail).

Step 1 will take hours, if not days. I have a lot of email, and it has been taking an hour to upload 2000 messages or so. Once all of this is done, I will be able to search my entire email archive from anywhere and delete messages from the inbox without having to worry that they won’t appear on my local Mail.app instance.

I hope it works!

| San Francisco, CA | link | trackback | Feb 10, 2011 12:11:26
  • http://twitter.com/tonyylu Tony Lu

    Hey Eric,

    Sorry to comment on such an old post – just wondering if you have any updates on how this went.  I’m finally coming around to using a local mail client now that I have to travel more for work, and I’m looking to aggregate my local archive.

  • http://echeng.com/ Eric Cheng

    Hey, Tony. The system is working very well. No complaints. :)

ARCHIVES
Journal Home
Where is Eric? (password)
Stuff for Sale
February 2014 (2)
December 2013 (1)
October 2013 (1)
June 2013 (3)
May 2013 (2)
April 2013 (3)
March 2013 (1)
February 2013 (2)
January 2013 (3)
November 2012 (2)
October 2012 (3)
September 2012 (8)
August 2012 (8)
July 2012 (8)
June 2012 (8)
May 2012 (5)
April 2012 (8)
March 2012 (15)
February 2012 (7)
January 2012 (6)
December 2011 (8)
November 2011 (10)
October 2011 (12)
September 2011 (8)
August 2011 (14)
July 2011 (9)
June 2011 (9)
May 2011 (11)
April 2011 (11)
March 2011 (12)
February 2011 (23)
January 2011 (22)
December 2010 (16)
November 2010 (17)
October 2010 (26)
September 2010 (24)
August 2010 (24)
July 2010 (30)
June 2010 (26)
May 2010 (21)
April 2010 (26)
March 2010 (19)
February 2010 (17)
January 2010 (29)
December 2009 (21)
November 2009 (23)
October 2009 (32)
September 2009 (19)
August 2009 (34)
July 2009 (21)
June 2009 (30)
May 2009 (23)
April 2009 (18)
March 2009 (6)
February 2009 (25)
January 2009 (5)
December 2008 (6)
November 2008 (22)
October 2008 (27)
September 2008 (25)
August 2008 (34)
July 2008 (34)
June 2008 (32)
May 2008 (26)
April 2008 (15)
March 2008 (19)
February 2008 (31)
January 2008 (43)
December 2007 (33)
November 2007 (29)
October 2007 (29)
September 2007 (9)
August 2007 (19)
July 2007 (10)
June 2007 (17)
May 2007 (26)
April 2007 (38)
March 2007 (39)
February 2007 (13)
January 2007 (35)
December 2006 (35)
November 2006 (14)
October 2006 (6)
September 2006 (20)
August 2006 (24)
July 2006 (32)
June 2006 (17)
May 2006 (23)
April 2006 (16)
March 2006 (16)
February 2006 (26)
January 2006 (33)
December 2005 (17)
November 2005 (21)
October 2005 (18)
September 2005 (17)
August 2005 (5)
July 2005 (15)
June 2005 (20)
May 2005 (25)
April 2005 (7)
March 2005 (22)
February 2005 (20)
January 2005 (38)
December 2004 (6)
November 2004 (24)
October 2004 (16)
September 2004 (22)
August 2004 (12)
July 2004 (17)
June 2004 (15)
May 2004 (11)
April 2004 (35)
March 2004 (40)
February 2004 (29)
January 2004 (36)
December 2003 (20)
November 2003 (18)
October 2003 (10)
September 2003 (18)
August 2003 (10)
July 2003 (34)
June 2003 (12)
May 2003 (49)
April 2003 (42)
March 2003 (42)
February 2003 (15)
January 2003 (7)
December 2002 (17)
November 2002 (19)
October 2002 (24)
September 2002 (22)
August 2002 (20)
July 2002 (21)
June 2002 (14)
May 2002 (15)
April 2002 (11)
March 2002 (13)
February 2002 (20)
January 2002 (17)
December 2001 (16)
Even Older Journal
Travel Journals

CATEGORIES / TAGS
(25) (2) (1) (3) (1) (1) (1) (6) (2) (3) (11) (8) (3) (1) (1) (4) (2) (4) (2) (1) (6) (1) (1) (1) (6) (2) (1) (1) (1) (3) (1) (5) (1) (1) (23) (1) (1) (1) (1) (1) (14) (1) (10) (1) (1) (2) (1) (1) (1) (27) (6) (3) (2) (4) (4) (1) (1) (41) (11) (12) (4) (38) (1) (3) (2) (4) (1) (1) (1) (1) (2) (1) (1) (1) (1) (1) (10) (25) (8) (3) (2) (3) (2) (1) (5) (1) (1) (2) (1) (1) (14) (1) (5) (1) (1) (5) (43) (1) (1) (1) (3) (24) (1) (1) (1) (1) (5) (1) (4) (1) (1) (10) (1) (3) (1) (1) (1) (1) (6) (5) (1) (1) (1) (3) (1) (3) (1) (1) (1) (69) (4) (3) (7) (3) (1) (16) (6) (1) (29) (1) (7) (1) (4) (4) (4) (1) (1) (1) (1) (1) (1) (1) (10) (4) (4) (2) (1) (89) (14) (1) (2) (79) (2) (2) (1) (1) (1) (1) (1) (1) (3) (2) (3) (1) (1) (24) (3) (5) (4) (1) (2) (1)
MOST POPULAR
Most Popular Posts of All Time


Eric Cheng's RSS Journal Journal RSS
Eric Cheng's RSS Journal Comments RSS

proudly powered by wordpress
script exec time: 0.84s
i hate computers.