My MacBook is long out of warranty and its battery seems to be decaying, so I'm trying to move my personal computer setup to the T430s I've had sitting around as a backup, running Pop!_OS. (Is 2021 the year of the Linux desktop?)

I thought I would use this opportunity to try using Emacs to manage my email, and read a few tutorials for doing that setup. These are my notes, to help remember how I configured this house of dry spaghetti.

Create folders

Trying to keep things clean, I created a separate fastmail folder within a new mail folder, in case I wanted to have another folder later for other email accounts. 

mkdir -p ~/mail/fastmail

Password management

Having two factor authentication turned on for my Fastmail account means I need to generate an app password.

Since I'm using Lastpass anyway, I stored the app password there and I installed the lastpass-cli package. After logging in, I could retrieve the password with:: 

lpass show --password <identifier for mail service>

This will work fine, but the Lastpass CLI client will sometimes timeout, and we'll need to log in again, with a graphical prompt for the password. I cam across this tutorial with a lpass-wrapper.sh script to handle this case: 

#!/bin/sh
# lpass-wrapper.sh

username='$username'

status=$(lpass status)

if [ $? -ne 0 ]
then
    if [ "$status" = 'Not logged in.' ]
    then
    # Make sure DISPLAY is set
    DISPLAY=${DISPLAY:-:0} lpass login "$username" 1>&2
    else
    echo "Lastpass error: $status" 1>&2
    exit 1
    fi
fi

lpass $@

Saving this script somewhere on the path (like ~/.local/bin) means we can now get the password with: 

lpass-wrapper.sh show --password <identifier for mail service>

when the Lastpass client has timed-out, we'll automatically get a prompt for the Lastpass master password to log back in.

retrieving emails

There is no shortage of programs to fetch email from IMAP servers. The notmuch documentation (discussed later) provides a few examples, the first two of which were offlineimap and mbsync.

offlineimap

Many tutorials that I found online use OfflineIMAP, so it looked like the right choice. Unfortunately, manually writing some configuration files based on some tutorials, I got a Python NullPointerException without explanation and which turned out to be due to my misunderstandings of how to point offlineimap to SSL certificates. After fixing that path, I encountered a str vs bytes error, which revealed that OfflineIMAP is written in Python 2, and arguments about porting to Python 3 have caused a split in their community.

I have no snake in this fair, but I don't especially want to have an extra dependency on Python 2, which isn't part of the current standard installation of Ubuntu 20.10. I also encountered other posts complaining about the relatively slow performance of offlineimap, so I'll set this option aside for now and try it again if the others don't work out.

mbsync

mbsync looks like it may not have been updated in a few years, but it seems to still be working ok for now.

My .mbsyncrc was based on these instructions, with tweaks to the PassCmd setting: 

# First section: remote IMAP account
IMAPAccount fastmail
Host imap.fastmail.com
Port 993
User <username>
PassCmd "lpass-wrapper.sh show --password <identifier for mail service>"
SSLType IMAPS
SSLVersions TLSv1.2

IMAPStore fastmail-remote
Account fastmail

# This section describes the local storage
MaildirStore fastmail-local
Path ~/mail/fastmail/
Inbox ~/mail/fastmail/Inbox
# The SubFolders option allows to represent all
# IMAP subfolders as local subfolders
SubFolders Verbatim

# This section a "channel", a connection between remote and local
Channel fastmail
Master :fastmail-remote:
Slave :fastmail-local:
Patterns *
Expunge None
CopyArrivalDate yes
Sync All
Create Slave
SyncState *

With this configuration, running: 

mbsync -a

downloaded my mail to ~/mail/fastmail folder. Here, -a stands for "all defined channels". I could alternatively run mbsync fastmail to operate the fastmail channel, which is the only currently defined channel.

indexing and tagging emails and emacs clients

Some of the Emacs clients for mail that I've come across are notmuch and mu4e, both of which seem to be preferred over GNUS. What I didn't understand was that these are both actually frontends to email indexing software, which is a nice bonus for fast searches.

notmuch

notmuch was the first frontend that I tried, following this guide. It indexed the 6,000 messages I had in my Fastmail account pretty quickly. 

sudo apt install notmuch

notmuch setup walks through an interactive setup for generating a .notmuch-config file, which was nice. I also edited the notmuch hooks in ~/mail/.notmuch/hooks/pre-new to add a call to retrieve mail using mbsync -qq fastmail, kind of like the checkmail.sh file in this tutorial.

notmuch emacs mode

notmuch comes with an Emacs frontend, which is why I started using it in the first place. It works pretty nicely. I configured it with:

I put my mail configuration stuff into an init-mail.el file, which includes: 

(use-package notmuch
  :disabled
  :ensure nil
  :load-path "/usr/share/emacs/site-lisp/elpa-src/notmuch-0.31/"
  :bind (("<f12>" . 'mu4e))
  :commands (notmuch)
  :config
  (add-hook 'notmuch-hello-mode-hook
            (lambda ()
              (display-line-numbers-mode 0)))
  :custom
  (notmuch-fcc-dirs "~/mail/fastmail/Sent")
  (notmuch-crypto-process-mime t) ; Automaticlaly check signatures
  )

Then in my init.el file: 

(use-package init-mail
  :ensure nil
  :custom
  (user-mail-address "<my email address here>")
  (user-full-name "<my name here>")
  (mail-envelope-from 'header)
  (mail-specify-envelope-from t)
  (mail-user-agent 'message-user-agent)
  (sendmail-program "~/.local/bin/msmtp-enqueue.sh")
  (send-mail-function 'sendmail-send-it)
  (smtpmail-queue-dir "~/mail/queue/")
  (message-directory "~/mail/fastmail/Drafts")
  (message-send-mail-function 'sendmail-send-it)
  (message-sendmail-envelope-from 'header)
  (message-kill-buffer-on-exit t)
  )

As it turned out, notmuch is built around tagging and showing emails matching those tags. I can see how this would be a good fit for Gmail (and I might revisit notmuch if I start using gmail through this set up too), but my Fastmail account uses a more traditional folder-based setup, and I couldn't easily figure out how to get notmuch to show me a view corresponding to my current inbox.

afew

It turns out that notmuch doesn't do much, and relies on something like afew, which can apply a chain of filters to further apply tags to emails that can be read by the notmuch client for further sorting your emails. i tried this out and copied and pasted this into ~/.config/afew/config: 

#default filter chain
[SpamFilter]
[KillThreadsFilter]

[SentMailsFilter]
sent_tag = sent
[ArchiveSentMailsFilter]

[InboxFilter]

This helped a bit, but still wasn't what I was expecting.

maildir-utils

Looking at the other options, maildir-utils or mu seems to be a popular choice too, so why not give it a try? 

sudo apt install maildir-utils
mu init -m ~/mail/fastmail --my-address=<my email address here>
mu index

Seemed to index the emails about as quickly as notmuch. Also started a background process. Do I need to restart it when I reboot? Hmm.

mu4e

Turns out the Emacs frontend mu4e is not automatically installed with maildir-utils: 

sudo apt install mu4e

Here's the Emacs configuration code, which I also put into init-mail.el: 

(use-package mu4e
  :ensure nil
  :load-path "/usr/share/emacs/site-lisp/mu4e/"
  :config
  (add-to-list 'mu4e-bookmarks
               '(:name "Inbox"
                 :key ?i
                 :query "maildir:/Inbox"))
  :custom
  (mu4e-get-mail-command "mbsync -qq fastmail")
  (mu4e-headers-date-format "%d-%m-%Y %H:%M")
  (mu4e-change-filenames-when-moving t)
  (mu4e-drafts-folder "/Drafts")
  (mu4e-refile-folder "/Archive")
  (mu4e-sent-folder "/Sent")
  (mu4e-trash-folder "/Trash")
  (mu4e-update-interval 300)
  (mu4e-headers-fields '((:human-date . 20)
                         (:flags . 6)
                         (:from . 22)
                         (:maildir . 8)
                         (:subject))))

This behaves more like what I expected. I added the bookmark for the Inbox folder, because otherwise I would have to manually "jump" to that folder, and Inbox isn't the first on the list.

I also had to set the individual folders because mu4e defaults to lowercase names like "drafts" and "sent".

I had some issues of the form Maildir error: duplicate UID NN. I found this post by someone who had the same issue, and found that it was because mu4e wasn't renaming files when moving, which causes problems for mbsync. This was fixed by setting mu4e-change-filenames-when-moving to t.

Wanderlust

I haven't tried it yet, but another option would be to avoid downloading emails altogether and, instead, use an IMAP client in Emacs that just talks to a remote IMAP server. Wanderlust does this.

sending emails

msmtp

Following these instructions again, I set up msmtp to send mail. The tls_fingerprint is the SHA-256 fingerprint from Fastmail's page about certificates. 

# Set default values for all following accounts.
defaults
tls on
tls_starttls off
tls_fingerprint AF:01:8E:7F:FF:36:61:06:C6:F9:D4:38:D1:4E:08:98:78:C8:27:C5:E5:C0:A3:97:49:37:F2:76:90:B1:27:59
logfile ~/.msmtp.log

# Fastmail
account fastmail
host smtp.fastmail.com
port 465

from <my from email address here>
auth on
user <my email provider username here>
passwordeval "lpass-wrapper.sh show --password <identifier for mail service> </dev/null"

# Set a default account
account default : fastmail

msmtpqueue scripts

The instructions refer to using msmtpqueue scripts from the Arch Wiki. As it turns out, these were already installed at /usr/share/doc/msmtp/examples/msmtpqueue/ when i installed msmtp. I copied these to ~/.local/bin as well. The msmtp-enqueue.sh script is referred to in the above configuration of the sendmail-program variable in the init-mail use-package declaration.

AppArmor woes

Trying ta actually send an email raised a permission denied error. Searching online, I found a relevant StackOverflow question and answer.

So this was solved with: 

sudo ln -s /etc/apparmor.d/usr.bin.msmtp /etc/apparmor.d/disable/
sudo apparmor_parser -R /etc/apparmor.d/usr.bin.msmtp