Interacting with a JMAP API

IMAP is one of the predominant email protocols out there, but it’s a) very old and b) somewhat painful to work with. To address some of these problems, JMAP was designed as IMAP-for-the-21st-ceentury, particularly for client usage.

Although it’s an official IETF standard, it doesn’t seem to have got that much traction - but is being used and supported by Fastmail, who also happen to be my long-term (nearly 20 years) email provider of choice.

That means I needed to get to grips with it in order to hack together my overly-complicated blog-content-from-markdown hack.

The problem with not being widely-adopted is the dearth of actual practical examples for ~~copy-and-pasting~~ inspiration. What follows is my get-things-up-and-running process, hopefully as a springboard for more complex usecases.

Basics

You interact with the API by sending HTTP POST requests with an authorisation header and a JSON payload. The payload contains the command you want to run along with any necessary criteria such as search terms.

Fastmail’s authorisation requires an Authorization header with a value in the format Bearer <API token>)
The API token needs to be created on the Fastmail dashboard
The method calls are sent as JSON text with an application/json type in the POST body

Command format

The POST body has two parts:

a using array, which doesn’t change from method to method and defines the spec being used
a methodCalls array which contains the actual methods you want to call, along with their parameters

The method calls have a format in the shape of object/action - objects are Mailbox, Email, Thread and so on; actions are verbs like get, set, query etc.

The main commands we’ll be using are Mailbox/get, Email/query and Email/get.

The overall shape of the body looks like this:

{
  "using": [
    "urn:ietf:params:jmap:core",
    "urn:ietf:params:jmap:mail"
  ],
  "methodCalls": [
    <calls go here>
  ]
}

Getting your account ID

The first step is to figure out your Fastmail account ID. This requires making an authenticated GET request to https://api.fastmail.com/.well-known/jmap, which returns a JSON response containing details of your account and what rights the API key allows.

Buried in middle somewhere is a primaryAccounts key, which should look something like this:

"primaryAccounts": {
    "urn:ietf:params:jmap:core": "u123456ab",
    "urn:ietf:params:jmap:mail": "u123456ab"
  },

Make a note of the value of the <xxx>:mail key, you’ll need it for all further requests.

Request a list of folders

Most processes are going to involve getting a list of available messages in a specific folder, and the prerequisite for that is a folder ID.

The request

Let’s start by getting a list of folders (aka mailboxes in JMAP-speak). Send this JSON body in a POST request:

{
  "using": [
    "urn:ietf:params:jmap:core",
    "urn:ietf:params:jmap:mail"
  ],
  "methodCalls": [
    [
      "Mailbox/get",
      {
        "accountId": "u123456ab",
        "ids": null
      },
      "abc"
    ]
  ]
}

Breaking down the methodCalls section, there’s three parts:

the command we’re sending, in this case Mailbox/get
the parameters for this command, in this case the accountId and the folders ids we want to match. By sending null, the server will respond with a list of all folders/mailboxes
the trailing abc is effectively a variable name - the results of this Mailbox/get command will be available to subsquent commands by referencing the abc variable, which allows you to create composite commands that are chained together

The response

The respose is a blob of JSON containing the methodResponse array. That in turn contains a list of folders and their various properties - the keys we’re interested in look like this:

{
  "methodResponses": [
    [
      "Mailbox/get",
      {
        "list": [
          {
            ...
            "name": "Inbox",
            "id": "abcdef1234",
            ...
          },
        ...
      },
      "abc"
    ]
  ],
  "latestClientVersion": "",
  "sessionState": "cyrus-0;p-e21c23889a;s-6606885adc92256b"
}

The name key contains the human-readable folder name, and the id is the corresponding folder ID

Getting a list of emails

Retrieving a specific email is a two-stage process: finding its ID, then retrieving the email itself:

Retrieving a list of emails in a specific folder

This uses the Email/query command with a filter (this example also throws in a sort order for good measure). It’s the same process - make a POST request with the auth headers, and send over the JSON payload in the body.

This is the full payload:

{
  "using": [
    "urn:ietf:params:jmap:core",
    "urn:ietf:params:jmap:mail"
  ],
  "methodCalls": [
    [
      "Email/query",
      {
        "accountId": "u123456ab",
        "filter": {
          "inMailbox": "abcdef1234"
        },
        "sort": [
          {
            "property": "receivedAt",
            "isAscending": false
          }
        ],
        "limit": 500
      },
      "xzy"
    ]
  ]
}

Breaking this down, we have:

a filter, which searches in a specific mailbox with ID abcdef1234
a sort order, which in this case is by property receivedAt and most-recent first
a limit, in case there’s a lot of results. If that happens, you can start using paging, but that’s beyond the scope of this simple example.

The result is a JSON response which looks like this:

{
  "methodResponses": [
    [
      "Email/query",
      {
        "ids": [
          "Mcae60028353b175d88882abe",
          "M999a6d73062302c1592daebe",
          "M38493717a69686cbb4fc4d6f",
        ],
        "sort": [
          {
            "isAscending": false,
            "property": "receivedAt"
          }
        ],
        "filter": {
          "inMailbox": "abcdef1234"
        },
        "position": 0,
        "collapseThreads": false,
        "queryState": "1871939:0",
        "canCalculateChanges": true,
        "accountId": "u123456ab",
        "total": 3
      },
      "xyz"
    ]
  ],
  "latestClientVersion": "",
  "sessionState": "cyrus-0;p-e21c23889a;s-6606885adc92256b"
}

The first and most useful part is the ids array, which contains a list of IDs for actual emails. You also get a confirmation of the sort order and filter that was used, and some metadata about the query itself.

position relates to the paging if that took place; collapseThreads relates to how message threads are handled, and the total value is the number of records returned which can be useful when processing the results.

Retrieving a specific email

Now that we’ve got message IDs, we can finish off by retrieving a specific one. This uses the Email/get method, to which you pass a specific message ID(s):

{
  "using": [
    "urn:ietf:params:jmap:core",
    "urn:ietf:params:jmap:mail"
  ],
  "methodCalls": [
    [
      "Email/get",
        {
          "accountId": "abcdef1234",
          "properties": null,
          "ids": [
            "Mcae60028353b175d88882abe"
          ]
        },
      "pqr"
    ]
  ]
}

The result (assuming the message exists) should be a lump of JSON that contains the message itself along with a pile of metadata:

{
  "methodResponses": [
    [
      "Email/get",
      {
        "accountId": "abcdef1234",
        "state": "1872326",
        "notFound": [],
        "list": [
          {
            "attachments": [],
            "mailboxIds": {
              "abcdef1234": true
            },
            "size": 91829,
            "htmlBody": [
              {
                "language": null,
                "location": null,
                "blobId": "G67f2b026af5a35d77985d0dd57edb93364d2736",
                "name": null,
                "type": "text/html",
                "cid": null,
                "size": 61902,
                "charset": "utf-8",
                "partId": "2",
                "disposition": null
              }
            ],
            "blobId": "G438f25a1713b0b3445b3ecd76a2d33d43337856",
            "from": [
              {
                "email": "sz-magazin@newsletter.sueddeutsche.de",
                "name": "SZ-Magazin Das Beste"
              }
            ],
            "textBody": [
              {
                "disposition": null,
                "size": 13300,
                "charset": "utf-8",
                "partId": "1",
                "cid": null,
                "type": "text/plain",
                "name": null,
                "language": null,
                "location": null,
                "blobId": "G05569b6bd4104d589a7efdece3d2f5d7dcbfe1e"
              }
            ],
            "inReplyTo": null,
            "id": "M438f25a1713b0b4451b3ecd",
            "sentAt": "2024-03-30T07:59:53+01:00",
            "to": [
              {
                "name": null,
                "email": "tim@duckett.de"
              }
            ],
            "receivedAt": "2024-03-30T06:59:59Z",
            "sender": null,
            "hasAttachment": false,
            "threadId": "Td27cfc407a520026",
            "keywords": {
              "$x-me-annot-2": true,
              "$ismailinglist": true,
              "$canunsubscribe": true
            },
            "preview": "Außerdem: Warum das Wort »später« so gefährlich ist Sollte der Newsletter nicht korrekt angezeigt werden, klicken Sie bitte hier Illustration: iStock / by Malte Mueller",
            "subject": "Lust auf ein neues Leben?",
            "cc": null,
            "bcc": null,
            "messageId": [
              "0.0.1A.354.1DA826FDE08E6A0.0@uspmta120023.emarsys.net"
            ],
            "replyTo": null,
            "bodyValues": {},
            "references": null
          }
        ]
      },
      "a"
    ]
  ],
  "latestClientVersion": "",
  "sessionState": "cyrus-0;p-e21c23889a;s-6607b80d3ff0a0fe"
}

This is an example of an HTML-formatted message - there’s a text preview which is contained in the preview key, but there’s one more step required to get the content of the message itself. This is available in two formats:

HTML, which is referred to by the blobId contained in the htmlBody section
text, which is referred to by the blobId contained in the textBody section

Confusingly, there’s a third blobId in the list section which refers to the message itself in case you want retrieve that as a block of text.

Retrieving the contents of a specific email

This final step is different to the others, because it’s a straight-forward GET request:

GET https://www.fastmailusercontent.com/jmap/download/<accountId>/<blobId>/<filename>?type=<type>

The parameters here are:

accountId - the account ID as used before
blobId- the ID of the specific blob you want, either HTML or text
filename - this appears to have no effect as far as I can tell, but needs to be present; you can send any string
type - the MIME type in which you want the content encoded. This makes no practical difference in terms of the content returned, but does show up in the content-type response header in case your code needs this. In this example, it makes most sense to send application/html

Getting email attachments

Although that final GET request is the end point if what you want is the message, my usecase was slightly different and I wanted any attachments.

The good news here is that the process of getting attachments is the same as the process of getting email contents - you send a GET request with a blob ID.

Attachment blob IDs will show up in the attachments array at the top of the email payload:

{
  "methodResponses": [
    [
      "Email/get",
      {
        "accountId": "abcdef1234",
        "state": "1872326",
        "notFound": [],
        "list": [
          {
            "attachments": [
              "G67f2b026af5a35d77985d0dd45edb93364d2736"
            ],
            "mailboxIds": {
              "abcdef1234": true
            },
            <SNIP>
}

Grab the blob ID(s) from the attachments array, feed these into the GET requests, and you can download and process attachments as needed.

Finishing up

This is only the minimal basics needed to get messages and attachments; there’s a lot more that the protocol can do. The docs are here, and you can use that information to extend the examples above to build a completely-functional email client if that’s what you need.

There’s a Postman collection of the relevant API calls here.

Interacting with a JMAP API - a rapid-start