mrb's blog

A Decentralized API to Personal Information

Keywords: decentralization email standard

"This blew my mind. Why the f*ck isn't this being done yet?"Reddit comment

An idea hit me: email is such a pervasive tool used by so many people and supported by so many software stacks that it is the ultimate vendor-neutral platform to build programmatic services between applications and persons: automatic exchange of contact information, PGP key, Bitcoin address, automating two-factor authentication, and so much more!

Here is an example: imagine you sign up on a web forum as bob@example.com. The site could send you a "special hidden email" to request your preferred avatar image, nickname, language, and timezone. And your mailbox automatically replies (like a vacation autoresponder) with the information formatted in a specific way, so that there is no need to re-enter the same information over and over on every web forum! (Like Gravatar but decentralized.) This would happen completely transparently, behind the scenes, without you even seeing the automatic email exchange.

Another example: encrypted email has failed to see wide adoption. Why? Because none of the setup steps are automated, so its use is cumbersome. First you have to ask the recipient if he even knows what PGP is, then if he wants to use encrypted email or not, and where his key can be obtained (key servers help but not always.) Instead imagine if the first time you composed an email to a new friend and hit "send", the email application would first send a "special hidden email" to your friend asking for his PGP key. Your friend's mailbox automatically replies "here is my key" (or "no key was configured"). And your mailbox receives the attached key, uses it to encrypt the email you composed, and saves it for future needs! With email encryption being negotiated automatically like that, its use would be more widespread.

This concept is what I call programmable email: email requests sent and replied to, automatically, in order to exchange personal information in a well-defined format without relying on a central database. In other words, this is a decentralized application programming interface (API) to personal information.

Programmable email would solve quite a few problems. Many other examples come to mind:

  • A request sent to a recipient's professional mailbox could return his business information: name, employer, job title, phone, address, corporate head shot. So you meet a person at a conference, and as you add his email address to your contact list, you magically see all the other fields being autocompleted with his full contact information!
  • A request could return the recipient's availability, according to his/her calendar, to help find appropriate times to schedule meetings. (My friend Bertrand came up with this scenario.)
  • A request could return the recipient's resume. No need to rely anymore on closed platforms like LinkedIn or Monster!
  • A request could return the recipient's GPS location. You arrive at a restaurant, sit at a table waiting for your brother to meet you there. You request your brother's location. His smartphone's mail app sees the request and replies, allowing you to see he is 10 min away. Great, you avoided texting/calling him while he was driving!
  • A request could return your medical history (this one would probably require manual approval as it is sensitive.) For example your doctor would type your email address in his computer, you see and approve the request on your smartphone, and he would receive your medical history, allergy information, or whatever information you configured to be returned (your choice).
  • A request could return your Bitcoin address. For example your dad might send 0.1 bitcoins to you by merely typing "bob@example.com" as the recipient in his Bitcoin client. This is much easier than copy/pasting a 30+ character long Bitcoin address.
  • A request could help confirm a two-factor authentication code in a more streamlined way. Often, authentication codes are read on a smartphone: HTML emails must be manually opened, take a lot of screen space to be displayed, and either a link is clicked or a code is read and retyped somewhere else. This is not very user-friendly. But with programmable email, because the format is machine-parsable, a smartphone would know this is an authentication request, so it could display a short notification in the status bar with an "approve/deny" button. Clicking "approve" would either send an automated email response, or simulate a link click (an HTTP GET request sent in the background) with no further user interaction needed, no waiting for a web page to load.

Some of the scenarios described in my examples are possible today using proprietary services, like LinkedIn or Gravatar. But what makes programmable email special is that it is a concept that is open, free to use, decentralized, vendor-neutral, and relying on a worldwide standard, email (SMTP), a technology that has the deepest market penetration compared to pretty much anything else. So its potential userbase is much larger. Indeed not everybody is on LinkedIn or Gravatar, but everybody has email.

Furthermore, LinkedIn, Gravatar, and other proprietary services can shut down, or disappear, or start charging money, or fail at any moment's notice. But programmable email is fundamentally resilient: it is a technology that will always exist and always be free.

All that is needed to make programmable email a reality is a specification on what data format the email requests and replies need to follow, and it can be implemented by many different applications and services while all being compatible with each other. I have a live demonstration available (see FAQ) where I used textual keywords in the subject and JSON data in the body.

What could be the disadvantages of programmable email? I can think of one: technically email is not instant. It takes a few seconds to transit between major webmail providers, or longer to second-class domains. This is not a problem for many of the usages I describe, but could be slightly annoying for some. What else?

I think programmable email could be amazing :-) Let me know what you think.

FAQ

1. Do you have a demonstration available?

Yes! I did it with Gmail, using filters and canned responses. The filters look for certain keywords in the subject like "autoresponder:fullname" and reply to them with a canned response which is JSON-formatted data. I created these filters on the test address pm33at@gmail.com. Try it out, email it!

  • autoresponder:online_identity returns information typically needed by a web forum: avatar image, language, etc
  • autoresponder:fullname returns a full name
  • autoresponder:phone returns a phone number

The key point to realize is that programmable email can be used right now, by leveraging features already provided by many webmail providers like filters and automatic responses. It is quite technical to set up, but in the future we could imagine webmails implementing easy-to-use configuration options. For example there could be a settings page with a checkbox "enable programmable email" and fields to define your avatar image, nickname, etc.

2. Would programmable emails pollute my inbox?

No, they could be hidden and filed away in separate folders, like spam. For example my filters on pm33at@gmail.com automatically archive the emails.

3. What about initial adoption, and interfacing with mailboxes not compatible with programmable email?

A great aspect of programmable email is that it downgrades surprisingly gracefully.

Example 1: a request for a PGP key should also put human-readable text in the subject and mail body ("Can you send me your PGP key?") If the mailbox of the recipient does not support programmable email, the recipient would at least see the email in the inbox. If he replies to it manually, the application that made the request will receive the key, as if the mailbox truly implemented programmable email!

Example 2: if you request the GPS location, the human-readable text could be "Where are you?". If you request it from your grandma and she happens to have a device without GPS capabilities, she would see the mail and perhaps reply "I am at my house", giving at least some information back to you. In these days where software like Google Maps can understand the voice command "navigate home" as referring to your address, it is even plausible that software could understand that your grandma saying "I am at my house" refers to her address.

4. Is sharing so much information over programmable email a recipe for privacy violation disasters?

It is no more dangerous than what people already share on their web home page or social media. It would be up to you to decide how to configure programmable email. The test address pm33at@gmail.com only shares an online identity, full name, and phone, because this could be all this person is comfortable sharing. If you would like nobody to be able to request a specific piece of information about you, do not configure it. Or maybe you would like only people from your university to be able to request it, so mailboxes could implement a feature where you could allow "*@university.edu" to access the information. Or maybe you would like to allow anybody to request it, but only after manually approving the request. In this case the email application could display a special notification to approve/deny each request coming in. Furthermore, the information exchanged over programmable email could be encrypted with PGP (because programmable email itself would drive wider adoption of PGP thanks to automatic exchange of keys.)

5. How would PGP keys exchanged over programmable email be secure?

It is true that naively implemented there would be some vectors of attack: plaintext SMTP is often used, and SMTP over TLS is usually susceptible to MitM attacks due to certificate validation being less strict. These facts allow, for example, a malicious Internet Service Provider to alter the attached PGP key before it reaches your mailbox. As a solution, one could implement an open framework similar to certificate transparency to monitor and audit PGP keys distributed over programmable email.

6. What about people spamming programmable email requests?

The same anti-spam technologies that apply to regular email would apply to programmable email. A spammer's requests would be flagged as spam, and would be ignored.

Comments

Jim Conte wrote: "autoresponder:fullname"

Instead of having the command in the subject, I'd prefer to keep it within the body, in the JSON data, that way, a practically infinite number of commands can be listed within just one email, which then just receives one email response, with all the data requested. (i.e. GPS location AND medical history).

I think subject lines are limited to a certain number of characters, aren't they?
15 Jul 2014 14:24 UTC

mrb wrote: Jim: yes I think multiple requests per email should be allowed. Per RFC 5322 the subject header (like any header) can spread multiple lines eg. "Subject: line1\r\n line2\r\n line3\r\n ...". So in theory its length is unlimited, but I am sure that some buggy SMTP implementations/services have low limits. So yes moving the commands to the body would help. I just want to make sure that accidental triggering of the commands happen rarely. So maybe commands should be required to be on the first non-blank lines of the mail body.

I was also thinking of placing the commands in a custom header ("X-Programmable-Email: fullname") but IMHO it is crucial to help adoption that we design programmable email in a way that commands can be easily tested and triggered, even manually, by people who just want to play with it, even from a webmail that does not allow setting custom headers.
15 Jul 2014 20:14 UTC

Steve wrote: Have you seen the WebFinger protocol? It seems like it's capable of similar things without relying on sending & receiving email.

http://tools.ietf.org/html/rfc7033
02 Sep 2014 18:51 UTC

Hugo Rodger-Brown wrote: Great idea - esp. given that access to an email account is becoming a de facto proof of identity (via password resets - if you 'own' the email account, you own the identity).

Google has started experimenting with in-inbox actions (https://developers.google.com/gmail/actions/actions/actions-overview) to try and extend email's utility - but this is a much simpler, x-platform idea.

I like it.
03 Sep 2014 11:05 UTC

hn:arethuza wrote: Great idea

What about treating the subject line like the first line of an HTTP request with
03 Sep 2014 11:16 UTC

mrb wrote: Steve: I was not aware of WebFinger, thanks for the pointer. I will talk to one of the authors of the RFC (turns out we both work for Google, heh).

arethuza: looks like your message was corrupted (sorry this blog software sucks).
03 Sep 2014 16:31 UTC

Michiel Trimpe wrote: If you like this then you'll probably love this: https://speakerdeck.com/mtrimpe/graphel-the-meaning-of-an-immutable-world

It's a presentation I'm working about a concept very similar to yours but with email as 'just a transport protocol.'

The core message about designing a computing paradigm *based on* the immutability of sent messages is still the same though.
13 Oct 2014 18:32 UTC

mrb wrote: Michiel: this slide deck sounds a little abstract to me (I understand this is work in progress), but it seems interesting. 15 Oct 2014 17:52 UTC