Google’s “smart reply” feature in Android is a pretty neat application of machine learning. Google’s servers scan your incoming text messages or e-mails and write replies for you. Smart replies hang out at the bottom of an app like Gmail or Google Inbox, and you can pick from several replies based on the context of the message. Now Google is experimenting with making smart replies even faster, by embedding reply options directly into Android notifications.
The experiment comes from Google’s new “Area 120” group, an idea incubator inside the company. Users that signed up for the group’s early-access program got an e-mail yesterday announcing the new feature, which is an app the team is just calling “Reply.” The app isn’t out yet, but the e-mail shows off two concept images and gives users a link to sign up.
The images show a notification from Hangouts and Android Messages with the expected text and image, but below them, right in the notification panel, are a few machine-produced replies. Someone asks “Are you at a restaurant?” and you can fire back a quick reply with a single tap.
With the second example, the e-mail claims the system will go beyond the usual smart reply fare: In response to “When can you be home?” the system pre-populates a reply that says “13 min,” complete with a car emoji. This is something that would be really impressive if it actually works. Google would first have to figure out your location (easy with GPS), figure out your home location (something you can tell the Assistant and Google Maps), then run a Google Maps query for traffic, calculate the drive time, and write the message. It’s all something Google normally does with a single voice query, but now it will do it in response to someone else’s query.
Best of all, Reply won’t just work with Google apps; the announcement calls out “Hangouts, Allo, WhatsApp, Facebook Messenger, Android Messages, Skype, Twitter DMs, and Slack” as compatible apps.
How this is (probably) going to work
The causal observer might be confused about how this can possibly work when Google doesn’t own apps like Twitter, Slack, and Facebook. How can Google change the features of apps it doesn’t own?
The answer lies in the pictures, if you look closely. Google shows notifications with the icons and names of “Hangouts” and “Android Messages,” but those notifications don’t come from those apps. As the first, colored word in the notification says, those notifications are actually from the app “Reply.” It’s all a clever usage of Android’s existing APIs.
First, Google will build a notification listener app. In Android, there exists an API that can allow an app to read, reply, and basically take over the entire notification panel. It was originally written with Android Wear in mind, allowing the Wear app to take your notification information and beam it to the watch, where it can be repackaged into a smartwatch form factor.
The Reply app will do something along the same lines as Android Wear—it will grab all the text and images from your messaging notifications through the notification-listener API and rebuild those notifications. But while Wear was doing it for a smartwatch format, Reply will do it so it can slipstream the smart=reply options into your notification panels. Just as Android Wear did not require any work from a third-party to support notifications, Reply shouldn’t require any extra work, either. That’s how it can work with WhatsApp, Slack, and the other third-party apps.
Reply will need to come up with some solution to not display the original apps notification and Reply’s duplicate, cloned notification alongside each other, possibly by just dismissing the original app notification.
The notification=listener API still doesn’t allow a third-party app to reply to your messages. For that there’s the “RemoteInput” API, which is another API originally meant for Android Wear and Android Auto that will probably be repurposed for Reply. Android’s watch and car interface is intended to be used for voice input; this API allows messaging apps to receive a block of text from Wear or Auto and send it to the appropriate contact. In Android 7.0, Google expanded the use of the API for the “in-line reply” option. Just tap on a notification and a keyboard would pop up, along with a tiny in-line text box. Reply will probably be an extension of this API but with single-tap machine language input instead of with a voice or keyboard.
Of course, we’ll confirm this is the way this all works when the Reply app comes out, but this is the most obvious way to make the app work. It’s not an ideal setup—ideally you’d want Android to take care of this natively, but remember this is just an experimental 20 percent project. If people like it, maybe someday it can be added to a future version of Android, so Google can avoid this hack of a setup.
We’re in the early access program and have signed up for the preview. Hopefully it arrives sometime soon.