The web platform has been evolving at a very rapid pace. One particular feature that has been in limelight in recent years is the Web Push API. Web push notifications allow users to opt-in to timely updates from their favorite sites and effectively re-engage with relevant content. A capability that was exclusive to native apps was opened up to the web platform. Many sites started embracing the notification capability and the adoption steadily grew. Over a period of time, the sites that prompt the browser permission dialog became overwhelming. It has, in fact, reached a point that Firefox added a preference that will block sites from even asking for permission to users. Instant notification permission dialogs are now considered as an anti-pattern.
We used a checklist-based approach to enable Web Push. This checklist encapsulates our learnings and research to provide the best user experience when dealing with notifications. In this post, we will outline that checklist.
Remove the annoyance
Our most important goal was to not annoy users with the permission pop-ups. The entire checklist was created with this goal in mind.
- Avoid instant permission prompt for new users: The most common mistake is to instantly show the browser permission dialog as soon as users land on a site. This has been the biggest source of annoyance. Users have no context on why the browser is prompting for permission and in some cases, they don’t even know what the website is for. To avoid this frustration, at eBay we decided to build a custom UI module that provides the necessary context. Only when a user interacts with it, we will prompt the browser permission dialog. The content of the custom module is algorithmically configurable, which allows us to experiment with various context injections. This enables us to customize the module based on the customer’s purchase journey.
Figure 1. Custom UI banner to enable browser notifications
- Make Web Push a progressive enhancement: We do all the required browser API checks (navigator.serviceWorker and window.Notification) before enabling this feature.
- Enable the feature only to users who have context: We show the custom UI module only for signed-in users. Registered and signed-in users obviously have more context about eBay and are more likely to engage with the module. We have not yet enabled this feature for guest users. Context varies between guest and signed-in users, and the message we show for signed-in users may not be appealing to guests. They may also be first time users who lack sufficient background on why this module is showing up. The customizable aspect of the module has allowed us to experiment with various content that may be relevant for guests. Once we identify a range of use cases, we will enable it.
- Enable the feature only on pages that benefit users: The module is currently shown only on the homepage. Displaying the same module on every page of the user journey would be highly redundant and also annoying to some extent. Homepage acts as a hub and a good place to experiment. In the future, we plan to expand the feature to other pages where users may benefit. For example, the checkout success page, where users would be interested to track the status of the item they just purchased or the listing success page, were sellers might want to know on how their item is performing. For these pages even the instant permission dialog will be beneficial, as users already have a thorough context.
- Respond to implicit user feedback: We also apply some heuristics on the module display frequency, based on user behavior. For instance, if the user does not interact with the module for 2 consecutive sessions, we do not show the module for 2 weeks. It is pretty clear that the user is not interested in it. If the same thing happens after the 2 week snooze period, we again hide the module for an even longer duration, and the pattern continues. We also configure a max cap for non-interactiveness, after which the module is never shown. A similar logic applies when the user clicks on the “Maybe later” button. These heuristics are not set in stone. We keep tweaking them as we see fit.
- Honor the user’s existing preference: Before displaying the module we always check the current permission status using the Notification.permission JavaScript API. We do not show the module if the permission status is denied, even if our backend system qualifies the user to be asked for permission. This can happen if the user has explicitly blocked permission access through browser preferences. In that case, we honor the user’s choice and do not display the module. (It would anyway not work as the permission is already denied.) There are also scenarios where a user can “Allow” or “Deny” notification access when the browser prompts, but can later change their decision through browser preference. We handle this by first checking if the user’s prior decision and the Notification.permission JavaScript API are in sync. If they are not, we update our system to reflect the JavaScript API status and sync the UI treatment accordingly.
- Handle failure gracefully: Subscribing a user to push notifications involves a sequence of steps. Each step is prone to failure. Efficient error handling in each of these steps is key to a pleasant user experience. We do not want to be in a scenario where a user opts in, but due to unhandled errors, the grant is not recorded and later displays the same permission module again. To minimize this scenario, we make sure every possible error is handled in a distinctive way that doesn't disrupt the user flow. Firebase Cloud Messaging (FCM) is used for managing our notifications, and the whole subscription code can be found here. But at a high level, the following 5 steps are required for subscription, and each step has to handle failure gracefully.
Figure 2. Subscription steps
- Don’t overdo it: And finally, just because users granted permission for notification, it doesn't mean we can bombard them with notifications. Too many push messages can also lead to annoyance. Our backend systems are designed to be courteous about this and batch the notifications accordingly.
The entire checklist evolves around this one aspect — avoiding user friction. Along those lines, the whole implementation is designed in a way that when the checklist is not met, the feature is not enabled.
Results
As mentioned earlier, we enabled the push feature for a subset of desktop web users for A/B testing. Our first metric to consider was the bounce rate. Adding a new module usually causes variation in bounce rates. We did not want the homepage bounce rate to increase due to this feature. To our surprise, there was no difference in bounce rate between test and control. This sort of validates our checklist — users were not annoyed by web push.
There are two interactions a user needs to perform to enable push — first on the custom permission module and next on the browser permission dialog. We wanted to get some numbers around this. We found that 21% of the users who interacted with the custom module clicked on “Yes please.” And of this 21%, close to 95% chose “Allow” in the browser permission dialog. Again a validation to our checklist — when the module is shown, it is contextual and appropriate. We are ramping up the feature to more users, targeting 100% in the next couple of weeks.
Web push notifications, when used wisely, can be an incredible feature for customers. There is already a lot of good literature around web push best practices. We hope this checklist will be another add-on to that list. Huge thanks to my colleagues Jonathan Calvin, Sathish Shanmugam, Vivek Muraka, Deepak Kumar, and Nishant Banore for partnering on this effort and launching to site.