Editor's note: The following article was originally published on The New Stack on April 29, 2021.
Though RESTful APIs remain the mainstay of the programmable world, there is rapid adoption of reactive event-driven architecture and a distinct shift from the traditional polling-based legacy integrations. The considerations of an event-based approach are not just limited to the obvious candidates, such as designing a system that reacts to changes in real time, but also include things like increasing adoption in resilient, highly decoupled microservices architectures.
When it comes to the asynchronous world, the complexity in designing a specification that abstracts (and thereby unifies) multiple protocols and supports diverse formats, brokers, channels and publish-and-subscribe perspectives is a formidable task — and an ask that AsyncAPI Specification 2.0 admirably meets.
In this article, we will discuss AsyncAPI Specification 2.0 and its adoption in eBay’s developer ecosystem, why eBay chose AsyncAPI Specification-based contracts to power its latest notification platform, and how AsyncAPI Specification standardizes and simplifies representation of a highly decoupled and resilient microservices topology.
What Is AsyncAPI?
AsyncAPI addresses the need for a unified, open source, protocol-agnostic asynchronous specification that is both human-readable and machine-readable, while also being backed by a diverse and rich tooling ecosystem. With its maturity and elegant abstractions, the AsyncAPI Specification has emerged as the industry standard for defining asynchronous, event-driven APIs.
AsyncAPI started as an offshoot from the OpenAPI Specification and maintains compatibility with it. A discussion about the AsyncAPI Specification would therefore be incomplete without an acknowledgement of the contribution of the OpenAPI Specification to the industry in general, as well as eBay in particular. (Learn more about why eBay adopted the OpenAPI Specification for its RESTful APIs in 2017, and eBay’s subsequent journey, in a previous article.)
Here are some of the key AsyncAPI Specification features that we found particularly useful:
- Clean separation between channels, operations and servers. This allows a complete representation of the event-driven topology (the producer, consumer and message perspectives), resulting in standardized and precise representation of a message-driven ecosystem.
- Support for traits and external references. This promotes reusability.
- Support for correlationIds using dynamic runtime expressions.
- Extensibility of the specification, for those one-off customization needs.
- Support for a wide range of asynchronous protocols to satisfy most industry needs.
- Compatibility with the OpenAPI Specification. This enables re-use of schema definitions from existing models, which leads to a short adoption cycle for organizations that have already standardized on the OpenAPI Specification.
Adopting AsyncAPI for eBay’s Asynchronous API Contracts
Late last year eBay initiated work on a new event notification platform, designed to meet current and future demands for asynchronous communication to API partners.
Aside from the usual considerations of scale, delivery guarantees, monitoring, playback and recovery, the new platform was also designed for data security; Elliptic Curve Cryptography- based message integrity verification; and support for multiple protocols and payload schema versions. Some of the advanced features also include multi-tenancy with isolation guarantees between different “tenants” of this system, as well as internally separating use-case-specific concerns from core platform concerns.
When it came to exposing the event streams of this new platform as a contract to eBay’s external developer community, the Async API 2.0 Specification was the unanimous choice.
In March 2021, eBay launched the first AsyncAPI-based contracts for the new business event notification capabilities. The AsyncAPI contract for eBay’s initial marketplace account deletion use case based on HTTP is here. This contract addresses the following integration concerns:
- Topic available for subscription;
- Supported protocols;
- Schema definition for the payload; and
- Message headers.
We found AsyncAPI extremely helpful because of the ease of use that results from delineation of channels, protocols and bindings; separation of concerns between publish and subscribe; and separation of protocol-specific and application-specific headers.
Applicability to Microservices Evolution
Most software systems start as a monolith and evolve over time to a distributed microservices-based approach. The first phase of this transition is the decomposition of the monolith to granular distributed services, with an entity services layer representing persistence. A typical outcome of this phase is often a business process tier orchestrating calls to a single or multiple underlying service tiers in a synchronous manner.
Orchestration, however, has issues of scale due to the inherently synchronous nature of this approach. An unexpected aggravation on any of the dependencies, for example, can cascade to a major degradation of functionality without adequate safeguards. A high degree of coupling between the components also introduces significant maintenance overheads when specific components or dependencies change.
The next phase of this evolution is a transition to an event-driven paradigm. An event-driven system entails a set of autonomous components that react to specific events or state changes. At a high level this system could be thought of as a set of event emitters and listeners that interact though a message bus. As a consequence of the decoupling, the emitters are not burdened with the listeners of the events they produce, nor do the listeners need to know about the event producers. Each can independently scale. Some of the major pitfalls of an orchestration-based approach are addressed. For further information, here is a great article by Martin Fowler on event collaboration.
A simplistic illustration of decoupled microservices is given below:
Figure 1: A typical decoupled microservices layout as a set of event emitters and listeners.
The illustration above shows a set of emitters and listeners interacting with a message broker.
However, representing a complex system of such autonomous components can be daunting. The event streams or channels at the core of this reactive system need to be represented along with details of the mechanism by which information is exchanged — for example, payload schema definitions, protocols and associated headers. Message brokers need to be presented along with URIs, protocol, security configuration and bindings that apply. Operations need to be unambiguous.
The AsyncAPI Specification adequately covers all of the key elements in the preceding paragraph, simplifying the representation of this complex topology. An event stream is represented as a channel. The message definition allows both payload schema and header information. The message bus (or broker) is represented in an AsyncAPI contract as the server. Servers can have associated security requirements. The set of event emitters are defined as producers and listeners as consumers.
For industry adoption, however, it’s not just the richness of the specification that matters — it’s also the tooling that comes with it. As with OpenAPI, AsyncAPI tooling incorporates powerful visualizers that allow architects and engineers to collaborate on the design. A complex layout can be simplified. Collaboration on design is simplified through standardized taxonomy and AsyncAPI visualizers.
Because AsyncAPI provides a machine-readable contract, it means that it can provide code generation for producers and consumers. Here are a few references:
- Java-spring-cloud-stream-template is an awesome codegen utility to spring up microservices from an AsyncAPI contract. It also comes loaded with quite a few useful customization options.
- Microcks is an open source Kubernetes mock-and-test framework that supports AsyncAPI.
- Solace offerings include AsyncAPI support.
- Postman has long enabled the API world and recently partnered with AsyncAPI.
There are more tooling references here.
Conclusion
We expect that eBay’s decision to standardize on and publish the AsyncAPI Specification 2.0-based contracts for event notifications will make integrations simple for eBay’s external developer community.
And from our experience with AsyncAPI so far, we definitely recommend using it.
Interested in pursuing a career at eBay? We are hiring! To see our current job openings, please visit: http://ebay.to/Careers.