The APIs evolved and grew with the business and enabled expansion to new experiences. They are built by developers for developers and treated as first-class products at eBay. We delivered a simple, consistent, and well-documented API portfolio rather than an API maze where it is challenging to discover capabilities.
Looking back over the past two years, there are so many lessons that we have learned about building reliable, secure, and scalable APIs.
Standards and patterns
In the API world, the goal is to create a vision for the APIs that also includes technical vision. Standards and specifications play an important role here, but they are nothing without proper process. The governance part is crucial to ensure that the APIs fit the technical vision. Agreeing on standards is challenging. Keeping the process up to date, enforcing the specifications, and making sure the engineering teams are aligned is difficult. So the governance is all about finding a balance: define standards but be flexible on the how part.
We continued in the direction of relying on existing patterns (somebody has already solved your problem) and defining our own: reuse rather than reinvent the wheel on every new API. This approach brings agility, speed, and consistency to the API development process.
One of our latest patterns is an asynchronous request used for capabilities that might take too long to complete. A client initiates a long-running job and retrieves the current state of the process. If the process is successfully initialized, the resource representation, which APIs send back, contains an expected completion time and other relevant data. If the process is successful, the client retrieves the newly created or updated resource.
We followed this pattern in the Marketing API that enables sellers to manage their campaigns and get comprehensive performance data. As part of the new product-based shopping experience, sellers are invited to suggest edits to the eBay product catalog. Because the process to approve catalog change requests is asynchronous, the recently released Catalog API leverages the same pattern.
POST https://api.ebay.com/sell/marketing/v1/ad_report_task
{
"reportType": "CAMPAIGN_PERFORMANCE_REPORT",
"dateTo": "2017-01-30T00:00:00.000Z",
"dateFrom": "2017-01-10T00:00:00.000Z",
"metricKeys": [ "CLICKS" ],
…
"marketplaceId": "EBAY-US",
"reportFormat": "TSV_GZIP",
"campaignIds": [ "10000000019" ]
}
HTTP/1.1 202 Accepted
Location: https://api.ebay.com/sell/marketing/v1/ad_report_task/abc
GET https://api.ebay.com/sell/marketing/v1/ad_report_task/abc
HTTP/1.1 200 OK
{
"reportTaskId": "10000049014",
"reportType": "ACCOUNT_PERFORMANCE_REPORT",
"reportName": "10000049014.tsv.gz",
"reportTaskStatus": "SUCCESS",
…
"reportId": "1049014",
"reportHref": "https://api.ebay.com/sell/marketing/v1/ad_report/1049014"
}
For large binary resources, we introduced a partial response pattern. This pattern helps to deal with unreliable and intermittent connections and to improve response time. Our Feed API, which exposes eBay inventory, leverages the partial response pattern. This approach allows downloading large feed files in chunks. (We have more than 1B listings on eBay!) A client specifies the byte range in the Range standard HTTP header.
GET https://api.ebay.com/buy/feed/v1/item?feed_scope=NEWLY_LISTED&category_id=625&date=20180811 Range: bytes=0-2499
On the response side, we also leverage standard HTTP headers. The Content-Length header has the size of the chunk in the response. The Content-Range header indicates the byte range contained in the response and the total size of the resource content.
HTTP/1.1 206 Partial Content Content-Type: text/tab-separated-values Content-Length: 2500 Content-Range: bytes 0-2499/4580 Content-Encoding: gzip Transfer-Encoding: chunked ...
Developer experience
“An API without a developer portal is like a hackathon without pizza; like a room of developers without awkward conversations. Or a developer’s laptop without stickers.” - Apigee
A developer portal is a front door to the API program. It provides everything external developers need to integrate with our services. Delivering APIs is only half of the work done. Developer experience is the heart and soul of a successful API program.
For most of our modern RESTful APIs, we exposed contracts based on the OpenAPI specification. This specification is a standardized, community-driven, and both human- and machine-readable way to describe the API contract. There are plenty of mature open source tools, one of them being Swagger, which has SDK support for more than 40 technology stacks. In 2017, we officially joined the OpenAPI Initiative (OAI) and later became one of the first in the industry to publish API contracts based on the OpenAPI 3.0 specification. Partner integrations with eBay APIs are streamlined and simplified, with significantly reduced time to market.
Our ultimate goal for eBay APIs is to be intuitive and easy to discover, understand, and consume. (And, difficult to misuse!) Names matter: we truly believe that each endpoint is a sentence where resources are nouns and HTTP methods verbs. OpenAPI-based contracts are for sure a jump-start to integrating with an API. It literally takes minutes to invoke a read-only capability. This approach puts a smile on any developer’s face. Still, contracts are not documentation and are insufficient for a good developer experience. Providing samples, quick start guides, and documentation that tells a story is what really makes the face of the API program.
Lessons learned
APIs are forever—there is one chance to get it right. Once there are live integrations, it becomes impossible to change APIs without impacting partners’ business.
In the API world, there is no such thing as one-size-fits-all. APIs should do one thing and do it well, without offering surprises. Gather requirements with a healthy degree of skepticism. Understand the problem statement and find a solution. (And iterate!)
APIs represent the consumer’s view of the capabilities. They are the front door to the portfolio of internal microservices (producer’s view) and have requirements tailored to external clients. Typically, the APIs are complex orchestrators that construct responses by leveraging fine-grained capabilities of backend services. Internal APIs or legacy backend functionalities should not constrain API design. Also, the APIs are not supposed to reflect organizational charts.
Plan and design expandable interfaces that will evolve and grow with the business. APIs express the value of a business, so they must be designed with a long-term vision in mind.
Customer satisfaction is one of the most important metrics to measure the success of the API program.
Have a good reason to introduce a new (major) version of your API. New APIs are not supposed to be slimmed-down legacy services.
Developers invested significant effort over the years to integrate with old APIs, so migration to the new modern family of services is a long-term process. Evangelize API adoption, act on customer feedback, and support partners and new integrations.
What is next?
The art of crafting and shipping APIs that developers love continues. Our vision is to have the latest eBay marketplace features, including the emerging field of AI, exposed through public APIs. These APIs will attract partners across the globe to leverage the new capabilities and create successful integrations and fantastic experiences for our buyers and sellers. The ultimate goal is to keep the marketplace open and have public APIs acting as lingua franca for our partners. We are announcing improvements and additions to our API portfolio next week at eBay Connect. Stay tuned!