Today was like every workday since I joined eBay nine months ago. Complex, challenging, and thought-provoking. Our data is huge, our customers are demanding, our competition is aggressive, and any improvements need to be implemented in a way that doesn’t disrupt the world’s largest online marketplace. I love every minute of it. Like most of my techie friends, I love tough problems — the provocative ones that answer every question with three new questions and steal part of your brain even when you’re not supposed to be thinking about work.
Let me give you an example — let’s talk about data. If I printed out a list of every query our users sent to us today, and I decided I’d spend 18 hours a day reading that list until I knew what every person looked for today, I would have an answer in 30 years — if I didn’t take any holidays or weekends off. Think things are easier on the inventory side? If I take a snapshot of all the items for sale on eBay today and resolved to read them all on the same rigorous schedule, I’d better make sure I’ve got a few generations lined up to continue my work, since it will take 190 years.
And yet, it’s my job to make sense of all this data. I run a team called Search Quality, which is responsible for matching the potential buyers doing searches on eBay with the most useful set of items. We’re doing our job well when the marketplace is efficient and productive for both buyers and sellers. But you can’t do that by sitting down and examining every query and item — it’s beyond human ability. Which is where the fun stuff comes in. Machine learning, data mining, information retrieval, graph theory, image analysis, natural language processing — Search Quality is where it all comes together. It’s enough to make a geek’s heart skip a few beats.
Some parts of the domain feel familiar. I worked in web search for six years, with most of that spent managing the Web Search Relevance team at Yahoo! But search on eBay is different. The content changes constantly, like in real time search. There is a rich community, like you’d see in a social network. The query volume and number of documents indexed are close to those you’d see in web search. And the constant changes in pricing, supply versus demand, and auction bidding sometimes feel like the biggest open-air bazaar imaginable. There are elements of so many domains that in isolation constitute whole businesses or eco-systems on their own, and they all make up just a part of eBay.
Which is why we’ve created this blog: to share the things we discover, to discuss our open challenges, and to engage with a broader community of academics, engineers, researchers, and technophiles.