Building a Marketplace — Search and Recommendation at Eventbrite

These are exciting times for Eventbrite! Since our beginning Eventbrite has served a platform for event organizers. We allow organizers to market their events, message their attendees, and most importantly we take care of the hassles associated with selling tickets. But recently we have began to take a more active role of connecting potential attendees to the events that they will enjoy and in doing so, we are becoming the marketplace of events!

Personalized Discovery – the Center of the Marketplace

And at the heart of this transition to a marketplace of events is the notion of discovery – the ability for users of our site to serendipitously discovery events that they will enjoy. At Eventbrite, we think of discovery as occurring in two predominant modes: search and recommendation. In search, the user is actively telling us what type of event they are looking for, and it is our job to return results that are most relevant to their search. In recommendation, Eventbrite will provide passive users with recommendations for events that they might be interested in. Personalization is an important aspect of discovery as we hope to provide users not only with recommendations of popular events or search results that match their query, but we also want to tune our recommendations and search results to match individual user preference based upon their previous interactions with Eventbrite and the things that they have explicitly told us about themselves.

But this is no small undertaking, and Eventbrite has some interesting challenges to overcome. A chief concern is that, unlike movies or consumer goods, events are an unusually short-lived type of product. Netflix and Amazon use customer interactions to build rich recommendation models of their products through a technique called collaborative filtering. But when an event is published to Eventbrite there is no user interaction data and by the time the event is finished we may have only started to adequately understand how an event may be matched to our users. Though this situation will be challenging, we believe that is can be overcome via a hybrid approach – a combination of collaborative filtering and content-based recommendation and search. Read on…

Blueprints for Personalized Discovery

We are still in the early stages of building out our discovery platform, but we have some good ideas about where we want to go and how we plan to get there.

A major component of our strategy is collaborative filtering. By watching how users interact with the events in our inventory (for instance through clicks, purchases, shares, etc.) we can build a model of user-to-event affinity. Currently we are using this affinity model to send personalized event recommendations to users that receive our event follow-up emails. And right now we are rolling out a new “related events” feature on our event landing pages that is driven by this same user-to-event affinity model. The model is quite flexible, so in the future we may use it to recommend events customized for a group of users (think “night out on the town” with friends), or we might tune our related events feature so that it the related events are further personalized for the viewing user.

But, as alluded to above, one of the short-comings of collaborative filtering is that building a model requires quite a bit of user interaction data, and at the moment an event goes live there is no interaction data and thus no chance that the new event will be recommended. Because of this we will also rely upon content-based strategies for recommendation and search. Initially this will be based largely upon the organizer-specified categorization for the events and the text we find in the title and description fields (along with filters on things like location). But we are also looking at creating an event tagging system which will allow for much more finely tuned categorization of events. We are also building up user profile pages and we may eventually allow users to tag themselves with the same system being used for events. This would make content-based recommendation as simple as finding events that have the same tags as the user.

There are some problems though in having two competing recommendation systems. How do you combine the results provided by each system? Is it really a good idea to maintain two competing systems which probably have their own technology stacks? And do you really want to query both systems every time a recommendation is required? We think this seems silly. So, we’ve decided to roll both collaborative-filtering and content-based recommendation into one system and we’re doing this with search technologies. Think about it this way, search engines are natively good at content-based search and recommendation – you just store the text and categorization for each event in the search engine and then you query it as you normally would. Then, if we want to combine this with collaborative-filtering, we have to find a way of including a new field or making a new type of query that incorporates the information from the user-to-event affinity model. As it turns out, there are a few different ways to do this; but that’s a discussion for a different blog post! Once content and collaborative filtering information are in the search engine then the two sources of information can be balanced in much the same way that matches in the title and description field would be balanced in more mundane uses of search. Ultimately search and recommendation become one and the same thing and both are all the more relevant because they incorporate content-based and collaborative filtering strategies.

Will it Work?

Well, this is the most exciting question of all, because we don’t know yet if this will all work as we’ve planned. In a previous life as a search consultant I built a prototype that closely resembles the system we’re working right now, and the results were very encouraging. But this time we’re building for much larger scale. However I have seen search engines perform some amazing feats of scaling in the past and I have a strong suspicion that search technologies will not let us down here.

If you’d like to see how we’re progressing, then check back in to the Eventbrite engineering blog from time to time. In the coming months I plan to blog fairly regularly about our progress. So pull up a chair, you’ll have a front-row seat to all of the action.


 

P.S. In case you’re interested in the broader discussion of search relevance, then you should check out Taming Search, a book I’m writing with Doug Turnbull. The techniques I’m using here will be covered in depth in a later chapter of the book. If you decided to buy the book, then please be sure to use the code mlturnbull to get 50% off the price of the book.

Type python, press enter. What happens?

Philip James and Asheesh Laroia discussed how the Python interpreter starts, from the perspective of the operating system (OS). Together, we will see the ins & outs of processes: fork(), exec(), stdin, and stdout.

It focused on OS concepts and requires no background knowledge, using analogies to Python data structures. (The talk does not discuss Python’s own initialization, such as site.py or global variables.)

Finding Spammers & Scammers through Rate Tracking with Python & Redis

We made it back from PyCon to the considerably warmer climate we’re accustomed to in San Francisco. Mica Swyers and Jay Chan from the Eventbrite Integrity team provides an introduction to rate tracking as well as an explanation of a particularly cool way to implement it. They go over what rate tracking is, why you would want to do it, and then how you can use build a Redis-backed “velocity engine” in Python to do just that.

D3.js Resources to Level Up

I have gotten a lot better at D3.js development over the past few years, and can trace most of my improvement to coming across a few key tutorials, blogs, books and other resources on the topic. They’ve been a huge help for me, and I’ve gathered a bunch of my favorites in this post to hopefully help others improve their D3 experience.

Here it goes:

Assessing your level

First, let’s define four general D3.js levels:

  • Complete Beginner: You have no previous experience with D3.js or any front end technologies (HTML/CSS).
  • Basic: You have some HTML/CSS/JS skills and have played around with some D3.js examples, but don’t completely understand the patterns and mechanics it uses.
  • Intermediate You know how to customize D3.js graphs using examples found in search engines, but you struggle to reuse them and aren’t quite happy with the quality of the code itself.
  • Proficient: You have build a lot of different graphs, tests and integrated them with different technologies or libraries. You’ve even created packages to easily share logic between projects.

Complete Beginner

Books

Scott Murray’s Interactive Data Visualization for the Web. Available free online through O’Reilly, it covers most of the basics needed for building your first interactive graphs.

Tutorials and Hands-on Learning
Reference links

Christophe Viau’s D3.js Gallery, is a pretty extensive list of charts made on D3 and sorted by graph type (bars, pie, bubble, etc.)

Courses

Udacity’s Data Visualization with D3.js, Communication with Data. It’s free and looks sweet, what else can you ask for?!

D3 Pie Chart

This chart employs a number of D3 Features: d3.csv, d3.scale.ordinal, d3.svg.arc & d3.layout.pie. From bl.ocks.org

 

Basic

Books

Getting started with D3.js: a short introduction (under 100 pgs) targeted for web developers.

Tutorials and Hands-on Learning

Introductions & Core Concepts sections of the D3 wiki tutorials.

Reference links
This simple bar chart is constructed from a TSV file storing the frequency of letters in the English language. The chart employs conventional margins and a number of D3 features. From bl.ocks.org

This simple bar chart is constructed from a TSV file storing the frequency of letters in the English language. The chart employs conventional margins and a number of D3 features. From bl.ocks.org

Intermediate

Books
Tutorials and Hands-on Learning

Introductions & Core Concepts section of the D3 wiki tutorials

Reference links
Videos

Ian Johnson’s bayd3 youtube playlist

coenraets.org

coenraets.org

 

Proficient

Books

Some of the content in Mastering D3.js may be new for you, but I haven’t found a lot published at this level.

Tutorials and Hands-on Learning

Check out the Specific Techniques section of the D3 wiki tutorials.

Reference links
Blogs
Interactive D3 chart from the NY TImes

Interactive D3 chart from the NY Times

I hope this roundup of links will help level up your D3.js skills. This is by no means an exhaustive list, if you have others you’ve found useful share them with me on twitter @golodhros. I hope this is a great starting point to get you started and building more exciting visualizations!

Eventbrite Engineering at SXSW 2015

Photo by CleftClips on Flickr

Photo by CleftClips on Flickr

If you are at SXSW this year you are undoubtedly already aware Eventbrite has a huge presence ticketing most of the parties, what you may not know though is that some of our engineering team are also out in Austin flying the eventbrite flag.

Our very own Daniel ‘Cubes’ Silverstein is giving a talk tomorrow (17th March) at 11am with Katie Dill, Head of Experience Design at Airbnb in the JW Marriot Salon C.

The Remaking of Airbnb and Eventbrite

IMG_0305_4x5In mid-2014 Airbnb and Eventbrite both underwent major redesigns of their websites. These efforts touched virtually every page on the sites and affected every person in each company. Find out why each company made it a priority to undertake such massive efforts, and how design, engineering, and product teams came together to accomplish these feats. This panel will include Daniel C. Silverstein from Eventbrite and Katie Dill from Airbnb to tell the inside story of why and how they did it.

If you are staying on for SXSW Music, we recommend checking out South Fry South Friendy’s, a gig hosted by the multi talented Eventbrite Nashville Engineer Jesse Keogh

If you are at SXSW we hope you get chance to eat lots of BBQ food, drink Margaritas and most importantly get to see Dan’s talk and Jesse’s gig! Have fun in Austin!

CSSConf Oakland review

The inaugural CSSConf Oakland took place at The New Parkway Theater in December 2014. It was expertly co-ordinated by Mikeal Rogers, Karolina Szczur, Kristina Schneider and Alex Sexton as part of JSFest, a week long series of front-end events in Oakland, California.

The relaxed setting meant it felt like chilling in the tatty sofas of a cosy living room, hanging out with 100 or so close personal friends. The sessions were excellently curated to form a well rounded agenda, with healthy diversity in the lineup and women representing seven out of 12 speakers.

Tab Atkins of the CSS working group spoke with such bubbling enthusiasm for new capabilities coming to CSS that we couldn’t help but be carried on a journey to the glittering future. A future where an image can be the output of a function, you can create source-order independent layouts with flexbox and even programatically adjust colour values live in the core language. Even media queries are getting a facelift with the ability to target the accuracy of the pointer or if a device has hover capabilities.

Two other highlights were talks bourne of experience gleaned through hard graft on real-world projects. Jessica Dillon told the story the implementation of Bugsnag, a visual CSS testing framework. Jennifer Wong shared pitfalls and lessons learned in her quest for beautiful responsive emails. Both talks elegantly encapsulated their respective projects and presented valuable take-home lessons.

Unequivocally my favourite session came from Elyse Holladay, Front-End Architect at RetailMeNot, whose eloquent and raw talk spoke directly to the soul of everyone there. Holladay mused that as we gain experience we are no longer safely coddled in the confidence of inexperience, we panic about what we know we don’t yet know. She proposes that instead of agonising over not yet being good at something, we should be kind to ourselves, relish the challenge of learning and always stay curious.

Photo of Jessica Dillon by Karolina Szczur

Originally published in the April edition of Net Magazine, written by Natalie Downe photo credit to Karolina Szczur

Deep linking an Android app

This article was posted on SD Times

To weave a more intuitive mobile Web, individual companies and developers must navigate the various technologies and hurdles in forging those intra-app connections. As senior Android manager at online ticketing platform Eventbrite, Juan Gomez was integral in implementing mobile deep links in the Eventbrite Android app over the past year and a half. He spoke with SD Times about the difference in establishing deep links on Android, the benefits and challenges of mobile deep linking, and why he thinks Android has a leg up on iOS.

SD Times: Can you talk about what Eventbrite is doing with mobile deep linking in the Android development space?

Gomez: When I came in, we were not doing anything on deep linking at all. So my first job was to make the Android app feel more Android-y, if you will, so more in tune to the Android ecosystem, and part of that is deep linking. One of the important things about deep linking for Android is that Android has relatively good SEO. We were showing up on Google results, we just needed the last piece of the puzzle of sending people from the Google results into our app.

Then Google contacted us when they launched App Indexing. This was before they announced it at I/O. App Indexing is basically showing a button and the logo of your app on the Google search results when a user searches for something on their Android phone.

Continue reading