Intercom Engineering Insights - Scale and Reliability 2015

To complement our popular "On Product" events, which give a broad view of all aspects of building world class product, we're starting a series of technology focused evening events, each with a specific theme we are passionate about.

Our first one will be on Thursday, December 10th at 6.15pm, focusing on Scale and Reliability Engineering.

Click here to register

What is Scale & Reliability Engineering?

Scale and Reliability Engineering is the art of high impact, product-focused software engineering, lead by an operational excellence mindset. We’re excited to talk about it.

Working in a 🚀growth environment brings interesting infrastructure and product engineering challenges. When your customer base, traffic and data are all growing at 🚀 pace, you’re constantly evolving your infrastructure and architecture to keep your application healthy. You’re constantly making decisions about whether you should keep doing the fast, simple things to scale or whether it’s time to take on a more ambitious, longer term project that changes the game. The best engineers can always find ways, exercising exquisite judgement, to make small changes that have massive positive impacts to scalability, latency, cost, reliability and customer value.

We’ll share the thought process and values we used to solve three specific Scale & Reliability engineering challenges, and the technology lessons learned. As well as sharing the insights and lessons we’ve learned with the community, we’d also love to get feedback and insights from the community on how we can improve going forward.

Why are we doing this?

These sessions are really valuable to us, as they embody two of our core engineering values.

Represent Intercom with pride
We are extremely proud of what we have built at Intercom. But we’re even prouder of engineers who build great products, and then share that knowledge with the world. That’s why all engineers at Intercom are encouraged to share their knowledge with others, and do that in ways that enrich the community. These events are a great way to show what we do and how we work, and also to learn from those in the audience.

Crave feedback and improvement
Feedback helps us shape and improve our work, our product, and ourselves. Engineers at Intercom are expected to question and be curious about whether their work had the impact we expected, and what we can improve now and for next time. Our Insights events are a great way for us to get community feedback on what we’ve worked on, and also to give constructive feedback to attendees.

We’ll be charging a €10 donation for all attendees. All proceeds will go to St. Vincent de Paul, a charity dedicated to fighting poverty in Ireland. We’ll be covering pizza and beer for the night, so your donation will go entirely to the good cause.

We do hope you can join us for what will be a great night.

Speakers & Talks

Scaling MongoDB to one million client connections and beyond
Intercom lets you store custom data attributes about your customers. In technology terms, this means storing, processing, and retrieving thousands of updates per second from hundreds of millions of users on billions of data points.

Using MongoDB as the store for this data, Niall Sheridan and Bariša Obradovic will talk about how we overcame the challenges of scaling MongoDB, while continuing to ship new code to production up to 100 times a day.

Scaling Elasticsearch from zero to one billion user documents
As our user base grew, so did the volume of unstructured data we needed to store and process. We also continued to come up with new product features that could be created by enabling complex, fast querying of the data. It became clear that we needed more capabilities than MongoDB alone could provide. Enter Elasticsearch, a distributed, scalable, highly available real-time search engine.

Matthew Barrington and Alex Gerlic will talk through our initial Elasticsearch implementation, the 3 major challenges we faced while scaling, and what we learned about operating a distributed datastore.

Faster, Safer, Easier Continuous Deployment, dozens of times a day
Ciaran Lee, CTO, says: “To me, Intercom is a place where: It’s as easy as possible to ship code to production; we are never afraid to deploy; and we ship ambitious projects as a series of small, safe steps”. Darragh Curran, VP of Engineering, has previously talked about how Shipping is your Company’s Heartbeat.

Eugene Kenny will talk through the mechanics and philosophies of how shipping code to production dozens of times a day actually improves our reliability, rather than threatens it.

Click here to register