It's a familiar problem for all fast-moving businesses. How do you keep your core technologies manageable for the growing number of teams that depend on them? As Intercom's product matured, our product teams needed to extend the Latest Mailing Database depth and breadth of their technology expertise to support the features we were developing for our customers. This showed up as a growing percentage of our product teams' time spent in operations or deep diving into understanding our small set of core technologies. The way our core technologies were used was also divergent, leading to unnecessarily increased complexity in our systems. Our product teams were slowed down and our operational health was taking a hit.
"This is the story of how we have grown and expanded our team over the past few years as the business has grown rapidly, and how we have expanded our responsibilities" Once we get to a certain size, we could justify dedicating a small team to Latest Mailing Database focus exclusively on enabling teams to more effectively use the technologies we're building Intercom on. Thus, at the end of 2018, Team Datastores was created, initially with a fairly conservative area of responsibility. It wouldn't be an operations team as such – the product teams would still own their infrastructure. Instead, it would be an internal core technology team that would take a long-term view of how we build and scale Intercom.
This is the story of how we have grown and expanded our team over the past few years as the company has grown rapidly, and how we have expanded our responsibilities. Our first challenge – Elasticsearch As we trained and figured out where to start, we had an obvious first technology: Elasticsearch, which was originally introduced to Latest Mailing Database support advanced search capabilities around attributes of personalized data and is the only database that we fully manage. Over the next few years, as Intercom became a more powerful product, Elasticsearch was used to enable more functionality. Product teams typically set up and own their own Elasticsearch infrastructure with little guidance, technical depth, or use of best practices.