State of the Nation

At Intercom, we run our own in-house billing system that we began developing at the start of 2015. The reasons for building and not buying could fill a blog post, but that’s a story for another day. Billing systems are a crucial component to the smooth running and growth of every business. All bugs are serious: even minor glitches impact revenue or reporting, and most seriously, erode customer trust.

Carefully modeled data, with strongly enforced constraints, rules out an entire class of bug. To this end, we imposed as much referential integrity in our database as possible when building our billing system. However we found ourselves repeatedly needing a time-bound constraint which wasn’t possible to enforce at the database level, nor easily at the application level.

Example

It’s important to have a complete history of every change a customer has made to their account to bill accurately and fairly.

In our system, we create "Assignment" records each time a customer changes their billing plan. These records specify which plan is active for a given period of time, as well as who created or removed the plan. For example, if I start a new subscription on the Small plan and Linda upgrades two weeks later to Medium, then we'd have the following records:

Subscription(id: 1)
PlanAssignment(
  name: "small", 
  subscription_id: 1,
  assigned_at: "2015-01-01",
  assigned_by: "patrick",
  unassigned_at: "2015-01-15",
  unassigned_by: "linda"
)
PlanAssignment(
  name: "medium",
  subscription_id: 1,
  assigned_at: "2015-01-15",
  assigned_by: "linda",
)

Ensuring Consistency

Having all these records is great, but only if the history that they describe is consistent. Any overlap in assignment records for different active plans is a source of dangerous bugs: what would we bill for at the end of the month?

Unfortunately, creating database level validations for this time consistency is not possible. We use MySQL, which doesn’t have the indexing power to do this. Even unique indexes for the start and end timestamp values won’t work.

StateOfTheNation

We faced this problem when modelling several objects in our system, including plans, coupons, credit cards, trials. Today, we’re open-sourcing a gem we developed to enforce the constraints required at an application level in Rails.

StateOfTheNation provides an easy way to model this one to many relationship between the parent (here, Subscription) and child (PlanAssignment) records over time, validating at each update that records don’t wrongly overlap. It also provides methods to fetch records active at any point in time, reducing the likelihood of querying bugs.

subscription.active_plan_assignment(Date.new(2015, 1, 10))
# => PlanAssignment(name: “small”, assigned_by: "patrick"…

If something goes wrong, and we try to insert invalid data, StateOfTheNation steps in to save the day:

subscription.plan_assignments.create!(
  name: "medium",
  assigned_at: Date.new(2015, 1, 14)
)
# => StateOfTheNation::ConflictError

We’ve been using this in production over the last year, and it’s helped us maintain a high bar for data quality in our system. Over time, we’ve also optimized its performance – this release includes built-in support for Shopify’s IdentityCache gem.

If you come across this pattern in your data, we hope that StateOfTheNation can help you out too. And if working on interesting systems like billing or large scale email delivery sounds interesting to you, Intercom is hiring for engineering positions in San Francisco and Dublin.