Why NextBillion.ai Chose Rust to Enhance Its Routing APIs

Some of the technologies used in the geospatial world are pretty old school. The histories of certain projects can be traced back to the 1960s. Back then, people weren’t spoilt for choice when it came to coding languages as we are today, so it’s not surprising that many routing APIs are coded in C/C++.

At NextBillion.ai, we modify and customize some of these projects for use in our routing API offerings. As you might expect, this is not a simple proposition, because of the use of C/C++. The most obvious option to consider was to continue coding the APIs fully in C/C++.

However, there are challenges in doing so:

1. It’s hard to recruit engineers who have C/C++ experience these days
2. It’s difficult to ensure memory safety and code quality, and to use modern programming techniques and frameworks

So, here’s how we did it instead.

We split our dev into two parts. The first part had to do with changes that absolutely must go into C/C++ code; the second part could be coded using something less painful, which integrates with C/C++ code through FFI.

We then asked ourselves, what’s the best way to call a C/C++ library at runtime? This is where Rust entered the picture.

There are so many programming tools/languages that provide FFI as a feature, but in different ways. Rust, the shiny new toy that everyone keeps talking about, is one of them.

Rust is quite popular these days; there are a number of user satisfaction and success stories, such as: https://discord.com/blog/why-discord-is-switching-from-go-to-rust

Coming back to NextBillion.ai’s dilemma, we now had to ask ourselves: Could something as new as Rust be married with old-school routing engines/APIs?

Let’s explore some of the pros and cons of Rust and why we went with it for NextBillion.ai’s routing APIs.

Pros

Rust calls a C library like C calls a C library

Rust has no inherent overhead when calling a C library. This is not the case for Java/Python/Node, or even Golang. First of all, Rust is not a GC language and smartly manages memory safety at compile time. Thus it strikes an almost perfect balance between runtime overhead and memory safety. This means we get to enjoy its runtime performance while not worrying too much about data racing, segmentation fault and memory leaking.

Rust has fearless concurrency

Rust enforces its ownership model, which addresses most concurrency problems at runtime. You would typically find it significantly harder to write a piece of code that has concurrency bugs in Rust compared to almost all other programming languages. On top of that, Rust does not have built-in abstraction for multitasking. You can choose what makes most sense for you, be it CSP (Golang), Actor Model, Async Await or Raw Multi-Threading. You’re spoilt for choice!

Rust is blazing fast

Rust frameworks appear very high in web-API performance ranking lists. In fact, Rust often has similar performance compared to programs coded using C/C++.

Rust supports modern tool-chains

Say goodbye to makefiles, because Rust has Cargo. Also, it’s enjoyable Unitesting Rust code.

Cons

Rust is “slow” at compile time

Every modern programming language provides memory safety and programmer-friendly features at a cost. For example, Golang gives goroutine and GC, but does allow you to know which thread your code is running on. Python gives you multi-threading, but locks everything with GIL. Node.js is lock-free, but removes multi-threading completely. In Rust’s case, it gives fabulous runtime performance and memory safety, but makes you suffer at compile time. Newbie Rustaceans have to learn to manage its steep learning curve and persevere through that period of time when it just makes them want to smash their keyboards.

Rust is too flexible and powerful for beginners

Rust offers a lot of flexibility and can be married with almost all programming concepts and models. New Rustaceans can get seriously confused or overwhelmed with the options at hand. You need to know what you’re doing!

We completed our full evaluation by putting all of these pros and cons on the table, and decided that it was absolutely worth trying in our situation.

Our journey with Rust so far

There are two sets of problems in routing APIs/engines:

1. Less to no flexibility, but the ability to calculate real fast!
2. More flexibility, but with performance penalties.

Obviously, the more flexible your APIs, the less pre-calculation and caching can be done beforehand, and more things to be done at runtime. Let’s focus on problem set number 1 today.

In the early days, when we quickly prototyped a solution for problem set number 1, we directly modified the C/C++ library where it was absolutely necessary and did everything else with a GC language. The result was a working API with acceptable performance.

And then new challenges arose from client requests.

Some examples:

1. Calculate A→B driving distance, ETA and route with a very good P95 latency under certain request concurrencies
2. Calculate ETAs for 4000×4000 origins and destinations (AKA large distance matrix) as fast as possible.

We realized that it was very hard to solve these challenges with the fast prototype API. When you want your API to perform as fast as possible, you have to muster all your strengths, including the choice of the programming language itself.

GC languages are inherently slower; we also wanted to maintain full control of how our program runs, so we decided that we needed the power and control that Rust grants.

It was not an easy road, though. We did everything from scratch, such as learning Rust, passing its stringent compile time checks, writing a proper FFI layer, choosing the right concurrency model and more. At the end of it all, we were able to craft a solution that had 5ms P95 latency for simple routing requests, as demanded by some clients. Obviously, there are more variables in the whole latency equation, but marrying our routing engine with Rust paved the way for us to achieve the overall goal.

Are you evaluating various programming tools for a project that requires C FFI? Hopefully the pros and cons listed in this blog could be a good starting point for you!