How Asergo Is Building a Fault-Tolerant Future with BEAM and Gleam

Introduction

Picture this: It’s Monday morning, and the Asergo team is logged in from across Europe for our weekly stand-up. Coffee mugs in hand, we’re all on camera, ready to tackle another week. During the meeting, the same familiar challenge comes up. One of our senior developers is deep in maintaining a legacy module he wrote years ago because, let’s be honest, he’s the only one who truly understands it. Meanwhile, our newest team member is finding it tough to get up to speed and contribute meaningfully. Why? Because our codebase spans multiple languages and frameworks, each with its own quirks.

This scene is all too familiar. After 25 years of innovation and growth, our software stack has become a complex web of over 200 repositories written in various languages. While this diversity once served us well, it’s now causing bottlenecks. Developers often find themselves waiting on each other because only a handful of people can maintain specific parts of the system. Onboarding new team members takes longer than we’d like, and development slows to a crawl as we navigate the intricacies of our polyglot environment.

We’ve come to realize that having everyone mostly using the same language and leveraging a shared, cross-corporation framework isn’t just a nice-to-have—it’s a necessity for our future growth.

In this blog, we’ll explore how we’re addressing these challenges by adopting BEAM and Gleam, the benefits and risks of this transition, and our pragmatic migration path.

The Challenge: A Complex Legacy

Our current software stack, while rich in diversity, presents several challenges:

Maintenance Bottlenecks: Only a few developers can maintain specific parts of the system, leading to dependencies and slower development cycles.
Onboarding Delays: New team members take longer to become productive due to the complexity and variety of languages and frameworks.
Scalability Issues: The intricate web of repositories and languages makes scaling and maintaining the system increasingly difficult.

Recognizing these issues, we’ve decided to make a strategic shift to improve our development process and system reliability. While we will continue to use languages like Go, Python, and Java for years to come, we will primarily focus on one language for most of our development. This approach aims to streamline our processes, enhance collaboration, and ensure long-term maintainability.

What is BEAM

In the mid-1980s, engineers at Ericsson were working on a programming language that could handle concurrent activities efficiently. This language was Erlang, and it was designed to manage telecommunication systems that needed to run continuously without failures.

To bring Erlang to life, Ericsson created the BEAM virtual machine. BEAM is the runtime environment that executes Erlang code, much the same way that the JVM is the runtime for the Java programming language.

What makes BEAM special is its ability to handle real-world problems where things can go wrong, but the system needs to keep running. This makes it a great choice for applications in telecommunications, banking, and other critical applications, such as Kubernetes infrastructure management, where uptime and reliability are crucial.

The Solution: BEAM as Our Strategic Foundation

After careful evaluation, we’re making a strategic shift to BEAM - the Erlang virtual machine that powers some of the world’s most reliable systems. Unlike traditional runtimes, BEAM is purpose-built for distributed, fault-tolerant applications with features like:

Lightweight processes that enable true isolation between components
Supervision trees that automatically restart failed processes
Hot code swapping that allows zero-downtime updates
Message-passing architecture that eliminates shared-state concurrency issues

This isn’t just an incremental improvement - it’s a fundamental shift toward an architecture designed for resilience from the ground up. BEAM’s proven track record with companies handling massive scale at WhatsApp, Ericsson, and Goldman Sachs demonstrates its capability to meet our reliability requirements.

Why change programming language

While BEAM provides the runtime, we needed a language that aligns with our goals for technical excellence and developer productivity.

The straightforward choice would be to stick with technologies we already know well, such as Go, the compiled language that forms the foundation of Kubernetes. Alternatively, we could leverage our extensive experience with the JVM and opt for Java or Kotlin. However, after careful consideration and a process that has been underway for several years, we’ve chosen Gleam as our new primary language.

Why Gleam?

Our impression is that Gleam offers compelling advantages that align with our goals:

Static Typing: Catches errors at compile time, reducing runtime failures and enhancing reliability.
Modern and Clean Syntax: With only 15 reserved words, Gleam is simple and easy to pick up, particularly for developers familiar with languages like Java and Scala.
Multi-Platform Compilation: Compiles to BEAM, JavaScript, and potentially WebAssembly, offering versatility for different deployment scenarios.
Seamless Interoperability: Allows access to a rich ecosystem of libraries and tools written in Gleam, Erlang, and Elixir, leveraging proven actor-based patterns.
Developer Experience: Offers a fantastic compiler and toolchain, including broad editor support via LSP.
Concurrent and Fault-Tolerant: Leverages the robust Erlang ecosystem, making it suitable for building scalable, fault-tolerant applications.
Community and Support: Boasts a vibrant and welcoming online community with growing popularity.

By choosing Gleam, we are also embracing the risks involved with adopting a newer language. One such risk is that the smaller ecosystem and community may result in fewer resources and tools available. This could mean that we might encounter challenges in finding pre-built solutions or getting quick support.

However, the upside for us is worth taking the risk. Gleam’s simplicity, modern syntax, and strong foundation on the Erlang/OTP platform provide a robust basis for development. The language’s clean and straightforward syntax can make coding more intuitive and efficient. Additionally, the growing community, though smaller, is vibrant and supportive, which can help mitigate some of the risks associated with adopting a newer language.

In summary, the potential benefits of improved code quality, developer productivity, and the ability to build scalable, fault-tolerant applications make Gleam an attractive choice. The versatility of multi-platform compilation and seamless interoperability with Erlang and Elixir further enhance its appeal.

Our Pragmatic Migration Path

Recognizing our resource constraints and the need for a smooth transition, we’re taking an incremental approach focused on strategic components. Our migration path is designed to minimize risk while maximizing the benefits of BEAM and Gleam.

Starting with the Consumption Billing System

The first system we have tackled is not a new system but a rewrite of our consumption billing system. This system is responsible for tracking and processing overuse of bandwidth and storage consumption for our remote storage systems. It collects data usage from various sources, processes this data, and then pushes it onto our message bus to be picked up by our invoice system.

Why We Chose This System:

Template for Developers: As a rewrite, this project provides our developers with a template and a practical starting point for working with Gleam and BEAM. It allows the team to familiarize themselves with the new language and runtime in a controlled environment.
Non-Critical System: The consumption billing system is not critical to our core operations. This means that any teething issues encountered during the migration will not impact our customers’ deployments. It’s an ideal candidate for us to learn and iterate without the pressure of affecting mission-critical systems.
Isolated but Integrative: While it is isolated in terms of its impact on core services, the system still integrates with our message bus and invoice system. This allows us to test and refine our integration strategies with existing systems.

By starting with the consumption billing system, we aim to build confidence and competence within the team while laying a solid foundation for the migration of more critical systems in the future.

Looking Ahead: The Journey with Gleam

The road ahead may present its share of challenges, but the promise of enhanced reliability, boosted developer productivity, and a scalable foundation makes every step worthwhile. We’re thrilled to keep sharing our progress as we evolve our architecture, ensuring we continue to deliver outstanding value to our customers.

Stay tuned!