Running simulations on thousands of graphics processing units (GPUs) in parallel is critical to Level 5’s success in testing and improving how AVs respond to various driving situations. “Simulation is one of the key ways we improve the safety of our software before it goes anywhere—even a test track,” says Timothy Perrett, senior staff engineer at Lyft Level 5. Exploring the simulation space (such as varying the speed, position, or vehicle dynamics) requires repeated testing and thus a lot of computing flexibility.

Early on, it was clear that Level 5 would have very different computing needs than Lyft’s rideshare business. “Level 5 has different needs and constraints,” says Perrett. “Most of our computing needs are in servicing large, batch-style workloads that have a very spiky profile. We need the ability to burst up to high peak loads and then quickly turn everything down when we’re not using it.”

Lyft could have invested in on-premises central processing units and GPUs, but the Lyft team’s prior experience on AWS made the AWS Cloud its first choice. So the testing began. Level 5 engineers started by utilizing capacity from Amazon EC2 On-Demand Instances, in conjunction with Amazon EKS, the fully managed Kubernetes service that AWS offers.

After experimenting with running simulations using On-Demand Instances, Lyft’s Level 5 team quickly realized it could improve efficiency and reduce costs by shifting to Amazon EC2 Spot Instances. Now more than 90 percent of the simulations run on Amazon EC2 Spot Instances, including Amazon EC2 P3 Instances powered by NVIDIA V100 Tensor Core GPUs, and that enables Lyft to take advantage of unused Amazon EC2 capacity in the AWS Cloud at up to a 70 percent discount compared to On-Demand pricing. “When we experimented with running on Amazon EC2 Spot Instances, we realized that as our program was growing quickly, there was an opportunity to significantly reduce our operational costs,” says Perrett.

The Level 5 team distributes its simulation workload in what Perrett calls a “clever dance” to ensure that simulations still run even when Amazon EC2 Spot Instances aren’t available because of high demand. Engineering staff observed which clusters—and pools within those clusters—operated efficiently and took into account regional zone usage. “We became smarter about how we allocate work and how we relocate jobs in a given resource pool on a given day,” Perrett notes. The team used Amazon EKS to prioritize and scale resource pools so jobs were efficiently using instances.

The engineering team was also careful to design systems so that simulations would function on a variety of hardware, depending on what was available—something Lyft calls fleet diversity. Perrett explains, “We put a lot of effort into making our stack work on whichever type of instance is available—Amazon EC2 P3 Instances versus the Amazon EC2 P2 Instances, for example.” This flexibility helps Level 5 engineers avoid having to wait to schedule simulations, even when demand is high.

Lyft also has to manage a massive amount of data gathered from simulations and from its AV fleet, and it takes advantage of Amazon Simple Storage Service (Amazon S3) to store and access an ever-expanding dataset as Lyft increases the number of sensors on its test vehicles. Gathering and storing all that information from its AVs and simulations amount to petabytes of data, and transferring that amount of data directly to the cloud, as the Level 5 team did in the early days, was costly. To reduce that cost, Lyft uses AWS Direct Connect, a dedicated network connection between its Level 5 engineering center and its cloud systems. “We have a very high-capacity network that connects to the places where we operate our AV fleet,” Perrett notes. “And then we upload the data for a much lower cost per petabyte.”

By carefully partitioning and directing its simulation traffic on Amazon EC2 Spot Instances, Lyft’s Level 5 engineering team reduced the cost of simulations to just pennies for each execution. “About 77 percent of our computing fleet across all Level 5 workloads—and over 90 percent of our AV simulation workload—is now on Amazon EC2 Spot Instances, and the cost savings overall has been around two-thirds,” says Perrett. “We were able to scale up our computing capacity significantly while reducing the overall cost of operation.”

Benefits of AWS