How did Bose solve it’s smart speaker bottleneck using Kubernetes?

What is Kubernetes?

Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications.

What does a “50 year old audio company” have anything to do with Kubernetes?

Bose was looking for ways to support its smart-speaker line of products and was not satisfied with the existing solutions. They chose to partner with a Canadian product development company called Connected, to use Kubernetes as part of the solution.

Problem

The cloud architecture manager for Bose, Dylan O’Mahony, presented this problem on stage in 2018. He asked the nearby Alexa Echo Dot device to play a song, and the nearby Bose smart speaker started playing the tune. One would think that since both the devices are wireless, it would be easier to make it work. Unfortunately, that wasn’t the case, he said. There is no direct interface between the two devices, hence they must communicate in the cloud. To put it in better terms, it would take thousands of miles of underwater cables to bridge the three-foot gap between two devices.

Amazon smart speakers handle all their traffic in the cloud(AWS), which then hands over the instructions to the Bose cloud. The speaker is not directly exposed on the internet; it can send out messages, but it is unable to receive random messages from the net. The easiest way to handle that is to have the speaker make a persistent connection to the Bose cloud when it powers up. MQTT was chosen as the protocol; a persistent bidirectional WebSocket connection is made between each speaker and the cloud service.

The “crux of the problem” is scaling; solutions abound for thousands of connected devices. When they looked around a few years ago for Internet of Things (IoT) products, they couldn’t find any that could handle the five-million (or more) connections envisioned for the system. Some managed services would scale to hundreds of thousands of connected devices, but not to millions. That is why Bose engaged with Connected, which was able to help prototype a system that could handle that many connections using Kubernetes.

Solution

There were a lot of choices they could have made for the different parts that made up the stack, but their methodology was to find the first thing that would work and run with it. The Bose cloud infrastructure was essentially Kubernetes on AWS; it does not use Amazon’s managed Kubernetes product (EKS). That part was handled by a small team at Bose. MQTT is the dominant communications protocol for IoT applications because it is lightweight. The devices set up a persistent WebSocket connection to the cloud, but something other than “magic” is needed to handle all those connections. For that, they used VerneMQ.

There are plenty of examples of configuring Kubernetes for HTTP ingress handling, but there were not many examples for ingress handling of straight TCP, he said. They were not using HTTP, but could not find examples of what they were trying to do so they chose HAProxy since it is the de facto standard for proxying and load balancing. It has been a good choice; it is “rock solid” and uses a “ridiculously low” amount of resources for a huge number of connections.

They put it all together in Kubernetes. The container images were all built from scratch using Alpine Linux, which resulted in small images. Because they are clustering services, VerneMQ and Cassandra are run as Kubernetes StatefulSets; that way, Kubernetes brings them up in order and gives them a consistent name in DNS. HAProxy was configured as a DaemonSet to ensure that it was deployed on each ingress node. The listening service, Prometheus, and Grafana were handled as Deployments; the latter two provided visibility into the cluster for management and diagnosing problems.

Conclusions

Starting out at scale is different than getting there via organic growth. When the system is growing, it will hit certain walls, as they did, but each needs to be surmounted before (sometimes almost immediately) smacking into the next one. Effortlessly scaling a system is a boring feature, but it is a “killer feature of Kubernetes”. It is incredible to be able to simply double the size of a cluster at the snap of your fingers, he said.

Finally, they were surprised by how much cheaper their solution is compared to the alternatives; they expected the cost to be less, but the projections show it to be more expensive at the start, but decreasing (on a cost per device per year measure) over time to a much lower cost than alternatives. No actual numbers were shown, but that is what Bose has calculated. All of that was “only possible through the flexibility of Kubernetes”, Doyle concluded.

In the Q&A, O’Mahony noted how happy Bose is with this work and that moving to Kubernetes is a “really good choice” for the company.