Zopa is both a ‘finance’ and a ‘technology’ company. It is our passion for technology that enables us to deliver better and more innovative financial services and products. Our platforms are complex curations of in-house developed applications, third-party software, libraries and tools, and all the infrastructure needed to support them.
In this post, I’ll cover three reasons why we chose Kubernetes to power our application platforms.
A challenge we have faced at Zopa has been continuing to deliver infrastructure on time, as both the company and number of technologies have experienced rapid growth. Any infrastructure process which is not entirely automated becomes difficult to scale and leads to delays and disappointment for developers blocked, waiting for their infrastructure to be provisioned.
Enter Kubernetes
Kubernetes is an orchestration platform that builds upon Docker – a containerization technology that we previously adopted with great success. With Kubernetes, you can build a fully automated application platform with deployment, routing, service discovery, high availability, self-healing and auto-scaling and all delivered by a good REST API which is easy to automate against.
Kubernetes is a rapidly evolving technology, with a huge amount of activity, and we have already found many benefits and compelling reasons for choosing it to power our platform, but here are my top three:
1. Full automation at last!
Kubernetes benefits everyone by automating more of the application deployment lifecycle without having to develop our own code. Applications and services can be provisioned by developers without needing any support from infrastructure teams, meaning fewer tickets and goodbye to endless waiting for infrastructure to be provisioned.
Some of the benefits from Kubernetes automation: -
- Automation across wide array of platform technologies via a common and consistent restful API.
- New application provision now measured in seconds.
- Automated deployments and roll-back.
- Less boring and repetitive work for infrastructure engineers.
- Give more control to happier developers (how DEVOPS!)
2. Scheduled infrastructure can heal!
At the heart of Kubernetes is a scheduler which decides what applications and jobs need to run and where. Unlike traditional manually planned environments, it can choose to run an application instance on any node (server) in the platform that has enough available resources. It is always running, inspecting the platform and making any changes where it needs to achieve the desired state – and it can heal and it can grow in response to application load.
In the diagram below, we can see a platform with three nodes (the servers that run our stuff) and we also have three applications creatively named ‘App A’, ‘App B’ and ‘App C’. We have told the platform that we desire two instances of ‘App A’ but one of the nodes suddenly and quite spectacularly halts, catches fire, and burns to the ground! The scheduler is informed and sees that we still desire two instances of ‘App A’ and so it schedules a copy to run on the next available/best node, and it does this in a fraction of the time it would have taken to page the support staff about node failure.
Scheduling gives us the following benefits: -
- Better utilization of physical servers.
- Enforce high availability by scheduling application containers to different physical hosts, datacentres or availability zones.
- Repair hardware failures by re-scheduling application containers to other available servers.
- Makes capacity management far easier as we only need to manage total cluster capacity.
- Can optionally federate applications across different geographic regions or cloud providers.
3. Leverage the Community
Kubernetes is one of the top open source projects on github and the most active. It is backed by Google, Redhat, CoreOS, Rancher and other major vendors, and is under the management of the Cloud Native Computing Foundation. Anyone who has been in the industry as long as I will appreciate the wisdom of not rolling-your-own solution for every problem you encounter, as great solutions today grow into nightmares to support tomorrow. Just imagine the size of infrastructure/development team you would need to keep up with the most active project on github?!
Here’s how we benefit: -
- An extremely large and active developer community.
- Avoid re-inventing the wheel with an inferior home-rolled solution.
- Codify design patterns and industry best practices and best of breed security.
- Talent pool and training out in the market.
Platforms are complex and Kubernetes is no exception – it’s packed full of features and patterns with lots more to talk about as to how it delivers on the benefits espoused here. We have built several Kubernetes environments and are using them to provide more automation and more resilient platforms through scheduling.
I hope that this has been a useful introduction to the why we at Zopa are adopting this exciting technology. There will be more on the how in later posts.
(Image credit to Michael Gaida - Pixabay
David is an infrastructure specialist, consultant, and sometimes manager, specialising in Linux and Web platforms with 15+ years’ experience of building large scale mission critical platforms for telecommunications and financial trading companies. He 'loves' DEVOPS, infrastructure automation, AWS, configurations management and continuous delivery, and firmly believes that containerization, platform-as-a-service and believes that scheduled application platforms are key to future improvements in application platforms and organisational agility. He joined Zopa in 2016 to help implement our Kubernetes platforms.