Cloud Infrastructure
The Vertex Platform is designed to run as a cloud native application on a Kubernetes cluster.
Cloud environment
All modules of the Vertex Platform are designed to be cloud agnostic, which means that we can run on any Kubernetes clusters both in the public-, hybrid- or in the private-cloud. We can use managed services from a specific cloud provider like an object store, key vault or any OpenId provider as authorization and authentication provider. If these managed services are not available we can run Kubernetes based alternatives for these services. These services can provide these services from within the Kubernetes cluster itself.
To run the Vertex Platform we need the following resources.
Kubernetes cluster
The main resources that is needed by the Vertex platform is a Kubernetes (K8S) cluster. Embracing Kubernetes as our orchestrator enables us to create a scalable high-available application with autoscaling possibilities using commodity hardware / services from different cloud providers. Another befit is that most data-intense companies already have a support structure to support k8s clusters. It also means that delivery pipelines can be made out of common pipeline solutions. All Vertex modules are setup for a cloud native deployment to a Kubernetes cluster (for more information see the CI/CD documentation). The cluster should provide the basic kubernetes components like a load balancer, a reliable performant kubernetes network and providing the storage solution for a persisted volume claim (pvc). The k8s cluster should have a pool of nodes that provide sufficient resources to run, update and failover the application. Ideally this node pool is automatic provisioning, updated and spread across availability zones providing High Availability support for the cluster.
Object store
An object store is needed to persist unstructured data like documents, messages, datasets and snapshots. This object store should be able to provide persistent storage with availability, storage capacity, redundancy, backup and restore options that match the requirements of the objectives of the Vertex deployment. Ideally the object store implement an S3 like interface directly but a integration with minio is sufficient to be used and should include all major object stores.
MSSQL database
One of the main databases used in the Vertex Platform is a Microsoft SQL Server database. The cloud environment should be able to provide a manged MSSQL instance or have the option to create a Virtual Machine that is capable of running an MSSQL setup. Depending on the objectives of the Vertex deployment this managed instance or VM setup should be high available and split across multiple data centers and/or regions.
Key vault
A key vault is used to store secrets for the application configuration and for the infrastructure. This makes it possible to use secrets in the deployment and to keep them in a safe place. Ideally a managed key vault is used from the cloud provider as permissions can then be easily used by the customer and the people handling the actually secret can be kept as small as possible. If the customer already has a key-rotation protocol/automation in place the chance is big that it already includes the managed key-vault solution of the cloud provider. It is possible to use different key vaults for for the infrastructure and the application to further limit the people who have access to secrets. The following 2 vaults can be distinguished:
Infrastructure vault
The infrastructure vault contains secrets to access container registries, git repositories and credentials for access to other cloud resources. With a key vault secrets provider in Kubernetes the secrets can be mounted as a local filesystem where the secrets are available as simple text files.
Application vault
The application vault is used to store business secrets. This vault can be used to hold all sensitive application configuration like connection strings, API credentials or encryption keys.
Container registry
See the CI/CD documentation
Git repository
See the CI/CD documentation
Cloud providers
The Vertex Platform has been designed to be cloud agnostic, however sometimes a specific integration might be needed. We have these type of dependencies interchangeable from the GitOps configuration. For example we might use a specific key vault of the cloud provider instead of using the generic Kubernetes hosted vault solution and might need to add a specific implementation to our applications or kubernetes configuration.
Microsoft Azure Cloud
For the Azure cloud we make active use of the following components:
- Azure Kubernetes Services (AKS)
- Azure Storage Account
- Azure Key Vault
- Azure Managed SQL
Cloud architectures
There are multiple soltions on how to setup a cloud environment.
Hub spoke network model
We have good experience with a hub-spoke model build in the Azure cloud. In this architecture the spokes are not directly connected to the internet but both in- and outgoing connections are routed through the hub that contains a Azure Firewall. This the AKS cluster itself is not directly internet-connected. Azure resources can be reached within the same virtual network using private endpoint links. We use hub-spoke as a networking model for efficiently managing common communication or security requirements. The hub containing the firewall and a service (Azure Bastion) for accessing VMs is maintained by the customer, the landing zone (including policy enforcement) for the spokes as well. Energy Essentials can automated the deployment of Azure resources into a spoke in an automated manner, using Terraform scripts and an Azure pipeline. Each DTAP environment is separated into a separate spoke, making it possible to set separate network policies per environment and keeping isolated environments.