As Reaction Commerce partners, we often encounter questions about how to deploy Reaction Core and its dependent services (Reaction Identity, Reaction Hydra, and Reaction Admin). There are many different ways to approach deploying software and, as is often the case, the best method for you depends on what you’re trying to achieve. The infrastructure required to support a small t-shirt shop can (and should) look very different to the infrastructure required to support an enterprise retailer with millions of customers.
Here at Slingshot Labs, we are huge fans of both Kubernetes (pronounced koo-burr-NET-eez) and Reaction Commerce. What’s even more exciting, is we think Reaction and Kubernetes fit together perfectly. On the one hand you have Reaction Commerce, an open-source, modular, headless commerce stack. On the other, you have Kubernetes, an open-source system for automating deployment, scaling, and management of containerized applications. In short, k8s is built to manage Reaction at scale. While k8s sometimes get a bad rap for being more complex than it really is, k8s was built on the shoulder of giants, like Google, who have been battle-testing it for over a decade. Once you wrestle with the initial learning curve, it unlocks a world of opportunity and capability that can give you the scalability and stability you need to sleep at night knowing your shop and customers are in good hands.
Today, the Slingshot Labs infrastructure team explores what it takes to deploy the open source (OSS) components of Reaction on Kubernetes (k8s). Kubernetes offers auto-scaling groups, logical volume management, and a declarative approach to defining resources and deployment, all out of the box. It makes automation a breeze! We believe running Reaction to top of Kubernetes is ideal for mid to enterprise scale deployments. We are also pleased to announce we are open sourcing the Helm chart we’ve developed to do exactly that! The chart provides an easy way to bootstrap a deployment to k8s and provides the flexibility to fine tune for product environments.
The below steps lays out in detail how you can deploy RC on k8s. We also outline what it takes to setup DNS, get the platform running with proper SSL termination, and more. You’ll need some basic, working knowledge of helm, kubectl and docker as these commands won’t be explained in detail here. Briefly, Docker is the most popular container platform in the world and allows you to isolate and pack self-contained applications. Kubernetes (accessible via the CLI tool kubectl) is a orchestration platform for docker and other containerization frameworks. Helm is a package manage that sits on-top of Kubernetes and helps standardize the deployment and release flow therein.
1. Clone the Reaction OSS Helm chart repo:
git clone https://github.com/slingshotlabs/reaction-oss-helm-chart.git
2. Make a copy of the values.yaml file:
cd reaction-oss-helm-chart && cp ./values.yaml ./my_values.yaml
The values file contains the configuration arguments. The chart provides sane defaults for most things out of the box. It is strongly recommended that you change the values of keys with a value of ‘changeMe’ if you intend to use the referenced functionality. You can also optionally create a values file with a subset of the required arguments and chain the files together with multiple “-f” flags in helm; this is useful for continuous deployment pipelines that serve multiple environments.
3. Edit the global section in “./my_values.yaml”
The only values that are required for full functionality are the stripe and segment api keys. Out of convenience, the chart packages a MongoDB replica set. By using this replica set, the values for the MongoDB items in the global section are not required. It is recommended that you do not operate databases (or any kind of persistence layer) on k8s. If you wish to bring your own replica set, set these values using the mongo replica set connection string format.
4. Edit the admin section
Set the host key to the FQDN (full qualified domain name) of where you would like to host the admin interface, it is recommended that this is a subdomain of your root domain. In the example above, we’ve set it to
admin.example.shop. If you’re using a more traditional TLD, it might look like
admin.slingshot.com. Optionally, you can configure
replica count and
SSL. Replica count denotes the number of instances to run behind your service. The SSL key generates “https” and “wss” (secure web socket) URLs for the public facing configuration variables inside of RC, set to
true to turn on.
5. Edit the api section
Once again, you should set
host to the FQDN of where you would like to host the api service. In the example above, we set ours to
api.example.shop. If you have decided to bring your own MongoDB replica set, you can optionally initialize that replica set using the
initReplicaSet key. To enable email functionality in RC, you can pass an SMTP connection string to the
6. Set the FQDN in the identity section
Rinse and repeat the FQDN directions in step 5 for the
7. (Optional) Set the FQDN in the web section
For evaluation purposes, the RC example storefront has been included as part of the helm chart. You can choose to disable this via the
enabled key in the
web section. If you choose to deploy the example storefront, make sure to set the FQDN.
8. (Optional) Set the FQDN in the mongodb section
If you have chosen to bring your own MongoDB replica set, disable the inbuilt deployment via the enabled key in the
9. Configure ingress deployments
Every deployment in the chart with the exception of Postgres and Mongo, require a public address. The easiest way to achieve this is through an ingress. An ingress exposes routes outside the cluster and forwards them to services deployed on the cluster.
Go through each section and set
enabled: true under
ingress and (optionally) set
enabled: true. Most ingress controllers require specifying some sort of annotation on the deployment as a means of picking it up.
If you’re using an nginx controller, the annotation to use would be:
On the admin ingress you might also want to increase buffer size and read-timeout. Buffer size because the request headers for the interface exceed the default max of the controller. Read-timeout to ensure that websockets stay alive.
In nginx, this would be:
annotations: kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/ssl-redirect: "false" nginx.ingress.kubernetes.io/proxy-buffer-size: "128k" nginx.ingress.kubernetes.io/proxy-buffers-number: "4" nginx.ingress.kubernetes.io/proxy-send-timeout: 3600 nginx.ingress.kubernetes.io/proxy-read-timeout: 3600
Deploying the Chart
You are now ready to deploy the chart on your cluster, make sure to have your ~/.kubeconfig file set up with your credentials for accessing the cluster. If you’re using a cloud provider, they usually provide a CLI tool for making the process of generating the file simpler.
1. Pull down chart dependencies:
helm dependency update
2. Deploying the chart is fairly simple:
helm install reactioncommerce . -f ./my_values.yaml
3. You might want to give your configuration values a quick test first:
helm install reactioncommerce . -f ./my_values.yaml --dry-run
This will generate kubernetes manifests (based on your config) and print them to stdout.
Assuming the deployment was successful, you should now be able to see your running pods, services and ingress:
Services are abstractions that define a logical set of pods:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 7d4h nginx-ingress-nginx-controller LoadBalancer 10.100.35.91 <REDACTED> 80:31935/TCP,443:32226/TCP 7d3h nginx-ingress-nginx-controller-admission ClusterIP 10.100.203.35 <none> 443/TCP 7d3h reactioncommerce-admin ClusterIP 10.100.198.228 <none> 4080/TCP,9231/TCP 7d reactioncommerce-api ClusterIP 10.100.175.173 <none> 3000/TCP 7d reactioncommerce-hydra ClusterIP 10.100.137.162 <none> 4444/TCP,4445/TCP,5555/TCP 7d reactioncommerce-identity ClusterIP 10.100.53.39 <none> 4100/TCP 7d reactioncommerce-mongodb ClusterIP 10.100.236.195 <none> 27017/TCP 7d reactioncommerce-mongodb-headless ClusterIP None <none> 27017/TCP 7d reactioncommerce-postgresql ClusterIP 10.100.97.137 <none> 5432/TCP 7d reactioncommerce-postgresql-headless ClusterIP None <none> 5432/TCP 7d reactioncommerce-web ClusterIP 10.100.69.221 <none> 4000/TCP 7d
Exposing the platform to the outside world
Now that we have RC deployed, it’s time to make sure we can access our services from the outside world. The helm chart creates our ingress resources for us, but those are just constructs - in order to proxy those resources we need an ingress controller.
Nginx is probably the most popular controller (second only to HAProxy). We can deploy a controller fairly simply from the nginx helm repo:
1. Add the repo:
$ helm repo add nginx-stable https://helm.nginx.com/stable$ helm repo update
2. Install the chart:
$ helm install nginx nginx-stable/nginx-ingress
3. Get the public facing address of our ingress controller:
Now that we have our controller up and running, we should be able to get the public facing domain (or ip) of our controller:
[[email protected] ~]$ kubectl get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 7d4h nginx-ingress-nginx-controller LoadBalancer 10.100.35.91 <REDACTED>.us-east-2.elb.amazonaws.com 80:31935/TCP,443:32226/TCP 7d3h
The “EXTERNAL-IP” column against the controller will either give us the IPv4 address or the FQDN of the load balancer that is terminating our ingress controller, in the output above the value we are looking for is ‘<REDACTED>.us-east-2.elb.amazonaws.com’.
4. (Optional) Get the underlying IPv4 address of the load balancer:
If the value is a FQDN, we’ll need to get the IPv4 address that the FQDN resolves to, otherwise skip ahead to the next section. To so so run the command:
You will get an output similar to the below:
; <<>> DiG 9.11.18-RedHat-9.11.18-1.fc32 <<>> example.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29393 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;example.com.???IN?A ;; ANSWER SECTION: example.com.??21118?IN?A?220.127.116.11 ;; Query time: 21 msec ;; SERVER: 192.168.1.1#53(192.168.1.1) ;; WHEN: Tue May 26 21:56:08 BST 2020 ;; MSG SIZE rcvd: 56
The answer section in our dig output will give us the IPv4 address, based on the example above we are looking for
Now that we have a public address, it’s time to configure our DNS. As all services are being terminated via our nginx controller, we only have to contend with a single address in DNS. From this point, all routing happens through virtual hosts.
Most domain name registrars provide a mechanism to configure DNS. If they do not, there are a number of free/paid options out there like no-ip. I would recommend Cloudflare, which isn’t strictly a DNS provider but can serve as one for this use case.
Within your DNS management interface:
- Create an A record for your domain pointing to the public facing IPv4 address of the ingress controller, in the above example that would be ‘18.104.22.168’.
- Create CNAME records for each of the “host” keys you specified in your chart config, these should target your own root domain, so if your storefront is hosted ‘example.shop’, the value of these CNAME records should also be ‘example.shop’.
You should now be able to access all of your RC services publicly via the URLs that you specified as part of the configuration. To begin the process of setting up a shop, visit your admin URL and click register; the first registration will be marked as the owner of the store:
This is our ‘Getting Started’ approach to deploying RC on k8s. We have not covered production hardening or any of the additional tooling that you would need for an enterprise deployment (ie automated release management, continuous deployment, auto-scaling, etc.).
If you are evaluating Reaction Commerce, looking for help with a production deployment, or just want to chat software, e-commerce, or kubernetes, feel free to reach out to us.