Blue/Green deployment strategy

film-storage architecture

To achieve Blue/Green deployment with Cloud Native applications using Argo Rollouts, we have designed this architecture.

This is the initial situation after deploying the application:

Initial status

OpenShift Components - Online

  • Routes and Services declared with the suffix -online.

  • Routes mapped only to the online services.

  • Services mapped to pods created by the rollout.

OpenShift Components - Offline

  • Routes and Services declared with the suffix -offline.

  • Routes mapped only to the offline services.

  • Services mapped to pods created by the rollout.

We will define an active or online service 'film-storage-online' and a preview or offline service 'film-storage-offline'. The idea is that the frontend will always use 'film-storage-online' and 'film-storage-offline' will only be used to test our service before triggering the rollout.

Generate film-storage APP GitOps Deployment Model

First of all, it is required to review a set of objects in our forked git repository to configure the deployment correctly for deploying our application following a GitOps model.

Please follow the next section to understand the required objects and configurations.

Review Services

As the previous picture shows, it is required to generate a couple of services that will be used to redirect the traffic to the correct application version.

Please take a look at the files ./blue-green/film-storage-service-offline.yaml and ./blue-green/film-storage-service-online.yaml to understand the objects and get the names of these elements.

Modify Rollout strategy

The Rollout object controls the application deployment. If you look closely, it is possible to find many similarities between an Argo Rollout object and a Kubernetes Deployment. The main difference between the previous objects referenced is the rollout strategy field.

Please edit the file named ./blue-green/film-storage-back-rollout.yaml in order to add the required <SERVICE_OFFLINE> and <SERVICE_ONLINE> service names:

  strategy:
    blueGreen:
      activeService: <SERVICE_ONLINE>
      autoPromotionEnabled: false
      previewService: <SERVICE_OFFLINE>

Modify Analysis Template

We have created an AnalysisTemplate 'film-storage-analysis-template' that just validates the health of the application, for production environments a better analysis should be done. Argo Rollouts uses this AnalysisTemplate to validate a new version and set whether it is ready to be promoted or not.

Please edit the file named ./blue-green/film-storage-analysis-template.yaml to add the required <USERNAME> that helps to reference the user namespace in the healthcheck URL.

apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  labels:
    app: film-storage
  name: film-storage-analysis-template
spec:
  metrics:
    - name: film-storage-webmetric
      provider:
        web:
          jsonPath: '{$.status}'
          url: >-
            http://film-storage-offline.<USERNAME>-blue-green.svc.cluster.local:8080/api/v1/health
      successCondition: result == 'UP'
For more information about AnalysisTemplate, please visit the following link.

Commit and Push the Changes

Once all the files are configured correctly, it is time to commit and push all changes to the repository that you have just forked.

git add .
git commit -m "Configured film-storage blue/green deployment files"
git push

Argo CD Application

We are going to create the ArgoCD application film-storage-blue-green, to apply the files modified previously to test Blue/Green deployment. Please, edit the .yaml file named ./argocd/film-storage-blue-green.yaml, and set your own GitHub repository in the repoURL.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: film-storage-blue-green
  namespace: %USER%-gitops-argocd
spec:
  destination:
    name: ''
    namespace: %USER%-blue-green
    server: 'https://kubernetes.default.svc'
  source:
    path: blue-green
    repoURL: https://github.com/change_me/workshop-argo-rollouts-resources
    targetRevision: HEAD
  project: default
  syncPolicy:
    automated:
      prune: false
      selfHeal: true
oc apply -f argocd/film-storage-blue-green.yaml
application.argoproj.io/film-storage-blue-green created

Looking at the Argo CD dashboard, you would notice that we have a new film-storage-blue-green application.

film-storage-blue-green
It is important to wait for Healthy and Synced status in the Argo CD Application

Test film-storage application

We have deployed the film-storage with ArgoCD. We can test the online and offline services to ensure that they are up and running.

To test the online service, it is required to extract the URL:

oc get route film-storage-online -n %USER%-blue-green -o jsonpath='{.spec.host}'

In addition, regarding testing the offline service, it is required to get the offline route:

oc get route film-storage-offline -n %USER%-blue-green -o jsonpath='{.spec.host}'

We can test the endpoint with the following request:

curl -s $(oc get route film-storage-offline -n %USER%-blue-green -o jsonpath='{.spec.host}')/api/v1/info | json_pp

The result:

{
   "app" : {
      "buildTime" : "2023/01/10 15:17:57",
      "name" : "film-storage",
      "version" : "0.2"
   }
}

Notice that in each microservice response we have added metadata information to see better the version of each application. This will help us to see the changes while we do the Blue/Green deployment. Because right now we have both routers against the same rollout revision we will have the same response with version 0.2:

We can also see the rollout’s status.

Argo Rollouts offers a Kubectl plugin to enrich the experience with Rollouts https://argoproj.github.io/argo-rollouts/installation/#kubectl-plugin-installation 
oc argo rollouts get rollout film-storage-blue-green --watch -n %USER%-blue-green
NAME                                                 KIND        STATUS     AGE  INFO
⟳ film-storage-blue-green                            Rollout     ✔ Healthy  60s
└──# revision:1
   └──⧉ film-storage-blue-green-644f5f6df7           ReplicaSet  ✔ Healthy  47s  stable,active
      └──□ film-storage-blue-green-644f5f6df7-bhq6z  Pod         ✔ Running  46s  ready:1/1

film-storage Blue/Green deployment

We have split a Cloud Native Blue/Green deployment into two steps:

  1. Deploy a new version.

  2. Promote a new version

To test Blue/Green deployment we are going to do changes in the backend application film-storage.

We already deployed the film-storage version 0.2, and we have a new film-storage version 0.3 ready to be deployed.

Step 1 - Deploy a new version

We will deploy a new version 0.3. To do so, we have to edit the files ./blue-green/film-storage-back-rollout.yaml to modify the value 0.2 to 0.3:

./blue-green/film-storage-back-rollout.yaml
          image: quay.io/dborrego/film-storage:0.2 <---

And push the changes

git add .
git commit -m "Change film-storage version to 0.3"
git push

ArgoCD will refresh the status after some minutes. If you don’t want to wait you can refresh it manually from ArgoCD UI.

Refresh film-storage blue/green

Argo Rollouts will automatically deploy the new film-storage version and execute the prePromotionAnalysis.

oc argo rollouts get rollout film-storage-blue-green --watch -n %USER%-blue-green
NAME                                                 KIND         STATUS        AGE    INFO
⟳ film-storage-blue-green                            Rollout      ॥ Paused      19m
├──# revision:6
│  ├──⧉ film-storage-blue-green-7c455c98c8           ReplicaSet   ✔ Healthy     4m36s  preview
│  │  └──□ film-storage-blue-green-7c455c98c8-m7bwh  Pod          ✔ Running     36s    ready:1/1
│  └──α film-storage-blue-green-7c455c98c8-6-pre     AnalysisRun  ✔ Successful  27s    ✔ 1
└──# revision:5
   └──⧉ film-storage-blue-green-644f5f6df7           ReplicaSet   ✔ Healthy     19m    stable,active
      └──□ film-storage-blue-green-644f5f6df7-bhq6z  Pod          ✔ Running     19m    ready:1/1

If the prePromotionAnalysis goes well, we can see that offline applications have version 0.3 and the new attribute description, but the online version has not changed.

This is our current status:

"film-storage blue/green Step 1

You can check the deployment with the following requests:

curl -s "$(oc get routes film-storage-online -n %USER%-blue-green --template='http://{{.spec.host}}')/api/v1/info" | json_pp
{
   "app" : {
      "buildTime" : "2023/01/10 15:17:57",
      "name" : "film-storage",
      "version" : "0.2"
   }
}
curl -s "$(oc get routes film-storage-offline -n %USER%-blue-green --template='http://{{.spec.host}}')/api/v1/info" | json_pp
{
   "app" : {
      "buildTime" : "2023/01/23 21:43:23",
      "name" : "film-storage",
      "version" : "0.3"
   }
}

Functional testers can execute tests to validate this new 0.3 version.

Step 2 - Promote a new version

As we have the promotion turned on to manual, we’re going to promote the new version to final users.

Argo Rollouts provides this command to promote film-storage:

oc argo rollouts promote film-storage-blue-green -n %USER%-blue-green

First Argo Rollouts will just change the service to use the new release (ReplicaSet). We minimize downtime because it just changes the service label.

oc argo rollouts get rollout film-storage-blue-green --watch -n %USER%-blue-green
NAME                                                 KIND         STATUS        AGE  INFO
⟳ film-storage-blue-green                            Rollout      ✔ Healthy     43m
├──# revision:6
│  ├──⧉ film-storage-blue-green-7c455c98c8           ReplicaSet   ✔ Healthy     28m  stable,active
│  │  └──□ film-storage-blue-green-7c455c98c8-m7bwh  Pod          ✔ Running     24m  ready:1/1
│  └──α film-storage-blue-green-7c455c98c8-6-pre     AnalysisRun  ✔ Successful  23m  ✔ 1
└──# revision:5
   └──⧉ film-storage-blue-green-644f5f6df7           ReplicaSet   ✔ Healthy     43m  delay:20s
      └──□ film-storage-blue-green-644f5f6df7-bhq6z  Pod          ✔ Running     43m  ready:1/1

This is our current status:

film-storage step2 initial

And after scaleDownDelaySeconds Argo Rollouts will scale down the first replicaSet (0.2).

oc argo rollouts get rollout film-storage-blue-green -n %USER%-blue-green
NAME                                                 KIND         STATUS        AGE  INFO
⟳ film-storage-blue-green                            Rollout      ✔ Healthy     45m
├──# revision:6
│  ├──⧉ film-storage-blue-green-7c455c98c8           ReplicaSet   ✔ Healthy     30m  stable,active
│  │  └──□ film-storage-blue-green-7c455c98c8-m7bwh  Pod          ✔ Running     26m  ready:1/1
│  └──α film-storage-blue-green-7c455c98c8-6-pre     AnalysisRun  ✔ Successful  26m  ✔ 1
└──# revision:5
   └──⧉ film-storage-blue-green-644f5f6df7           ReplicaSet   • ScaledDown  45m

This is our final status:

film-storage blue/green step 2

To test the new online service, you can use again both URL (online and offline):

curl -s "$(oc get routes film-storage-offline -n %USER%-blue-green --template='http://{{.spec.host}}')/api/v1/info" | json_pp
curl -s "$(oc get routes film-storage-online -n %USER%-blue-green --template='http://{{.spec.host}}')/api/v1/info" | json_pp