Setting up Selenium Hub in Kubernetes

Femi Alashi -

Setting up Selenium Hub in Kubernetes
This tutorial assumes that you can write and set up a basic selenium test automation framework. This tutorial will focus on setting up selenium hub in kubernetes infrastructure, explaining some of the basic terminologies and how to set it up.

Definitions –

  1. Kubernetes – is an open-source system for automating deployment, scaling, and management of containerized applications
    Another definition that I like is – k8 is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation.
  2. Cluster – A Kubernetes cluster consists of a set of worker machines, called nodes, that run containerized applications. Every cluster has at least one worker node.
    The worker node(s) host the Pods that are the components of the application workload. The control plane manages the worker nodes and the Pods in the cluster. As seen in the diagram below:

Tools

  1. GitHub Repo
  2. IDE – (VS Code Suggested)

Installation
In this tutorial we will be going through how to set up kubernetes on your local machine as well as in cloud – AWS and GC.

  1. Local environment:
    Install VSCode, install the Kubernetes plug and its dependencies and go through the steps below:

    1. Brew – install here
      /bin/bash -c “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)”
    2. VM: Hyperkit Driver – Install (Optional)
      1. brew install docker-machine-driver-hyperkit
      2. sudo chown root:wheel ~/.minikube/bin/docker-machine-driver-hyperkit
      3. sudo chmod u+s ~/.minikube/bin/docker-machine-driver-hyperkit
    3. Minikube – Installation link
      1. brew install minikube
      2. sudo mv minikube /usr/local/bin
    4. Kubernetes CLI –
      1. brew install kubernetes-cli (This is automatically installed with minikube)
    5. Kubectl Install here (Upgrade)
      1. brew install kubectl
      2. brew install kubectl 1.18.0
      3. chmod +x ./kubectl
      4. sudo mv ./kubectl /usr/local/bin/kubectl
      5. Verify version – kubectl version –client
    6. Start Cluster
      1. minikube start (will automatically download hyperkit)
      2. minikube dashboard
    7. Setting up Selenium Hub needed to run test in Kubernetes
      1. We need to create a deployment app in a deploy.yml file that has all the set up needed to create a selenium-hub. The file contains the following:
        1. The application version – appVersion
        2. What kind of app we wanna establish – Deployment
        3. The metadata to identify the pod. In this case we use – name
        4. The spec
          1. The spec has a template format, where we need to set up the image that needs to be deployed in the container.
          2. The name of the container and the port that we want to expose and for selenium we will be using the default – 4444
      2. To deploy a pod
        1. There are several ways to deploy or start a pod within the namespace of a cluster. We are going to deploy our pods directly from the file. Using the command:
          Kubectl create -f
        2. To get the list of the pods in the cluster, we are going to use the command:
          Kubectl get pods
        3. To get all the information about a particular pod or function, we are going to use the command:
          Kubectl describe pod
        4. You’ll get the pod name from the metadata name tag or from the list provided from “kubectl get pods”
      3. Set up Service to expose the application
        1. When the selenium-hub application is deployed, we need a way to expose the 4444 to other nodes that want to connect to the hub.
        2. So we create a service application and name it whatever you like but in this case I named it – service.yml
        3. The service.yml file contains the following with some explanation of the parameters mean.
          1. The apiVersion, as the name implies is the apiVersion provided by the Kubernetes team.
          2. What kind of app are we looking to set up; this kind is – Service
          3. Metadata – we need a way to identify the pod and within the metadata we have –
            1. Name – The name of the pod.
            2. Label – We also add a label to identify the application, because the service is an application.
          4. Spec – We also add the spec to provide more information on the application as well. The Service spec has the following –
            1. A selector – which is used to define the application that we want to expose its port to other services within the namespace or cluster.
              1. App – the name of the app that the service needs to select and expose its ports.
            2. Ports – The application identified in the selector has its ports and the new port(s) that we are going to map it to for other services to use as a point/port of access
              1. Port -The original port of the application that the service is exposing its port and in this case its the selenium-hub which is 4444.
              2. nodePort – we are using nodePort to expose the port of the application by mapping the new port. It is good practice to use ports in the 30000-39999 range.
              3. Nodeport – We need to tell the service application to use nodeport.
        4. Create a Replica Controller –
          1. We are going to use the replica controller to ensure that we have control over the pods lifecycle. The replica controller helps to ensure that the number of pods we declared and instantiated remains the same. Like in the replica.yml you’ll notice that we declared 2 selenium-chrome to be live and running in the namespace or cluster.
          2. In summary the replica controller will do these things for us –
            1. Create the replica controller kind with the number of browser nodes needed to stay alive, running and active.
            2. Connect to the service to provide the bridge between the grid and the individual nodes.
          3. The replica.yml contains the following –
            1. apiVersion – By now you understand what the apiVersion is used for in the yml file.
            2. Kind – The feature kind here is replica controller and in summary, is used to maintain the lifecycle of pods declared.
            3. Metadata – because we need to identify the replica controller pod
              1. Name – we use the name of the pod to identify the pod via its metadata. The name in our replica.yml file is – selenium-rep
            4. Spec – Lets define the specification of the application
              1. Replicas – As its name implies, it is used to determine the number of replicas of the pod that is needed to be up and running always.
            5. Selector – This is used to identify the application or container that needed to be selected for a particular action.
                1. App – The app in the case is the: selenium-chrome which is defined within the “template” first
          4. Please note that the replica controller does not consist of just these parameters, we are only declaring the ones that we need for this topic.
            Create the replica controller kind to determine the number of browser node needed.
            Connect to the service to provide the bridge between the grid and the individual nodes
    8. Run tests in the Selenium Hub.
      1. To get the IP – there are several ways to get the IP, the one I recommend which will also be easier for you to remember is using the kubectl to describe the selenium-hub.
        1. Kubectl describe pod selenium-hub
        2. All the information about the selenium-hub will be rendered. You want to pay attention to the value of the Node – minikube/XXX.XX.XX.X
        3. The IP value will be listed as the value of Node
      2. Selenium test suite –
        1. The document assumes that you know how to run selenium tests remotely using the remote function provided by selenium.
        2. To run tests in the Kubernetes cluster; your remote url needs to be updated with the IP obtained from describing service.yml and the nodePort provided. For example ⇒ http://XXX.XX.XX.X:30001/wd/hub
        3. Run your tests with your usual commands.
        4. TIP – Write a script to check that the hub’s availability before running test – http://XXX.XX.XX.X:30001/wd/hub/status
    9. Quick Kubectl commands
      1. Kubectl get pods
      2. Kubectl create -f
      3. Kubectl delete pod
      4. kubectl delete replicationcontroller selenium-rep
      5. kubectl delete deployment selenium-hub
      6. kubectl delete svc selenium-srv
      7. Kubectl describe <function>
        1. Functions can be – deploy, svc – kubectl describe svc
        2. Kubectl describe pod kubectl describe replicationcontroller selenium-rep
    10. Debugging
      1. Download VNC from Chrome
      2. Replace POD_NAME with the name of the pod you want to connect to.
      3. kubectl port-forward $POD_NAME 5900:5900
      4. Then connect to localhost:5900 with your VNC client using the password “secret”
  2. Cloud Environment – Coming soon.
    1. Amazon EKS – Managed Kubernetes Service or
    2. Google Kubernetes Engine.
author img

Femi Alashi

Over 7 years of excellent, broad experience and knowledge on the client-server and web-based application in Functional/ Non-functional testing using all test automation frameworks from Java-selenium, RSpec, Minitest, NightwatchJS to mention a few.

Leave a Reply

Your email address will not be published. Required fields are marked *