Game Day 5th Edition

We had the 5th Game Day organised by ManoMano, and if I recall well the second one in collaboration with AWS. The main theme this time was about sustainability: how to reduce resource expenses and carbon footprint by right sizing, moving to ARM architecture, analyzing and compressing data, etc.

AWS Quests

We were split in different groups and had to complete different quests for a fictitious company which goal was to work in a more sustainable way.

The first quest was to reduce the number of instances of a service that were consuming messages. The instances were not even queuing messages since they had more compute power than needed. For SQS to be efficient, we should have some message queuing. Also, we observed through CloudWatch that the CPU usage of an instance was around 15%. Given the different collected metrics, we tried to right size the instances. We had also some auto scaling mechanism if the instances could not process properly the message traffic. Indeed, we set up an autoscaling group (ASG) with the desired state property set to 0. A scaling up policy based on the CloudWatch alarm (created previoulsy) was spinning up a new instance when the SQS queue reached more than 50 messages. This helped us to even reduce more the cost.

The second quest was about the configuration of an EKS cluster and the deployment of a service (x86 architecture). We used kubectl commands to apply some configurations (port exposition, a load balancer service, etc.):

kubectl apply -f deploy.yml

where the deploy.yml was something like:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gameday-deployment
  labels:
    app: nginx
    architecture: x_86
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
        service: gameday
    spec:
      containers:
        - name: app
          image: <repository>/gameday-images:x86_latest
          ports:
            - containerPort: 3000

## we first tried with the service below
apiVersion: v1
kind: Service
metadata:
  name: gameday-service
spec:
  selector:
    service: gameday
  ports:
    - protocol: TCP
      port: 80
      targetPort: 3000

# then we decided to use a loadbalancer service 
# (hint from quest)
apiVersion: v1
kind: Service
metadata:
  name: gameday-loadbalancer-service
spec:
  selector:
    service: gameday
  ports:
    - port: 80
      targetPort: 3000
  type: LoadBalancer

We had some port exposure issues on the load balancer service which resulted in unhealthy checks. We had to force the port exposure using the following command:

kubectl expose deployment/gameday-deployment \ 
--port=80 --target-port=3000 --type LoadBalancer

After getting our service up and running, we moved to Graviton processors (ARM architecture) in order to reduce carbon footprint and optimize cost.

The two last quests were about analyzing data and optimize its storage. The data was originally stored in S3 buckets and we had to move them to database to be able to easily run queries using Athena. We didn't succeed to have points on these two quests but we got a glimpse of the functionalities of AWS Glue and Athena. Indeed, we used a Glue crawler to replicate the S3 buckets data to a database which offer more flexibility for data manipulation using Athena.

To conclude, I liked the format of the game day where I learnt a lot about the different possibilities to optimize our resources using the different existing services and techniques. We had a great colleague who acted as a mentor and guided us through the different decisions we took. It was fun and thanks to him, we managed to finish second on the overall leaderboard.

Unfortunately, I could not attend two other gameday sessions in the afternoon dedicated to two internal tools but from what I heard, it was also a great experience.