Avil Page (Posts about devops)

Build/Deploy Full Stack Web Apps with only Config

Anand Reddy Pandikunta — Sat, 28 Mar 2026 06:17:29 GMT

The best code is the code you don't have to write

Config First Tools

Any tool that provides first class support for configuration files can be considered a Config First Tool. These tools allow developers to setup/build/deploy applications by writing configuration files instead of writing code.

In DevOps world, tools like Terraform, Ansible, and Kubernetes are examples of Config First Tools. These kinds of tools are not that popular in the web development world.

In this article, lets see how we can build a full stack web application using only configuration files.

Todo - Web App

A simple backend API that allows users to manage todo lists. A simple frontend that uses the backend API. A simple script to deploy the backend and frontend applications.

Backend API

PocketBase is a backend API that provides a simple way to create and manage databases, authentication, and file storage. It is a great tool for building simple backend APIs without writing any code.

$ brew install pocketbase
$ pocketbase serve

We can create a collection called Todo with required fields and pocketbase will automatically generate the API for us.

Frontend

Lowdefy is a config first tool that allows us to build web applications using yaml config.

$ npm install -g lowdefy
$ lowdefy init
$ lowdefy dev

For basic crud apps, instead of us writing the boilerplate config, we can ask an AI agent to write the config for us.

Deployment

For deployment, we can use Ansible to deploy both the backend and frontend applications. Here also we can just AI agent to write the playbook for us.

$ pip install ansible
$ ansible-playbook deploy.yml

OliveTin

Even though we can use ansible to deploy the applications, it is not that convenient to run ansible playbooks on mobile. OliveTin is a config first tool that allows us to create web UIs for our scripts.

  - title: "Deploy ToDo App"
    popupOnStart: execution-dialog
    shell: ansible-playbook deploy.yml
    timeout: 6000
    icon: "🤖"

We can create a simple web UI for our deployment script with just 5 lines.

Conclusion

For quick prototyping and simple applications, config first tools can be a great choice as they allow us to build applications without writing any code. They also allow us to focus on the application logic rather than the implementation details.

With the rise of AI, it is much easy to maintain 500 lines of config than 5000 lines of code.

In Praise of OliveTin

Anand Reddy Pandikunta — Fri, 30 Jan 2026 20:13:11 GMT

In a world of complex internal tools, OliveTin bridges the gap between raw CLI and usable web UI with just yaml config.

Server Admin Panel

I used admin panels like CockPit, Ajenti, etc to provide simple web UI for non-developers to provide access to services. Developing custom widgets is time consuming and requires programming skills as well.

With OliveTin, I can provide web UI directly with just yaml config. This saves a lot of time and effort. It also provides a clean UI as it starts from scratch.

Backend Driven UI

For existing python scripts, I often use tools like Streamlit, NiceGUI, etc to create web UIs. Organizing the UI components for multiple scripts is time consuming.

OliveTin can provide web UI directly python scripts and they can be organized in groups with just yaml config.

Mobile Friendly

To do a simple deployment on mobile, I need to open an app that supports ssh, ssh into the server, navigate to a directory, run a script to deploy.

OliveTin provides a "Deploy" button on mobile browser which is way convenient.

Authentication

OliveTin provides authentication(local users, OAuth2, JWT, etc), authorization(ACLs) & accounting(logs) out of the box. With just yaml config, we can secure the web UI.

Conclusion

There are hundreds of other features provided by OliveTin like scheduling, file uploads, webhooks, etc.

If you ever want to provide a simple web UI for scripts with low code or no code, give OliveTin a try!

Remote Access to k3d/k3s Kubernetes cluster

Anand Reddy Pandikunta — Wed, 17 Dec 2025 03:10:39 GMT

Introduction

We learnt how to deploy kubernetes cluster anywhere with a single k3d command. By default, k3d cluster is accessible only from the host machine.

Remote Access

Create new cluster with

$ k3d cluster create cloud-k8s \
  --api-port 6443:6443@loadbalancer \
  --k3s-arg "--tls-san=<remote-ip>@server:*"

--api-port 6443:6443@loadbalancer maps port 6443 on the host machine to the cluster's API server load balancer. This can be changed (e.g., 8080:6443) if 6443 is taken on the host.
--k3s-arg "--tls-san=..." adds the host's public IP to the certificate, preventing SSL errors later.
Ensure remote server firewall allows incoming traffic on the chosen port (e.g., 6443 or 8080). To verify, run telnet <remote-ip> 6443 from local machine.

On the remote server, get kubeconfig:

$ k3d kubeconfig get cloud-k8s
---
apiVersion: v1
clusters:
- cluster:
    server: https://0.0.0.0:6443
  name: k3d-rk
   ... # truncated for brevity

Copy this kubeconfig, replace 0.0.0.0 with the remote server's public IP.

Open Free Lens, paste this config and connect to the remote k3d cluster.

Conclusion

You can use your favorite Kubernetes tools (kubectl, FreeLens, k9s, etc.) to manage the remote k8s cluster.

Migrating from Nginx to Caddy

Anand Reddy Pandikunta — Fri, 30 May 2025 15:51:21 GMT

When deploying web apps in production, a web server is essential for serving static files, handling reverse proxying, and managing SSL/TLS certificates.

I have been using Nginx for a long time, but I have recently switched to Caddy as it provides automatic HTTPS by default. In addition, Caddy has a simpler configuration syntax and is easier to set up for basic use cases.

Nginx vs Caddy

To illustrate the differences, here are some examples of how to configure a simple web server in both Nginx and Caddy.

Nginx Configuration

server {
    listen 80;
    server_name avilpage.com;

    location / {
        root /var/www/html;
        index index.html index.htm;
    }

    location /api {
        proxy_pass http://localhost:3000;
    }
}

With Nginx, you need to manually handle SSL/TLS certificates, which can be cumbersome. You typically use tools like Certbot to obtain and renew certificates.

Caddy Configuration

avilpage.com {
    root * /var/www/html
    file_server

    reverse_proxy /api localhost:3000
}

Caddy automatically manages SSL/TLS certificates for you, so you don't need to worry about obtaining or renewing them. It also provides default redirection from HTTP to HTTPS, making it easier to secure your site.

Conclusion

Caddy's automatic HTTPS and simpler configuration syntax make it a compelling choice for web servers, especially for those who want to get started quickly without dealing with the complexities of SSL/TLS management.

Free DockerHub Alternative - ECR Public Gallery

Anand Reddy Pandikunta — Sun, 09 Feb 2025 16:08:34 GMT

DockerHub started rate limiting¹ anonymous docker pulls. When testing out a new CI/CD setup, I hit the rate limit and had to wait for an hour to pull the image. This was a good time to look for alternatives.

AWS ECR Public Gallery² is a good alternative to DockerHub as of today(2025 Feb). It is free and does not have rate limits even for anonymous users.

Once we find the required image from the gallery, we can simply change the image name in the docker pull command to pull the image from ECR Gallery.

docker pull public.ecr.aws/ubuntu/ubuntu

In Dockerfile, we can use the image from ECR Gallery as follows:

FROM public.ecr.aws/ubuntu/ubuntu

That is a quick way to avoid DockerHub rate limits.

How (and when) to use systemd timer instead of cronjob

Anand Reddy Pandikunta — Mon, 05 Aug 2024 07:37:50 GMT

Introduction

* * * * * bash demo.sh

Just a single line of code is sufficient to schedule a cron job. However, there are some scenarios where I find systemd timer more useful than cronjob.

How to use systemd timer

We need to create a service file(contains the script to be run) and a timer(contains the schedule).

# demo.service
[Unit]
Description=Demo service

[Service]
ExecStart=bash demo.sh

# demo.timer
[Unit]
Description=Run myscript.service every 1 minutes

[Timer]
OnBootSec=1min
OnUnitActiveSec=1min

[Install]
WantedBy=multi-user.target

We can copy these files to /etc/systemd/system/ and enable the timer.

$ sudo cp demo.service demo.timer /etc/systemd/system/

$ sudo systemctl daemon-reload

$ sudo systemctl enable --now demo.timer

We can use systemctl to see when the task is executed last and when it will be executed next.

$ sudo systemctl list-timers --all

Use Cases

Singleton - In the above example, lets say demo.sh takes ~10 minutes to run. With cron job, in ten minutes we will have 10 instances of demo.sh running. This is not ideal. With systemd timer, it will ensure only one instance of demo.sh is running at a time.
On demand runs - If we want to test out the script/job, systemd allows us to immediately run it with usual systemctl start demo without needing to run the script manually.
Timer - With cron, we can run tasks upto a minute precision. Timer can run tasks till second level precision.

[Timer]
OnCalendar=*-*-* 15:30:15

In addition to that, we can run tasks based on system events. For example, we can run a script 15 minutes from reboot.

[Timer]
OnBootSec=15min

Conclusion

Systemd timer is a powerful tool that can replace cronjob in many scenarios. It provides more control and flexibility over cronjob. However, cronjob is still a good choice for simple scheduling tasks.

Mastering Kraken2 - Part 2 - Performance Optimisation

Anand Reddy Pandikunta — Sun, 28 Jul 2024 05:21:30 GMT

Mastering Kraken2

Part 1 - Initial Runs

Part 2 - Classification Performance Optimisation (this post)

Part 3 - Build custom database indices

Part 4 - Build FDA-ARGOS index

Part 5 - Regular vs Fast Builds (upcoming)

Part 6 - Benchmarking (upcoming)

Introduction

In the previous post, we learned how to set up kraken2¹, download pre-built indices, and run kraken2. In this post, we will learn various ways to speed up the classification process.

Increasing RAM

Kraken2 standard database is ~80GB in size. It is recommended to have at least db size RAM to run kraken2 efficiently². Let's use 128GB RAM machine and run kraken2 with ERR10359977³ sample.

$ time kraken2 --db k2_standard --report report.txt ERR10359977.fastq.gz > output.txt
Loading database information... done.
95064 sequences (14.35 Mbp) processed in 2.142s (2662.9 Kseq/m, 402.02 Mbp/m).
  94816 sequences classified (99.74%)
  248 sequences unclassified (0.26%)
kraken2 --db k2_standard --report report.txt ERR10359977.fastq.gz >   1.68s user 152.19s system 35% cpu 7:17.55 total

Now the time taken has come down from 40 minutes to 7 minutes. The classification speed has also increased from 0.19 Mbp/m to 402.02 Mbp/m.

The previous sample had only a few reads, and the speed is not a good indicator. Let's run kraken2 with a larger sample.

$ time kraken2 --db k2_standard --report report.txt --paired SRR6915097_1.fastq.gz SRR6915097_2.fastq.gz > output.txt
Loading database information... done.
Processed 14980000 sequences (2972330207 bp) ...
17121245 sequences (3397.15 Mbp) processed in 797.424s (1288.2 Kseq/m, 255.61 Mbp/m).
  9826671 sequences classified (57.39%)
  7294574 sequences unclassified (42.61%)
kraken2 --db k2_standard --report report.txt --paired > output.txt  526.39s user 308.24s system 68% cpu 20:23.86 total

This took almost 20 minutes to classify ~3 Gbp of data. Out of 20 minutes, 13 minutes was spent in classification. The remaining time in loading the db into memory.

Let's use k2_plusPF⁴ db, which is twice the size of k2_standard and run kraken2.

$ time kraken2 --db k2_plusfp --report report.txt --paired SRR6915097_1.fastq.gz SRR6915097_2.fastq.gz > output.txt
Loading database information...done.
17121245 sequences (3397.15 Mbp) processed in 755.290s (1360.1 Kseq/m, 269.87 Mbp/m).
  9903824 sequences classified (57.85%)
  7217421 sequences unclassified (42.15%)
kraken2 --db k2_plusfp/ --report report.txt --paired SRR6915097_1.fastq.gz  >   509.71s user 509.51s system 55% cpu 30:35.49 total

This took ~30 minutes to complete, but the classification took only 13 minutes similar to k2_standard. The remaining time was spent in loading the db into memory.

Preloading db into RAM

We can use vmtouch⁵ to preload db into RAM. kraken2 provides --memory-mapping option to use preloaded db.

$ vmtouch -vt k2_standard/hash.k2d k2_standard/opts.k2d k2_standard/taxo.k2d
           Files: 3
     Directories: 0
   Touched Pages: 20382075 (77G)
         Elapsed: 434.77 seconds

When Linux requires RAM, it will incrementally evict the db from memory. To prevent this, we can copy the db to shared memory (/dev/shm) and then use vmtouch to preload the db.

$ cp -r k2_standard /dev/shm

$ vmtouch -t /dev/shm/*.k2d

Now, let's run kraken2 with --memory-mapping option.

$ time kraken2 --db k2_standard --report report.txt --memory-mapping --paired SRR6915097_1.fastq.gz SRR6915097_2.fastq.gz > output.txt
Loading database information... done.
17121245 sequences (3397.15 Mbp) processed in 532.486s (1929.2 Kseq/m, 382.79 Mbp/m).
  9826671 sequences classified (57.39%)
  7294574 sequences unclassified (42.61%)
  kraken2 --db k2_standard --report report.txt --paired SRR6915097_1.fastq.gz   >  424.20s user 11.76s system 81% cpu 8:54.98 total

Now the classification took only ~10 minutes.

Multi threading

kraken2 supports multiple threads. I am using a machine with 40 threads.

$ time kraken2 --db k2_standard --report report.txt --paired SRR6915097_1.fastq.gz SRR6915097_2.fastq.gz --memory-mapping --threads 32 > output.txt
Loading database information... done.
17121245 sequences (3397.15 Mbp) processed in 71.675s (14332.5 Kseq/m, 2843.81 Mbp/m).
  9826671 sequences classified (57.39%)
  7294574 sequences unclassified (42.61%)
kraken2 --db k2_standard --report report.txt --paired SRR6915097_1.fastq.gz      556.58s user 22.85s system 762% cpu 1:16.02 total

With 32 threads, the classification took only 1 minute. Beyond 32 threads, the classification time did not decrease significantly.

Optimising input files

So far we have used gzipped input files. Let's use unzipped input files and run kraken2.

$ gunzip SRR6915097_1.fastq.gz
$ gunzip SRR6915097_2.fastq.gz

$ time kraken2 --db k2_standard --report report.txt --paired SRR6915097_1.fastq SRR6915097_2.fastq --memory-mapping --threads 30 > output.txt
Loading database information... done.
17121245 sequences (3397.15 Mbp) processed in 34.809s (29512.0 Kseq/m, 5855.68 Mbp/m).
  9826671 sequences classified (57.39%)
  7294574 sequences unclassified (42.61%)
kraken2 --db k2_standard --report report.txt --paired SRR6915097_1.fastq    30   565.03s user 17.12s system 1530% cpu 38.047 total

Now the classification time has come down to 40 seconds.

Since the input fastq files are paired, interleaving the files also takes time. Lets interleave the files and run kraken2.

To interleave the files, lets use seqfu tool.

$ conda install -y -c conda-forge -c bioconda "seqfu>1.10"

$ seqfu interleave -1 SRR6915097_1.fastq.gz -2 SRR6915097_2.fastq.gz > SRR6915097.fastq

$ time kraken2 --db k2_standard --report report.txt --memory-mapping SRR6915097.fq --threads 32 > output.txt
Loading database information... done.
34242490 sequences (3397.15 Mbp) processed in 20.199s (101714.1 Kseq/m, 10090.91 Mbp/m).
  17983321 sequences classified (52.52%)
  16259169 sequences unclassified (47.48%)
kraken2 --db k2_standard --report report.txt --memory-mapping SRR6915097.fq  32  618.96s user 18.24s system 2653% cpu 24.013 total

Now the classification time has come down to 24 seconds.

Conclusion

In terms of classification speed, we have come a long way from 0.1 Mbp/m to 1200 Mbp/m. In the next post, we will learn how to optimise the creation of custom indices.

Headlamp - k8s Lens open source alternative

Anand Reddy Pandikunta — Sun, 23 Jun 2024 20:18:02 GMT

Since Lens is not open source, I tried out monokle, octant, k9s, and headlamp¹. Among them, headlamp UI & features are closest to Lens.

Headlamp

Headlamp is CNCF sandbox project that provides cross-platform desktop application to manage Kubernetes clusters. It auto-detects clusters and provides cluster wide resource usage by default.

It can also be installed inside the cluster and can be accessed using a web browser. This is useful when we want to access the cluster from a mobile device.

$ helm repo add headlamp https://headlamp-k8s.github.io/headlamp/

$ helm install headlamp headlamp/headlamp

Lets port-forward the service & copy the token to access it.

$ kubectl create token headlamp

# we can do this via headlamp UI as well
$ kubectl port-forward service/headlamp 8080:80

Now, we can access the headlamp UI at http://localhost:8080.

Conclusion

If you are looking for an open source alternative to Lens, headlamp is a good choice. It provides a similar UI & features as Lens, and it is accessible via mobile devices as well.

https://headlamp.dev/ ↩

macOS - Log & track historical CPU, RAM usage

Anand Reddy Pandikunta — Fri, 31 May 2024 20:18:02 GMT

In macOS, we can use inbuilt Activity Monitor or third party apps like Stats to check the live CPU/RAM usage. But, we can't track the historical CPU & memory usage. sar, atop can track the historical CPU & memory usage. But, they are not available for macOS.

Netdata

Netdata¹ is an open source observability tool that can monitor CPU, RAM, network, disk usage. It can also track the historical data.

Unfortunately, it is not stable on macOS. I tried installing it on multiple macbooks, but it didn't work. I raised an issue² on their GitHub repository and the team mentioned that macOS is a low priority for them.

Glances

Glances³ is a cross-platform monitoring tool that can monitor CPU, RAM, network, disk usage. It can also track the historical data.

We can install it using Brew or pip.

$ brew install glances

$ pip install glances

Once it is installed, we can monitor the resource usage using the below command.

$ glances

Glances can log historical data to a file using the below command.

$ glances --export-csv /tmp/glances.csv

In addition to that, it can log data to services like influxdb, prometheus, etc.

Let's install influxdb and export stats to it.

$ brew install influxdb
$ brew services start influxdb
$ influx setup

$ python -m pip install influxdb-client

$ cat glances.conf
[influxdb]
host=localhost
port=8086
protocol=http
org=avilpage
bucket=glances
token=secret_token

$ glances --export-influxdb -C glances.conf

We can view stats in the influxdb from Data Explorer web UI at http://localhost:8086.

Glances provides a prebuilt Grafana dashboard⁴ that we can import to visualize the stats.

From Grafana -> Dashboard -> Import, we can import the dashboard using the above URL.

Conclusion

In addition to InfluxDB, Glances can export data to ~20 services. So far, it is the best tool to log, track and view historical CPU, RAM, network and disk usage in macOS. The same method works for Linux and Windows as well.

Timestamp to Relative Time - Kibana Scripted fields

Anand Reddy Pandikunta — Wed, 10 Apr 2024 08:45:33 GMT

When browsing logs in Kibana, there will be a timestamp stamp field on the left for all the docs. It is difficult to read & comprehend the timestamp in the logs. It would be better if we can convert the timestamp to a human-readable relative time like 5 minutes ago, 1 hour ago, etc.

Kibana Scripted Fields

Kibana provides a feature called scripted fields to create new fields in the index pattern. We can use this feature to convert the timestamp to a relative time.

Go to Stack Management -> Index Patterns -> Create index pattern -> Select the index pattern -> Scripted fields, click on Add scripted field, add the below script.

long now = new Date().getTime();

long timestamp = doc['@timestamp'].value.toInstant().toEpochMilli();
long diff = now - timestamp;
if (diff > 7200000) {
  return Math.round(diff / 3600000) + " hours ago";
} else if (diff > 3600000) {
  return Math.round(diff / 3600000) + " hour ago";
} else if (diff > 120000) {
  return Math.round(diff / 60000) + " minutes ago";
} else if (diff > 60000) {
  return (Math.round(diff / 60000) + " minute ago");
} else {
  return Math.round(diff / 1000) + " seconds ago";
}

Once the field is saved, we can go back to Discover and see the new field in the logs. We can toggle the visibility of the Relative Time field to see the relative time.

Conclusion

Instead of looking at the timestamp and calculating the relative time in our head, we can use relative time in Kibana . This will make it easier to read & comprehend the logs.