Scalable Real-Time Chat with WebSockets

Overview:

This application is a scalable, real-time chat system built using modern web technologies and distributed systems principles. It allows users to join chat rooms and communicate instantly with other participants, leveraging WebSockets for real-time message delivery.

Motivation Behind Building the Application:

The journey to create this scalable real-time chat application stemmed from a combination of personal interest and professional development goals. As a Software Engineer with a growing fascination for distributed systems, I wanted to challenge myself by building something that would put theoretical concepts into practice.

Key Features:

Real-time messaging using WebSockets
Scalable architecture supporting multiple concurrent users
Load-balanced backend for improved performance and reliability
Containerized components for easy deployment and scaling

Let’s Go Through The Details

Architecture Components:

Frontend

Simple web interface for users to join chats and send/receive messages.
Hosted on Nginx for efficient static content serving.

HAproxy Load Balancer

Acts as a reverse proxy.
Distributes incoming WebSocket connections across multiple backend servers.
In its configuration, I set the mode to HTTP, which operates at Layer 7 (Application Layer) of the OSI model. This makes the load balancer stateless, meaning it treats each request independently. It then routes each request to one of the available backend servers based on the configured load balancing algorithm (Round-Robin Algorithm).

      frontend http
          bind *:8080
          mode http
          timeout client 1000s
          use_backend all
      backend all 
          mode http
          timeout server 1000s
          timeout connect 1000s
          server s1 ws1:8080
          server s2 ws2:8080
          server s3 ws3:8080
          server s4 ws4:8080

WebSocket Servers

In my App, four server instances handling real-time message processing.
Manage WebSocket connections with clients.
WebSocket: is a stateful, bidirectional communication protocol which keep the connection between client and server alive until one of them terminate this connection ws://localhost:8080 In my App, I used it because it provides:
1. Real-Time Messaging: allow instant message delivery without the need for clients to constantly poll the server for updates.
2. Reduced Latency: Once a connection is established, messages can be sent with minimal overhead, ensuring a responsive chat experience.
3. Efficient for mobile: Reduces battery and data usage on mobile devices compared to constant HTTP polling.

Redis Pub/Sub

In-memory database used for message distribution.
Implements publish/subscribe mechanism for efficient message broadcasting.

      import redis from 'redis';

      // Creating a SUB
      const subscriber = redis.createClient({
          port: 6379,
          host: 'rds' // Redis Container Server
      });
      // Creating a PUB
      const publisher = redis.createClient({
          port: 6379,
          host: 'rds' // Redis Container Server
      });
      // Setting and Getting Data
      const client = redis.createClient({
          port: 6379,
          host: 'rds' // Redis Container Server
      });

Docker

All components are containerized for consistency and easy scaling.
In this App, there are two Dockerfiles the first one is Dockerfile.front which build the image of frontend and the other one is Dockerfile.back which build the image of backend (WebSocket Servers).
I used docker-compose to make a network among containers, therefore they can communicate to each other.
I will provide three files here Dockerfile.front , Dockerfile.back and docker-compose if you want to learn how to setup the program, you can check the project Repository on Github in the Resources section below.

    # Frontend Dockerfile

    FROM nginx:alpine

    # Here The NGINX Look at the files
    WORKDIR /usr/share/nginx/html

    # Copy All Files
    COPY chat-front /usr/share/nginx/html

    # The Actual Container Port
    EXPOSE 80

    # Backend Dockerfile

    FROM node:20
    # My Docker working directory
    WORKDIR /live-chat/app
    # First Copying the json file
    COPY app/package.json /live-chat/app
    # Run this command after json copied to be sure that 
    # those two steps were cached as long as there is no 
    # changes in the json file
    RUN npm install
    # Then Copy all app files 
    COPY app /live-chat/app
    # Running Command
    CMD [ "npm", "run", "app" ]

    # The Docker-Compose File

    version: '3'

    services:
      chat:
        image: chatfront
        ports:
          - "1007:80"
      prserv:
        image: haproxy
        ports:
          - "8080:8080"
        volumes:
          - ./haproxy:/usr/local/etc/haproxy
      ws1:
        image: wsapp
        volumes:
          - ./app/index.mjs:/live-chat/app/index.mjs:ro
        environment:
          - APPID=1000
      ws2:
        image: wsapp
        volumes:
          - ./app/index.mjs:/live-chat/app/index.mjs:ro
        environment:
          - APPID=2000
      ws3:
        image: wsapp
        volumes:
          - ./app/index.mjs:/live-chat/app/index.mjs:ro
        environment:
          - APPID=3000
      ws4:
        image: wsapp
        volumes:
          - ./app/index.mjs:/live-chat/app/index.mjs:ro
        environment:
          - APPID=4000
      rds:
        image: redis:3

Let's dive into the most interesting part of this article

What does actually happen behind the scene?

Well, Let’s start from the beginning, The story begins when the user make a request to NGINX (web server) to load the frontend content http://localhost:1007/

Now, the content is loaded successfully and the user can SUBSCRIBE to the live-chat through writing their name, and hit the SUBSCRIBE button, now the user is already in the chat room. But let’s talk more about SUBSCRIBE button …

Well, once hitting the SUBSCRIBE button the request will go to HAproxy, first, it balances the request to one of the WebSocket Servers and then the WebSocket connection can be initiated ws://localhost:8080 so, how can this connection be initiated?

The WebSocket connection starts as an HTTP request. The client sends an HTTP request to the server, specifically requesting an upgrade to the WebSocket protocol, then the server responds with an HTTP status code 101 Switching Protocols, which indicates that the connection is successfully upgraded from HTTP to WebSocket. This process is called HTTP Handshake. For more clarification, you can check the image below ...

Now, the user connects to a WebSocket server, and this connection stays alive until it is terminated by either the client or the server.

Remember: We use in this application Redis In-Memory Database and use the PUB/SUB mechanism to distribute messages.

When running the application all WebSocket Servers subscribe to the channel in Redis called ‘livechat‘, therefore when any server publishes a message to this channel, all servers will receive this message as well.

Message Flow in the Live-Chat Application

Let's consider a scenario with 5 clients subscribed to the Live-Chat, each potentially connected to different WebSocket servers (as distributed by HAProxy). Here's how a message flows through the system:

Message Initiation: When Client-1 sends a message, it's sent to their established WebSocket connection.
HAProxy Routing: The message passes through HAProxy, which routes it to the specific WebSocket server that Client-1 is connected to. HAProxy maintains this connection information, ensuring consistent routing.
Server Processing: The WebSocket server receives the message from Client-1.
Redis Publication: The server then publishes this message to the 'livechat' Redis channel.
Message Distribution: All WebSocket servers subscribed to the 'livechat' Redis channel receive this message.
Client Broadcast: Each WebSocket server then sends the message to all clients connected to it, ensuring all chat participants receive the message.

This architecture creates a foundation for a responsive, efficient, and scalable real-time communication system. It allows messages to be quickly distributed to all participants, regardless of which server they're connected to.

Challenges Faced:

The big question that I faced is how to scale WebSocket, because WebSocket is stateful, it is difficult to horizontal scale, because once the server dead, all connections to it just dead.
I spend a bit much time to understand how to scale WebSocket with Docker Containers.
Another Interesting challenge, I faced is to make an interface for users to interact with the application.

Future Improvements:

User Authentication and Private Chats I will implement a user authentication system to allow for user accounts and private conversations.
Message Persistence I will add a database (e.g., PostgreSQL or MongoDB) to store chat messages.
Rich Media Support Enhancing the chat with support for rich media content, such as file sharing capabilities (images, documents, etc.)

Conclusion:

Building this real-time chat application has been an exciting journey into the world of distributed systems and modern web technologies. Through this project, I have explored the power of WebSockets for real-time communication, leveraged HAProxy for efficient load balancing, and utilized Redis for seamless message distribution across multiple servers.

I hope this blog post has provided valuable insights into building scalable, real-time web applications. Whether you're a seasoned developer or just starting out, I encourage you to explore these technologies and perhaps build upon this concept in your own projects.

Happy coding, and may your chats always be real-time!

Building a Scalable Real-Time Chat Application: A Journey Through WebSockets

Step-by-Step Guide to a Live Chat Application