By examining my personal journey from a NATS novice to mastering its intricacies, this long-form article aims to showcase the importance and applicability of NATS in the software development landscape. Through comprehensive exploration of various topics, readers will gain a solid foundation, advanced techniques, and best practices for leveraging NATS effectively in their projects.

Introduction

Our topic for today is about how you can get started with NATS. We are assuming that you are aware of why you need NATS and want to know the concepts of NATS along with a walkthrough for how to deploy those concepts/components in your organization.

The first part would include the basic concept and Installation Guide, and setup, admin-related CRUD operations, shell scripts which might not be needed immediately but would be good to have in your arsenal. Whereas the second part would be more developer-focused - applying NATS in application, etc. Let's Begin

Understanding NATS

In this section, we will delve into the fundamentals of NATS and its key components.

A. Definition and Overview

Nats, which stands for "Naturally Adaptable and Transparent System," is a lightweight, high-performance messaging system known for its simplicity and scalability. It enables the exchange of messages between applications in a distributed architecture, allowing for seamless communication and increased efficiency.

B. Architecture Diagram

To better understand the inner workings of NATS, let's take a closer look at its architecture. The diagram below illustrates the key components involved in a typical Nats deployment:

C. Key Features

NATS offers several key features that make it a powerful messaging system. These include:

• Publish-Subscribe Model: NATS follows a publish-subscribe model where publishers send messages to subjects and subscribers receive those messages based on their interest in specific subjects.
• This model allows for flexible and decoupled communication between different parts of an application or across multiple applications.‍
• Scalability: With support for horizontal scaling, NATS can handle high loads of message traffic, making it suitable for large-scale systems.‍
• Performance: NATS is built for speed, providing low-latency message delivery and high throughput.‍
• Reliability: NATS ensures that messages are reliably delivered to subscribers, even in the presence of network interruptions or failures.‍
• Security: NATS supports secure communication through various authentication and encryption mechanisms, protecting sensitive data.

D. Use Cases and Applications

NATS' simplicity and versatility make it suitable for a wide range of use cases and applications. Some common use cases include:

• Real-time data streaming and processing
• Event-driven architectures
• Microservices communication
• IoT (Internet of Things) systems
• Distributed systems and cloud-native applications

E. Concepts

To better grasp the various components and terminologies associated with NATS, let's explore some key concepts:

1. NATS server: The NATS server acts as the central messaging infrastructure, responsible for routing messages between publishers and subscribers.
2. NATS CLI: The NATS command-line interface (CLI) is a tool that provides developers with a command-line interface to interact with the NATS server and perform various administrative tasks.
3. NATS clients: NATS CLI and clients both are different. The NATS client is an API/code-based approach to access the NATS server. Clients are not as powerful as CLI but are mainly used along with source code to achieve a specified goal. We won't be covering this as it is not part of the scope.
4. Routes: Routes allow NATS clusters to bridge and share messages with other nodes within and outside clusters, enabling communication across geographically distributed systems.
5. Accounts: Accounts in NATS provide isolation and access control mechanisms, ensuring that messages are exchanged securely and only between authorized parties.
6. Gateway: Gateways list all the servers in different clusters that you want to connect in order to create a supercluster.
7. SuperCluster: SuperCluster is a powerful feature that allows scaling NATS horizontally across multiple clusters, providing enhanced performance and fault tolerance.

F. System Requirements

Before diving into NATS, it's important to ensure that our system meets the necessary requirements. The system requirements for NATS will vary depending on the specific deployment scenario and use case. However, in general, the minimum requirements include:

Hardware:

Network:

• All the VMs should be part of the same cluster.
• 4222, 8222, 4248, and 7222 ports should be open for inter-server and client connection.
• Whitelisting of GitHub EMU account on prod servers (Phase 2).
• Get AVI VIP for all the clusters from the network team.

Logs:

By default, logs will be disabled, but the configuration file will have placeholders for logs enablement. Some of the important changes include:

• debug: It will show system logs in verbose.
• trace: It will record every message processed on NATS.
• logtime, logfile_size_limit, log_file: As the name represents, it will show the time when recording the logs, individual file limit for log files (once filled, auto rotation is done by NATS), and the name of the file, respectively.

TLS:

I will be showing the configuration of how to use the certs. Do remember, this setup is being done for the development environment to allow more flexibility towards explaining things and executing it.

Getting Started with NATS

In this section, we will guide you through the installation and setup process for NATS.

Building the Foundation

First, we will focus on building a strong foundation in NATS by understanding its core concepts and implementing basic messaging patterns.

A. Understanding NATS Subjects

In NATS, subjects serve as identifiers that help publishers and subscribers establish communication channels. They are represented as hierarchical strings, allowing for flexibility in message routing and subscription matching.

B. Exploring Messages, Publishers, and Subscribers

Messages are the units of data exchanged between applications through NATS. Publishers create and send messages, while subscribers receive and process them based on their subscribed subjects of interest.

C. Implementing Basic Pub/Sub Pattern

The publish-subscribe pattern is a fundamental messaging pattern in NATS. It allows publishers to distribute messages to multiple subscribers interested in specific subjects, enabling decoupled and efficient communication between different parts of the system.

D. JetStream

JetStream is an advanced addition to NATS that provides durable, persistent message storage and retention policies. It is designed to handle high-throughput streaming scenarios while ensuring data integrity and fault tolerance.

E. Single Cluster vs. SuperCluster

NATS supports both single clusters and superclusters. Single clusters are ideal for smaller deployments, whereas superclusters provide the ability to horizontally scale NATS across multiple clusters, enhancing performance and fault tolerance.

Implementation

As this blog is about deploying NATS from an admin perspective. We will be using only shell script for this purpose.

Let's start with the implementation process:

Prerequisite

These commands are required to be run on all the servers hosting Nats-server. In this blog, we will cover a 3-node cluster that will be working at Jetstream.

Installing NatsCLI and Nats-server:

_{mkdir -p rpm}

_{# NATSCLI}

_{curl -o rpm/nats-0.0.35.rpm  -L https://github.com/nats-io/natscli/releases/download/v0.0.35/nats-0.0.35-amd64.rpm}

_{sudo yum install -y rpm/nats-0.0.35.rpm}

_{# NATS-server}

_{curl -o rpm/nats-server-2.9.20.rpm  -L https://github.com/nats-io/nats-server/releases/download/v2.9.20/nats-server-v2.9.20-amd64.rpm}

_{sudo yum install -y rpm/nats-server-2.9.20.rpm}

Local Machine Setup for JetStream:

_{# Create User}

_{sudo useradd --system --home /nats --shell /bin/false nats}

_{# Jetstream Storage}

_{sudo mkdir -p /nats/storage}

_{# Certs}

_{sudo mkdir -p /nats/certs}

_{# Logs}

_{sudo mkdir -p /nats/logs}

_{# Setting Right Permission}

_{sudo chown --recursive nats:nats /nats}

_{sudo chmod 777 /nats}

_{sudo chmod 777 /nats/storage}

Next, we will create the service file in the servers at /etc/systemd/system/nats.service

_{sudo bash -c 'cat <<EOF > /etc/systemd/system/nats.service}

_[Unit]

_{Description=NATS Streaming Daemon}

_{Requires=network-online.target}

_{After=network-online.target}

_{ConditionFileNotEmpty=/nats/nats.conf}

_[Service]

_#Type=notify

_User=nats

_Group=nats

_{ExecStart=/usr/local/bin/nats-server -config=/nats/nats.conf}

_{#KillMode=process}

_{Restart=always}

_{RestartSec=10}

_{StandardOutput=syslog}

_{StandardError=syslog}

_{#TimeoutSec=900}

_{#LimitNOFILE=65536}

_{#LimitMEMLOCK=infinity}

_[Install]

_{WantedBy=multi-user.target}

_EOF'

Full File will look like:

CODE: https://gist.github.com/velotiotech/888872c69f5d643402390acad78a3590.js

Creating conf file at all the servers at /nats directory

_{Server setup}

_{server_name=nts0}

_{listen: <IP/DNS-First>:4222 # For other servers edit the IP/DNS remaining in the cluster}

_{https: <DNS-First>:8222}

_{#http: <IP/DNS-First>:8222 # Uncommnet this if you are running without tls certs} 

_{JetStream Configuration}

_{jetstream {}

  _{store_dir=/nats/storage}

  _{max_mem_store: 6GB}

  _{max_file_store: 90GB}

_}

_{Intra Cluster Setup}

_{cluster {}

  _{name: dev-nats # Super Cluster should have unique Cluster names}

  _{host: <IP/DNS-First>}

  _{port: 4248}

  _{routes = [}

    _{nats-route://<IP/DNS-First>:4248}

    _{nats-route://<IP/DNS-Second>:4248}

    _{nats-route://<IP/DNS-Third>:4248}

  _]

_}

_{Account Setup}

_{accounts: {}

  _{$SYS: {}

    _{users: [}

      _{{ user: admin, password: password }}

    _]

  _},

  _{B: {}

    _{users: [}

      _{{user: b, password: b}}

    _],

    _{jetstream: enabled,}

    _{imports: [}

    _{# {stream: {account: "\$G"}}}

    _]

  _},

  _{C: {}

    _{users: [}

      _{{user: c, password: c}}

    _],

    _{jetstream: enabled,}

    _{imports: [}

    _]

  _},

  _{E: {}

    _{users: [}

      _{{user: e, password: e}}

    _],

    _{jetstream: enabled,}

    _{imports: [}

    _]

  _}

_}

no_auth_user: e # Change this on every server to have a user in the system which does not need password, allowing local account in supercluster

We can use “Accounts” to help us provide local and global stream separation, the configuration is identical except for the changes in the no_auth_user which must be unique for each cluster, making the stream only accessible from the given cluster without the need of providing credentials exclusively.

Gateway Setup: 

Intra Cluster/Route Setup and Account Setup remain similar and need to be present in another cluster with the cluster having the name “new-dev-nats.”

_{gateway {}

  _{name: dev-nats}

  _{listen: <IP/DNS-First>:7222}

  _{gateways: [}

    _{{name: dev-nats, urls: [nats://<IP/DNS-First>:7222, nats://<IP/DNS-Second>:7222, nats://<IP/DNS-Third>:7222]},}

    _{{name: new-dev-nats, urls: [nats://<NEW-IP/DNS-First>:7222, nats://<NEW-IP/DNS-Second>:7222, nats://<NEW-IP/DNS-Third>:7222]}}

  _]

_}

TLS setup

_{tls: {}

  _{cert_file: "/nats/certs/natsio.crt"}

  _{key_file: "/nats/certs/natsio.key"}

  _{ca_file: "/nats/certs/natsio_rootCA.pem"}

_}

NOTE: no_auth_user: b is a special directive within NATS. If you choose to keep it seperate accross all the nodes in the cluster, you can have a “local account” setup in supercluster. This is beneficial when you want to publish data which should not be accessible by any other server.

Complete conf file on <IP/DNS-First> machine would look like this:

CODE: https://gist.github.com/velotiotech/83f4eb51f26f5e0343c05143ba344728.js

Recap on Conf File Changes

The configuration file in all the nodes for all the environments will need to be updated, to support “gateway” and “accounts.”

• Individual changes on all the conf files need to be done.
• Changes for the gateway will be almost similar except for the change in the name, which will be specific to the local cluster of which the given node is part of.
• Changes for an “account” will be almost similar except for the “no_auth_user” parameter, which will be specific to the local cluster of which the given node is part of.
• The “nats-server --signal reload” command should be able to pick up the changes.

Starting Service

After adding the certs, re-own the files:

_{sudo chown --recursive nats:nats /nats}

Creating firewall rules:

_{sudo firewall-cmd --permanent --add-port=4222/tcp}

_{sudo firewall-cmd --permanent --add-port=8222/tcp}

_{sudo firewall-cmd --permanent --add-port=4248/tcp}

_{sudo firewall-cmd --permanent --add-port=7222/tcp}

_{sudo firewall-cmd --reload}

Start the service:

_{sudo systemctl start nats.service}

_{sudo systemctl enable nats.service}

Check status:

sudo systemctl status nats.service -l

Note: Remember to check logs of status commands in node2 and node3; it should show a connection with node1 and also confirm that node1 has been made the leader.

Setting up the context:

Setting up context will help us in managing our cluster better with NATSCLI.

_{# pass the --tlsca flag in dev because we do not have the DNS registered. In staging and Production the `tlsca` flag will not be needed because certs will be registered.}

_{nats context add nats --server <IP/DNS-First>:4222,<IP/DNS-Second>:4222,<IP/DNS-Third>:4222 --description "Awesome Nats Servers List" --tlsca /nats/certs/natsio_rootCA.pem --select}

_{nats context ls}

_{nats account info}

Complete file for starting service would like this:

CODE: https://gist.github.com/velotiotech/c4a41694a2216ddc3409f54d8782eabc.js

Validation: 

The account info command should shuffle among the servers in the Connected URL string.

Stream Listing:

Streams that will be available across the regions would require the credentials. The creds should be common across all clusters:

The same info can be obtained from the different clusters when the same command is fired:

To fetch local streams that are present under the no_auth_user:

And from the different clusters using the same command (without credentials), we should get a different stream:

Advanced Messaging Patterns with NATS

In this section, we will explore advanced messaging patterns that leverage the capabilities of NATS for more complex communication scenarios.

A. Request-Reply Pattern

The request-reply pattern allows applications to send requests and receive corresponding responses through NATS. It enables synchronous communication, making it suitable for scenarios where immediate responses are required.

B. Publish-Subscribe Pattern with Wildcards

Nats introduces the concept of wildcards to the publish-subscribe pattern, allowing subscribers to receive messages based on pattern matching. This enables greater flexibility in subscription matching and expands the possibilities of message distribution.

C. Queue Groups for Load Balancing and Fault Tolerance

Queue groups provide load balancing and fault tolerance capabilities in NATS. By grouping subscribers together, NATS ensures that messages are distributed evenly across the subscribers within the group, preventing any single subscriber from being overwhelmed.

Overcoming Real-World Challenges

In this section, we will discuss real-world challenges that developers may encounter when working with NATS and explore strategies to overcome them.

A. Scalability and High Availability in NATS

As applications grow and message traffic increases, scalability, and high availability become crucial considerations. NATS offers various techniques and features to address these challenges, including clustering, load balancing, and fault tolerance mechanisms.

B. Securing NATS Communication

Security is paramount in any messaging system, and NATS provides several mechanisms to secure communication. These include authentication, encryption, access control, and secure network configurations.

C. Monitoring and Debugging Techniques

Efficiently monitoring and troubleshooting a NATS deployment is essential for maintaining system health. NATS provides tools and techniques to monitor message traffic, track performance metrics, and identify and resolve potential issues in real time.

Recovery Scenarios in NATS 

This section is intended to help in scenarios when NATS services are not usable. Scenarios such as Node failure, Not reachable, Service down, or region down are some examples of such a situation.

Summary

In this article, we have embarked on a journey from being a NATS novice to mastering its intricacies. We have explored the importance and applicability of NATS in the software development landscape. Through a comprehensive exploration of NATS' definition, architecture, key features, and use cases, we have built a strong foundation in NATS. We have also examined advanced messaging patterns and discussed strategies to overcome real-world challenges in scalability, security, and monitoring. Furthermore, we have delved into the Recovery scenarios, which might come in handy when things don't behave as expected. Armed with this knowledge, developers can confidently utilize NATS to unlock its full potential in their projects.