Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker.

It supports various data structures such as Strings, Hashes, Lists, Sets etc. DCOS offers Redis as a service. 

Why Do We Use External Persistent Storage for Redis Mesos Containers?

Since Redis is an in-memory database, an instance/service restart will result in loss of data. To counter this, it is always advisable to snapshot the Redis in-memory database from time to time.

This helps Redis instance to recover from the point in time failure.

In DCOS, Redis is deployed as a stateless service. To make it a stateful and persistent data, we can configure local volumes or external volumes.

The disadvantage of having a local volume mapped to Mesos containers is when a slave node goes down, your local volume becomes unavailable, and the data loss occurs.

However, with external persistent volumes, as they are available on each node of the DCOS cluster, a slave node failure does not impact the data availability.

Rex-Ray

REX-Ray is an open source, storage management solution designed to support container runtimes such as Docker and Mesos.

REX-Ray enables stateful applications such as databases to persist and maintain its data after the life cycle of the container has ended. Built-in high availability enables orchestrators such as Docker Swarm, Kubernetes, and Mesos Frameworks like Marathon to automatically orchestrate storage tasks between hosts in a cluster.

Built on top of the libStorage framework, REX-Ray's simplified architecture consists of a single binary and runs as a stateless service on every host using a configuration file to orchestrate multiple storage platforms.

Objective: To create a Redis service in DC/OS environment with persistent storage.

Warning: The Persistent Volume feature is still in beta Phase for DC/OS Version 1.11.

Prerequisites:

• Make sure the rexray service is running and is in a healthy state for the cluster.

Steps:

• Click on the Add button in Services component of DC/OS GUI.

• Click on JSON Configuration.  

Note: For persistent storage, below code should be added in the normal Redis service configuration JSON file to mount external persistent volumes.

CODE: https://gist.github.com/velotiotech/2a23b12f1d96805c1abc8dcd25a3f940.js

• Make sure the service is up and in a running state:

If you look closely, the service was suspended and respawned on a different slave node. We populated the database with dummy data and saved the snapshot in the data directory.

When the service did come upon a different node 10.0.3.204, the data persisted and the volume was visible on the new node.

CODE: https://gist.github.com/velotiotech/8adadb858ad605f74d8014423a1e6048.js

•  Check the volume tab :

Note: For external volumes, the status will be unavailable. This is an issue with DC/OS.

The Entire Service JSON file:

CODE: https://gist.github.com/velotiotech/13fa86a37a02bc92f4724c8d439b6e89.js

Redis entrypoint

To connect with Redis service, use below host:port in your applications:

redis.marathon.l4lb.thisdcos.directory:6379

Conclusion

We learned about Standalone Redis Service deployment from DCOS catalog on DCOS.  Also, we saw how to add Persistent storage to it using RexRay. We also learned how RexRay automatically manages volumes over AWS ebs and how to integrate them in DCOS apps/services.  Finally, we saw how other applications can communicate with this Redis service.

References

• Persistent Volumes
• Storage Management Solution
• Redis Clients