“Hope this email finds you well” is how 2020-2021 has been in a nutshell. Since we’ve all been working remotely since last year, actively collaborating with teammates became one notch harder, from activities like brainstorming a topic on a whiteboard to building documentation.

Having tools powered by collaborative systems had become a necessity, and to explore the same following the principle of build fast fail fast, I started building up a collaborative editor using existing available, open-source tools, which can eventually be extended for needs across different projects.

Conflicts, as they say, are inevitable, when multiple users are working on the same document constantly modifying it, especially if it’s the same block of content. Ultimately, the end-user experience is defined by how such conflicts are resolved.

There are various conflict resolution mechanisms, but two of the most commonly discussed ones are Operational Transformation (OT) and Conflict-Free Replicated Data Type (CRDT). So, let’s briefly talk about those first.

Operational Transformation

The order of operations matter in OT, as each user will have their own local copy of the document, and since mutations are atomic, such as insert V at index 4 and delete X at index 2. If the order of these operations is changed, the end result will be different. And that's why all the operations are synchronized through a central server. The central server can then alter the indices and operations and then forward to the clients. For example, in the below image, User2 makes a delete(0) operation, but as the OT server realizes that User1 has made an insert operation, the User2’s operation needs to be changed as delete(1) before applying to User1.

OT with a central server is typically easier to implement. Plain text operations with OT in its basic form only has three defined operations: insert, delete, and apply.

Source: Conclave

“Fully distributed OT and adding rich text operations are very hard, and that’s why there’s a million papers.”

CRDT

Instead of performing operations directly on characters like in OT, CRDT uses a complex data structure to which it can then add/update/remove properties to signify transformation, enabling scope for commutativity and idempotency. CRDTs guarantee eventual consistency.

There are different algorithms, but in general, CRDT has two requirements: globally unique characters and globally ordered characters. Basically, this involves a global reference for each object, instead of positional indices, in which the ordering is based on the neighboring objects. Fractional indices can be used to assign index to an object.

Source: Conclave

As all the objects have their own unique reference, delete operation becomes idempotent. And giving fractional indices is one way to give unique references while insertion and updation.

There are two types of CRDT, one is state-based, where the whole state (or delta) is shared between the instances and merged continuously. The other is operational based, where only individual operations are sent between replicas. If you want to dive deep into CRDT, here's a nice resource.

For our purposes, we choose CRDT since it can also support peer-to-peer networks. If you directly want to jump to the code, you can visit the repo here.

Tools used for this project:

As our goal was for a quick implementation, we targeted off-the-shelf tools for editor and backend to manage collaborative operations.

• Quill.js is an API-driven WYSIWYG rich text editor built for compatibility and extensibility. We choose Quill as our editor because of the ease to plug it into your application and availability of extensions.
• Yjs is a framework that provides shared editing capabilities by exposing its different shared data types (Array, Map, Text, etc) that are synced automatically. It's also network agnostic, so the changes are synced when a client is online. We used it because it's a CRDT implementation, and surprisingly had readily available bindings for quill.js.

Prerequisites:

To keep it simple, we'll set up a client and server both in the same code base. Initialize a project with npm init and install the below dependencies:

CODE: https://gist.github.com/velotiotech/983eb97b92b560d4290165a96eb798eb.js

• Quill: Quill is the WYSIWYG rich text editor we will use as our editor.

• quill-cursors is an extension that helps us to display cursors of other connected clients to the same editor room.

• Webpack, webpack-cli, and webpack-dev-server are developer utilities, webpack being the bundler that creates a deployable bundle for your application.

• The Y-quill module provides bindings between Yjs and QuillJS with use of the SharedType y.Text. For more information, you can check out the module’s source on Github.

• Y-websocket provides a WebsocketProvider to communicate with Yjs server in a client-server manner to exchange awareness information and data.

• Yjs, this is the CRDT framework which orchestrates conflict resolution between multiple clients. 

Code to use

CODE: https://gist.github.com/velotiotech/b4e82fdca68d6f518ad541481a899240.js 

This is a basic webpack config where we have provided which file is the starting point of our frontend project, i.e., the index.js file. Webpack then uses that file to build the internal dependency graph of your project. The output property is to define where and how the generated bundles should be saved. And the devServer config defines necessary parameters for the local dev server, which runs when you execute “npm start”.

We'll first create an index.html file to define the basic skeleton:

CODE: https://gist.github.com/velotiotech/f96264879f6fed49035a8dfee99cceed.js

The index.html has a pretty basic structure. In <head>, we’ve provided the path of the bundled js file that will be created by webpack, and the css theme for the quill editor. And for the <body> part, we've just created a button to connect/disconnect from the backend and a placeholder div where the quill editor will be plugged.

• Here, we've just made the imports, registered quill-cursors extension, and added an event listener for window load:

CODE: https://gist.github.com/velotiotech/2e6fcbf543a357c990b096fe0d5774c6.js

• Let's initialize the Yjs document, socket provider, and load the document:

CODE: https://gist.github.com/velotiotech/ec1e994203ee2311b697556e8b6a60b6.js

• We'll now initialize and plug the Quill editor with its bindings:

CODE: https://gist.github.com/velotiotech/4a0d183bdc626a27424bb7c386c71eb5.js

• Finally, let's implement the Connect/Disconnect button and complete the callback:

CODE: https://gist.github.com/velotiotech/aa5730fba5f35fa78327ad61929c7321.js

Steps to run:

• Server:

For simplicity, we'll directly use the y-websocket-server out of the box.

NOTE: You can either let it run and open a new terminal for the next commands, or let it run in the background using `&` at the end of the command.

• Client:

Start the client by npm start. On successful compilation, it should open on your default browser, or you can just go to http://localhost:8080.

Show me the repo

You can find the repository here.

Conclusion:

Conflict resolution approaches are not relatively new, but with the trend of remote culture, it is important to have good collaborative systems in place to enhance productivity.

Although this example was just on rich text editing capabilities, we can extend existing resources to build more features and structures like tabular data, graphs, charts, etc. Yjs shared types can be used to define your own data format based on how your custom editor represents data internally.