SurrealDB Internal design — Part 1 — Welcome to Wonderland

SurrealDB logo (source: docs)

A full list of posts in the series can be found here.

One of the most hyped and anticipated Database technologies that has arisen recently is a product created by two individuals and that promises to be the DB for the future — that is none other than Surrealdb.

The code was open sourced a few months ago, and I’m proud to say I contributed a feature to this amazing project. What I’ve discovered while writing code in this ecosystem, is that the internal design of Surrealdb is not documented, and so I had to figure out where anything goes by debugging and clicking references in VSCode, not an ideal situation by any measure.

As an attempt to make the code more accessible to the community as a whole, and to allow more people to contribute, in this series of posts I will do my best to break the code design, focusing at one part at a time.

This post is an overview of Surrealdb, on the next posts we will dive into the code itself.

What is Surrealdb?

According to the documentation:

SurrealDB is an innovative NewSQL cloud database, suitable for serverless applications, jamstack applications, single-page applications, and traditional applications.

Let’s break that down.

NewSQL — a database that offers a NoSQL experience (with no structured data constraints), whole also maintaining the ACID guarantees (transactional execution, data consistency and rollback capability).

Serverless Cloud — surrealdb can be deployed on a distributed architecture and can run on all major providers, enabling the database to scale as needed.

Application Backend — surrealdb has additional features that allow it to be used directly from a customer facing client as well as from a custom server. These features include:

• An advanced premissions model
• Live updates via Websockets
• A REST API
• And more…

SurrealDB data model

Sound awesome! What can you store in this DB?

Basically — Anything.

SurrealDB supports three types of data paradigms, all working together to create a modern data model:

1. Unstructured data — Arbitrary objects with no defined structure, including nested references to other objects in the database
2. Tabular (structured) data — like classical SQL databases, storing rows with a defined and enforced structure
3. Graph connections — arbitrary connections between objects using a graph syntax, which allows the connection itself to have properties that can be queried

This means that we can save whatever data we want, decide if we want it to have a structure, and connect different objects into a graph that we can query.

All data is stored in groups that for simplicity’s sake are called “tables”, even when the data is unstructured.

Example Time

All operations on the database are pretty much intuitive for anyone who has dealt with any kind of database before.

(The example is based on the official documentation):

Working with unstructured data:

UPDATE person SET
	waist = <int> "34.59",
	height = <float> 201,
	score = <decimal> 0.3 + 0.3 + 0.3 + 0.1
;

Or with adding structure:

-- Create a schemafull table
DEFINE TABLE user SCHEMAFULL;

-- Specify fields on the user table
DEFINE FIELD name ON TABLE user TYPE object;
DEFINE FIELD name.first ON TABLE user TYPE string;
DEFINE FIELD name.last ON TABLE user TYPE string;
DEFINE FIELD email ON TABLE user TYPE string ASSERT is::email($value);

-- Add a unique index on the email field preventing duplicate values
DEFINE INDEX email ON TABLE user COLUMNS email UNIQUE;

On top of this data we can build our graph:

-- Add a graph edge between user:tobie and article:surreal
RELATE user:ori->write->article:welcome-to-wonderland
	SET time.written = time::now()
;

-- Add a graph edge between specific users and developers
LET $from = (SELECT users FROM site:medium);
LET $devs = (SELECT * FROM user WHERE tags CONTAINS 'surrealdb');
RELATE $from->like->$devs UNIQUE
	SET time.connected = time::now()
;

And all of this can be done via a REST API, custom client libraries in various languages, or websockets (see the docs).

Conclusion

I’ve only touched the surface of the functionality of this amazing tool, it can do so much more.

I encourage you to read the docs, or watch this short Youtube video if you only want a taste:

Future posts will dive into the code base itself (which is open source and can be found here) and will be focused on the community, with an aim of enabling more people to get their hands dirty and contribute.

Follow to get notified when the posts go live!