Happy New Year!

This is my final post for the 2021. This year I didn't post that much, but a lot of work was put into the blog to rewrite it using Grotksy. I hope everyone has a great 2022 and that next year is much better than the last one.

The inspiration for this blog post comes from the idea of building a tiny db that feels more natural to Javscript. All the databases that I've seen make a heavy use of methods like: db.get(), db.put(), db.scan(), db.query(). And many others that Im sure you have seen. I think it would be great to see something like:

const db = getDb("...")
// Create new user
const u = {username: "jdoe", email: "jdoe@example.com", id: 100}
// Store new user in the database
db.objects.users[u.username] = u
// Commit the changes to the database
db.actions.save()

In this blog post we will be building a much simpler version that stores everything in memory. Each change made to the objects will be stored in a log (called layers) and the final object will be composed of all the small layers present in the log.

We need to implement some generic getters/setters.

const objects = new Proxy({}, {
    get: function(obj, prop) {
        validate(prop, null)
        // Implementation
    },
    set: function(obj, prop, value) {
        validate(prop, value)
        // Implementation
    }
})

Let's define the validation function. In this case we want the objects to be able to be serialized to JSON.

const validate = (prop, value) => {
    // Make sure that the property and value are serializable
    // JSON.stringify throws an error if not serializable
    const l = {}
    l[prop] = value
    JSON.stringify(l)
    return l
}

This empty proxy will validate that the values and prop are serializable and do nothing else. Now we can start building on top of it.

We need a root object where we will store all the changes that are applied to an object. We will have a sort of tree structure to hold everything together. It will look something like this:

              rootObject({})  -> layers([{users: {jdoe: ...}}, {tokens: {tk1: ...}}])
                    |
        --------------------------
        |                        |
 child(.users{})          child(.tokens{})
        |                        |
       ...                      ...

The root object contains the layers with all the changes made from the beginning of the existence of the object. Each time a property of the root object is accessed a child is returned that internally holds a reference to the root. This way we can go through the entire chain of access and be able to reach the stored layers. By chain of access I mean the following: objects.users.jdoe.metadata.login.ip. As you can see, we need to traverse through many objects to be able to reach the ip field. But the layer that contains the information is only stored in the root, so each child needs to mantain a reference to the parent to be able to reach the root node.

Let's define a simple function to be able to create a new rootObject.

const wrapObject = (parent, key, current) => {
    const rootObj = {
        parent: Object.freeze(parent),
        layers: [Object.freeze({'value': current, 'previous': null})],
        pushLayer (l) {}, // Push new layer
        getLayer (ks) {}, // Get layer where information is stored based on given keys
        getValue (k) {} // Get value that matches given key
    }

    const rootProxy = {
        get: function(obj, prop) {
            validate(prop, null)
            const val = rootObj.getValue(prop)
            if (typeof val == 'object') {
                // If the value is an object we need to have a child instance
                // with a reference to the parent
                return wrapObject(rootObj, prop, val).objects
            }
            // If the value is other kind like a number or string we can safely return that
            return val
        },
        set: function(obj, prop, value) {
            const l = validate(prop, value)
            // Add new layer to the rootObj
            rootObj.pushLayer({'value': l})
        }
    }

    return {
        actions: {
            revert () {
                // Deleting the last layer will revert the changes
                const pop = rootObj.layers[rootObj.layers.length-1]
                rootObj.layers.splice(rootObj.layers.length-1, rootObj.layers.length)
                return pop
            }
        },
        objects: new Proxy({}, rootProxy)
    }
}

The layer format:

const layer = {
    value: {status: 'active'},
    previous: null // Reference to a previous layer that has the key 'status' in it
}

We would need a function to be able to tell if an object has a list of nested keys. Trust me for now, you'll see.

const nestedExists = (obj, ks) => {
    for (let j = 0; j < ks.length; j++) {
        let k = ks[j];
        if (!(k in obj)) {
            return false
        }
        obj = obj[k]
    }
    return true
}

Now we're almost done. Let's define the functions for handling the store and retrieval of layers.

const rootObj = {
    parent: Object.freeze(parent),
    layers: [Object.freeze({'value': current, 'previous': null})],
    pushLayer (l) {
        // If this is a child object we need to build the entire chain of access
        // from the bottom up
        if (parent) {
            const ll = {}
            ll[key] = l['value']
            // Search for a previous layer modifying the same key
            const previous = parent.getLayer([key])
            // Tell the parent object to push the new layer
            parent.pushLayer(Object.freeze({'value': ll, previous}))
        } else {
            // We are in the root object, add the layer to the array
            this.layers.push(Object.freeze(l))
        }
    },
    getLayer (ks) {
        // Search through the entire list of layers to see if one contains all the keys
        // that we are looking for. Start from the end of the array (top of the stack)
        for (let i = this.layers.length - 1; i >= 0; i--) {
            let v = nestedExists(this.layers[i]['value'], ks)
            if (v) {
                return this.layers[i]
            }
        }
        if (parent) {
            // If we are in a child object, look through all the previous layers
            // and see if the key we're looking for is contained in one of them.
            let ll = parent.getLayer([key].concat(ks))
            while (ll) {
                let a = nestedExists(ll['value'][key], ks)
                if (a) {
                    return Object.freeze({'value': ll['value'][key]})
                }
                ll = ll.previous
            }
        }
    },
    getValue (k) {
        // Straightforward, get layer and return value
        const l = this.getLayer([k])
        if (l) {
            return Object.freeze(l['value'][k])
        }
    }
}

Let's try the final result. The source code is loaded in this page, so you can open a dev console in the browser and try for yourself.

const store = wrapObject(null, null, {})

// Create new user
const user = {username: 'jdoe', email: 'jdoe@example.com', name: 'John Doe', id: 100}

// Add new user
store.objects.users = {}
store.objects.users[user.username] = user

// Print user email
console.log(store.objects.users.jdoe.email)

// Change user email and print
store.objects.users.jdoe.email = 'jdoe2@example.com'
console.log(store.objects.users.jdoe.email)

// Revert last change and print email again
store.actions.revert()
console.log(store.objects.users.jdoe.email)

That's it for now. We defined a Javascript object that contains the entire history of changes that were made to itself. And at any point we can revert the changes and go back to a previous state. Everything is stored in an array and is easily serializable. If we wanted to take this to the next level, each change could be written to a persistence storage (s3, sqlite, mysql, ...)

The full source code is available in a public gist.