Recently, I’ve been dabbling in server-side Swift with a side project. Currently I’m using Vapor 3, which has its own ORM called Fluent. With my little side project I had the desire to store some data in the database (PostgreSQL in this case) as JSONB. I knew PostgreSQL could handle this, but I didn’t know if Fluent could.

Unfortunately, documentation on Vapor 3 and Fluent is sparse. I couldn’t find any docs on how to add a JSONB column for the current version of Vapor. From random GitHub issue posts, it looked like there used to be a protocol you could implement to do it, but, alas, that protocol no longer exists. There is a Discord community for Vapor, but my question (along with most others) went unanswered.

After a lot of trail and error and digging through the source code, I figured out a way that works. The approach is to define a custom type that can serialize itself to a native PostgreSQL type. In my case I used JSONB, but it could theoretically be any PostgreSQL type.

Example

Here’s an example:


import Foundation
import Fluent
import FluentPostgreSQL

struct MyDataType: Codable, Equatable {
    let foo: Int
    let bar: String
}

extension MyDataType: ReflectionDecodable {
    static func reflectDecoded() throws -> (MyDataType, MyDataType) {
        return (
            MyDataType(foo: 42, bar: "towel"),
            MyDataType(foo: 42, bar: "mostly harmless")
        )
    }
}

extension MyDataType: PostgreSQLDataConvertible {
    static func convertFromPostgreSQLData(_ data: PostgreSQLData) throws -> MyDataType {
        let decoder = JSONDecoder()
        if let binary = data.binary {
            return try decoder.decode(MyDataType.self, from: binary[1...])
        } else {
            throw PostgreSQLError(identifier: "Null data", reason: "Beats me")
        }
    }

    func convertToPostgreSQLData() throws -> PostgreSQLData {
        let encoder = JSONEncoder()
        let data = try encoder.encode(self)
        return PostgreSQLData(.jsonb, binary: [0x01] + data)
    }
}

struct MyModel: PostgreSQLUUIDModel {
    var id: UUID?
    let name: String
    let myData: MyDataType
}

In this case, the myData property will end up as a JSONB column on MyModel‘s table. Now I’ll try to explain each of the required protocols as best as I understand them from the source code.

Codable

Codable is actually required for a couple of things. First, it’s required by PostgreSQLUUIDModel (and variants) for database serialization/deserialization. Second, it’s used in the implementation of the PostgreSQLDataConvertible protocol to actually convert my custom type to JSON.

ReflectionDecodable and Equatable

ReflectionDecodable and Equatable are used to implement reflection leveraging the Decodable machinery. Reflection is apparently required on all database model properties. Without this protocol, things built fine, but I would get runtime errors saying my type (e.g. MyDataType) didn’t conform to ReflectionDecodable. The protocol is an odd duck, because although there are headerdocs describing how to implement it, there’s zero discussion on how/why it is used. Here’s my attempt at the how:

Big picture, Fluent leverages Decodable to generate reflection data (i.e. key path and type) for the properties in a data type (e.g. MyDataType). It does this by implementing a custom Decoder that records each property’s key path and the type it encodes. Although Fluent only cares about the type in this case, it needs a placeholder value of the correct type in order to satisfy the Decoder protocol. That’s where ReflectionDecodable comes in; it provides the values that the custom Decoder can use.

There are two ways the custom reflection Decoder is used. The first is to generate reflection data for all the properties in the given data type. The second is to generate reflection data for a property at a given key path. The second way is more complicated and is why reflectDecoded() returns two values.

Fluent looks for the property at the given key path by iterating the data type (e.g. MyDataType) one property at a time. It does this by running the custom Decoder for each property “offset” (an integer). It marks the active property offset by value. This is why reflectDecoded() has two values, and the headerdocs make a big deal about them not being equal. The property at the active offset decodes using the first value in reflectDecoded(). All other properties decode using the second value in reflectDecoded(). After the custom Decoder is done decoding for a property offset, Fluent takes the returned value, and compares the value at the given key path to the first value in reflectDecoded() (which is where Equatable is used). If they are equal, then the correct property has been found and it’s reflection data can be returned.

tl;dr: ReflectionDecodable is necessary to provide placeholder values for a custom Decoder. The first value is used to mark a given property as “active” by a search algorithm. The second value is used in all other cases.

PostgreSQLDataConvertible

The final protocol is PostgreSQLDataConvertible. It is used to serialize and deserialize from PostgreSQL. In my case, I’m choosing to serialize to JSONB, but really any Postgres data type could be used. JSONB is just JSON data with a version byte at the beginning. That’s why the first byte is skipped when deserializing, and [0x01] is prepended when serializing.