Safe from the Losing Fight

As I’m getting further into my Vapor side project, I’m learning how writing Swift iOS/macOS apps is different from writing Swift server apps. One of the ways they are different is integration tests. For iOS apps, it’s usually done via automated UI testing. For Vapor, it’s API testing, or sometimes I see it called controller testing. Regardless of its name, I find myself writing more integration tests than unit tests for my server. I think there are two reasons for that:

1. APIs tend to be well defined and easy to interact with, at least compared to UIs
2. The integration tests give me a lot more confidence than unit tests that my server is behaving that way I intended it to

Of course, the downside is integration tests are much slower to run. So far, that’s a trade off I’ve been willing to make.

In any case, I want to talk about how I set up integration testing for my Vapor 3 app. My goal was to write test cases where calling the API under test was simple, and looked as much like real client code as possible. To accomplish that, I wrote some helper types called TestApplication and TestResponse to hide boilerplate code. I also created some fixtures to easily seed the database with data, along with some utility methods to reset the database after each test. Finally, I had to figure out how use Docker to stand up a PostgreSQL database I could run the tests against.

Example testcase

I’m going to start with a test case showing how I wrote my integration tests, then work backwards explaining the infrastructure I set up to support that.

As an example, I’m going to test an API that allows an authenticated user to invalidate all auth tokens that have been issued for their account, thereby forcing all devices to re-authenticate. Here’s the test:

class UserControllerTests: XCTestCase {
    ...
    func testResetTokens_isTrue_success() throws {
        let request = UserController.UserEnvelope(user: UserController.UpdateRequest(resetTokens: true))
        let response = try app.put("/api/users/\(user.id!)", headers: .withAuthorization(app.apiKey, jwt), body: request)
        XCTAssertEqual(response.status, .ok)

        let responseBody = try response.content(decodeTo: UserController.UserEnvelope<UserController.UserResponse>.self).user

        XCTAssertEqual(responseBody.id, user.id!)
        XCTAssertEqual(responseBody.email, user.email)

        let showResponse = try app.get("/api/users/\(user.id!)", headers: .withAuthorization(app.apiKey, jwt))
        XCTAssertEqual(showResponse.status, .unauthorized)
    }
    ...
}

Right now I’m using XCTestCase because it doesn’t rely on any external dependencies. I’ve been entertaining the idea of using Quick & Nimble so I can share more setup between tests, use custom matchers, and make the assertions a touch more readable. But for now, I just name my test functions with a specific pattern: API name, plus parameter values, plus expected results. I also toss in the app state if it’s relevant for the test.

The first line of the test is creating the request that I want to validate. The app specific request types are currently defined in the controller code, which is why there’s namespacing. The next line makes the request via TestApplication and then waits for a response. The request is performing a PUT on /api/users/:user_id with the request parameters encoded in the body. The .withAuthorization(app.apiKey, jwt) is a fixture on HTTPHeaders that creates headers with a valid authorization for this user.

Once the API performs the token reset, it returns the user object. The test validates this by decoding the response body and asserting the id and email match what’s expected. Then, to fully verify that all tokens were reset, the test makes another request (this time a GET request) with the existing token. It asserts that the request is rejected with an unauthorized status.

At this point there are a lot of undefined things that make this test function. I will eventually get to them all. But for now, I’ll start with the setup code that creates the user and JWT that are used by this test. Here’s what I got for that:

import Foundation
import XCTest
import Vapor
import FluentPostgreSQL
import JWT
@testable import App

class UserControllerTests: XCTestCase {
    var app: TestApplication!
    var connection: PostgreSQLConnection!
    var user: User!
    var jwt: String!

    override func setUp() {
        super.setUp()

        try! TestApplication.reset()
        app = try! TestApplication()
        connection = app.connection

        user = User.user(on: connection)

        let payload = AuthJWTPayload(uid: UserIDClaim(value: user.id!),
                                     iat: IssuedAtClaim(value: Date()))
        jwt = try! JWTSerialization().encode(payload, on: app.container)
    }

    override func tearDown() {
        super.tearDown()

        _ = user.delete(on: connection)

        connection.close()
    }
    ...
}

Since I’m using XCTest, I’m using the standard setUp() and tearDown() methods. In the setup I start by resetting the app via TestApplication.reset(); this resets the database. Then I create a TestApplication instance, which is the test’s interface into the app. I’ll need to add data to the database as well verify data was created, so I go ahead and create a database connection. Then I immediately use it create a user using the User.user(on:) fixture. The final bit of setup is creating a valid JWT for the user that can be used for making API calls. I wrote about using JWTs in Vapor API servers earlier.

My tearDown() is simple. I delete the user I created in the set up, and then close the database connection. This is a bit redundant in that I reset the app on each test, but I like to do it because it feels like good hygiene.

Now I have a complete test case, but there’s still a lot of infrastructure to build. The example test above used a couple of fixtures, so I’ll talk about those next.

Fixtures

My goal with fixtures is to provide a quick and easy way to create model data to test with. My fixtures tend to change a lot from app to app because they’re tied to the model types, and those change from app to app. That said, I do have a couple of patterns that I follow for fixtures. First, I prefer to make the fixtures static methods on the data type they’re creating instances of. In my experience, this scales a bit better than having one fixtures namespace or type that all fixtures are created from. Second, while I have the fixture methods take the model’s properties as parameters, I do prefer to default those parameters to sane values. I have found that reduces test maintenance when I need to add a property to a model type.

To provide a concrete example, here’s the user fixture used in example test above:

import Foundation
import Vapor
@testable import App

extension User {
    static func user(email: String = "bob.jimbob@example.com", on connection: DatabaseConnectable) -> User {
        let user = try! User(email: email)
        return try! user.create(on: connection).wait()
    }
}

There’s not much here, but I find even simple fixtures like this save time and boilerplate. The fixture creates a User instance, then saves it to the database. The most notable thing here is that the fixture waits on the database save to complete before returning. It does this by calling wait() on the promise.

The other thing that’s probably worth commenting on is my liberal use of force unwraps in the tests. Normally, I would not suffer a force unwrap to live (with a couple of exceptions). But I view tests as controlled environments that real users will never have to experience, and if something crashes it was probably a programmer error anyway.

TestApplication

Everything that I’ve written about so far has been pretty specific to my particular app. But there’s some testing infrastructure that I built that should be fairly portable. The next piece I want to talk about is the TestApplication.

The TestApplication represents the system under test. Its main function is to receive requests and send back responses. There’s some boilerplate code needed to make that ergonomic for tests. I chose to make TestApplication its own type (as opposed to an extension on Application) for a few reasons. First, I feel a bespoke type makes it clear which methods and properties are useful for tests, as opposed to production code. Second, having a separate type allows other testing properties and methods to have a convenient place to live.

With all that said, here’s the init and deinit of my TestApplication type:

import Foundation
import Vapor
import FluentPostgreSQL
@testable import App

class TestApplication {
    private let application: Application
    private let configuration: TestingConfiguration

    lazy var connection: PostgreSQLConnection = {
        return try! application.newConnection(to: .psql).wait()
    }()
    var container: Container {
        return application
    }

    init(arguments: [String] = CommandLine.arguments) throws {
        var env = Environment.testing
        env.arguments = arguments

        self.application = try App.app(env)
        self.configuration = try application.make(TestingConfiguration.self)
    }

    deinit {
        try! application.releaseCachedConnections()
        try! application.syncShutdownGracefully()
    }
}

The TestApplication type mainly wraps Vapor’s Application type. However, it also contains the TestingConfiguration, which is helpful for tests wanting to grab the API key, or fake services, or any other piece of configuration data. The TestApplication also exposes a database connection, lazily created, and a dependency injection container. These objects are needed to do anything interesting in a Vapor app.

The init creates a testing environment and sets the command line arguments on it. The command line argument functionality will be used later, when resetting the database between tests. But for now, init ensures the application will be stood up using the TestingConfiguration. After the application is created, a TestingConfiguration instance is pulled out and stored in a property.

I didn’t write a deinit method to start with. After I had written severals tests I started getting test failures because all the Postgres connections were being used, and after that no test could acquire a new one. So the deinit manually clears any cached connections (Vapor seems to keep a pool around), then waits synchronously until the Vapor app fully shuts down.

Making test requests

I’ve created my TestApplication now, and while it does some helpful things, I haven’t covered the main thing it does: send requests and receive responses. There were couple of pieces to implementing this. First, I needed to put together a method that can do basic send and receive. Then I added several methods to make it ergonomic to call from tests.

To start, I built the basic workhorse method:

class TestApplication {
  ...
  private func request<T: Content>(_ path: String, method: HTTPMethod, headers: HTTPHeaders, body: T?) throws -> TestResponse {
       let responder = try application.make(Responder.self)
       let httpRequest = HTTPRequest(method: method, url: URL(string: path)!, headers: headers)
       let request = Request(http: httpRequest, using: application)
       if let body = body {
           try request.content.encode(body)
       }
       return try TestResponse(response: responder.respond(to: request).wait())
  }
  ...
}

The first thing I’ll point out here is the method signature. It takes a path to the API to be called, the HTTP method to use, headers, and an optional body. The body can be any type as long as it conforms to Vapor’s Content protocol. The method can throw, and returns a TestResponse if it is successful. These parameters cover all the options that my tests care about.

The first thing request() does is create an object that conforms to the Vapor Responder protocol. This is the object that allows TestApplication to make requests and receive responses. Then request() creates the raw HTTPRequest using the parameters passed in, then the Request object based on that. The using parameter wants a dependency injection container, so I give it the application. If there is a non-nil body provided in the parameters, it is encoded into the request. Now that the request is fully created, it is given to the Responder which returns a promise to the response. Since this is for tests, I wait() on the response promise to be fulfilled before wrapping it in a TestResponse.

Although this method removes a lot of the boilerplate needed to make a request, I felt like it could be a bit cleaner. I figured that the HTTP method could be the method name instead of a parameter. So I added some methods that do that:

class TestApplication {
  ...
  func get<T: Content>(_ path: String, headers: HTTPHeaders = HTTPHeaders(), body: T?) throws -> TestResponse {
      return try request(path, method: .GET, headers: headers, body: body)
  }

  func put<T: Content>(_ path: String, headers: HTTPHeaders = HTTPHeaders(), body: T?) throws -> TestResponse {
      return try request(path, method: .PUT, headers: headers, body: body)
  }

  func post<T: Content>(_ path: String, headers: HTTPHeaders = HTTPHeaders(), body: T?) throws -> TestResponse {
      return try request(path, method: .POST, headers: headers, body: body)
  }

  func delete<T: Content>(_ path: String, headers: HTTPHeaders = HTTPHeaders(), body: T?) throws -> TestResponse {
      return try request(path, method: .DELETE, headers: headers, body: body)
  }
  ...
}

These methods also default the headers to an empty set of headers, in the cases where tests don’t care about them. These methods are pretty ergonomic, but there’s one more case I needed to handle.

Some of my APIs don’t send a body in the request. The workhorse request() method can handle that; the caller just passes in nil in that case. However, because body is a generic type, the compiler needs a type — any type — to compile the call. nil by itself won’t give the compiler enough information to infer the type. I needed an instance of an optional type set to nil to satisfy the type inference. I also wanted to avoid forcing the test from having to pass that instance in. If a body parameter wasn’t provided, the TestApplication methods should assume there isn’t one.

Fortunately this can be done easily, albeit with a bit more boilerplate:

class TestApplication {
  ...
  struct Empty: Content {
     static let instance: Empty? = nil
  }

  func get(_ path: String, headers: HTTPHeaders = HTTPHeaders()) throws -> TestResponse {
     return try get(path, headers: headers, body: Empty.instance)
  }

  func delete(_ path: String, headers: HTTPHeaders = HTTPHeaders()) throws -> TestResponse {
     return try delete(path, headers: headers, body: Empty.instance)
  }
  ...
}

I defined a placeholder type called Empty and conformed it to Vapor’s Content protocol. The protocol conformance was to satisfy request()‘s constraints on the body parameter. Then I declared an optional static instance of the type, and set it to nil. I could then use this static instance to represent an empty request body.

Now that I had the Empty.instance instance created, I used it to remove the body parameter. Here, I’ve only provided versions of the get and delete methods without a body, but the same technique works for put and post.

At this point I could now make requests like I showed in my example test. But how about validating the response I got back?

TestResponse

The main goal of TestResponse was to make it easy for tests to validate the response. I chose to wrap Vapor’s Response type instead of providing an extension on it for the same reason TestApplication wraps Application. The main things my tests want to do with a response is: verify the HTTP status, verify the body, and maybe verify a header or two.

import Foundation
import Vapor

struct TestResponse {
    private let response: Response

    init(response: Response) {
        self.response = response
    }

    var status: HTTPResponseStatus { return response.http.status }

    func content<T: Decodable>(decodeTo type: T.Type) throws -> T {
        return try response.content.decode(type).wait()
    }

    func header(_ name: HTTPHeaderName) -> String? {
        return response.http.headers.firstValue(name: name)
    }
}

TestResponse is simple. It exposes the HTTP status in status, provides a synchronous way to decode the body via the content() method, and looks up a given header with the header() method. For more complicated tests it might need to expose more information, but this works for all my tests.

Resetting the database

So far, I’ve been mainly concerned with sending requests, receiving responses, and validating them. But for tests to work, I needed to be able to reset the database between test runs. Otherwise tests would interfere with one another.

First, I needed to set up some Fluent commands in my configuration. These added command line arguments that can reset the database for me.

public func configure(_ config: inout Config, _ env: inout Environment, _ services: inout Services, _ configuration: ConfigurationType) throws {
  ...
  var commandConfig = CommandConfig.default()
  commandConfig.useFluentCommands()
  services.register(commandConfig)
  ...
}

There’s not much to this. Toward the end of my configure() method, I created a default Vapor CommandConfig, told it to use Fluent commands, then registered it as a service.

Now that the Fluent commands were set up, I needed to add a couple of methods to TestApplication to have it reset the database.

class TestApplication {
  ...
  func run() throws {
     try application.asyncRun().wait()
  }

  static func reset() throws {
     try TestApplication(arguments: ["vapor", "revert", "--all", "-y"]).run()
     try TestApplication(arguments: ["vapor", "migrate", "-y"]).run()
  }
  ...
}

The run() method looks a bit odd by itself. It synchronously runs the application. Which turned out to be handy, if it has some commands to run from the command line.

The reset() static method provided those command line arguments. First, it creates an instance of the app that reverts the database, and runs it. Then, it it creates an instance of the app that runs all migrations.

That testing database

I’ve blissfully ignored that all this needs a real test database to talk to. I could have set up a local Postgres instance on my laptop (which I’ve done before), but that doesn’t scale well if I have multiple apps. So I’ve been using Docker to handle standing up a test database.

I already had Homebrew installed, so I used it to install Docker:

brew cask install docker

Then I started a Postgres instance running in Docker for the test suite to talk to with this command:

docker run --name myapp-postgres-test -e POSTGRES_DB=myapp-test -e POSTGRES_USER=myapp -e POSTGRES_PASSWORD=password -p 5433:5432 -d postgres

This names the instance myapp-postgres-test so I can refer to it later. The -e flag sets environment variables, and POSTGRES_DB, POSTGRES_USER, POSTGRES_PASSWORD values all match the values I set in my TestingConfiguration. The -p flag tells Docker to publish Postgres’s default port (5432) on port 5433, which is where my TestingConfiguration was pointing. The -d flag tells Docker to run the container in the background, and postgres argument is the Docker image to use.

At this point, I can run all tests in Xcode (menu: Product > Test) or run vapor test from the command line.

Sometimes during development I changed the database schema in way that my migrations didn’t catch. In these cases, I was lazy, and I just reset the Docker image:

docker stop myapp-postgres-test
docker rm myapp-postgres-test

This first stops the container, then removes it, leaving me free to re-execute the docker run command above to stand up a fresh Postgres database.

As a final testing infrastructure tip, I’ll start by observing that Swift on Linux doesn’t currently have reflection capabilities. That means test cases have to be manually given to the testing framework as a parameter. I find this is tedious and error prone, especially given how forgetful I am. Therefore, I let the Swift Package Manager autogenerate the necessary information for me:

swift test --generate-linuxmain

Unfortunately the generated code is static, so I have re-run the command anytime I add or remove a test case.

Also, I’m terrible at remembering all these Docker and Swift Package Manager commands, so I added them to my app’s README.md file.

Conclusion

With API servers, I find writing integration tests more helpful than unit tests. However, in my experience there’s some boilerplate code needed to make creating integration tests in Vapor comfortable. My approach is create a TestApplication class to model the system under test, and make to it ergonomic to send requests and synchronously receive responses. Likewise, I use a TestResponse type to make validating responses easier. I also leverage fixtures for seeding the test database, and a utility function for resetting the database between test runs. Finally, I make use of Docker to make it convenient to stand up and reset the Postgres testing database.

When I started this post, I was planning on talking about Vapor integration testing. But in the thick of that, I found myself on a substantial rabbit trail talking about environment configuration. That’s because I wanted to change configuration values based on if I was in the testing environment or not. As it turns out, there’s not an established mechanism in Vapor for doing that that I like. So I built one that I like slightly better.

To clarify, what I mean by environment configuration is static global settings like: database URL, email server configuration, api keys, etc. That is, things that will change based on environment (e.g. production, staging, testing).

I’d like to have a few things for my environment configuration. First, I’d like all the configuration to be in one file, so it’s easy to see in one place. In my experience, most web frameworks provide this. They have config files that are specific to production, testing, and development. Second, I’d like the configuration to be statically checked at compile time, so I have confidence that all the values have been provided. I want to avoid the burden of having to run it and see it crashes at start up because something was not registered as a service. Finally, I’d like my global configure() method to not have conditionals in it. i.e. no ifs checking to see if the environment is testing and then doing something different than production; or changing the configuration code flow if the environment is release.

My plan for building this out is to first describe what I want the final environment configuration to look like in Swift. Then, I’ll figure out how to create and load that configuration based on the Vapor Environment. Finally, I modify the global configure() method to make use of the new configuration data type.

Start with where I want to end

I’ll first start by defining the testing configuration I’d like to use for running integration tests:

import Foundation
import Vapor
import FluentPostgreSQL

struct TestingConfiguration: ConfigurationType {
    let apiKey = "fake-api-key"
    let databaseConfig = PostgreSQLDatabaseConfig(
            hostname: "localhost",
            port: 5433,
            username: "myapp",
            database: "myapp-test",
            password: "password")
    let mailProvider = TestMailProvider()

    func configure(_ services: inout Services) throws {
        try services.register(mailProvider)
    }
}

This is as declarative as I can get it currently. The api key is fake since this is for tests, and the database is pointing to a test database on the localhost. I also want to point out I’m using native Swift types in my configuration like PostgreSQLDatabaseConfig. Finally, a fake test provider for the mailer is created and registered as a service. This will allow me to test sending email without actually sending mail to a real 3rd party service.

Ideally, I wouldn’t need the configure(_:) method here, and could do everything completely declaratively. But given the production mail provider that I’m using, I have to swap the provider out entirely. There’s not a configuration that can be given to the production provider that allows testing.

Another approach would be to register both testing and production mail providers and then use Vapor’s Config.prefer() to choose the correct one at runtime. But that would still require some sort of configuration method on the testing and production configurations to call Config.prefer() on the correct type. And, to me, that’s less clear than simply registering the provider that I want.

Now, here’s my production configuration:

import Foundation
import Vapor
import FluentPostgreSQL

struct ProductionConfiguration: ConfigurationType {
    let apiKey: String
    let databaseConfig: PostgreSQLDatabaseConfig

    init() throws {
        self.apiKey = try Environment.require("SECRET_API_KEY")
        guard let config = try PostgreSQLDatabaseConfig(url: Environment.require("DATABASE_URL")) else {
            throw Abort(.internalServerError, reason: "DATABASE_URL not configured correctly")
        }
        self.databaseConfig = config
    }

    func configure(_ services: inout Services) throws {
        try services.register(MailgunProvider(mailgunConfig: mailgun))
    }
}

The production configuration is a bit more complicated and less declarative because it needs to pull values out of the environment. It does use a small helper method I added to Environment to make fetching a required variable a little less verbose.

import Foundation
import Vapor

extension Environment {
    static func require(_ key: String) throws -> String {
        guard let value = get(key) else {
            throw Abort(.internalServerError, reason: "Missing value for \(key)")
        }
        return value
    }
}

Environment.require() works the same as Environment.get(), except it returns a non-optional, and throws an internal error if the value is missing.

I also have a DevelopmentConfiguration that I used for running the service locally. However, it’s not as interesting as the testing or production configurations, so I’m not showing it’s implementation here.

The final bit I need for my configurations is a protocol that they’re required to conform to. This is what allows the compiler to statically check that all the values are provided.

public protocol ConfigurationType: Service {
    var apiKey: String { get }
    var databaseConfig: PostgreSQLDatabaseConfig { get }

    func configure(_ services: inout Services) throws
}

The protocol simply declares the two config values that are used, and the configure() method. I’ll also note that it derives from the Service protocol so that I can register it with the dependency injection framework.

How to load the correct configuration based on the environment

At this point, I have three environment configurations (production, testing, and development) but I need a way of loading the correct one at runtime. I know that I want to pass the loaded environment configuration to the global configure() method for it to be used. So, for that reason, I chose to load it in the app(_:) method in app.swift.

import Vapor

/// Creates an instance of Application. This is called from main.swift in the run target.
public func app(_ env: Environment) throws -> Application {
    var config = Config.default()
    var env = env
    var services = Services.default()
    let configuration = try registerConfiguration(for: env, services: &services)
    try configure(&config, &env, &services, configuration)
    let app = try Application(config: config, environment: env, services: services)
    try boot(app)
    return app
}

All I did here was call a new method called registerConfiguration() and pass the result of that into the global configure() method. Here’s the definition of that:

private func registerConfiguration(for env: Environment, services: inout Services) throws -> ConfigurationType {
    if env.isRelease {
        return try register(ProductionConfiguration(env: env), services: &services)
    } else if env.isTesting {
        return register(TestingConfiguration(), services: &services)
    } else {
        return try register(DevelopmentConfiguration(env: env), services: &services)
    }
}

private func register<T: ConfigurationType>(_ configuration: T, services: inout Services) -> ConfigurationType {
    services.register(configuration, as: ConfigurationType.self)
    return configuration
}

This is where I choose which configuration to load and return. It’s not a clean mechanism, in that it asks a series of somewhat ambiguous questions to determine which configuration it should load. First, if the environment claims that it is a release environment, I choose the production configuration. If it’s not a release environment, I ask if it is a testing environment (kind of, see below). If it is neither production or testing, then I assume it should use a development configuration.

There are some tradeoffs to this approach. One benefit is that there will always be a valid environment configuration loaded. One downside is that this only supports three configurations. If I wanted to add a staging configuration, this code would have to change, in addition to creating a StagingConfiguration type. There’s also the matter of determining what kind of environment I have. isRelease is a Bool property on Environment, and multiple environments could have set that to true. That might be ok, but it does make the determination more ambiguous.

On the other hand the isTesting property is something I made up entirely:

import Foundation
import Vapor

extension Environment {
    var isTesting: Bool {
        return name == Environment.testing.name
    }
}

isTesting checks to see if the name is the same name as the static testing instance on Environment. So unlike production, the testing configuration will only be loaded for one specific environment, named a specific name. I did it this way because Environment only provides a name property and a isRelease flag. Other than the name, I didn’t have any way to differentiate between non-release environments.

An alternative way to do the loading would be by the name only. I’ve seen this with web frameworks for dynamic languages. If they have an environment named “spackle”, the framework will try to load a configuration file named something like “spackle.config” from a known location. This approach loses the static compile time checking (assuming the language had any), but it’s straight forward and flexible. In Swift, I could do the same, except use JSON as the config file format. That would mean I couldn’t use non-JSON (or non-Decodable) types in my configuration (like PostgreSQLDatabaseConfig), and dynamically selecting which providers to register would be trickier. A compromise for Swift could be to register the ConfigurationTypes in a global table by name. This would allow rich Swift datatypes to be used and static type checking to happen at compile time. However, the downside to all these approaches is if an environment name is given that doesn’t exactly match any of the configurations, the app will crash. That could be an acceptable tradeoff though, since one could argue that’s a progammer error.

What configure() looks like now

One of the goals I started out with was to remove all the conditionals in the global configure() method. Setting up all the necessary configuration is already a complicated enough task, and adding conditionals increases the complexity. The conditionals implicitly encode the differences in environment configuration, which makes discerning the differences difficult. It also is difficult to see when the differences were intentional vs. accidental. Since the configuration is implicit and spread out across a large file, it becomes trivial to make a mistake when updating an environment configuration.

Here’s a redacted version of my resulting global configure() method:

import FluentPostgreSQL
import Vapor
import Authentication
import Leaf

/// Called before your application initializes.
public func configure(_ config: inout Config, _ env: inout Environment, _ services: inout Services, _ configuration: ConfigurationType) throws {

    /// Register providers first
    try configuration.configure(&services)

    ...

    services.register(StaticApiKeyConfig(apiConfig: configuration), as: ApiKeyConfig.self)

    ...

    // Configure database
    var databases = DatabasesConfig()
    let databaseConfig = configuration.databaseConfig
    let database = PostgreSQLDatabase(config: databaseConfig)
    databases.add(database: database, as: .psql)
    services.register(databases)

    ...
}

First off, I let the environment configuration register any Services that it needs to. For the test of the method, it’s simply a matter of the configuration code pulling configuration values off the ConfigurationType. There are no conditionals, only straight line code.

It would be nice if Vapor provided some infrastructure for this out of the box. However, until then, this is how I create and load the environment configuration in Vapor. I first create the needed configurations as a native Swift types. Then I load them in the global app() method, before passing them to the global configure(). The end result is a statically checked configuration, all in one file, and a global configure() method with no conditionals.

The last time I got to yammering on, I mentioned I was building a side project in Vapor 3. When I got to setting up authentication for this project, I decided to use JWTs for reasons that are probably inexplicable to anyone with a clue.

Fortunately my cluelessness didn’t deter me. I quickly found that Vapor has an Authentication framework and a JWT framework. Unfortunately, I also found that they don’t work together out of the box, and there’s not much documentation on either. Cue my patented flailing around and trying to grok someone else’s code. Eventually, figured out something that worked.

Here’s my plan for marrying JWT with Authentication: first, figure out what to put in my JWT. Then I’ll need some code to authenticate that JWT, and some middleware to call it. Finally, I’ll use the results of the middleware to see if the holder of the JWT is authorized to perform the request they made.

Defining the JWT Payload

First, I had to define what my JWT looked like. A JWT is essentially a signed JSON payload, so I had to decide what should go in that payload. The main thing my app needed to know was the user making the request.

Here’s an abridged version of what I ended up doing:

import Foundation
import Vapor
import JWT
import Authentication

struct UserIDClaim: JWTClaim {
    var value: UUID
}

struct AuthJWTPayload: JWTPayload {
    func verify(using signer: JWTSigner) throws {
        // nothing for now
    }

    let uid: UserIDClaim
    let iat: IssuedAtClaim
}

The JWTPayload is a protocol defined in the JWT framework. Each property in the payload is supposed to be a “claim” that can be verified, and the verify method is supposed to do the verifying. However, since the verify method only takes the signer, there’s not much verifying it can actually do. I suppose it could verify that the IssuedAtClaim isn’t older than a given date.

Speaking of claims, there’s a standard set that the JWT framework defines. The IssuedAtClaim is an example. However, it’s possible to define your own, which is what I did with UserIDClaim. As the name suggests, it represents the user ID that the holder of the JWT operates as. All JWTClaims must be Codable, and define a mutable property named value.

The weird three letter property names appear to be a convention for claims. My assumption is it’s to keep the resulting JSON, and therefore the JWT, small.

Verifying a JWT Payload

Now that I had a payload defined, I needed something to verify it. The Authentication framework defines this kind of type as Authenticatable. I decided to be a bit fancy and define a protocol for JWT authenticatables before I wrote an implementation to authenticate my specific JWT payload.

import Foundation
import Vapor
import Authentication
import JWT

protocol JWTAuthenticatable: Authenticatable {
    associatedtype JWTType: JWTPayload

    static func authenticate(jwt: JWTType, on connection: DatabaseConnectable) -> Future<Self?>
}

The Authenticatable protocol itself has no constraints. I had JWTAuthenticatable declare a generic type that conforms to JWTPayload, and a static authenticate method to verify it. The return value will be an instance of the JWTAuthenticatable if the payload validates, nil otherwise.

Now I could define my actual JWTAuthenticatable type to verify the JWT payload:

import Foundation
import Vapor
import Authentication
import JWT

struct AuthJWTAuthenticatable: JWTAuthenticatable {
    typealias JWTType = AuthJWTPayload

    let user: User

    static func authenticate(jwt: JWTType, on connection: DatabaseConnectable) -> Future<AuthJWTAuthenticatable?> {
        return User.query(on: connection)
            .filter(\.id == jwt.uid.value)
            .first()
            .map { maybeUser in
                return maybeUser.map { AuthJWTAuthenticatable(user: $0) }
            }
    }

}

There are a couple of things to note here. First, if authenticate returns an instance of Authenticatable, the middleware (described later) is going to stash it on the request. Meaning Authenticatable is a great place to stash information about the authenticated “caller” because it can be accessed by other middleware or controllers. In my case, the caller represents a user, so I stash the user on the Authenticatable so my controllers can use it later.

The second thing of note is that I’m hitting the database to verify the JWT. This negates one of benefits of using a JWT, which is being able to verify a token without hitting the database. However, for my app I always need to have the user struct in memory, so it’s easier to go ahead and load it here. An alternative approach to avoid hitting the database during authentication would be:

struct AuthJWTAuthenticatable: JWTAuthenticatable {
    typealias JWTType = AuthJWTPayload

    let userId: UUID

    static func authenticate(jwt: JWTType, on connection: DatabaseConnectable) -> Future<AuthJWTAuthenticatable?> {
        return AuthJWTAuthenticatable(userId: jwt.uid.value)
    }
}

With this approach, I’d have to fetch the user in each controller that needed the full User.

Creating Some Middleware to Authenticate Incoming Requests

Now that I have a payload and the ability to authenticate it, I need to hook into the request handling and authenticate all incoming requests. As with most web frameworks, Vapor provides middleware as a solution for hooking into the request handling. With that in mind, here’s my middleware for authenticating incoming requests:

import Foundation
import Vapor
import JWT
import Authentication

struct JWTAuthenticationMiddleware<A: JWTAuthenticatable>: Middleware {

    func respond(to req: Request, chainingTo next: Responder) throws -> Future<Response> {
        // If the request is already authenticated, go to next middleware
        if try req.isAuthenticated(A.self) {
            return try next.respond(to: req)
        }

        // This parses the JWT string from the Authorization header
        guard let tokenString = req.http.headers.bearerAuthorization?.token else {
            // If there isn't a header, that's fine. Just go to the next middleware
            return try next.respond(to: req)
        }
        // Use my helper type to decode the string into my JWTPayload struct
        let token = try JWTSerialization<A.JWTType>().decode(tokenString, on: req)

        // Ask the JWTAuthenticatable if this is a valid JWT
        return A.authenticate(jwt: token, on: req).flatMap { a in
            if let a = a {
                // The JWT was valid, so this method call stashes the JWTAuthenticatable on the request
                try req.authenticate(a)
            }
            // Regardless of if the JWT is valid or not, go to the next middleware
            return try next.respond(to: req)
        }
    }
}

extension JWTAuthenticationMiddleware: ServiceType {
    static func makeService(for worker: Container) throws -> JWTAuthenticationMiddleware {
        return JWTAuthenticationMiddleware()
    }
}

I based my authentication middleware on the authentication middlewares defined in the Authentication framework. The main difference is my middleware uses my JWTAuthenticatable protocol to do the authentication. The goal of the middleware is to authenticate any Authorization header the request might have, and, if valid, store the resulting Authenticatable on the request for future use. Notably, the middleware doesn’t try to authorize the caller (i.e. it doesn’t require a valid Authorization header). This is because I don’t yet know at this stage what the request is trying to do, and what level of authorization that would require.

I factored out the JWT deserialization into a helper type to keep the middleware a bit more focused. Here’s what the deserialization code looks like:

import Foundation
import Vapor
import JWT

enum AuthJWTError: Error {
    case invalidDataEncoding
}

struct JWTSerialization<T: JWTPayload> {

    func encode(_ payload: T, on container: Container) throws -> String {
        let jwt = JWT(payload: payload)
        let config = try container.make(JWTConfig.self)
        let encoded = try jwt.sign(using: config.signer())
        guard let jwtString = String(data: encoded, encoding: .utf8) else {
            throw AuthJWTError.invalidDataEncoding
        }
        return jwtString
    }

    func decode(_ string: String, on container: Container) throws -> T {
        let config = try container.make(JWTConfig.self)
        let jwt = try JWT<T>(from: string, verifiedUsing: config.signer())
        return jwt.payload
    }
}

There’s not much to the decoding of JWTs. The JWT type is defined by Vapor’s JWT framework and can both sign and verify the signature of a JWT. The JWTConfig is a type that I created so I could abstract out the signing key and signing algorithm. I used Vapor’s dependency injection mechanism to fetch it. Here’s what it looks like:

import Foundation
import Vapor
import JWT

protocol JWTConfig: Service {
    func signer() throws -> JWTSigner
}

class HmacJWTConfig: JWTConfig {
    private let apiConfig: ApiConfigurationType

    public init(apiConfig: ApiConfigurationType) {
        self.apiConfig = apiConfig
    }

    func signer() throws -> JWTSigner {
        return try JWTSigner.hs256(key: apiConfig.jwtKey.hexEncodedData())
    }
}

The JWTConfig is a Service protocol that creates a JWTSigner (defined in the JWT framework) on demand. I only have one concrete implementation in my app: HmacJWTConfig. HmacJWTConfig is pretty simple; it creates a JWTSigner.hs256 signer (which uses HMAC-SHA256) with a key it pulls from the app config. The config implementation isn’t that interesting; on production it pulls a hex encoded key from an environment variable.

I’ve almost got my app where it can authenticate any incoming requests. The last piece is the configuration:

try services.register(AuthenticationProvider())
services.register(JWTAuthenticationMiddleware<AuthJWTAuthenticatable>.self)
services.register(HmacJWTConfig(apiConfig: configuration.api), as: JWTConfig.self)
...
middlewares.use(JWTAuthenticationMiddleware<AuthJWTAuthenticatable>.self)
...

This is simply registering all the services and middleware I defined earlier.

Using the Authenticatable

Finally, I’m ready to use the Authenticatable to see if a request is properly authorized. For my app, the requests coming in are user-scoped, meaning the user ID being acted on is in the request URL. A user should only be able to act on their own account. Therefore, my authorization method should make sure the request’s user ID parameter matches my JWT payload’s user ID.

Here’s what I do in my controller:

private func authorizedUser(_ request: Request) throws -> User {
    let user = try request.requireAuthenticated(AuthJWTAuthenticatable.self).user
    let userIdParameter = try request.parameters.next(UUID.self)
    guard userIdParameter == user.id else {
        throw Abort(.unauthorized, reason: "Unauthorized")
    }
    return user
}

The first line pulls out the AuthJWTAuthenticatable that my middleware stashed there. If it’s missing, then requireAuthenticated will throw an error. Next, I pull the request’s user ID parameter out. Finally, I compare the two to make sure they match. If they don’t, I abort with an unauthorized error.

This isn’t the only way to handle authorization. If I just wanted to make sure certain routes had authentication of some kind (i.e. requireAuthenticated doesn’t throw an error), then I could make use of GuardAuthenticationMiddleware defined in the Authentication framework.

And that’s about it for using JWTs to do authentication in Vapor 3. While Vapor has all the tools for JWT and authentication, they’re not put together in a way that works out of the box. For that reason, I’ve done a walkthrough here on how I put them together in my app.