Validating JSON input in Rust web services

February 15, 2021 by Evaldas Buinauskas

One of the key features of a good web service is user input validation. External input cannot be trusted and application must prevent malicious data from being processed.

It’s not only a matter of security but as well as for service usability and development experience. When something goes wrong, at the very least service should respond with 400 Bad Request, but good service will also respond with exact details of what went wrong.

In order not to invent its own error response format, a service should use RFC7807. RFC 7807 provides a standard format for returning problem details from HTTP APIs.

In this tutorial, we’ll implement a web service in Rust using warp web framework and add request validation using validator

Warp is a super-easy, composable, web server framework for warp speeds.

Validator is a simple validation library for Rust structs.

Project creation

For starters, create a new project using cargo

cargo new warp-validation --bin
cd warp-validation

Dependencies

Then edit Cargo.toml file and add these dependencies:

[dependencies]
tokio = { version = "1", features = ["full"] }
warp = "0.3"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
validator = { version = "0.12", features = ["derive"] }
http-api-problem = { version =  "0.21.0", features = ["hyper", "warp"] }

tokio is asynchronous runtime for Rust applications
warp is web framework of choice
serde is serialization and deserialization framework
validator is validation library
http-api-problem implements RFC7807 Problem Details for HTTP APIs

Now that when all dependencies are ready, we can start by implementing the most basic service.

Implementing web service

Basic web service

This is the most basic web service you can build in Warp. All it does is responds to GET /hello/:name requests with Hello, :name, where :name is path variable. We’ll build on top of that.

use warp::Filter;

#[tokio::main]
async fn main() {
    let hello = warp::path!("hello" / String)
        .map(|name| format!("Hello, {}!", name));

    warp::serve(hello)
        .run(([127, 0, 0, 1], 3030))
        .await;
}

So if we called our endpoint using cURL:

curl http://localhost:3030/hello/evaldas

This would be the response:

Hello, evaldas!.

Accepting `POST` requests

The fundamental building block of warp is the Filter: they can be combined and composed to express rich requirements on requests.

Thanks to its Filter system, warp provides these out of the box:

Path routing and parameter extraction

Header requirements and extraction

Query string deserialization

JSON and Form bodies

Multipart form data

Static Files and Directories

Websockets

Access logging

Gzip, Deflate, and Brotli compression

A prebuilt list of filters can be found here.

In order to accept POST requests with JSON payloads, the following pieces are necessary:

Request struct needed for strongly typed requests. Notice that it derives the Deserialize trait, meaning, it can be deserialized from JSON
warp::post() filter to accept only POST requests
warp::body::json() to read JSON payload
.and(..) filter to chain them together and ensure that a successful request must match all of the required filters

use serde::Deserialize;
use warp::Filter;

#[derive(Deserialize, Debug)]
pub struct Request {
    pub name: String,
    pub email: String,
    pub age: u8,
}

#[tokio::main]
async fn main() {
    let hello = warp::path!("hello" / String)
        .and(warp::post())
        .and(warp::body::json())
        .map(|name: String, request: Request| {
            format!("Hello, {}! This is request: {:?}", name, request)
        });

    warp::serve(hello).run(([127, 0, 0, 1], 3030)).await;
}

See what happens when we send a valid request

curl http://localhost:3030/hello/evaldas \
	-X POST \
	-H "Content-Type: application/json"
	-d '{"name":"name","email":"email","age":1}'

We’d get this response back:

Hello, evaldas! This is request: Request { name: "name", email: "email", age: 1 }

Validating requests

Validating requests using the validator library is a breeze. All we really need to do is to derive the Validate trait and add #validate attributes to our fields.

#[derive(Deserialize, Debug, Validate)]
pub struct Request {
    #[validate(length(min = 1))]
    pub name: String,
    #[validate(email)]
    pub email: String,
    #[validate(range(min = 18, max = 100))]
    pub age: u8,
}

And then we can call .validate() method on our request to validate it.

#[tokio::main]
async fn main() {
    let hello = warp::path!("hello" / String)
        .and(warp::post())
        .and(warp::body::json())
        .map(|name: String, request: Request| {
            let result = request.validate();

            format!(
                "Hello, {}! This is request: {:?} and its validation result: {:?}",
                name, request, result
            )
        });

    warp::serve(hello).run(([127, 0, 0, 1], 3030)).await;
}

Sending exactly the same request would result in a response with a validation failure.

Hello, evaldas! This is request: Request { name: "name", email: "email", age: 1 } and its validation result: Err(ValidationErrors({"email": Field([ValidationError { code: "email", message: None, params: {"value": String("email")} }]), "age": Field([ValidationError { code: "range", message: None, params: {"value": Number(1), "max": Number(100.0), "min": Number(18.0)} }])}))

This is already great, but in order to completely implement validation, all we really need to do is to make it a part of the request chain.

Implementing a custom validation filter

To achieve that, we’ll start by creating a custom Error enum type that contains only ValidationErrors.

#[derive(Debug)]
enum Error {
    Validation(ValidationErrors),
}

It will also implement warp::reject::Reject, this is needed to turn an Error into a custom warp Rejection.

impl warp::reject::Reject for Error {}

Then we’ll create a custom method that accepts a generic value constrained to implement the Validate trait. It’s going to validate the value and either return the value itself or validation errors.

fn validate<T>(value: T) -> Result<T, Error>
where
    T: Validate,
{
    value.validate().map_err(Error::Validation)?;

    Ok(value)
}

And finally we’ll create a custom with_validated_json filter which does a couple of things:

Limits payload sizes to 16KiB using warp::body::content_length_limit
Uses built-in warp::body::json() filter to deserialize JSON into a struct
And then passes deserialized value onto validate method we just built

fn with_validated_json<T>() -> impl Filter<Extract = (T,), Error = Rejection> + Clone
where
    T: DeserializeOwned + Validate + Send,
{
    warp::body::content_length_limit(1024 * 16)
        .and(warp::body::json())
        .and_then(|value| async move { validate(value).map_err(warp::reject::custom) })
}

It’s now enough to replace warp::body::json() filter in our request chain with with_validated_json(). We also can remove validation from handler.

Sending that exact same request results in a validation failure, but this time it doesn’t even reach the handler and returns early.

Unhandled rejection: Validation(ValidationErrors({"email": Field([ValidationError { code: "email", message: None, params: {"value": String("email")} }]), "age": Field([ValidationError { code: "range", message: None, params: {"value": Number(1), "max": Number(100.0), "min": Number(18.0)} }])}))

Notice that it says that rejection is unhandled - this is the case when warp doesn’t know what to do with the Rejection and simply returns it. Luckily for us, there’s an easy way to handle them and return appropriate responses. There’s a excellent rejection example. We’ll improve it by making sure that rejections return an instance of Problem Details instead of only status code and plain error text.

We’ll use Rejection::find method to find whether our custom Error is the reason why request has failed, convert that into Problem Details, and in every other case return internal server error.

async fn handle_rejection(err: Rejection) -> Result<impl Reply, Infallible> {
    let response = if let Some(e) = err.find::<Error>() {
        handle_crate_error(e)
    } else {
        HttpApiProblem::with_title_and_type_from_status(StatusCode::INTERNAL_SERVER_ERROR)
    };

    Ok(response.to_hyper_response())
}

fn handle_crate_error(e: &Error) -> HttpApiProblem {
    match e {
        Error::Validation(errors) => {
            let mut problem =
                HttpApiProblem::with_title_and_type_from_status(StatusCode::BAD_REQUEST)
                    .set_title("One or more validation errors occurred")
                    .set_detail("Please refer to the errors property for additional details");

            let _ = problem.set_value("errors", errors.errors());

            problem
        }
    }
}

We then finally need to use Filter::recover to recover from the rejection and pass handle_rejection as an argument to it.

If we send exactly the same request again, this time it would return an instance of problem details with an appropriate status code:

{
  "type": "https://httpstatuses.com/400",
  "status": 400,
  "title": "One or more validation errors occurred",
  "detail": "Please refer to the errors property for additional details",
  "errors": {
    "age": [
      {
        "code": "range",
        "message": null,
        "params": {
          "max": 100.0,
          "min": 18.0,
          "value": 1
        }
      }
    ],
    "email": [
      {
        "code": "email",
        "message": null,
        "params": {
          "value": "email"
        }
      }
    ]
  }
}

The complete application looks as follows:

use http_api_problem::{HttpApiProblem, StatusCode};
use serde::de::DeserializeOwned;
use serde::Deserialize;
use std::convert::Infallible;
use validator::{Validate, ValidationErrors};
use warp::{Filter, Rejection, Reply};

#[derive(Deserialize, Debug, Validate)]
pub struct Request {
    #[validate(length(min = 1))]
    pub name: String,
    #[validate(email)]
    pub email: String,
    #[validate(range(min = 18, max = 100))]
    pub age: u8,
}

#[tokio::main]
async fn main() {
    let hello = warp::path!("hello" / String)
        .and(warp::post())
        .and(with_validated_json())
        .map(|name: String, request: Request| {
            format!("Hello, {}! This is request: {:?}", name, request)
        })
        .recover(handle_rejection);

    warp::serve(hello).run(([127, 0, 0, 1], 3030)).await;
}

fn with_validated_json<T>() -> impl Filter<Extract = (T,), Error = Rejection> + Clone
where
    T: DeserializeOwned + Validate + Send,
{
    warp::body::content_length_limit(1024 * 16)
        .and(warp::body::json())
        .and_then(|value| async move { validate(value).map_err(warp::reject::custom) })
}

fn validate<T>(value: T) -> Result<T, Error>
where
    T: Validate,
{
    value.validate().map_err(Error::Validation)?;

    Ok(value)
}

#[derive(Debug)]
enum Error {
    Validation(ValidationErrors),
}

impl warp::reject::Reject for Error {}

async fn handle_rejection(err: Rejection) -> Result<impl Reply, Infallible> {
    let response = if let Some(e) = err.find::<Error>() {
        handle_crate_error(e)
    } else {
        HttpApiProblem::with_title_and_type_from_status(StatusCode::INTERNAL_SERVER_ERROR)
    };

    Ok(response.to_hyper_response())
}

fn handle_crate_error(e: &Error) -> HttpApiProblem {
    match e {
        Error::Validation(errors) => {
            let mut problem =
                HttpApiProblem::with_title_and_type_from_status(StatusCode::BAD_REQUEST)
                    .set_title("One or more validation errors occurred")
                    .set_detail("Please refer to the errors property for additional details");

            let _ = problem.set_value("errors", errors.errors());

            problem
        }
    }
}

Vinted Engineering

These are the voyages of code tailors that help create Vinted

Validating JSON input in Rust web services

Project creation

Dependencies

Implementing web service

Basic web service

Accepting `POST` requests

Validating requests

Implementing a custom validation filter

Project creation

Dependencies

Implementing web service

Basic web service

Accepting POST requests

Validating requests

Implementing a custom validation filter

Accepting `POST` requests