Sometimes things don't turn out the way we want. For instance, I never should have looted that (supposedly cursed) jewel-encrusted sceptre from the Tomb of Rosetta.

I keep losing followers so I guess the curse is real after all

— Owen Nelson (@theomn) October 31, 2019

Error handling in Rust, with its extra type-system concerns, can feel awkward when coming from a language like Python.

In this post, we'll pretend to write a program that will report how many followers you've lost on some imaginary social network (since the last time you checked). Using this premise we'll compare some differences and similarities in the error handling patterns in Python and Rust.

If you're already familiar with error handling patterns in Python, my hope is this article will help you more easily find your footing in Rust, at least as far as error handling goes.

The flow of the imaginary program will be to:

• fetch a list of followers from the social network's public web API.
• read the follower list we fetched last time from disk cache.
• compare the two lists (identifying the unfollowers).
• update the disk cache with the updated follower list.

In terms of the failure conditions, we're looking at:

• If the initial fetch fails, we'll be forced to exit.
• If the cache read fails, we should be okay, but we won't be able to report unfollows.
• If the cache update fails, I guess we're in trouble and should probably exit with an error.

Exceptions in Python ◈

To set the scene, let's first look at some Python.

Using our make-believe "who unfollowed me" program as a stand-in for something worthy of being packaged up, we might define some custom exceptions to help advertise the failure modes of the library.

Libraries frequently provide custom exception types since this gives callers a good sense of the ways the library can fail, and a "scope of failure" to try/except around.

We might not be able to identify all the ways this program can fail, but the parts that rely on the network, external systems or disk are high on the list of things that can go wrong.

class WhoUnfollowedError(Exception):
    """Root error type for the library."""

class HttpError(WhoUnfollowedError):
    """Failed to complete an HTTP request."""

class DiskCacheReadError(WhoUnfollowedError):
    """Failed to read the cache file."""

class DiskCacheWriteError(WhoUnfollowedError):
    """Failed to update the cache file."""

Providing a "root" exception type for library along with several subclasses help consumers of the library to isolate errors in broad strokes, then get more granular as required.

Where possible, we'd like all the exceptions raised by our library to be one of our own or from the Python stdlib, not an exception type from our dependencies.

In our library internals, the functions dealing with disk have been written to raise our own exceptions and uses exception chaining to associate them with the underlying cause (probably some sort of OSError).

Unfortunately, the code that deals the web API will raise one of the many exception types provided by the ever-popular requests library, so we'll have to handle the chaining ourselves.

Chaining can help separate anticipated versus unexpected failures, but more than that it'll afford us some privacy in our public API. It's better for folks to be try/excepting our HttpError than one of the exceptions from requests.

With our custom exceptions defined we can write a function to run through the flow.

import sys
from requests.exceptions import RequestException

def print_report():
    """Print a report of who unfollowed you since the last time you checked.

    :raises WhoUnfollowedError:
    """
    try:
        current_followers = fetch_current_followers()
    except RequestException as exc:
        # Repackage the ``RequestException`` as one of our own types using
        # chaining.
        raise HttpError from exc

    try:
        previous_followers = read_cache()
    except DiskCacheReadError:
        print("Unable to read from cache file.", file=sys.stderr)
        # Fallback to an empty list if the file wasn't readable.
        # The file could have been deleted, so we'll try to repopulate with
        # the call to ``update_cache()`` at the end.
        previous_followers = []

    unfollowed = find_unfollows(current_followers, previous_followers)

    print(f"{len(unfollowed)} unfollowed since last run.")

    for record in unfollowed:
        print(f"{record.id} - {record.login}")
    else:
        # Nobody unfollowed so you must be doing something right!
        print("Nice job!")

    # This could raise ``DiskCacheWriteError`` if we don't have permission to
    # write to the cache directory, or if the disk is full.
    # Since there's nothing we can do about these problems ourselves, leave
    # the exception unhandled (possibly killing our program).
    update_cache(current_followers)

That's the Python scene.

Exceptions are raised, possibly re-raised, from various points in the call stack.

Custom exceptions help draw the borders around library code and advertise failure modes.

If an exception is raised but not handled by an except block, it'll terminate the Python script causing the process to exit.

Result Types in Rust ◈

Python uses exceptions for error handling, but Rust doesn't. What does Rust have instead?

In Rust, errors come in a couple different flavors:

• Panics which represent fatal errors that can't be handled.
• Results which model an outcome as a value, either good or bad.

Panics can't be handled, immediately killing the thread they happen in. By their very nature they are ill-fitted for the handling of errors.

As a general rule panics should be avoided at all costs, but this is a rule to be bent and broken depending on the context.

Chapter 9 of The Rust Programming Language has a great section, To panic! or Not to panic!, on how to decide when panicking might be a good choice.

While panics are not a good replacement for exceptions, the Result type is.

If you've looked at Rust's Option type, the idea of Result similar. Where Option represents a value that may or may not exist, Result is for values that may or may not be successful. This tracks more closely to the experience we get with exceptions in Python, so this will be our focus.

We're going to see Result in action throughout the remainder of the post but for more background, take a look at Recoverable Errors with Result, also from Chapter 9 of The Rust Programming Language.

Result in Practice ◈

Let's take a look at how we might approximate the Python implementation in Rust.

Since Rust doesn't have classes, we can't do the whole class inheritance thing we did with the exception types in Python. How do we represent our errors with the same sort of polymorphism we had with those classes?

A typical solution is to define an error type as an Enum. Enums are a single type that can represent multiple variants. It's not identical to inheritance, but it's decent substitute in this case.

use std::error::Error;
use std::fmt;

#[derive(Debug)]
enum WhoUnfollowedError {
    /// Failed to complete an HTTP request.
    Http { source: reqwest::Error },
    /// Failed to read the cache file.
    DiskCacheRead { source: std::io::Error },
    /// Failed to update the cache file.
    DiskCacheWrite { source: std::io::Error },
}

impl fmt::Display for WhoUnfollowedError {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        match self {
            WhoUnfollowedError::Http { .. } => {
                write!(f, "Fetch failed")
            }
            WhoUnfollowedError::DiskCacheRead { .. } => {
                write!(f, "Disk cache read failed")
            }
            WhoUnfollowedError::DiskCacheWrite { .. } => {
                write!(f, "Disk cache write failed")
            }
        }
    }
}

impl Error for WhoUnfollowedError {
    fn source(&self) -> Option<&(dyn Error + 'static)> {
        match self {
            WhoUnfollowedError::Http { source } => Some(source),
            // The "disk" variants both have the same type for their source
            // fields so we can group them up in the same match arm.
            WhoUnfollowedError::DiskCacheRead { source }
            | WhoUnfollowedError::DiskCacheWrite { source } => Some(source),
        }
    }
}

impl From<reqwest::Error> for WhoUnfollowedError {
    fn from(other: reqwest::Error) -> WhoUnfollowedError {
        WhoUnfollowedError::Http { source: other }
    }
}

In the above code block, we define an enum to represent the different failure modes, echoing the original Python exception types.

The enum derives the Debug trait, then manually implements Display.

The Debug and Display traits are sort of like the __repr__() and __str__() magic methods in Python, allowing our custom type to be printed.

Any type that wants to implement the Error trait must also implement these (it's a requirement of the trait).

We've opted to provide an implementation for Error's source() method which is effectively how Rust handles "chaining" of errors. Any Error can offer up another Error as an underlying source.

Additionally, we've implemented From for the error type in reqwest (no relation to requests) which is our HTTP client this time around, allowing automatic conversion into our error type via ?.

Technically, implementing Error is not a requirement for use with Result, but it sets you up to be compatible with the larger rust error handling ecosystem.

If the code for defining custom error types looks like a bunch of boilerplate, I can't disagree. Thankfully, there are a number of packages available to streamline the process. My favorite is thiserror, which can automate the Display and Error trait implementations including the Error::source() method. It can even implement the From trait to allow automatic conversion from other Error types into yours.

If we were using thiserror, our WhoUnfollowedError code might look like this:

use thiserror::Error;

#[derive(Error, Debug)]
enum WhoUnfollowedError {
    #[error("Failed to complete an HTTP request")]
    Http { #[from] source: reqwest::Error },
    #[error("Failed to read the cache file")]
    DiskCacheRead { source: std::io::Error },
    #[error("Failed to update the cache file")]
    DiskCacheWrite { source: std::io::Error },
}

Much more concise!

Now that we've got custom a custom error type, we can build the function for the entire flow.

fn fetch_current_followers() -> Result<Vec<User>, reqwest::Error> {
    /* ... */
}

fn read_cache() -> Result<Vec<User>, WhoUnfollowedError> {
    /* ... */
}

fn update_cache(records: &[User]) -> Result<(), WhoUnfollowedError> {
    /* ... */
}

fn find_unfollows<'a>(
    current: &'a [User],
    previous: &'a [User],
) -> &'a [User] {
    /* ... */
}

fn print_report() -> Result<(), WhoUnfollowedError> {
    // The `?` converts `reqwest::Error` into a `WhoUnfollowedError::Http`
    // automatically via the `From` impl.
    // It also immediately ends the function, returning its `Error`, if
    // something went bad.
    let current = fetch_current_followers()?;

    // The `match` keyword can be used in a similar way as the `except`
    // blocks in our Python version.
    let previous = match read_cache() {
        // When the Result is "ok", we take the value.
        Ok(records) => records,
        // When it's a `DiskCacheRead` error, we "handle it."
        Err(WhoUnfollowedError::DiskCacheRead { .. }) => {
            eprintln!("Unable to read cache file.");
            // Fallback to an empty list if the file wasn't readable.
            // The file could have been deleted, so we'll try to repopulate
            // with the call to `update_cache()` at the end.
            vec![]
        }
        // Propagate if something other than `DiskCacheRead` happened.
        Err(other) => return Err(other),
    };

    let unfollowed = find_unfollows(&current, &previous);

    println!("{} unfollowed since last run.", unfollowed.len());

    if unfollowed.is_empty() {
        // Nobody unfollowed so you must be doing something right!
        println!("Nice job!");
    }

    for record in unfollowed {
        println!("{} - {}", record.id, record.login);
    }

    // The `?` will propagate the error if there's a problem. 
    update_cache(&current)?;

    // If we made it this far, I guess everything is good!
    Ok(())
}

The Rust version looks closer to the Python than we might have imagined.

The function signatures (at the top of the code block) show clearly whether the operation is fallible (ie, returns a Result).

One of the bigger difference might be on the use of the ? operator( discussed below) which helps us to focus on the "happy path" in simple cases.

Another thing to note is how the Rust version counts on there being a single Error type as cited by the function's signature.

With Python, you can raise any sort of exception from any place in your program, but this Rust function expects to return a specific type of Error. Where the custom exception types were a nice to have for the Python version, the aggregation we get with our enum is vital here.

For library authors, using an explicit error type like we have written is preferred. Still, it is possible to express the Error type in completely abstract terms, like "any type that implements Error" but this is more or less the same using except Exception in Python. Often we aim to except with a more narrow type to avoid accidentally "swallowing" exceptions that should be handled more explicitly.

If you're writing non-library code, and you'd rather live a more carefree life, Anyhow (from the author of thiserror) is a great library to help streamline this workflow. To do this without an extra dependency, check out the Rust by Example guide on Boxing Errors.

Handle, Propagate, Panic ◈

Each time your code is faced with a Result you are being asked to make a choice:

• Handle the error then and there (often by providing some sort of fallback in the event of an error).
• Propagate the error, returning it to your caller, asking them face this choice for you.
• Panic by calling Result::unwrap() or Result::expect().

In the print_report() function, we are doing a combination of handling and propagating. Since this is the most common way to deal with a Result, Rust provides special syntax to streamline this flow using the ? operator.

The ? operator sweetens the deal. ◈

When a Result is followed with ?, Rust will effectively inject code that will:

• Produce the inner "successful" value if the Result is Ok.
• Force an early return of the function with the Error otherwise.

Andrew Gallant wrote an in-depth post on the topic of Rust error handling in which he calls out the specifics of something I'd noticed in practice, but could not find mentioned in the rust docs.

In the section titled The real try macro/? operator he shows how examples for the try! macro (the precursor to ?) shown in the Rust docs is really a simplified version of the actual code generated by the macro.

The Rust docs would have you think the assignment like

let happy_value = my_result?;

would be rewritten as

let happy_value = match my_result {
    Ok(val) => val,
    Err(e) => return Err(e),
}

In reality, the Err(e) match arm is not just returning the Error itself. Instead, it's also attempting to convert the type of the Error. The code generated by the macro is not this, but it's closer to this:

let happy_value = match my_result {
    // Call `.into()` on the error type to try and convert to whatever the 
    // function signature says the error type is.
    Ok(val) => val,
    Err(e) => return Err(e.into()),
}

The conversion part of this is a huge win. If we settle on a single error type for a project, we can capture errors from any source simply by implementing the From trait for the other error into your error type.

If you're using a library like thiserror to help write your error types, this dovetails nicely! Remember that thiserror can generate From implementations for you using small annotations via attributes on your enum variants.

In this way, you can stitch together all the possible failures that could arise and ascribe some extra meaning to them by wrapping them with your own Error type.

Why is Rust this way? ◈

To summarize difference between Result and exceptions:

A Result is a value so propagation up the call stack is manual whereas exceptions are fire and forget.

Exceptions will travel up the call stack until caught making them more implicit and indirect. It's possible the caller of a function will have no idea it could fail, and the number of ways things can fail (possibly) increases with each new function call required for that first function to return.

You may not be able to anticipate the need to try/except whatever the exception type is if it originates from some deep function call, many steps removed from the call site in your code. In Python systems, this can lead to "surprise" errors that show up at runtime, often in production.

A Result, being a value, encodes failure modes explicitly in the return types of functions. Having failure modes visible in this way reduces those "surprise" errors.

Programs dealing with concurrency is another place where we see the benefits of treating errors as values. One of Rust's aims (originally a part of the sales pitch) was "fearless concurrency", so it makes sense that this would be given weight during the language's design.

In a way, exceptions are spring-loaded. For programs with concurrency there's the risk of one task failure accidentally killing the rest of the pending tasks before they complete. This is often not desired!

Lynn Root wrote series of blog posts titled asyncio: We Did It Wrong, all about async programming in Python. The third post in the series, Exception Handling in asyncio, digs into the problems you'll face when you mix an exception model with tasks that are running simultaneously.

The solutions Lynn offers lean towards treating errors as values to the point looking sort of similar to how async Rust code doing a similar task might look.

Perhaps Python and Rust are not so different after all!