Exceptions

Built-in C++ to Python exception translation

When Python calls C++ code through pybind11, pybind11 provides a C++ exception handler that will trap C++ exceptions, translate them to the corresponding Python exception, and raise them so that Python code can handle them.

pybind11 defines translations for std::exception and its standard subclasses, and several special exception classes that translate to specific Python exceptions. Note that these are not actually Python exceptions, so they cannot be examined using the Python C API. Instead, they are pure C++ objects that pybind11 will translate the corresponding Python exception when they arrive at its exception handler.

Exception thrown by C++

Translated to Python exception type

std::exception

RuntimeError

std::bad_alloc

MemoryError

std::domain_error

ValueError

std::invalid_argument

ValueError

std::length_error

ValueError

std::out_of_range

IndexError

std::range_error

ValueError

std::overflow_error

OverflowError

pybind11::stop_iteration

StopIteration (used to implement custom iterators)

pybind11::index_error

IndexError (used to indicate out of bounds access in __getitem__, __setitem__, etc.)

pybind11::key_error

KeyError (used to indicate out of bounds access in __getitem__, __setitem__ in dict-like objects, etc.)

pybind11::value_error

ValueError (used to indicate wrong value passed in container.remove(...))

pybind11::type_error

TypeError

pybind11::buffer_error

BufferError

pybind11::import_error

import_error

Any other exception

RuntimeError

Exception translation is not bidirectional. That is, catching the C++ exceptions defined above above will not trap exceptions that originate from Python. For that, catch pybind11::error_already_set. See below for further details.

There is also a special exception cast_error that is thrown by handle::call() when the input arguments cannot be converted to Python objects.

Registering custom translators

If the default exception conversion policy described above is insufficient, pybind11 also provides support for registering custom exception translators. To register a simple exception conversion that translates a C++ exception into a new Python exception using the C++ exception’s what() method, a helper function is available:

py::register_exception<CppExp>(module, "PyExp");

This call creates a Python exception class with the name PyExp in the given module and automatically converts any encountered exceptions of type CppExp into Python exceptions of type PyExp.

It is possible to specify base class for the exception using the third parameter, a handle:

py::register_exception<CppExp>(module, "PyExp", PyExc_RuntimeError);

Then PyExp can be caught both as PyExp and RuntimeError.

The class objects of the built-in Python exceptions are listed in the Python documentation on Standard Exceptions. The default base class is PyExc_Exception.

When more advanced exception translation is needed, the function py::register_exception_translator(translator) can be used to register functions that can translate arbitrary exception types (and which may include additional logic to do so). The function takes a stateless callable (e.g. a function pointer or a lambda function without captured variables) with the call signature void(std::exception_ptr).

When a C++ exception is thrown, the registered exception translators are tried in reverse order of registration (i.e. the last registered translator gets the first shot at handling the exception).

Inside the translator, std::rethrow_exception should be used within a try block to re-throw the exception. One or more catch clauses to catch the appropriate exceptions should then be used with each clause using PyErr_SetString to set a Python exception or ex(string) to set the python exception to a custom exception type (see below).

To declare a custom Python exception type, declare a py::exception variable and use this in the associated exception translator (note: it is often useful to make this a static declaration when using it inside a lambda expression without requiring capturing).

The following example demonstrates this for a hypothetical exception classes MyCustomException and OtherException: the first is translated to a custom python exception MyCustomError, while the second is translated to a standard python RuntimeError:

static py::exception<MyCustomException> exc(m, "MyCustomError");
py::register_exception_translator([](std::exception_ptr p) {
    try {
        if (p) std::rethrow_exception(p);
    } catch (const MyCustomException &e) {
        exc(e.what());
    } catch (const OtherException &e) {
        PyErr_SetString(PyExc_RuntimeError, e.what());
    }
});

Multiple exceptions can be handled by a single translator, as shown in the example above. If the exception is not caught by the current translator, the previously registered one gets a chance.

If none of the registered exception translators is able to handle the exception, it is handled by the default converter as described in the previous section.

See also

The file tests/test_exceptions.cpp contains examples of various custom exception translators and custom exception types.

Note

Call either PyErr_SetString or a custom exception’s call operator (exc(string)) for every exception caught in a custom exception translator. Failure to do so will cause Python to crash with SystemError: error return without exception set.

Exceptions that you do not plan to handle should simply not be caught, or may be explicitly (re-)thrown to delegate it to the other, previously-declared existing exception translators.

Handling exceptions from Python in C++

When C++ calls Python functions, such as in a callback function or when manipulating Python objects, and Python raises an Exception, pybind11 converts the Python exception into a C++ exception of type pybind11::error_already_set whose payload contains a C++ string textual summary and the actual Python exception. error_already_set is used to propagate Python exception back to Python (or possibly, handle them in C++).

Exception raised in Python

Thrown as C++ exception type

Any Python Exception

pybind11::error_already_set

For example:

try {
    // open("missing.txt", "r")
    auto file = py::module_::import("io").attr("open")("missing.txt", "r");
    auto text = file.attr("read")();
    file.attr("close")();
} catch (py::error_already_set &e) {
    if (e.matches(PyExc_FileNotFoundError)) {
        py::print("missing.txt not found");
    } else if (e.match(PyExc_PermissionError)) {
        py::print("missing.txt found but not accessible");
    } else {
        throw;
    }
}

Note that C++ to Python exception translation does not apply here, since that is a method for translating C++ exceptions to Python, not vice versa. The error raised from Python is always error_already_set.

This example illustrates this behavior:

try {
    py::eval("raise ValueError('The Ring')");
} catch (py::value_error &boromir) {
    // Boromir never gets the ring
    assert(false);
} catch (py::error_already_set &frodo) {
    // Frodo gets the ring
    py::print("I will take the ring");
}

try {
    // py::value_error is a request for pybind11 to raise a Python exception
    throw py::value_error("The ball");
} catch (py::error_already_set &cat) {
    // cat won't catch the ball since
    // py::value_error is not a Python exception
    assert(false);
} catch (py::value_error &dog) {
    // dog will catch the ball
    py::print("Run Spot run");
    throw;  // Throw it again (pybind11 will raise ValueError)
}

Handling errors from the Python C API

Where possible, use pybind11 wrappers instead of calling the Python C API directly. When calling the Python C API directly, in addition to manually managing reference counts, one must follow the pybind11 error protocol, which is outlined here.

After calling the Python C API, if Python returns an error, throw py::error_already_set();, which allows pybind11 to deal with the exception and pass it back to the Python interpreter. This includes calls to the error setting functions such as PyErr_SetString.

PyErr_SetString(PyExc_TypeError, "C API type error demo");
throw py::error_already_set();

// But it would be easier to simply...
throw py::type_error("pybind11 wrapper type error");

Alternately, to ignore the error, call PyErr_Clear.

Any Python error must be thrown or cleared, or Python/pybind11 will be left in an invalid state.

Handling unraisable exceptions

If a Python function invoked from a C++ destructor or any function marked noexcept(true) (collectively, “noexcept functions”) throws an exception, there is no way to propagate the exception, as such functions may not throw. Should they throw or fail to catch any exceptions in their call graph, the C++ runtime calls std::terminate() to abort immediately.

Similarly, Python exceptions raised in a class’s __del__ method do not propagate, but are logged by Python as an unraisable error. In Python 3.8+, a system hook is triggered and an auditing event is logged.

Any noexcept function should have a try-catch block that traps class:error_already_set (or any other exception that can occur). Note that pybind11 wrappers around Python exceptions such as pybind11::value_error are not Python exceptions; they are C++ exceptions that pybind11 catches and converts to Python exceptions. Noexcept functions cannot propagate these exceptions either. A useful approach is to convert them to Python exceptions and then discard_as_unraisable as shown below.

void nonthrowing_func() noexcept(true) {
    try {
        // ...
    } catch (py::error_already_set &eas) {
        // Discard the Python error using Python APIs, using the C++ magic
        // variable __func__. Python already knows the type and value and of the
        // exception object.
        eas.discard_as_unraisable(__func__);
    } catch (const std::exception &e) {
        // Log and discard C++ exceptions.
        third_party::log(e);
    }
}

New in version 2.6.