Generating C Sources for CFFI Extensions ¶

CFFI ships a command-line tool, cffi-gen-src, that produces the same output as FFI.emit_c_code(): a .c source file ready to be compiled into a CPython c-extension module. This is useful for projects that want CFFI to generate source code for a later compilation step without depending on setuptools.

Installing CFFI installs the cffi-gen-src script; running python -m cffi.gen_src invokes the same command line and behaves identically. Use the former where a script with a Python shebang makes more sense (e.g. cross-compiling) and use the latter when the console script is not on PATH but a Python interpreter is.

In a packaged project, cffi-gen-src is usually called by a script or rule executed by a build backend, and the build backend then consumes the generated C file. Any Python build backend that can run a source generator, such as meson-python, scikit-build-core, or similar, can use cffi-gen-src the same way. The rest of this page uses meson-python in the examples.

The command line is the only supported interface; the Python API inside cffi._cffi_gen_src is private.

The implementation is based on the cffi-buildtool project by Rose Davidson (@inklesspen on GitHub). It is included in CFFI with permission of the original author.

The `cffi-gen-src` Command-line Tool ¶

cffi-gen-src has two subcommands. The first, exec-python is most useful if you already have a Python script that sets up an FFI definition. The second, read-sources is most useful if you are wrapping a large API surface and want a more structured way to specify a set of FFI definitions.

`cffi-gen-src exec-python`¶

This mode executes a Python script that creates a cffi.FFI object as a module-level global and calls FFI.set_source() on it, then emits the generated C source. By default cffi-gen-src looks for a global named ffibuilder; pass --ffi-var if the object has another name. Apart from being run by the tool instead of by hand, it is the same script the CFFI docs show under Main mode of usage.

Let’s say we want to create an extension module that wraps a single C function named square. The square function has the following signature:

int square(int n);

Let’s also say this function definition is exposed inside a header named square.h. We could create a set of FFI bindings for this function given this _squared_build.py:

from cffi import FFI

ffibuilder = FFI()

ffibuilder.cdef("int square(int n);")

ffibuilder.set_source(
    "squared._squared",
    '#include "square.h"',
)

To generate the source code for the C extension, you would run:

$ cffi-gen-src exec-python _squared_build.py _squared.c

Many CFFI FFI definition scripts have an if __name__ == "__main__" section that triggers a compilation step. This is not needed for a script run by cffi-gen-src, which only writes the generated C source. If the script does have such a section it is harmless: the script is executed with __name__ set to "cffi.gen_src", so the block is skipped and an existing FFI definition script works unchanged.

If the cffi.FFI is bound to a name other than ffibuilder, pass --ffi-var. To make that concrete, let’s say your FFI definition script creates an FFI object named make_ffi:

from cffi import FFI

make_ffi = FFI()

In that case, you would pass --ffi-var=make_ffi to cffi-gen-src:

$ cffi-gen-src exec-python --ffi-var=make_ffi _squared_build.py _squared.c

Note

cffi-gen-src emits C source for out-of-line API mode modules. It cannot emit source for ABI mode modules where FFI.set_source() is called with None.

Note

CFFI’s setuptools integration supports passing libraries=, library_dirs=, include_dirs=, and extra_compile_args= arguments to FFI.set_source(). When using cffi-gen-src, these arguments are ignored. Link and include settings are the responsibility of the compilation step that happens after generating the C extension sources. If you are using meson-python following the examples in this section, you would express them through the dependencies, include_directories, and c_args arguments of py.extension_module().

`cffi-gen-src read-sources`¶

For larger modules, keeping the FFI definition and any necessary C source prelude in separate files tends to be easier to work with – you can configure your editor to treat them as plain C, and write presubmit tooling that parses the FFI definition directly without extracting it from a Python script.

Given squared.cdef.txt:

int square(int n);

and squared.csrc.c:

#include "square.h"

you would run the following command to generate the C source code for a CFFI extension:

$ cffi-gen-src read-sources squared._squared squared.cdef.txt squared.csrc.c _squared.c

With all other details left exactly the same as the exec-python example.

The first positional argument passed to the read-sources command is the fully qualified module name that will be embedded in the generated C source code (equivalent to the first argument to FFI.set_source()).

A Worked Example Using `meson-python`¶

Project layout:

squared/
├── pyproject.toml
├── meson.build
└── src/
    ├── squared/
    │   ├── __init__.py
    │   └── _squared_build.py
    └── csrc/
        ├── square.h
        └── square.c

pyproject.toml:

[build-system]
build-backend = 'mesonpy'
requires = ['meson-python', 'cffi']

[project]
name = 'squared'
version = '0.1.0'
description = 'Small self-contained example project that builds a CFFI extension via meson-python.'
requires-python = '>=3.9'
dependencies = ['cffi']

meson.build:

project(
    'squared',
    'c',
    version: '0.1.0',
    default_options: ['warning_level=2'],
)

py = import('python').find_installation(pure: false)

install_subdir('src/squared', install_dir: py.get_install_dir())

cffi_gen_src = find_program('cffi-gen-src')

square_lib = static_library(
    'square',
    'src/csrc/square.c',
    include_directories: include_directories('src/csrc'),
)
square_dep = declare_dependency(
    link_with: square_lib,
    include_directories: include_directories('src/csrc'),
)

squared_ext_src = custom_target(
    'squared-cffi-src',
    command: [
        cffi_gen_src,
        'exec-python',
        '@INPUT@',
        '@OUTPUT@',
    ],
    output: '_squared.c',
    input: ['src/squared/_squared_build.py'],
)

py.extension_module(
    '_squared',
    squared_ext_src,
    subdir: 'squared',
    install: true,
    dependencies: [square_dep],
)

src/squared/__init__.py:

from ._squared import ffi, lib


def squared(n):
    return lib.square(n)

src/squared/_squared_build.py:

from cffi import FFI

ffibuilder = FFI()

ffibuilder.cdef("int square(int n);")

ffibuilder.set_source(
    "squared._squared",
    '#include "square.h"',
)

if __name__ == "__main__":
    ffibuilder.compile(verbose=True)

src/csrc/square.h:

#ifndef SQUARE_H
#define SQUARE_H

int square(int n);

#endif

src/csrc/square.c:

#include "square.h"

int square(int n) {
    return n * n;
}

Build and install the project with any Python build front-end. For example, with pip, in the root squared directory:

$ python -m pip install .
$ python -c "from squared import squared; print(squared(7))"
49

To switch this project to read-sources mode, replace _squared_build.py with two files, so that the project layout becomes:

squared/
├── pyproject.toml
├── meson.build
└── src/
    ├── squared/
    │   ├── __init__.py
    │   ├── squared.cdef.txt
    │   └── squared.csrc.c
    └── csrc/
        ├── square.h
        └── square.c

The first new file, squared.cdef.txt, contains the FFI definition:

int square(int n);

and the second, squared.csrc.c, contains the C source prelude:

#include "square.h"

then change two spots in the meson.build file. First, update the custom_target command to call cffi-gen-src read-sources with two input arguments:

command: [
  cffi_gen_src,
  'read-sources',
  'squared._squared',
  '@INPUT0@',
  '@INPUT1@',
  '@OUTPUT@',
],

and then list both of the FFI specification files under input:

input: ['src/squared/squared.cdef.txt', 'src/squared/squared.csrc.c']

Distributing CFFI Extensions using Setuptools ¶

You can (but don’t have to) use CFFI’s Distutils or Setuptools integration when writing a setup.py. For Distutils (only in out-of-line API mode; deprecated since Python 3.10):
# setup.py (requires CFFI to be installed first)
from distutils.core import setup

import foo_build   # possibly with sys.path tricks to find it

setup(
    ...,
    ext_modules=[foo_build.ffibuilder.distutils_extension()],
)
For Setuptools (out-of-line only, but works in ABI or API mode; recommended):
# setup.py (with automatic dependency tracking)
from setuptools import setup

setup(
    ...,
    setup_requires=["cffi>=1.0.0"],
    cffi_modules=["package/foo_build.py:ffibuilder"],
    install_requires=["cffi>=1.0.0"],
)
Note again that the foo_build.py example contains the following lines, which mean that the ffibuilder is not actually compiled when package.foo_build is merely imported—it will be compiled independently by the Setuptools logic, using compilation parameters provided by Setuptools:
if __name__ == "__main__":    # not when running with setuptools
    ffibuilder.compile(verbose=True)

Note that some bundler tools that try to find all modules used by a project, like PyInstaller, will miss _cffi_backend in the out-of-line mode because your program contains no explicit import cffi or import _cffi_backend. You need to add _cffi_backend explicitly (as a “hidden import” in PyInstaller, but it can also be done more generally by adding the line import _cffi_backend in your main program).

Note that CFFI actually contains two different FFI classes. The page Using the ffi/lib objects describes the common functionality. It is what you get in the from package._foo import ffi lines above. On the other hand, the extended FFI class is the one you get from import cffi; ffi_or_ffibuilder = cffi.FFI(). It has the same functionality (for in-line use), but also the extra methods described below (to prepare the FFI). NOTE: We use the name ffibuilder instead of ffi in the out-of-line context, when the code is about producing a _foo.so file; this is an attempt to distinguish it from the different ffi object that you get by later saying from _foo import ffi.

The reason for this split of functionality is that a regular program using CFFI out-of-line does not need to import the cffi pure Python package at all. (Internally it still needs _cffi_backend, a C extension module that comes with CFFI; this is why CFFI is also listed in install_requires=.. above. In the future this might be split into a different PyPI package that only installs _cffi_backend.)

Note that a few small differences do exist: notably, from _foo import ffi returns an object of a type written in C, which does not let you add random attributes to it (nor does it have all the underscore-prefixed internal attributes of the Python version). Similarly, the lib objects returned by the C version are read-only, apart from writes to global variables. Also, lib.__dict__ does not work before version 1.2 or if lib happens to declare a name called __dict__ (use instead dir(lib)). The same is true for lib.__class__, lib.__all__ and lib.__name__ added in successive versions.

Generating C Sources for CFFI Extensions ¶

The `cffi-gen-src` Command-line Tool ¶

`cffi-gen-src exec-python`¶

`cffi-gen-src read-sources`¶

A Worked Example Using `meson-python`¶

Distributing CFFI Extensions using Setuptools ¶

CFFI

Navigation

Related Topics

Generating C Sources for CFFI Extensions¶

The cffi-gen-src Command-line Tool¶

cffi-gen-src exec-python¶

cffi-gen-src read-sources¶

A Worked Example Using meson-python¶

Distributing CFFI Extensions using Setuptools¶

Generating C Sources for CFFI Extensions ¶

The `cffi-gen-src` Command-line Tool ¶

`cffi-gen-src exec-python`¶

`cffi-gen-src read-sources`¶

A Worked Example Using `meson-python`¶

Distributing CFFI Extensions using Setuptools ¶