Native machine code in .mpy files¶
This section describes how to build and work with .mpy files that contain native machine code from a language other than Python. This allows you to write code in a language like C, compile and link it into a .mpy file, and then import this file like a normal Python module. This can be used for implementing functionality which is performance critical, or for including an existing library written in another language.
One of the main advantages of using native .mpy files is that native machine code can be imported by a script dynamically, without the need to rebuild the main MicroPython firmware. This is in contrast to MicroPython external C modules which also allows defining custom modules in C but they must be compiled into the main firmware image.
The focus here is on using C to build native modules, but in principle any language which can be compiled to stand-alone machine code can be put into a .mpy file.
A native .mpy module is built using the mpy_ld.py
tool, which is found in the
tools/
directory of the project. This tool takes a set of object files
(.o files) and links them together to create a native .mpy files. It requires
CPython 3 and the library pyelftools v0.25 or greater.
Supported features and limitations¶
A .mpy file can contain MicroPython bytecode and/or native machine code. If it
contains native machine code then the .mpy file has a specific architecture
associated with it. Current supported architectures are (these are the valid
options for the ARCH
variable, see below):
x86
(32 bit)x64
(64 bit x86)armv7m
(ARM Thumb 2, eg Cortex-M3)armv7emsp
(ARM Thumb 2, single precision float, eg Cortex-M4F, Cortex-M7)armv7emdp
(ARM Thumb 2, double precision float, eg Cortex-M7)xtensa
(non-windowed, eg ESP8266)xtensawin
(windowed with window size 8, eg ESP32)
When compiling and linking the native .mpy file the architecture must be chosen and the corresponding file can only be imported on that architecture. For more details about .mpy files see MicroPython .mpy files.
Native code must be compiled as position independent code (PIC) and use a global offset table (GOT), although the details of this varies from architecture to architecture. When importing .mpy files with native code the import machinery is able to do some basic relocation of the native code. This includes relocating text, rodata and BSS sections.
Supported features of the linker and dynamic loader are:
executable code (text)
read-only data (rodata), including strings and constant data (arrays, structs, etc)
zeroed data (BSS)
pointers in text to text, rodata and BSS
pointers in rodata to text, rodata and BSS
The known limitations are:
data sections are not supported; workaround: use BSS data and initialise the data values explicitly
static BSS variables are not supported; workaround: use global BSS variables
So, if your C code has writable data, make sure the data is defined globally, without an initialiser, and only written to within functions.
Linker limitation: the native module is not linked against the symbol table of the
full MicroPython firmware. Rather, it is linked against an explicit table of exported
symbols found in mp_fun_table
(in py/nativeglue.h
), that is fixed at firmware
build time. It is thus not possible to simply call some arbitrary HAL/OS/RTOS/system
function, for example.
New symbols can be added to the end of the table and the firmware rebuilt.
The symbols also need to be added to tools/mpy_ld.py
’s fun_table
dict in the
same location. This allows mpy_ld.py
to be able to pick the new symbols up and
provide relocations for them when the mpy is imported. Finally, if the symbol is a
function, a macro or stub should be added to py/dynruntime.h
to make it easy to
call the function.
Defining a native module¶
A native .mpy module is defined by a set of files that are used to build the .mpy. The filesystem layout consists of two main parts, the source files and the Makefile:
In the simplest case only a single C source file is required, which contains all the code that will be compiled into the .mpy module. This C source code must include the
py/dynruntime.h
file to access the MicroPython dynamic API, and must at least define a function calledmpy_init
. This function will be the entry point of the module, called when the module is imported.The module can be split into multiple C source files if desired. Parts of the module can also be implemented in Python. All source files should be listed in the Makefile, by adding them to the
SRC
variable (see below). This includes both C source files as well as any Python files which will be included in the resulting .mpy file.The
Makefile
contains the build configuration for the module and list the source files used to build the .mpy module. It should defineMPY_DIR
as the location of the MicroPython repository (to find header files, the relevant Makefile fragment, and thempy_ld.py
tool),MOD
as the name of the module,SRC
as the list of source files, optionally specify the machine architecture viaARCH
, and then includepy/dynruntime.mk
.
Minimal example¶
This section provides a fully working example of a simple module named factorial
.
This module provides a single function factorial.factorial(x)
which computes the
factorial of the input and returns the result.
Directory layout:
factorial/
├── factorial.c
└── Makefile
The file factorial.c
contains:
// Include the header file to get access to the MicroPython API
#include "py/dynruntime.h"
// Helper function to compute factorial
STATIC mp_int_t factorial_helper(mp_int_t x) {
if (x == 0) {
return 1;
}
return x * factorial_helper(x - 1);
}
// This is the function which will be called from Python, as factorial(x)
STATIC mp_obj_t factorial(mp_obj_t x_obj) {
// Extract the integer from the MicroPython input object
mp_int_t x = mp_obj_get_int(x_obj);
// Calculate the factorial
mp_int_t result = factorial_helper(x);
// Convert the result to a MicroPython integer object and return it
return mp_obj_new_int(result);
}
// Define a Python reference to the function above
STATIC MP_DEFINE_CONST_FUN_OBJ_1(factorial_obj, factorial);
// This is the entry point and is called when the module is imported
mp_obj_t mpy_init(mp_obj_fun_bc_t *self, size_t n_args, size_t n_kw, mp_obj_t *args) {
// This must be first, it sets up the globals dict and other things
MP_DYNRUNTIME_INIT_ENTRY
// Make the function available in the module's namespace
mp_store_global(MP_QSTR_factorial, MP_OBJ_FROM_PTR(&factorial_obj));
// This must be last, it restores the globals dict
MP_DYNRUNTIME_INIT_EXIT
}
The file Makefile
contains:
# Location of top-level MicroPython directory
MPY_DIR = ../../..
# Name of module
MOD = factorial
# Source files (.c or .py)
SRC = factorial.c
# Architecture to build for (x86, x64, armv7m, xtensa, xtensawin)
ARCH = x64
# Include to get the rules for compiling and linking the module
include $(MPY_DIR)/py/dynruntime.mk
Compiling the module¶
The prerequisite tools needed to build a native .mpy file are:
The MicroPython repository (at least the
py/
andtools/
directories).CPython 3, and the library pyelftools (eg
pip install 'pyelftools>=0.25'
).GNU make.
A C compiler for the target architecture (if C source is used).
Optionally
mpy-cross
, built from the MicroPython repository (if .py source is used).
Be sure to select the correct ARCH
for the target you are going to run on.
Then build with:
$ make
Without modifying the Makefile you can specify the target architecture via:
$ make ARCH=armv7m
Module usage in MicroPython¶
Once the module is built there should be a file called factorial.mpy
. Copy
this so it is accessible on the filesystem of your MicroPython system and can be
found in the import path. The module can now be accessed in Python just like any
other module, for example:
import factorial
print(factorial.factorial(10))
# should display 3628800
Further examples¶
See examples/natmod/
for further examples which show many of the available
features of native .mpy modules. Such features include:
using multiple C source files
including Python code alongside C code
rodata and BSS data
memory allocation
use of floating point
exception handling
including external C libraries