NAME¶

nbdkit-plugin - how to write nbdkit plugins

SYNOPSIS¶

 #define NBDKIT_API_VERSION 2
 
 #include <nbdkit-plugin.h>
 
 #define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS
 
 static void *
 myplugin_open (void)
 {
   /* create a handle ... */
   return handle;
 }
 
 static struct nbdkit_plugin plugin = {
   .name              = "myplugin",
   .open              = myplugin_open,
   .get_size          = myplugin_get_size,
   .pread             = myplugin_pread,
   .pwrite            = myplugin_pwrite,
   /* etc */
 };
 
 NBDKIT_REGISTER_PLUGIN(plugin)

When this has been compiled to a shared library, do:

 nbdkit [--args ...] ./myplugin.so [key=value ...]

When debugging, use the -fv options:

 nbdkit -fv ./myplugin.so [key=value ...]

DESCRIPTION¶

An nbdkit plugin is a new source device which can be served using the Network Block Device (NBD) protocol. This manual page describes how to create an nbdkit plugin in C.

To see example plugins: https://github.com/libguestfs/nbdkit/tree/master/plugins

Plugins written in C have an ABI guarantee: a plugin compiled against an older version of nbdkit will still work correctly when loaded with a newer nbdkit. We also try (but cannot guarantee) to support plugins compiled against a newer version of nbdkit when loaded with an older nbdkit, although the plugin may have reduced functionality if it depends on features only provided by newer nbdkit.

For plugins written in C, we also provide an API guarantee: a plugin written against an older header will still compile unmodified with a newer nbdkit.

The API guarantee does not always apply to plugins written in other (non-C) languages which may have to adapt to changes when recompiled against a newer nbdkit.

To write plugins in other languages, see: nbdkit-lua-plugin(3), nbdkit-ocaml-plugin(3), nbdkit-perl-plugin(3), nbdkit-python-plugin(3), nbdkit-ruby-plugin(3), nbdkit-sh-plugin(3), nbdkit-tcl-plugin(3).

"#define NBDKIT_API_VERSION 2"¶

Plugins must choose which API version they want to use, by defining NBDKIT_API_VERSION to a positive integer prior to including "nbdkit-plugin.h" (or any other nbdkit header). The default version is 1 for backwards-compatibility with nbdkit v1.1.26 and earlier; however, it is recommended that new plugins be written to the maximum version (currently 2) as it enables more features and better interaction with nbdkit filters. Therefore, the rest of this document only covers the version 2 interface. A newer nbdkit will always support plugins written in C which were compiled against any prior API version.

"nbdkit-plugin.h"¶

All plugins should start by including this header file, after optionally choosing an API version:

 #include <nbdkit-plugin.h>

"#define THREAD_MODEL"¶

All plugins must define a thread model. See "THREADS" below for details. It is generally safe to use:

 #define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS

"struct nbdkit_plugin"¶

All plugins must define and register one "struct nbdkit_plugin", which contains the name of the plugin and pointers to callback functions.

 static struct nbdkit_plugin plugin = {
   .name              = "myplugin",
   .longname          = "My Plugin",
   .description       = "This is my great plugin for nbdkit",
   .open              = myplugin_open,
   .get_size          = myplugin_get_size,
   .pread             = myplugin_pread,
   .pwrite            = myplugin_pwrite,
   /* etc */
 };
 
 NBDKIT_REGISTER_PLUGIN(plugin)

The ".name" field is the name of the plugin.

The callbacks are described below (see "CALLBACKS"). Only ".name", ".open", ".get_size" and ".pread" are required. All other callbacks can be omitted. However almost all plugins should have a ".close" callback. Most real-world plugins will also want to declare some of the other callbacks.

The nbdkit server calls the callbacks in the following order over the lifetime of the plugin:

".load"

is called once just after the plugin is loaded into memory.

".config" and ".config_complete"

".config" is called zero or more times during command line parsing. ".config_complete" is called once after all configuration information has been passed to the plugin.

Both are called after loading the plugin but before any connections are accepted.

".open"

A new client has connected.

".can_write", ".get_size" and other option negotiation callbacks

These are called during option negotiation with the client, but before any data is served. These callbacks may return different values across different ".open" calls, but within a single connection, must always return the same value; other code in nbdkit may cache the per-connection value returned rather than using the callback a second time.

".pread", ".pwrite" and other data serving callbacks

After option negotiation has finished, these may be called to serve data. Depending on the thread model chosen, they might be called in parallel from multiple threads. The data serving callbacks include a flags argument; the results of the negotiation callbacks influence whether particular flags will ever be passed to a data callback.

".close"

The client has disconnected.

".open" ... ".close"

The sequence ".open" ... ".close" can be called repeatedly over the lifetime of the plugin, and can be called in parallel (depending on the thread model).

".unload"

is called once just before the plugin is unloaded from memory.

Flags¶

The following flags are defined by nbdkit, and used in various data serving callbacks as follows:

"NBDKIT_FLAG_MAY_TRIM"

This flag is used by the ".zero" callback; there is no way to disable this flag, although a plugin that does not support trims as a way to write zeroes may ignore the flag without violating expected semantics.

"NBDKIT_FLAG_FUA"

This flag represents Forced Unit Access semantics. It is used by the ".pwrite", ".zero", and ".trim" callbacks to indicate that the plugin must not return a result until the action has landed in persistent storage. This flag will not be sent to the plugin unless ".can_fua" is provided and returns "NBDKIT_FUA_NATIVE".

The following defines are valid as successful return values for ".can_fua":

"NBDKIT_FUA_NONE"

Forced Unit Access is not supported; the client must manually request a flush after writes have completed. The "NBDKIT_FLAG_FUA" flag will not be passed to the plugin's write callbacks.

"NBDKIT_FUA_EMULATE"

The client may request Forced Unit Access, but it is implemented by emulation, where nbdkit calls ".flush" after a write operation; this is semantically correct, but may hurt performance as it tends to flush more data than just what the client requested. The "NBDKIT_FLAG_FUA" flag will not be passed to the plugin's write callbacks.

"NBDKIT_FUA_NATIVE"

The client may request Forced Unit Access, which results in the "NBDKIT_FLAG_FUA" flag being passed to the plugin's write callbacks (".pwrite", ".trim", and ".zero"). When the flag is set, these callbacks must not return success until the client's request has landed in persistent storage.

ERROR HANDLING¶

If there is an error in the plugin, the plugin should call "nbdkit_error" to report an error message; additionally, if the callback is involved in serving data, the plugin should call "nbdkit_set_error" to influence the error code that will be sent to the client. These two functions can be called in either order. Then, the callback should return the appropriate error indication, eg. "NULL" or "-1".

If the call to "nbdkit_set_error" is omitted while serving data, then the global variable "errno" may be used. For plugins which have ".errno_is_preserved == 1" the core code will use "errno". In plugins written in non-C languages, we usually cannot trust that "errno" will not be overwritten when returning from that language to C. In that case, either the plugin must call "nbdkit_set_error" or hard-coded "EIO" is used.

"nbdkit_error" has the following prototype and works like printf(3):

 void nbdkit_error (const char *fs, ...);
 void nbdkit_verror (const char *fs, va_list args);

For convenience, "nbdkit_error" preserves the value of "errno".

"nbdkit_set_error" can be called at any time, but only has an impact during callbacks for serving data, and only when the callback returns an indication of failure. It has the following prototype:

 void nbdkit_set_error (int err);

FILENAMES AND PATHS¶

The server usually (not always) changes directory to "/" before it starts serving connections. This means that any relative paths passed during configuration will not work when the server is running (example: "nbdkit plugin.so disk.img").

To avoid problems, prepend relative paths with the current directory before storing them in the handle. Or open files and store the file descriptor.

 char *nbdkit_absolute_path (const char *filename);

 char *nbdkit_realpath (const char *filename);

CALLBACKS¶

 const char *name;

 const char *version;

 const char *longname;

 const char *description;

 void load (void);

 void unload (void);

 void dump_plugin (void);

 int config (const char *key, const char *value);

 const char *magic_config_key;

 int config_complete (void);

 const char *config_help;

 void *open (int readonly);

 void close (void *handle);

 int64_t get_size (void *handle);

 int can_write (void *handle);

 int can_flush (void *handle);

 int is_rotational (void *handle);

 int can_trim (void *handle);

 int can_zero (void *handle);

 int can_fua (void *handle);

 int pread (void *handle, void *buf, uint32_t count, uint64_t offset,
            uint32_t flags);

 int pwrite (void *handle, const void *buf, uint32_t count, uint64_t offset,
             uint32_t flags);

 int flush (void *handle, uint32_t flags);

 int trim (void *handle, uint32_t count, uint64_t offset, uint32_t flags);

 int zero (void *handle, uint32_t count, uint64_t offset, uint32_t flags);

THREADS¶

Each nbdkit plugin must declare its thread safety model by defining the "THREAD_MODEL" macro. (This macro is used by "NBDKIT_REGISTER_PLUGIN").

The possible settings for "THREAD_MODEL" are defined below.

"#define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_CONNECTIONS"

Only a single handle can be open at any time, and all requests happen from one thread.

Note this means only one client can connect to the server at any time. If a second client tries to connect it will block waiting for the first client to close the connection.

"#define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS"

This is a safe default for most plugins.

Multiple handles can be open at the same time, but requests are serialized so that for the plugin as a whole only one open/read/write/close (etc) request will be in progress at any time.

This is a useful setting if the library you are using is not thread-safe. However performance may not be good.

"#define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_REQUESTS"

Multiple handles can be open and multiple data requests can happen in parallel. However only one request will happen per handle at a time (but requests on different handles might happen concurrently).

"#define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL"

Multiple handles can be open and multiple data requests can happen in parallel (even on the same handle).

All the libraries you use must be thread-safe and reentrant. You may also need to provide mutexes for fields in your connection handle.

If none of the above thread models are suitable, then use "NBDKIT_THREAD_MODEL_PARALLEL" and implement your own locking using "pthread_mutex_t" etc.

SHUTDOWN¶

When nbdkit receives certain signals it will shut down (see "SIGNALS" in nbdkit(1)). The server will wait for any currently running plugin callbacks to finish and also call the ".unload" callback before unloading the plugin.

Note that it's not guaranteed this can always happen (eg. the server might be killed by "SIGKILL" or segfault).

PARSING SIZE PARAMETERS¶

Use the "nbdkit_parse_size" utility function to parse human-readable size strings such as "100M" into the size in bytes.

 int64_t nbdkit_parse_size (const char *str);

"str" can be a string in a number of common formats. The function returns the size in bytes. If there was an error, it returns "-1".

PARSING BOOLEAN PARAMETERS¶

Use the "nbdkit_parse_bool" utility function to parse human-readable strings such as "on" into a boolean value.

 int nbdkit_parse_bool (const char *str);

"str" can be a string containing a case-insensitive form of various common toggle values. The function returns 0 or 1 if the parse was successful. If there was an error, it returns "-1".

READING PASSWORDS¶

The "nbdkit_read_password" utility function can be used to read passwords from config parameters:

 int nbdkit_read_password (const char *value, char **password);

For example:

 char *password = NULL;
 
 static int
 myplugin_config (const char *key, const char *value)
 {
   ..
   if (strcmp (key, "password") == 0) {
     free (password);
     if (nbdkit_read_password (value, &password) == -1)
       return -1;
   }
   ..
 }

The "password" result string is allocated by malloc, and so you may need to free it.

This function recognizes several password formats. A password may be used directly on the command line, eg:

 nbdkit myplugin password=mostsecret

But more securely this function can also read a password interactively:

 nbdkit myplugin password=-

or from a file:

 nbdkit myplugin password=+/tmp/secret

(If the password begins with a "-" or "+" character then it must be passed in a file).

DEBUGGING¶

Run the server with -f and -v options so it doesn't fork and you can see debugging information:

 nbdkit -fv ./myplugin.so [key=value [key=value [...]]]

To print debugging information from within the plugin, call "nbdkit_debug", which has the following prototype and works like printf(3):

 void nbdkit_debug (const char *fs, ...);
 void nbdkit_vdebug (const char *fs, va_list args);

For convenience, "nbdkit_debug" preserves the value of "errno". Note that "nbdkit_debug" only prints things when the server is in verbose mode (-v option).

 int myplugin_debug_foo;
 int myplugin_debug_bar;
 ...
 if (myplugin_debug_foo) {
   nbdkit_debug ("lots of extra debugging about foo: ...");
 }

 nbdkit -f -v -D myplugin.foo=1 -D myplugin.bar=2 myplugin [...]

INSTALLING THE PLUGIN¶

The plugin is a "*.so" file and possibly a manual page. You can of course install the plugin "*.so" file wherever you want, and users will be able to use it by running:

 nbdkit /path/to/plugin.so [args]

However if the shared library has a name of the form "nbdkit-name-plugin.so" and if the library is installed in the $plugindir directory, then users can be run it by only typing:

 nbdkit name [args]

The location of the $plugindir directory is set when nbdkit is compiled and can be found by doing:

 nbdkit --dump-config

If using the pkg-config/pkgconf system then you can also find the plugin directory at compile time by doing:

 pkgconf nbdkit --variable=plugindir

PKG-CONFIG/PKGCONF¶

nbdkit provides a pkg-config/pkgconf file called "nbdkit.pc" which should be installed on the correct path when the nbdkit plugin development environment is installed. You can use this in autoconf configure.ac scripts to test for the development environment:

 PKG_CHECK_MODULES([NBDKIT], [nbdkit >= 1.2.3])

The above will fail unless nbdkit ≥ 1.2.3 and the header file is installed, and will set "NBDKIT_CFLAGS" and "NBDKIT_LIBS" appropriately for compiling plugins.

You can also run pkg-config/pkgconf directly, for example:

 if ! pkgconf nbdkit --exists; then
   echo "you must install the nbdkit plugin development environment"
   exit 1
 fi

You can also substitute the plugindir variable by doing:

 PKG_CHECK_VAR([NBDKIT_PLUGINDIR], [nbdkit], [plugindir])

which defines "$(NBDKIT_PLUGINDIR)" in automake-generated Makefiles.

WRITING PLUGINS IN OTHER PROGRAMMING LANGUAGES¶

You can also write nbdkit plugins in Lua, OCaml, Perl, Python, Ruby, shell script or Tcl. Other programming languages may be offered in future.

For more information see: nbdkit-lua-plugin(3), nbdkit-ocaml-plugin(3), nbdkit-perl-plugin(3), nbdkit-python-plugin(3), nbdkit-ruby-plugin(3), nbdkit-sh-plugin(3), nbdkit-tcl-plugin(3).

Plugins written in scripting languages may also be installed in $plugindir. These must be called "nbdkit-name-plugin" without any extension. They must be executable, and they must use the shebang header (see "Shebang scripts" in nbdkit(1)). For example a plugin written in Perl called "foo.pl" might be installed like this:

 $ head -1 foo.pl
 #!/usr/sbin/nbdkit perl
 $ sudo install -m 0755 foo.pl $plugindir/nbdkit-foo-plugin

and then users will be able to run it like this:

 $ nbdkit foo [args ...]

SEE ALSO¶

nbdkit(1), nbdkit-filter(3).

Standard plugins provided by nbdkit:

nbdkit-curl-plugin(1), nbdkit-data-plugin(1), nbdkit-example1-plugin(1), nbdkit-example2-plugin(1), nbdkit-example3-plugin(1), nbdkit-example4-plugin(1), nbdkit-ext2-plugin(1), nbdkit-file-plugin(1), nbdkit-floppy-plugin(1), nbdkit-guestfs-plugin(1), nbdkit-gzip-plugin(1), nbdkit-iso-plugin(1), nbdkit-libvirt-plugin(1), nbdkit-memory-plugin(1), nbdkit-nbd-plugin(1), nbdkit-null-plugin(1), nbdkit-partitioning-plugin(1), nbdkit-pattern-plugin(1), nbdkit-random-plugin(1), nbdkit-split-plugin(1), nbdkit-streaming-plugin(1), nbdkit-tar-plugin(1), nbdkit-vddk-plugin(1), nbdkit-xz-plugin(1), nbdkit-zero-plugin(1) ; nbdkit-lua-plugin(3), nbdkit-ocaml-plugin(3), nbdkit-perl-plugin(3), nbdkit-python-plugin(3), nbdkit-ruby-plugin(3), nbdkit-sh-plugin(3), nbdkit-tcl-plugin(3) .

AUTHORS¶

Eric Blake

Richard W.M. Jones

Pino Toscano

COPYRIGHT¶

Copyright (C) 2013-2018 Red Hat Inc.

LICENSE¶

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

• Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
• Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
• Neither the name of Red Hat nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.