FEAPI Documentation
Researchers in the field of Music Information Retrieval often require a large amount of different audio features to be computed and evaluated. Although many of these features are algorithmically not necessarily demanding, some development effort is required to adapt previously implemented features to new projects or to implement features that have been implemented by others before. Therefore, many researchers have to invest a lot of time into redundant and unnecessary programming work that may be avoided by using a generalized interface for low level feature extraction, which is proposed here.
FEAPI is a Feature Extraction plugin API (Application Programming Interface) with a C programming language interface. It is designed to extract low level features from successive input blocks of audio data. The advantages of a well defined API are:
-
fast implementation of new features due to pre-defined API on plugin side and host side
-
fast integration of plugins from other researchers either as source code or in binary format
-
avoidance of redundant development due to openly published feature extraction plugins from other research groups
A plugin API should be designed with two main goals in mind: on the one hand the definition of an easy to understand API with as few functions and parameters as possible to allow fast plugin development and easy host implementation, and on the other hand the definition of an API that is as general as possible to allow for the most diverse use cases and features. Unfortunately, these two requirements tend to contradict each other, so a compromise has to be found.
For the sake of simplicity, the following restrictions have been accepted:
-
the plugin shall not allocate memory that is used outside of the plugin
-
the plugin shall never call host functions, but only answer function calls by the host
-
the plugin shall not need any file handles or complex data structures passed to it from the host
-
the plugin shall work on succeeding blocks of audio data
-
one plugin library (dll, shared lib) can only contain exactly one plugin
Due to the diversity of features to be extracted, the API has to support:
-
multidimensional features
-
features with different sample rates and possibly non-equidistant sampling intervals
-
features with pre-defined as well as undefined processing window lengths
Further requirements are cross-platform compatibility of the API and compatibility with as many programming languages as possible, while using a widely used programming language that allows efficient implementation of signal processing routines. The best compromise here seems to be the programming language ANSI C, since C compilers are available on all computing platforms, it is widely known and it is often used in the field of audio signal processing. Furthermore, many other programming languages and environments allow for the integration of C-libraries.
The API declaration and short function descriptions can be found in the file FEAPI.h. Note that the function pointer declarations are used as synonyms for the function names themselves.
When talking about samples, the number of audio samples in one channel is meant. If the sample size is 32 bit, one sample has a memory usage of 4 bytes. Frames are defined to be the number of samples per channel. A block of frames of length N consists therefore of (N * channels) samples.
Configuration settings that can be defined by the plugin user are called parameters, the algorithm output or the features being extracted are called results. Input data as well as result data have the properties of so called signals.
Before using a plugin, an instance has to be created. After using a plugin, this previously created instance has to be destroyed to free the allocated memory and resources.
-
FEAPI_Error_t FEAPI_CreatePluginInstance_t (FEAPI_PluginInstance_t *phInstanceHandle)
Upon calling this function, a new instance is created. The handle to the new instance is written to the function parameter phInstanceHandle.
Note that this function is not intended to implement any time-consuming or heavy-weight stuff like memory allocation; this has to be done in FEAPI_InitializePlugin_t. The return value is FEAPI_kNoError in case of no error.
-
FEAPI_Error_t FEAPI_DestroyPluginInstance_t (FEAPI_PluginInstance_t *phInstanceHandle)
The instance pointed to in phInstanceHandle is destroyed with this function call.
The return value is FEAPI_kNoError in case of no error.
-
FEAPI_Error_t FEAPI_InitializePlugin_t (FEAPI_PluginInstance_t hInstanceHandle, float fInputSampleRate, int iNumberOfAudioChannels, int iHostApiMajorVersion, FEAPI_UserData_t *pstUserData)
This function initializes the plugin and does the internal memory allocation. fSampleRate gives the sample rate in Hz, while iNumberOfChannels gives the audio channel count. The sample rate for all input signals is identical and remains constant. iHostApiMajorVersion contains the major version number of the host's FEAPI implementation and pstUserData is either NULL or contains some special vendor specific data. Please note that neither the sample rate nor the number of channels can be modified after the instance creation. The function has to be called at least once before the first call of FEAPI_ProcessPlugin_t and after changing a non-realtime parameter. It is important to note that the plugin state is only completely defined after the call of FEAPI_InitializePlugin_t. For example, the number of results may depend on sample rate, number of channels, or (non-realtime) parameter settings.
The return value is FEAPI_kNoError in case of no error.
The functions described in this section allow to retrieve information about the plugin itself.
-
int FEAPI_GetPluginAPIVersion_t (FEAPI_VersionInfo_t eAPIMajorMinorOrSubVersion)
The API version is defined in the format major.minor.sub, each of them (major, minor, and sub) being an integer. This function returns the API version being used for the plugin for one of the three possibilities as specified by eAPIMajorMinorOrSubVersion.
-
FEAPI_Error_t FEAPI_GetPluginName_t (FEAPI_PluginInstance_t hInstanceHandle, char *pcPluginName)
Call this function to retrieve the name of the plugin. hInstanceHandle points to the current instance. The plugin name is copied into the buffer specified by function parameter pcPluginName. Note thath the host has to allocate at least 1024 bytes including the string terminator for the string.
The return value is FEAPI_kNoError in case of no error.
-
FEAPI_Error_t FEAPI_GetPluginVendor_t (FEAPI_PluginInstance_t hInstanceHandle, char *pcPluginVendor)
Call this function to retrieve the vendor string of the plugin. hInstanceHandle points to the current instance. The plugin vendor string is copied into the buffer specified by function parameter pcPluginVendor. Note that the host has to allocate at least 1024 bytes including the string terminator for the string.
The return value is FEAPI_kNoError in case of no error.
-
float FEAPI_GetPluginProperty_t (FEAPI_PluginInstance_t hInstanceHandle, FEAPI_PluginProperty_t ePluginProperty)
Call this function to retrieve information about the basic capabilities of the plugin. hInstanceHandle points to the current instance. The requested property is specified in ePluginProperty. For information about possible requests please look at FEAPI_PluginProperty_t .
The return value is the requested value in float format.
-
FEAPI_Error_t FEAPI_GetPluginId_t (FEAPI_PluginInstance_t hInstanceHandle, char *pcPluginId)
Call this function to retrieve the vendor specific ID of the plugin. hInstanceHandle points to the current instance. The plugin ID string is copied into the buffer specified by function parameter pcPluginId. Note that the host has to allocate at least 1024 bytes including the string terminator for the string.
The return value is FEAPI_kNoError in case of no error.
-
int FEAPI_GetPluginVendorVersion_t (FEAPI_PluginInstance_t hInstanceHandle, FEAPI_VersionInfo_t ePluginMajorMinorOrSubVersion)
hInstanceHandle points to the current instance. This function can be used in exactly the same way as FEAPI_GetPluginAPIVersion_t but returns the vendor version of the plugin instead of its API version.
-
FEAPI_Error_t FEAPI_GetPluginDescription_t (FEAPI_PluginInstance_t hInstanceHandle, char *pcPluginDescription)
Call this function to retrieve a description of what the plugin actually does. hInstanceHandle points to the current instance. The plugin description string is copied into the buffer specified by function parameter pcPluginDescription. Note that the host has to allocate at least 4096 bytes including the string terminator for the string.
The return value is FEAPI_kNoError in case of no error.
-
FEAPI_Error_t FEAPI_GetPluginCopyright_t (FEAPI_PluginInstance_t hInstanceHandle, char *pcPluginCopyright)
Call this function to retrieve copyright information about the plugin. hInstanceHandle points to the current instance. The copyright string is copied into the buffer specified by the function parameter pcPluginCopyright. Note that the host has to allocate at least 4096 bytes including the string terminator for the string.
The return value is FEAPI_kNoError in case of no error.
In most cases, the data inputs will be audio inputs and the number of data inputs will equal the number of audio channels. However, other input signals are allowed. The host can poll for information about all input channels in the format of structure FEAPI_SignalDescription_t.
-
int FEAPI_GetPluginNumOfInputs_t (FEAPI_PluginInstance_t hInstanceHandle)
Call this function to retrieve the number of inputs required by the plugin. hInstanceHandle points to the current instance. In most cases, the return value will equal the number of audio channels. It can never be smaller than the number of audio channels.
-
FEAPI_Error_t FEAPI_GetPluginInputDescription_t (FEAPI_PluginInstance_t hInstanceHandle, int iInputIndex, FEAPI_SignalDescription_t *pstInputDescription)
Call this function to retrieve a description of a specific input. hInstanceHandle points to the current instance. iInputIndex is the parameter index ranging from 0 for the first parameter to (number of inputs - 1). The detailed input description is copied into a structure of type FEAPI_SignalDescription_t pointed to by the function parameter pstInputDescription. Note that the host has to allocate the memory for the parameter description structure.
The return value is FEAPI_kNoError in case of no error.
Parameters can be used to control the plugin algorithm configuration. For each single parameter, several properties are defined in the FEAPI_ParameterDescription_t structure, which contains information like name, unit, range, default value, description, parameter quantization and "real-timeness". Each parameter has to be in float format. Note the property FEAPI_ParameterDescription_t::bIsChangeableInRealTime that, if it is set to non-realtime, requires a call to FEAPI_InitializePlugin_t after changing the parameter value.
-
FEAPI_Error_t FEAPI_ProcessPlugin_t (FEAPI_PluginInstance_t hInstanceHandle, const FEAPI_Signal_t **ppfInputBuffer, const FEAPI_TimeStamp_t *ptFEAPI_TimeStamp,int iNumberOfFrames)
The process function does the actual result/feature calculation. It has to be called with subsequent blocks of audio input data. hInstanceHandle points to the current instance, ppfInputBuffer points to the current block of audio data stored as a two dimensional array in the format [channels][samples]. A time stamp in seconds marking the beginning of the data may be given, to make synchronisation more easy. The number of frames is given in iNumberOfFrames and has to be compliant with the minimum and maximum frame size boundaries given in FEAPI_PluginProperty_t.
The return value is FEAPI_kNoError in case of no error.
-
FEAPI_Error_t FEAPI_ProcessPluginDone_t (FEAPI_PluginInstance_t hInstanceHandle)
Call this function after the last call of FEAPI_ProcessPlugin_t, i.e. when no more new audio data is available. hInstanceHandle points to the current instance.
The return value is FEAPI_kNoError in case of no error.
The Feature Extraction plugin API allows the calculation of one or more features at a time. For each single result, several properties are defined in the FEAPI_SignalDescription_t structure, which contains name, unit, range, description, result quantization and output sample rate (if it is constant).
A single result can be a single value, or can be an array of values (like e.g. a magnitude spectrum). Furthermore, depending on the size of the process input buffer, there may be several consecutive results with different time stamps at one time.
-
int FEAPI_GetPluginNumOfResults_t (FEAPI_PluginInstance_t hInstanceHandle)
Call this function to retrieve the number of results/features that the plugin is able to extract. hInstanceHandle points to the current instance. Even if a result is multidimensional, its count will be one. If no result is computed (an unlikely case), the function returns 0.
-
FEAPI_Error_t FEAPI_GetPluginResultDescription_t (FEAPI_PluginInstance_t hInstanceHandle, int iResultIndex, FEAPI_SignalDescription_t *pstResultDescription)
Call this function to retrieve the description of a special result/feature. hInstanceHandle points to the current instance. iResultIndex is the parameter index ranging from 0 for the first parameter to (number of results - 1). The detailed result description is copied into a structure of type FEAPI_SignalDescription_t pointed to by the function parameter pstResultDescription. Note that the host has to allocate the memory for the result description structure.
The return value is FEAPI_kNoError in case of no error.
-
int FEAPI_GetPluginSizeOfResult_t (FEAPI_PluginInstance_t hInstanceHandle, int iResultIndex)
Call this function before each call of FEAPI_GetPluginResult_t to retrieve the size of the result with index iResultIndex in float values. Note that this function always returns the size of one result, i.e. if the result is a single value, it will always return 1. For an N-dimensional result, it will return N. This function has to be called in combination with ::FEAPI_GetPluginResults_t until it returns 0.
-
int FEAPI_GetPluginResult_t (FEAPI_PluginInstance_t hInstanceHandle, int iResultIndex, FEAPI_Signal_t *pfResult, FEAPI_TimeStamp_t *ptFEAPI_TimeStamp)
Call this function to retrieve the result with index iResultIndex after one process call. The result is copied into the buffer pointed to by pfResult. Note that due to the push interface, it may be possible that the plugin has several succeeding results for the same result index. Therefore, call this function with the same iResultIndex until FEAPI_GetPluginSizeOfResult_t returns 0. hInstanceHandle points to the current instance. Note that the host has to allocate the memory for the result buffer according to the previously requested size of the result.
-
int FEAPI_GetPluginResultLatency_t (FEAPI_PluginInstance_t hInstanceHandle, int iResultIndex)
To make synchronization as easy as possible, this function returns the latency of each result/feature expressed in frames of the input sample rate. Since the host buffer size may vary, this is only the plugin internal latency.
-
int FEAPI_ResetPlugin_t (FEAPI_PluginInstance_t hInstanceHandle)
Call this function to reset the plugin instance given in hInstanceHandle. Reset means that all internal buffers and states are flushed. This is needed in cases where an already used instance is re-used for a new audio data stream.
The return value is FEAPI_kNoError in case of no error.
In the current state, the API functions are not intended to be thread safe. Therefore, host developers should avoid to call more than one function at the same time. For an example of how the API functions can be called, please read the FEAPI Host Tutorial. Note however, that all plugins still have to be capable of running multiple instances at the same time!
The Feature Extraction plugin API is designed to be platform independent. The specific implementation and usage of plugin and host may have to be platform dependent to some extent. Note that in its current state, FEAPI was only implemented and tested on Win32 and MacOSX. The following recommendations refer to these platforms.
The following directories should be the default directories for FEAPI plugins:
-
Win32+Win64: $ProgramFiles/FEAPI
-
MacOsX: /Library/Audio/Plug-Ins/FEAPI
This is however only the default location, and host implementations should allow the user to specify one or more additional directories to be searched for plugins.
The host has to test possible plugins for FEAPI compliance by checking the exported functions of the library. Therefore, the used file extension is not of much importance. It is recommended however, that the following file extensions are used:
-
Win32+Win64: .dll
-
MacOsX: .feapi
The Feature Extraction plugin API is released under a BSD license. Therefore, static linking of the API without publishing other sources of the project is allowed and the code may be used in open source as well as in commercial products with the small restrictions mentioned in the license itself (see FEAPI.h).
Please keep the authors or contributors informed about any extensions or fixes of the source files.
The exact license is:
Copyright (c) 2004-2007, FEAPI development team All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. Neither the name of the FEAPI development team nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Variable names have a prefix characters indicating their types:
| unsigned: | u |
| pointer: | p |
| array: | a |
| class: | C |
| bool: | b |
| char: | c |
| short (int16): | s |
| int (int32): | i |
| __int64: | l |
| float (float32): | f |
| double (float64): | d |
For example, a pointer to a buffer of unsigned ints will be named puiBufferName.
web: http://feapi.sourceforge.net
feapi mailing list: feapi-discussion@lists.sourceforge.net
Generated on Fri Mar 23 10:28:52 2007 for FEAPI Documentation by
1.3.9.1