ReLU Prototype and Function List¶

Description¶

This kernel represents Rectified Linear Unit (ReLU). It performs various types of the rectifier activation on input. The following types of ReLU are supported by this type of kernel:

identity functon \(y_{i} = x_{i}\)

General ReLU: \(y_{i} = MAX(x_{i},\ 0)\)

ReLU1: \(y_{i} = MAX(MIN\left( x_{i},1 \right),\ - 1)\)

ReLU6: \(y_{i} = MAX(MIN\left( x_{i},6 \right),\ 0)\)

Where:

\(x_{i}\) - \(i_{\text{th}}\) value in input tensor

\(y_{i}\) - \(i_{\text{th}}\) value in output tensor

Functions¶

Kernels which implement ReLU functions have the following prototype:

mli_status mli_krn_relu_<data_format>(
   const mli_tensor *in,
   const mli_relu_cfg *cfg,
   mli_tensor *out);

where data_format is one of the data formats listed in Table MLI Data Formats and the function parameters are shown in the following table:

ReLU Function Parameters¶
Parameter	Type	Description
`in`	`mli_tensor *`	[IN] Pointer to constant input tensor.
`cfg`	`mli_relu_cfg *`	[IN] Pointer to relu parameters structure.
`out`	`mli_tensor *`	[IN \| OUT] Pointer to output tensor. Result is stored here

mli_relu_cfg is defined as:

typedef struct {
   mli_relu_type type;
 } mli_relu_cfg;

mli_relu_cfg Structure Field Description¶
Field Name	Type	Enumeration Value	Description
`type`	`mli_relu_type` (enumeration)	`MLI_RELU_NONE`	No ReLU. Identity function
		`MLI_RELU_GEN`	General Rectifier function with output range from 0 to value maximum inclusively
		`MLI_RELU_1`	ReLU1 Rectifier function with output range [-1, 1]
		`MLI_RELU_6`	ReLU6 Rectifier function with output range [0, 6]

List of Available ReLU Functions¶
Function Name	Details
`mli_krn_relu_sa8`	All tensors data format: sa8
`mli_krn_relu_fx16`	All tensors data format: fx16

Conditions¶

Ensure that you satisfy the following general conditions before calling the function:

in and out tensors must be valid (see mli_tensor Structure Field Descriptions) and satisfy data requirements of the selected version of the kernel.

in and out tensors must be of the same shape

mem_stride of the innermost dimension must be equal to 1 for all the tensors.

For sa8 versions of kernel, in addition to general conditions, ensure that you satisfy the following quantization conditions before calling the function:

in tensor must be quantized on the tensor level. This implies that the tensor contains a single scale factor and a single zero offset.

Zero offset of in tensor must be within [-128, 127] range.

Ensure that you satisfy the platform-specific conditions in addition to those listed above (see the Platform Specific Details chapter).

Result¶

These functions modify:

Memory pointed by out.data.mem field.

el_params field of out tensor which is copied from in tensor.

It is assumed that all the other fields and structures are properly populated to be used in calculations and are not modified by the kernel.

The kernel supports in-place computation. It means that out and in tensor structures can point to the same memory with the same memory strides but without shift. It can affect performance for some platforms.

Warning

Only an exact overlap of starting address and memory stride of the in and out tensors is acceptable. Partial overlaps result in undefined behavior.

Depending on the debug level (see section Error Codes), this function performs a parameter check and returns the result as an mli_status code as described in section Kernel Specific Configuration Structures.