Transpose Convolution Prototype and Function List

Description

This kernel implements a general 2D transposed convolution operation which works by swapping the forward and backward passes of a convolution. For more details on calculations, see chapter 4 of A guide to convolution arithmetic for deep learning.

Optionally, a saturating ReLU activation function can be applied to the result of the convolution during the function’s execution. For more info on supported ReLU types and calculations, see ReLU Prototype and Function List.

The dilation_height and dilation_width parameter of mli_conv2d_cfg configuration structure is not applicable in MLI transposed convolution and must be equal to 1.

For example, in a HWCN data layout, if the in feature map is \((Hi, Wi, Ci)\) and the weights is \((Hk, Wk, Ci, Co)\), the output feature map is \((Ho, Wo, Co)\) tensor where the spatial dimensions comply with the following system of equations:

(1)\[ \begin{align}\begin{aligned}\begin{cases}\\\hat{Ph} = ({Hk}-1)*2-padding\_top-padding\_bottom\\\hat{Pw} = ({Wk}-1)*2-padding\_left-padding\_right\\\hat{Wi} = ({Wi}-1)*stride\_width+1\\\hat{Hi} = ({Hi}-1)*stride\_height+1\\{Wo} = \hat{Wi}+\hat{Pw}-{Wk}+1\\{Ho} = \hat{Hi}+\hat{Ph}-{Hk}+1\\\end{cases}\end{aligned}\end{align} \]

Where:

\(\hat{Wi}\), \(\hat{Hi}\) - effective in feature map width and height after applying \(stride\_*\) to the original width (\(Wi\)) and height (\(Hi\)).

\(\hat{Pw}\), \(\hat{Ph}\) - transposed width and height paddings.

\(Wo\), \(Ho\) - out feature map width and height.

\(Wk\), \(Hk\) - weights * kernel width and height.*

This is a MAC-based kernel which implies accumulation. See Quantization: Influence of Accumulator Bit Depth for more information on related quantization aspects. The Number of accumulation series is up to (\(Wk*Hk*Ci\)).

Functions

Kernels which implement Transpose Convolutions have the following prototype:

mli_status mli_krn_transpose_conv2d_hwcn_<data_format>(
   const mli_tensor *in,
   const mli_tensor *weights,
   const mli_tensor *bias,
   const mli_conv2d_cfg *cfg,
   mli_tensor *out);

where data_format is one of the data formats listed in Table MLI Data Formats and the function parameters are shown in the following table:

Transpose Convolution Function Parameters

Parameter	Type	Description
in	mli_tensor *	[IN] Pointer to constant input tensor.
weights	mli_tensor *	[IN] Pointer to constant weights tensor.
bias	mli_tensor *	[IN] Pointer to constant bias tensor.
cfg	mli_conv2d_cfg *	[IN] Pointer to convolution parameters structure.
out	mli_tensor *	[IN | OUT] Pointer to output feature map tensor. Result is stored here

The following table lists all the available Transpose Convolution functions:

List of Available Transpose Convolution Functions

Function Name	Details
mli_krn_transpose_conv2d_hwcn_sa8_sa8_sa32	In/out layout: HWC Weights layout: HWCN In/out/weights data format: sa8 Bias data format: sa32
mli_krn_transpose_conv2d_hwcn_fx16	In/out layout: HWC Weights layout: HWCN All tensors data format: fx16
mli_krn_transpose_conv2d_hwcn_fx16_fx8_fx8	In/out layout: HWC Weights layout: HWCN In/out data format: fx16 Wights/Bias data format: fx8
mli_krn_transpose_conv2d_hwcn_sa8_sa8_sa32_k2x2_str2	In/out layout: HWC Weights layout: HWCN In/out/weights data format: sa8 Bias data format: sa32 Width of weights tensor: 2 Height of weights tensor: 2 Stride across Width dimension: 2 Stride across Hight dimension: 2
mli_krn_transpose_conv2d_hwcn_fx16_k2x2_str2	In/out layout: HWC Weights layout: HWCN All tensors data format: fx16 Width of weights tensor: 2 Height of weights tensor: 2 Stride across Width dimension: 2 Stride across Hight dimension: 2
mli_krn_transpose_conv2d_hwcn_fx16_fx8_fx8_k2x2_str2	In/out layout: HWC Weights layout: HWCN In/out data format: fx16 Wights/Bias data format: fx8 Width of weights tensor: 2 Height of weights tensor: 2 Stride across Width dimension: 2 Stride across Hight dimension: 2
mli_krn_transpose_conv2d_hwcn_sa8_sa8_sa32_k4x4_str2	In/out layout: HWC Weights layout: HWCN In/out/weights data format: sa8 Bias data format: sa32 Width of weights tensor: 4 Height of weights tensor: 4 Stride across Width dimension: 2 Stride across Hight dimension: 2
mli_krn_transpose_conv2d_hwcn_fx16_k4x4_str2	In/out layout: HWC Weights layout: HWCN All tensors data format: fx16 Width of weights tensor: 4 Height of weights tensor: 4 Stride across Width dimension: 2 Stride across Hight dimension: 2
mli_krn_transpose_conv2d_hwcn_fx16_fx8_fx8_k4x4_str2	In/out layout: HWC Weights layout: HWCN In/out data format: fx16 Wights/Bias data format: fx8 Width of weights tensor: 4 Height of weights tensor: 4 Stride across Width dimension: 2 Stride across Hight dimension: 2

Conditions

Ensure that you satisfy the following general conditions before calling the function:

• in, out, weights and bias tensors must be valid (see mli_tensor Structure Field Descriptions) and satisfy data requirements of the selected version of the kernel.

• Shapes of in, out, weights and bias tensors must be compatible, which implies the following requirements:

  • in and out are 3-dimensional tensors (rank==3). Dimensions meaning, and order (layout) is aligned with the specific version of kernel.

  • weights is a 4-dimensional tensor (rank==4). Dimensions meaning, and order (layout) is aligned with the specific kernel.

  • bias must be a one-dimensional tensor (rank==1). Its length must be equal to \(Co\) (output channels OR number of filters).

  • Channel \(Ci\) dimension of in and weights tensors must be equal.

  • Shapes of in, out and weights tensors together with cfg structure must satisfy the equations (1)

  • Width and height (\(Wk, Hk\)) of the weights tensor must not exceed appropriate effective dimensions of the in tensor (see (1)).

• in and out tensors must not point to overlapped memory regions.

• mem_stride of the innermost dimension must be equal to 1 for all the tensors.

• padding_top and padding_bottom parameters must be in the range of [0, \(Hk\)).

• padding_left and padding_right parameters must be in the range of [0, \(Wk\)).

• stride_width parameter must be in range of [1, \(Wk\)).

• stride_height parameter must be in range of [1, \(Hk\)).

• dilation_height and dilation_width must be equal to 1.

For fx16 and fx16_fx8_fx8 versions of kernel, in addition to the general conditions, ensure that you satisfy the following quantization conditions before calling the function:

• The number of frac_bits in the bias and out tensors must not exceed the sum of frac_bits in the in and weights tensors.

For sa8_sa8_sa32 versions of kernel, in addition to the general conditions, ensure that you satisfy the following quantization conditions before calling the function:

• in and out tensor must be quantized on the tensor level. This implies that each tensor contains a single scale factor and a single zero offset.

• Zero offset of in and out tensors must be within [-128, 127] range.

• weights and bias tensors must be symmetric. Both must be quantized on the same level. Allowed Options:

  • Per Tensor level. This implies that each tensor contains a single scale factor and a single zero offset equal to 0.

  • Per \(Co\) dimension level (number of filters). This implies that each tensor contains separate scale point for each sub-tensor. All tensors contain single zero offset equal to 0.

• Scale factors of bias tensor must be equal to the multiplication of input scale factor broadcasted on weights array of scale factors. See the example for the similar condition in the Convolution 2D Prototype and Function List.

Ensure that you satisfy the platform-specific conditions in addition to those listed above (see the Platform Specific Details chapter).

Result

These functions only modify the memory pointed by out.data.mem field. It is assumed that all the other fields of out tensor are properly populated to be used in calculations and are not modified by the kernel.

Depending on the debug level (see section Error Codes) this function performs a parameter check and returns the result as an mli_status code as described in section Kernel Specific Configuration Structures.