Overview

Introduction

The Machine Learning Inference Library is the basis for machine learning inference for ARC VPX, ARC EMxD, and ARC HS4xD families of ARC DSP cores. Its purpose is to enable porting of machine learning models mostly based on NN to ARC processors.

The MLI library is a collection of ML algorithms (primitives) which roughly can be separated into the following groups:

MLI supported primitives are intended for

  • ease of use

  • inferring efficient solutions for small/mid-sized models using very limited resources

Purpose of This Document

The document describes

  • the usage of MLI primitives and

  • the technical details of MLI primitives such as

    • data formats

    • data layouts

    • quantization schemes

    • kernel naming convention

    • supported kernels

Memory Hierarchy

The functions in the MLI API operate on the lowest level in the memory hierarchy: the local memories (for example: DCCM, VCCM, and so on). All the memory movement needs to happen at a level above the MLI API. The data move functions can be used for the purpose. Inside the MLI kernels, there is no data movement. The strategy on data movement can be different for each platform or each application. For some platforms and some applications, all the data could fit in CCM, in which case no data movement is needed. Some platforms have a data cache; in those cases the data movement is done by the hardware in a transparent way. The caller/user of MLI APIs is responsible for ensuring that all data buffers passed to the MLI functions are in the CPU’s local memory prior to making the call.

The MLI library does not allocate any memory dynamically. The caller is responsible for providing the correct parameters and allocated memory. The MLI library might use internal statically allocated data.

Header Files

To ensure consistency for users, the following public header files are provided with MLI Implementation. These are intended to be included by the application code:

  • mli_types.h: This file contains definitions of the various data structures defined elsewhere in this document.

  • mli_config.h: This file describes configuration-related parameters as well as platform-specific data

  • mli_api.h: This file (or sub-include files) declares the MLI API functions

Terms and definitions

AGU

Address Generation Unit

API

Application Programming Interface

ARC EMxD

Family of 32-bit ARC Processor Cores. Single-core, 3-Step Pipeline, ARCv2DSP

ARC HS4xD

Family of 32-bit ARC Processor Cores. Multi-core, Dual-Issue, 10-Step Pipeline, ARCv2DSP

ARCv2DSP

Synopsys DesignWare® ARC® Processors Family of 32-bit CPUs

CCAC

MetaWare Compiler

CCM

Closely Coupled Memory

DCCM

Data Closely Coupled Memory

DMA

Direct Memory Access

DSP

Digital Signal Processor

FXAPI

Fixed-point API

LTO

Link-Time Optimization

MAC

Multiple Accumulate

MDB

MetaWare Debugger

MPY

Multiply Command

MWDT

MetaWare Development Toolset

nSIM

Instruction Set Simulator

OOB

Out-Of-the Box

PCM

Pulse Code Modulation

TCF

Tool Configuration File. Holds information about ARC processor build configuration and extensions.

VCCM

Vector Closely Coupled Memory

xCAM

Cycle Accurate Model