.. highlight:: shell Creating toolchain release ========================== Introduction ------------ ``release.mk`` is a makefile to create prebuilt packages of GNU Toolchain for ARC. It relies on other scripts in toolchain repository to build components. To build a release GNU Toolchain for ARC processors, several prerequisites should be installed and/or built before running a ``release.mk`` which does most of the work. List of prerequisites to build toolchain is list in toolchain ``README.md`` file. Note that there are extra dependencies to build toolchain for Windows hosts, in addition to those that are required to build toolchain for Linux hosts. To create Windows installer several MinGW and MSYS components are required (path set by :envvar:`THIRD_PARTY_SOFTWARE_LOCATION`). For a list of MinGW and MSYS packages, please refer to `windows-installer/README.md` section "Prerequisites". There are several variables that can be set to disable particular components, like Windows installer or OpenOCD, however those are not specifically tested, so may not really work. By default ``release.mk`` will build all of the possible components. It is also possible to invoke particular Make targets directly to get only a limited set of distributables, however it is not possible to make further targets like :option:`deploy` or :option:`upload` to use only this limited set of files (there is always an option to modify ``release.mk`` to get desired results). Building prerequisites ---------------------- Eclipse plugin for ARC ^^^^^^^^^^^^^^^^^^^^^^ .. warning:: This section doesn't cover a ``build/Makefile`` file in ``arc_gnu_eclipse`` repository which automates build process with ant. Build Eclipse plugin for ARC following those guidelines: https://github.com/foss-for-synopsys-dwc-arc-processors/arc_gnu_eclipse/wiki/Creating-Eclipse-plugins-release-zip-file Create and push respective git tag:: $ pushd arc_gnu_eclipse $ git tag arc-2016.03 $ git push -u origin arc-2016.03 $ popd Environment variables --------------------- Those are make variables which can be set either as a parameters to make, like ``make PARAM=VALUE`` or they can be specified in the ``release.config`` file that will be sourced by ``release.mk``. .. envvar:: CONFIG_STATIC_TOOLCHAIN Whether to build toolchain linked dynamically or statically. Note this affects the toolchain executable files, not the target libraries. Possible values ``y`` and ``n`` Default value ``n`` .. envvar:: DEPLOY_BUILD_DESTINATION Where to copy unpacked engineering build. Location is in format ``[hostname:]/path``. A directory named ``${RELEASE_TAG##-arc}`` will be created in the target path and will contain unpacked directories. Directories names are different from those that are in the tarballs - namely useless cruft is avoided and verion is not mentioned as well, so that it is easier to use those directories via symbolic links. For example, for tarball arc_gnu_2016.09-eng006_prebuilt_elf32_le_linux_install.tar.gz build directory will be elf32_le_linux. .. envvar:: DEPLOY_DESTINATION Where to copy release distributables. Location is in format ``[hostname:]/path``. A directory named ``${RELEASE_TAG##-arc}`` will be created in the target path and will contain all deploy artifacts. So for ``RELEASE_TAG = arc-2016.03-alpha1`` directory will be ``2016.03-alpha1``, while for ``RELEASE_TAG = arc-2016.03`` it will be ``2016.03``. .. envvar:: ENABLE_BIG_ENDIAN Whether to build and upload big endian toolchain builds. Big endian toolchain is required for IDE targets. Possible values ``y`` and ``n`` Default value ``y`` .. envvar:: ENABLE_DOCS_PACKAGE Whether to build separate packages with just documentation PDF files. Possible values ``y`` and ``n`` Default value ``n`` .. envvar:: ENABLE_IDE Whether to build and upload IDE distributable package. Note that build script for Windows installer always assumes presence of IDE, therefore it is not possible to build it when this option is ``n``. Possible values ``y`` and ``n`` Default value ``y`` .. envvar:: ENABLE_IDE_MACOS Whether to build and upload IDE distributable package on macOS. Possible values ``y`` and ``n`` Default value ``n`` .. envvar:: ENABLE_IDE_PLUGINS_BUILD Whether to build IDE plugins as part of ARC GNU Toolchain or download prebuilt ZIP. Possible values ``y`` and ``n`` Default value ``y`` .. envvar:: ENABLE_LINUX_IMAGES Whether to build and deploy Linux images built with this toolchain. This targets uses Buildroot to build rootfs and uImage for AXS103. Possible values ``y`` and ``n`` Default value ``y`` .. envvar:: ENABLE_LINUX_TOOLS Whether to build and deploy GNU Toolchain for Linux targets. Possible values ``y`` and ``n`` Default value ``y`` .. envvar:: ENABLE_GLIBC_TOOLS Whether to build and deploy GNU Toolchain for Linux/glibc targets. Possible values ``y`` and ``n`` Default value ``y`` .. envvar:: ENABLE_NATIVE_TOOLS Whether to build and upload native toolchain. Currently toolchain is built only for ARC HS Linux. Possible values ``y`` and ``n`` Default value ``y`` .. envvar:: ENABLE_OPENOCD Whether to build and upload OpenOCD distributable package for Linux. IDE targets will not work if OpenOCD is disabled. Therefore if this is ``n``, then :envvar:``ENABLE_IDE`` and :envvar:`ENABLE_WINDOWS_INSTALLER`` also must be ``n``. Possible values: ``y`` and ``n`` Default value: ``y`` .. envvar:: ENABLE_OPENOCD_WIN Whether to build and upload OpenOCD for Windows. This target currently depends on :envvar:`ENABLE_OPENOCD`, which causes source code to be cloned for OpenOCD. OpenOCD for Windows build will download and build libusb library and is a prerequisite for IDE for Windows build. Possible values ``y`` and ``n`` Default value ``y`` .. envvar:: ENABLE_PDF_DOCS Whether to build Toolchain PDF documentation. This affects only the "toolchain" repository - PDF documents from gcc, binutils, etc are always created, regardless of this option. Possible values ``y`` and ``n`` Default value ``y`` .. envvar:: ENABLE_SOURCE_TARBALL Whether to create a source tarball. Usually that should be true, so that release would include source tarball, however if release makefile is run multiple times on various machines to create packages for various target systems, then it makes sense to have this true only on one system. Possible values ``y`` and ``n`` Default value ``y`` .. envvar:: ENABLE_WINDOWS_INSTALLER Whether to build and upload Windows installer for toolchain and IDE. While building of installer can be also skipped simply by not invoking respective make targets, installer files still will be in the list of files that should be deployed and uploaded to GitHub, therefore this variable should be set to ``n`` for installer to be completely skipped. This variable also disables build of the toolchain for Windows as well. Possible values ``y`` and ``n`` Default value ``y`` .. envvar:: GIT_REFERENCE_ROOT Root location of existing source tree with all toolchain components Git repositories. Those repositorie swill be used as a reference when cloning source tree - this reduces time to clone and disk space consumed. Note that all of the components must exist in reference root, otherwise clone will fail. .. envvar:: IDE_PLUGIN_LOCATION Location of ARC plugin for Eclipse. This must be a directory and plugin file must have a name ``arc_gnu_${RELEASE_TAG##arc-}_ide_plugin.zip``. File will be copied with rsync therefore location may be prefixed with hostname separated by semicolon, as in ``host:/path``. This variable is used and must be set only if :envvar:``ENABLE_IDE_PLUGINS_BUILD`` is set to ``n``. .. envvar:: LIBUSB_VERSION Version of Libusb used for OpenOCD build for Windows. Default value 1.0.20 .. envvar:: RELEASE_NAME Name of the release, for example "GNU Toolchain for ARC Processors, 2016.03". .. envvar:: RELEASE_TAG Git tag for this release. Tag is used literally and can be for example, arc-2016.03-alpha1. .. envvar:: THIRD_PARTY_SOFTWARE_LOCATION Location of 3rd party software, namely Java Runtime Environment (JRE) and Eclipse tarballs. .. envvar:: WINDOWS_TRIPLET Triplet of MinGW toolchain to do a cross-build of toolchain for Windows. Default value i686-w64-mingw32 .. envvar:: WINDOWS_WORKSPACE Path to a directory that is present on build host and is also somehow available on a Windows host where Windows installer will be built. Basic scenario is when this location is on the Linux hosts, shared via Samba/CIFS and mounted on Windows host. Note that on Windows path to this directory, should be as short as possible , because Eclipse contains very long file names, while old NSIS uses ancient Windows APIs, which are pretty limited in the maximum file length. As a result build might fail due to too long path, if :envvar:`WINDOWS_LOCATION` is too long on Windows host. Make targets ------------ .. option:: build Build all distributable components that can be built on RHEL hosts. The only components that are not built by this target are: * OpenOCD for Windows - (has to be built on Ubuntu * ARC plugins for Eclipse - built by external job * Windows installer - created on Windows hosts. This tasks would depend on toolchain created by :option:`build` target. This target is affected by :envvar:`RELEASE_TAG`. .. option:: copy-windows-installer Copy Windows installer, created by ``windows-installer/build-installer.sh`` from :envvar:`WINDOWS_WORKSPACE` to ``release_output`` directory. .. option:: create-tag Create Git tags for released components. Required environment variables: :envvar:`RELEASE_TAG`, :envvar:`RELEASE_NAME`. OpenOCD must have a branch named ``arc-0.9-dev-${RELEASE_BRANCH}``, where ``RELEASE_BRANCH`` is a bare release, evaluated from the tag, so for :envvar:`RELEASE_TAG` of ``arc-2016.09-eng003``, ``RELEASE_BRANCH`` would be ``2016.09``. .. option:: deploy Deploy build artifacts to remote locations. It deploys same files as those that are released, and a few extra ones (like Windows toolchain tarballs). This target just copies deploy artifacts to location specified by :envvar:`DEPLOY_DESTINATION`. This target depends on :envvar:`DEPLOY_DESTINATION` and on :envvar:`WINDOWS_WORKSPACE`. .. option:: distclean Remove all cloned sources as well as build artifacts. .. option:: prerequisites Clone sources of toolchain components from GitHub. Copy external components from specified locations. Is affected by following environment variables: :envvar:`RELEASE_TAG`, :envvar:`GIT_REFERENCE_ROOT` (optional), :envvar:`THIRD_PARTY_SOFTWARE_LOCATION`. .. option:: push-tag Push Git tags to GitHub. .. option:: upload Upload release distributables to GitHub Releases. A new GitHub "Release" is created and bound to the Git tag specified in :envvar:`RELEASE_TAG`. This target also depends on :envvar:`RELEASE_NAME` to specify name of release on GitHub. .. option:: windows-workspace Create a workspace to run ``windows-installer/build-installer.sh`` script. Location of workspace is specified with :envvar:`WINDOWS_WORKSPACE`. ``build-installer.sh`` script will create an installer in the workspace directory. To copy installer from workspace to ``release_output`` use :option:`copy-windows-installer`. Invocation ---------- Release process consists of several sequential steps that should be done in the specified order. Some custom modifications can be done in between those steps. First, create directory-workspace:: $ mkdir arc-2016.03 $ cd arc-2016.03 Clone the ``toolchain`` repository:: $ git clone -b arc-dev \ https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain.git That command uses an HTTPS protocol to do Git clone - other protocols may be used as well. This documentation assumes the default case where ``arc-dev`` branch is the base for the release. .. note:: Currently ``tag-release.sh`` script used in the release process has a check that ensures that current branch is a development branch by checking that branch name ends in ``-dev``. First setup required make variables in the ``release.config`` file that will be sourced by ``release.mk`` (``...`` must be replaced with an actual paths):: $ cat release.config RELEASE_TAG=arc-2016.03 THIRD_PARTY_SOFTWARE_LOCATION=... GIT_REFERENCE_ROOT=... WINDOWS_WORKSPACE=... Fetch prerequisites (git repositories and external packages):: $ make -f release.mk prerequisites Create git tags:: $ make -f release.mk create-tag Build toolchain:: $ make -f release.mk build Prepare workspace for Windows installer build script. Note that target location, as specified by :envvar:`WINDOWS_WORKSPACE` should be shared with Windows host on which installer will be built. :: $ make -f release.mk windows-workspace On Windows host, build installer using ``windows-installer/build-installer.sh`` script. Note that this script requires a basic cygwin environment. :: $ RELEASE_BRANCH=2016.03 toolchain/windows-installer/build-installer.sh Copy Windows installer from :envvar:`WINDOWS_WORKSPACE` into ``release_output``:: $ make -f release.mk copy-windows-installer Deploy toolchain to required locations. This target may be called multiple times with different :envvar:`DEPLOY_DESTINATION` values:: $ make -f release.mk deploy DEPLOY_DESTINATION= $ make -f release.mk deploy DEPLOY_DESTINATION= Similarly, unpacked builds can be deployed to multiple locations:: $ make -f release.mk deploy-build DEPLOY_BUILD_DESTINATION= $ make -f release.mk deploy-build DEPLOY_BUILD_DESTINATION= Push tags to remote repositories:: $ make -f release.mk push-tag Finally, upload assets to GitHub Releases:: $ make -f release.mk upload .. vim: set tw=80 expandtab sts=3 sw=3 ts=3: