NetBSD pkgsrc framework is a collection of make files, scripts and patches, used for downloading, configuring and building mostly open source software packages. See https://pkgsrc.org for more details.
Every quarter a new pkgsrc release is made. To keep up to date with the recent releases, download them from https://cdn.netbsd.org/pub/pkgsrc/.
Pkgsrc can also be used on Solaris or Illumos, however there are a few extra steps required to bootstrap it before it can be used. There are many different ways in how pkgsrc can be configured, depending on personal preferences. The notes below are mainly for myself and describe how I tend to use it on Solaris.
1. Decide which toolchain to use
On Solaris there may be multiple toolchains installed: Sun/Oracle studio compilers, GNU compilers, or LLVM compilers. Pkgsrc can support different toolchains, however a lot of open source software builds with the least problems using GNU compilers. For that reason I would recommend using GCC C and C++ compilers.
GCC compilers can be configured to use either Sun original binutils or GNU binutils. It is recommended to use GCC compilers configured with Sun binutils, as this seems to result in the least problems when building software from source. I have another article which describes how to build GCC compilers on Solaris from scratch. At the time of writing this article, gcc-14.X causes build failure with some packages as it has more pedantic error checking, hence using gcc-13.X provides a less painful experience.
Pkgsrc includes several GCC compilers, which can be built from source using pkgsrc framework and installed as pkgsrc packages, however in order to build a compiler from pkgsrc, you need to bootsrap pkgsrc with a native compiler. Bootstrapping pkgsrc builds various support binaries and if the native compiler is GCC, it links the binaries to GCC native compiler's shared libraries. Later when you build a new compiler from pkgsrc and tell it to use this compiler, then it will link later binaries to pkgsrc GCC compiler's share libraries. In the end, you end up with some binaries linked to one compiler and another set to another compiler. There are ways to hack around it, but on Solaris I always build GCC compilers from source and use them as the native compilers. This way I can have multiple compilers and have the freedom to install whichever GCC version that works best for me, instead of relying on pkgsrc for this task.
I normally install GCC compilers under /opt and then setup /opt/gcc symlink to point to a working compiler version.
2. Decide on pkgsrc install location and directory name
I normally install pkgsrc packages under /opt/pkg-<release_ver> and then setup /opt/pkg symlink to point the current stable packages directory. This way, I can use the same Solaris zone for building multiple pkgsrc releases concurrently. For example, I can use current /opt/pkg-2024Q2 packages while building and installing /opt/pkg-2024Q3 packages. If the new packages build and run correctly, I can later simply update the symlink and may be a few rc.d start scripts, as they hardcode the absolute paths to binaries.
3. Download pkgsrc stable release
Here "stable" is a relative term and probably applies mostly to NetBSD. Some packages may or may not build and work on Solaris/Illumos.
cd /opt &&
wget https://cdn.netbsd.org/pub/pkgsrc/pkgsrc-2024Q3/pkgsrc.tar.gz &&
gtar -xf pkgsrc.tar.gz &&
mv pkgsrc pkgsrc-2024Q3 &&
chown -RP root:root pkgsrc-2024Q3
4. Define environment variables for bootstrapping pkgsrc
Adjust these based on your own preferences.
MAKE_JOBS=128 &&
GCC_BASE="/opt/gcc" &&
PKGSRC_VER="2024Q3" &&
PKGSRC_BASE="/opt/pkgsrc-${PKGSRC_VER:?}" &&
PKG_BASE="/opt/pkg-${PKGSRC_VER:?}" &&
LD_OPTIONS="\
-L${PKG_BASE:?}/lib-gcc -R${PKG_BASE:?}/lib-gcc \
-L${PKG_BASE:?}/lib-gcc/sparcv9 -R${PKG_BASE:?}/lib-gcc/sparcv9" &&
PATH="${GCC_BASE:?}/bin:${PKG_BASE:?}/bin:${PKG_BASE:?}/sbin:/bin:/usr/bin:/sbin:/usr/sbin" &&
export LD_OPTIONS PATH
GCC_BASE is the base directory where native GCC compilers are installed. This consists of various sub-directories like bin/, lib/, etc.
PKGSRC_BASE is the base directory where the pkgsrc tree is located.
PKG_BASE is the base directory where binary pkgsrc packages get installed.
LD_OPTIONS is set for Solaris linker to hardcode the path for GCC shared libraries. See pkg/57685 why the usual LDFLAGS variable does not work here.
PATH should contain directories with: GCC compiler binaries, pkgsrc installed binaries and all other system binaries.
4. Copy GCC libraries to pkgsrc install directory
When building packages from source, some of them get linked to GCC shared libraries. We need to make sure that:
- We tell pkgsrc where to find GCC libraries. For that we export Solaris linker LD_OPTIONS environment variable.
- Packages continue to work correctly if the GCC compiler used to build them is later removed from the system. For that we physically copy required libraries to pkgsrc install directory.
test -d "${PKG_BASE:?}/lib-gcc/sparcv9" || \
mkdir -p "${PKG_BASE:?}/lib-gcc/sparcv9" &&
gtar -C "${GCC_BASE:?}/lib" -cf - $(ls "${GCC_BASE:?}/lib/"lib*.so* | \
while read i; do basename "$i"; done) | \
gtar -C "${PKG_BASE:?}/lib-gcc" -xpf - &&
gtar -C "${GCC_BASE:?}/lib/sparcv9" -cf - $(ls "${GCC_BASE:?}/lib/sparcv9/"lib*.so* | \
while read i; do basename "$i"; done) | \
gtar -C "${PKG_BASE:?}/lib-gcc/sparcv9" -xpf -
5. Bootstrap pkgsrc
Here we build pkgsrc support binaries, using /opt/pkg-<release_ver>.objects as the base directory for object files and /opt/pkg-<release_ver>.distfiles as the base directory for downloaded source archives.
rm -rf "/opt/pkg-${PKGSRC_VER:?}.objects/bootstrap" &&
env \
MAKECONF="" \
PKGMAKECONF="" \
DISTDIR="/opt/pkg-${PKGSRC_VER:?}.distfiles" \
PKGSRC_COMPILER=gcc \
USE_NATIVE_GCC=yes \
CC="${GCC_BASE:?}/bin/gcc" \
CXX="${GCC_BASE:?}/bin/g++" \
CFLAGS="-O2 -mcpu=v9" \
CXXFLAGS="-O2 -mcpu=v9" \
"${PKGSRC_BASE:?}/bootstrap/bootstrap" \
--prefer-pkgsrc yes \
--make-jobs "${MAKE_JOBS:?}" \
--workdir "/opt/pkg-${PKGSRC_VER:?}.objects/bootstrap" \
--prefix "${PKG_BASE:?}" \
--pkgdbdir "${PKG_BASE:?}/db/pkg"
6. Generate pkgsrc mk.conf
The mk.conf file is used by pkgsrc when building software packages from source. It contains various preference settings that specifies file locations, build options and compiler flags.
mv "${PKG_BASE:?}/etc/mk.conf" "${PKG_BASE:?}/etc/mk.conf.orig" &&
cat > "${PKG_BASE:?}/etc/mk.conf" << EOF
.ifdef BSD_PKG_MK # begin pkgsrc settings
# Build packages concurrently. On Solaris this requires installing pkgtools/shlock package
#PKGSRC_LOCKTYPE= sleep
#PKGSRC_SLEEPSECS= 60
OBJHOSTNAME= yes
ABI= 64
UNPRIVILEGED= no
X11_TYPE= modular
MAKE_JOBS= 16
# WARNING: Changing PREFER_* after bootstrap will require rebuilding all
# packages with a dependency that switched between native/pkgsrc.
PREFER_PKGSRC= yes
SKIP_LICENSE_CHECK= yes
DEPENDS_TARGET= package-install
LOCALBASE= /opt/pkg-${PKGSRC_VER:?}
PKG_DBDIR= \${LOCALBASE}/db/pkg
SYSCONFBASE= \${LOCALBASE}/etc
VARBASE= \${LOCALBASE}/var
PKG_TOOLS_BIN= \${LOCALBASE}/sbin
PKGINFODIR= info
PKGMANDIR= man
# Use Solaris /usr/bin/bash as the shell
TOOLS_PLATFORM.sh?= /usr/bin/bash
# Use pkgsrc install
TOOLS_PLATFORM.install?= \${LOCALBASE}/bin/bsdinstall
# Use pkgsrc sed. May need to be uncommented on older versions of Solaris.
# Check pkgsrc etc/mk.conf.orig to see if this line was enabled.
#TOOLS_PLATFORM.sed?= \${LOCALBASE}/bin/nbsed
# Enable when using native GCC
PKGSRC_COMPILER= gcc
USE_NATIVE_GCC= yes
CC= /opt/gcc/bin/gcc
CXX= /opt/gcc/bin/g++
# Enable when using pkgsrc GCC
#USE_PKGSRC_GCC= yes
#USE_PKGSRC_GCC_RUNTIME= yes
#GCC_REQD= 13
CFLAGS+= -O2 -mcpu=v9
CXXFLAGS+= -O2 -mcpu=v9
DBG= # prevent DBG from adding default optimizer flags
PACKAGES= /opt/pkg-${PKGSRC_VER:?}.packages
WRKOBJDIR= /opt/pkg-${PKGSRC_VER:?}.objects
DISTDIR= /opt/pkg-${PKGSRC_VER:?}.distfiles
MASTER_SORT= .uk .be .fr .de .dk .ch .it .fi .pl .no .nl .se
.endif # end pkgsrc settings
EOF
7. Build some packages
🖙 Before building more packages, it may be prudent to save current pkgsrc bootstrap state into tar archive, so that later it could be extracted on a different machine without bootstrapping it from scratch
gtar -zcf "/opt/pkg-${PKGSRC_VER:?}_bootstrap.tgz" "${PKG_BASE:?}"
It is best to create a script for this
PKGSRC_VER="2024Q3" &&
PKGSRC_BASE="/opt/pkgsrc-${PKGSRC_VER:?}" &&
PKG_BASE="/opt/pkg-${PKGSRC_VER:?}" &&
PKG_DBDIR="${PKG_BASE:?}/db/pkg" &&
LD_OPTIONS="\
-L${PKG_BASE:?}/lib-gcc -R${PKG_BASE:?}/lib-gcc \
-L${PKG_BASE:?}/lib-gcc/sparcv9 -R${PKG_BASE:?}/lib-gcc/sparcv9" &&
PATH="${GCC_BASE:?}/bin:${PKG_BASE:?}/bin:${PKG_BASE:?}/sbin:${PKG_BASE:?}/libexec:/bin:/usr/bin:/sbin:/usr/sbin" &&
export PKG_DBDIR LD_OPTIONS PATH
PKG_LIST='
meta-pkgs/modular-xorg-protos
meta-pkgs/modular-xorg-libs
meta-pkgs/modular-xorg-fonts
meta-pkgs/modular-xorg-utils
meta-pkgs/modular-xorg-apps
'
for i in ${PKG_LIST:?}
do
cd "${PKGSRC_BASE:?}/${i}" && bmake package-install || break
done
# For pkgsrc support commands to work correctly, PKG_DBDIR must be set correctly
pkg_info -a
