Spack: a Package Manager for Scientific Software
Massimiliano Culpo - Sylabs, Inc.Sylabs, Singularity and Spack
Spack in a nutshell
Spack is a package manager
- Lead developer: Todd Gamblin (LLNL)
- Tracks metadata for installed software
- Multiple configurations of the same software
- Support for supercomputers, linux and MacOS
- Install from source or binary packages
- Very active and engaging community
- Dual license Apache-2.0 or MIT
Who can profit from Spack?
- End users of HPC software
- HPC application teams
- Package developers
- User support teams at HPC centers
Spack is used worldwide!
Clone it and you're ready to go!
$ git clone https://github.com/spack/spack.git
$ . spack/share/spack/setup-env.sh
$ spack install hdf5
Packages are Python classes
class Kripke(Package):
"""Kripke is a simple, scalable, 3D Sn
deterministic particle transport mini app.
"""
homepage = "https://codesign.llnl.gov/kripke.php"
url = "https://codesign.llnl.gov/kripke-1.1.tar.gz"
version('1.1', '7fe6f2b26ed983a6ce5495ab701f85bf')
version('1.0', 'f4247dde07952a5ff866b24e45b5cdd1')
variant('mpi', default=True, description='Build with MPI.')
depends_on('mpi', when="+mpi")
Easy to work with packages
$ # Create a new package in the built-in repository
$ spack create <url>
$ # Modify an existing package
$ export EDITOR='emacs -nw'
$ spack edit <package-name>
$ # Scrape for versions of an existing package
$ spack versions <package-name>
Spack core concepts
Spec syntax + Directives + Concretizer
The spec syntax describes user's needs
$ # Install a specific version by appending @
$ spack install hdf5@1.10.1
$ # Specify a compiler (and optional version), with %
$ spack install hdf5@1.10.1 %gcc@4.7.3
$ # Add special boolean compile-time options with +
$ spack install hdf5@1.10.1 +szip
$ # Add custom compiler flags
$ spack install hdf5@1.10.1 cppflags="-O3 -floop-block"
$ # Cross-compile on a Cray or Blue Gene/Q
$ spack install hdf5@1.10.1 target=backend
Directives model allowed configurations
class Openblas(MakefilePackage):
"""OpenBLAS: An optimized BLAS library"""
homepage = 'http://www.openblas.net'
url = 'http://github.com/OpenBLAS/v0.2.19.tar.gz'
version('0.3.4', sha256='4b4b4453251')
version('0.3.3', sha256='49d88f4494a')
variant('shared', default=True,
description='Build shared libraries')
variant('ilp64', default=False,
description='64 bit integers')
conflicts('%intel@16', when='@0.2.15:0.2.19')
The concretizer fills in missing details
Each configuration is installed in its own prefix
Spack fights combinatorics with combinatorics!
Database of installed packages
Installed configurations are stored in a JSON file
{
"database": {
"installs": {
"ivqu252fvh7r5iar6zwx4fmeoxiykln7": {
"explicit": true,
"installation_time": 1548272929.178339,
"ref_count": 0,
"installed": true,
"path": "/home/mculpo/PycharmProjects/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-8.2.0/zlib-1.2.11-ivqu252fvh7r5iar6zwx4fmeoxiykln7",
"spec": {
"zlib": {
"version": "1.2.11",
"arch": {
"platform": "linux",
"platform_os": "ubuntu18.04",
"target": "x86_64"
},
"compiler": {
"name": "gcc",
"version": "8.2.0"
},
"namespace": "builtin",
"parameters": {
"optimize": true,
"pic": true,
"shared": true,
"cflags": [],
"cppflags": [],
"cxxflags": [],
"fflags": [],
"ldflags": [],
"ldlibs": []
}
}
}
}
},
"version": "0.9.3"
}
}
opt/spack/.spack-db/index.json
Provenance is preserved for each configuration
$ tree $(spack location -i hdf5)/.spack
<prefix>/.spack
├── archived-files
│ └── config.log
├── build.env
├── build.out
├── repos
│ └── builtin
│ ├── packages
│ │ ├── hdf5
│ │ │ └── package.py
│ │ └── zlib
│ │ └── package.py
│ └── repo.yaml
└── spec.yaml
6 directories, 7 files
Query what's installed from the command line
$ spack find zlib
==> 1 installed package
-- linux-ubuntu18.04-x86_64 / gcc@8.2.0 ----
zlib@1.2.11
$ spack find --start-date 'a month ago'
==> 3 installed packages
-- linux-ubuntu18.04-x86_64 / gcc@8.2.0 ----
hdf5@1.10.4 openblas@0.3.5 zlib@1.2.11
Uninstall anything in an easy and safe way
$ spack find zlib
==> 2 installed packages
-- linux-ubuntu18.04-x86_64 / gcc@8.2.0 ----
zlib@1.2.8 zlib@1.2.11
$ spack uninstall zlib@1.2.8
==> The following packages will be uninstalled:
-- linux-ubuntu18.04-x86_64 / gcc@8.2.0 ----
yxoie27 zlib@1.2.8%gcc+optimize+pic+shared
==> Do you want to proceed? [y/N] y
==> Successfully uninstalled zlib@1.2.8%gcc@8.2.0 [...] /yxoie27
Generate module files for HPC sites
modules:
lmod:
core_compilers:
- gcc@4.8.5
hierarchy:
- mpi
- lapack
hash_length: 0
all:
suffixes:
+mpi: mpi
+openmp: openmp
...
A thin wrapper script mediates compilation
Compiler options can be filtered out or added
New features since last year
Chain together multiple instances of Spack
Downstream instances can read upstream DBs
Spack Environments: portable and reproducible builds
Spack environments virtualize a Spack instance
$ spack env create my-project
==> Created environment 'my-project' in [...]/my-project
$ spack env activate my-project
$ spack env status
==> In environment my-project
$ spack find
==> In environment my-project
==> No root specs
==> 0 installed packages
virtualenvs but for anything!
All environments are based on a manifest file
$ spack config get
# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
# add package specs to the `specs` list
specs: [hdf5~mpi, python, mpich]
mirrors: {}
modules:
enable: []
repos: []
packages: {}
config: {}
Use spack.yaml to distribute an environment
- It's the configuration file we have seen before
- Does not have to live inside Spack tree
- Can be bundled with anything
- Can be manipulated directly if it is in the
pwd
Fully concretized specs are in spack.lock
{
"concrete_specs": {
"teneqii2xv5u6zl5r6qi3pwurc6pmypz": {
"xz": {
"version": "5.2.4",
"arch": {
"platform": "linux",
"platform_os": "ubuntu16.04",
"target": "x86_64"
},
...
spack.yaml or spack.lock you can recreate an environment
Environment might have associated views
|
|
Spack Stacks: cartesian products of specs
spack:
definitions:
- compilers: ['%gcc@7.1.0', '%gcc@4.9.3']
- when: target == 'x86_64'
compilers: ['%intel']
specs:
- matrix:
- [zlib, libelf, libdwarf]
- [$compilers]
exclude:
- libdwarf%gcc@4.9.3
- cmake
Spack can optimize for specific microarchitectures
Spack 0.12.X: support was coarse grained
$ # Based on platform.machine() which is basically uname -m
$ spack arch
linux-ubuntu18.04-x86_64
$ spack -d install zlib
[...]
==> [2019-09-26-12:52:54.178762] Successfully installed zlib
Fetch: 0.01s. Build: 3.87s. Total: 3.88s.
[+] /home/culpo/PycharmProjects/spack/opt/spack/linux-ubuntu18.04-x86_64/gcc-9.0.1/zlib-1.2.11-tn4qvs7g6r45dipq6kvg4ggs6rts3m7s
$ cat spack-cc-zlib-tn4qvs7.out.log
[cc] /usr/bin/gcc-9 -c ztest28029.c
[cc] /usr/bin/gcc-9 -c -fPIC -O2 ztest28029.c
[cc] /usr/bin/gcc-9 -c -fPIC -O2 ztest28029.c
[...]
Spack >= 0.13.X: finer notion of architectures
$ spack arch
linux-ubuntu18.04-broadwell
$ spack arch --known-targets
Generic architectures (families)
aarch64 ppc64 ppc64le x86 x86_64
IBM - ppc64
power7 power8 power9
GenuineIntel - x86_64
nocona nehalem sandybridge haswell skylake skylake_avx512 cascadelake
core2 westmere ivybridge broadwell mic_knl cannonlake icelake
[...]
Microarchitecture specific optimizations
$ spack -d install zlib
[...]
==> [2019-09-26-13:57:08.721455] Successfully installed zlib
Fetch: 0.01s. Build: 3.22s. Total: 3.23s.
[+] /home/culpo/PycharmProjects/spack/opt/spack/linux-ubuntu18.04-broadwell/gcc-9.0.1/zlib-1.2.11-os3djb57dqdzhaqsl2n7iefz3gwombyi
$ cat spack-cc-zlib-os3djb5.out.log
[cc] /usr/bin/gcc-9 -march=broadwell -mtune=broadwell -c ztest30073.c
[cc] /usr/bin/gcc-9 -march=broadwell -mtune=broadwell -c -fPIC -O2 ztest30073.c
[cc] /usr/bin/gcc-9 -march=broadwell -mtune=broadwell -c -fPIC -O2 ztest30073.c
[...]
The concretizer knows about compiler support
$ spack arch
linux-ubuntu18.04-broadwell
$ spack spec zlib %gcc@4.8
[...]
Concretized
--------------------------------
==> Warning: gcc@4.8 cannot build optimized binaries for "broadwell". Using best target possible: "haswell"
zlib@1.2.11%gcc@4.8+optimize+pic+shared arch=linux-ubuntu18.04-haswell
$ spack spec zlib %gcc@4.8 target=broadwell
[...]
==> Error: cannot produce optimized binary for micro-architecture 'broadwell' with gcc@4.8 [supported compiler versions are 4.9:]
Simplified recipes with more powerful tests
# The colon denotes an open range. The dependency
# is attached to any x86_64 compatible target
depends_on('libfoo', when='target=x86_64:')
# In recipes we can use Microarchitecture objects
# directly and test them for compatibility or features
if spec.target >= 'broadwell':
config_args.append('--from-broadwell-on')
if 'avx512' in spec.target:
config_args.append('--enable-avx512')
Spack integrates with Gitlab CI pipelines
CI infrastructure for source and binary packages
Spack can generate recipes to build container images
Extension to Spack Environments
spack:
specs:
- hdf5+mpi
- mpich
container:
format: docker
strip: true
base:
image: "centos:7"
spack: develop
packages:
- libgomp
labels:
app: "gromacs"
mpi: "mpich"
A command prints recipes to stdout
$ ls
spack.yaml
$ spack containerize
# Build stage with Spack pre-installed and ready to be used
FROM spack/centos7:latest as builder
# What we want to install and how we want to install it
# is specified in a manifest file (spack.yaml)
RUN mkdir /opt/spack-environment \
&& (echo "spack:" \
&& echo " specs:" \
&& echo " - hdf5~mpi" \
&& echo " concretization: together" \
&& echo " config:" \
&& echo " install_tree: /opt/software" \
&& echo " view: /opt/view") > /opt/spack-environment/spack.yaml
# Install the software, remove unecessary deps
RUN cd /opt/spack-environment && spack install && spack gc -y
...
Support for Singularity def files
Bootstrap: docker
From: spack/ubuntu-bionic:latest
Stage: build
%post
# Create the manifest file for the installation in /opt/spack-environment
mkdir /opt/spack-environment && cd /opt/spack-environment
cat << EOF > spack.yaml
spack:
specs:
- hdf5~mpi
packages:
all:
target:
- broadwell
concretization: together
config:
install_tree: /opt/software
view: /opt/view
EOF
# Install all the required software
. /opt/spack/share/spack/setup-env.sh
spack install
spack gc -y
spack env activate --sh -d . >> /opt/spack-environment/environment_modifications.sh
...
Spack supports concurrent or parallel builds
Synchronization via filesystem locks
Coming soon in Spack
Concretization prefers already available binaries
