Goblin StoreTechnical documentation

Field notes from the cave

Installing Goblin Store

This page builds Goblin Store from source on Ubuntu and installs the server and pool-preparation helper in /usr/local/bin. Goblin Store currently has no Debian package and the top-level build does not install the server automatically.

Goblin Store manual: Getting started · Installation · Command-line reference · Ubuntu service

Requirements

Goblin Store targets Linux kernel 5.19 or newer and requires:

The server’s default async data path requires liburing. A build without liburing retains portable logic for tests and library consumers but cannot run the production async server. Ubuntu 24.04 LTS provides GCC 14 in the Universe component and CMake 3.28.3; later Ubuntu releases also satisfy the minimums.

Install Ubuntu packages

On Ubuntu 24.04 or newer, enable the Universe component if your image does not already provide it, then install:

sudo apt-get update
sudo apt-get install -y \
    git \
    g++-14 \
    cmake \
    ninja-build \
    pkg-config \
    libssl-dev \
    liburing-dev \
    libcurl4-openssl-dev

Add the native RDMA development packages only when that endpoint is needed:

sudo apt-get install -y libibverbs-dev librdmacm-dev

The configure summary reports whether io_uring, TLS, the mirror client, and RDMA were found. Older Ubuntu releases may need a newer compiler or CMake from an Ubuntu-supported toolchain repository. Check before building:

g++-14 --version
cmake --version
uname -r

Clone and build

main is the recommended build and the repository’s default branch. It advances through reviewed development branches merged into it:

git clone https://github.com/adamdeprince/goblin-store.git
cd goblin-store
git switch main
git pull --ff-only origin main

cmake -S . -B build -G Ninja \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_CXX_COMPILER=g++-14

cmake --build build --parallel
ctest --test-dir build --output-on-failure

Review the configure line. A production Linux build should report io_uring=1. Features explicitly requested at runtime fail with a clear configuration error when their build dependency is absent.

Useful CMake switches

Option Default Purpose
GOBLIN_WERROR ON Treat compiler warnings as errors.
GOBLIN_SANITIZE ON Enable AddressSanitizer and UndefinedBehaviorSanitizer in Debug builds.
GOBLIN_ENABLE_RDMA ON on Linux Build native RDMA when libibverbs and librdmacm are installed.
GOBLIN_ENABLE_EXASOCK OFF Build the explicitly selected ExaSock backend. ExaSock must already be installed and is never vendored.
GOBLIN_BUILD_ROCKSDB_BENCHMARK OFF Build the optional Goblin-versus-RocksDB benchmark; requires RocksDB development files.
BUILD_TESTING ON Build the dependency-free test executable.

Example development build:

cmake -S . -B build-debug -G Ninja \
    -DCMAKE_BUILD_TYPE=Debug \
    -DCMAKE_CXX_COMPILER=g++-14
cmake --build build-debug --parallel
ctest --test-dir build-debug --output-on-failure

Install the server binaries

The server and helper are standalone executables. Install them with the standard install utility:

sudo install -o root -g root -m 0755 \
    build/goblin-store \
    build/goblin-store-path-prep \
    /usr/local/bin/

Verify the installed parser:

/usr/local/bin/goblin-store --help

The optional goblin-bench, embedded C++ storage library, C++ memcache client, and Python package have separate consumers and are not required to run the server.

Prepare storage

Every Goblin-managed directory must be empty when it is prepared:

sudo install -d -m 0750 -o "$USER" -g "$USER" /var/lib/goblin-store

goblin-store-path-prep \
    /var/lib/goblin-store/ssd \
    /var/lib/goblin-store/heads

goblin-store-path-prep creates each directory, verifies that it is empty, and writes .goblin-store-marker. The server refuses to wipe or recover a directory without that marker. Prepare one directory for each physical SSD/HDD path supplied on the command line. A persistent --heads-path must be separate from every SSD and HDD pool.

For a production installation, stop here and follow Run as an Ubuntu service; that guide creates the service account and prepares paths with the final ownership.

Upgrade

Build and test the new revision before replacing a running binary:

git fetch --tags origin
git switch main
git pull --ff-only

cmake -S . -B build -G Ninja \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_CXX_COMPILER=g++-14
cmake --build build --parallel
ctest --test-dir build --output-on-failure

sudo systemctl stop goblin-store
sudo install -o root -g root -m 0755 \
    build/goblin-store \
    build/goblin-store-path-prep \
    /usr/local/bin/
sudo systemctl start goblin-store

The default cache is ephemeral and starts empty. A deployment using --heads-path recovers valid head/SSD/HDD combinations after an ordinary restart. Read the release notes and architecture changes before moving between versions that alter on-disk formats.

Next