magnus/zsh-plugin-manager-benchmarks
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Reproducible Docker benchmarks for zsh plugin managers — companion to the oh-my-zsh alternatives article on magnus919.com
3 commits 1 branch 0 tags 36 KiB
  • Shell 93.7%
  • Dockerfile 6.3%
Find a file
Exact
Magnus Hedemark 8075db3dd2 Rewrite benchmark harness in native zsh 
- bench.sh → bench.zsh with #!/usr/bin/env zsh
- Uses zmodload zsh/datetime EPOCHREALTIME instead of bash time + bc
- No more bash wrappers, no bc dependency, no time output parsing
- Dockerfile stripped: removed hyperfine, fzf, wget (unused deps)
- Removed __zsh_timed helper captures elapsed time in zsh-native math
- All globs use (N) null-glob qualifier for err_exit safety
- README updated to note the script is written in zsh
2026-05-18 04:12:36 -04:00
results Initial benchmarks: Dockerfile, bench.sh, results, and references 2026-05-18 02:12:05 -04:00
bench.zsh Rewrite benchmark harness in native zsh 2026-05-18 04:12:36 -04:00
Dockerfile Rewrite benchmark harness in native zsh 2026-05-18 04:12:36 -04:00
LICENSE Initial commit 2026-05-18 02:11:13 -04:00
README.md Rewrite benchmark harness in native zsh 2026-05-18 04:12:36 -04:00
references.md Initial benchmarks: Dockerfile, bench.sh, results, and references 2026-05-18 02:12:05 -04:00

README.md

Zsh Plugin Manager Benchmarks

Reproducible Docker benchmarks for zsh plugin manager startup performance.

This repo accompanies the article Oh My Zsh Alternatives on magnus919.com. The benchmarks measure shell startup time (time zsh -i -c exit) for several plugin managers with a standardized set of plugins, running inside a Docker container on Ubuntu 22.04.

What's Measured

  • Cold start: First shell load after clearing .zcompdump and cache files. Represents the first shell you open after a reboot.
  • Warm start: Subsequent loads, averaged over 5 runs. Represents what you see in day-to-day use after the first terminal is opened.

Important caveat: time zsh -i -c exit measures total process runtime, not user-visible latency. For a more accurate picture of perceived performance (first prompt lag, first command lag), see romkatv/zsh-bench.

Plugin Managers Tested

Manager Version Install Method
Oh My Zsh latest from HEAD Git clone
Antidote latest from HEAD Git clone
Zinit latest from HEAD Git clone
Zim latest from HEAD Git clone

Sheldon was omitted from the Docker benchmarks because prebuilt binaries aren't available for arm64. If you're on x86_64, uncomment the Sheldon section in bench.sh to include it.

Standard Plugin Set

All managers (except the "minimal OMZ" case) load these plugins:

  • colored-man-pages, docker-compose, gh, git, rsync, ssh, brew (via OMZ)
  • zsh-users/zsh-autosuggestions
  • zsh-users/zsh-syntax-highlighting

This matches the plugin profile from the original oh-my-zsh alternatives article.

How to Run

Prerequisites: Docker.

# Clone this repo
git clone https://git.brandyapple.com/magnus/zsh-plugin-manager-benchmarks.git
cd zsh-plugin-manager-benchmarks

# Build the Docker image
docker build -t zsh-bench .

# Run the benchmarks
docker run --rm zsh-bench

The benchmark takes ~5 minutes to run (mostly cloning repos and warming caches). Results are printed to stdout.

The benchmark script (bench.zsh) is written in native zsh — no bash wrappers, no bc for math, no time output parsing. It uses zsh/datetime's EPOCHREALTIME sub-second timers and zsh-native arithmetic.

Results (May 2026)

Run on: Docker for Mac (Apple Silicon) — Linux arm64, zsh 5.8.1

Configuration Cold Load Warm Avg (5 runs)
Bare zsh (baseline) 0.121s 0.037s
Oh-My-Zsh (stock, 7 plugins) 0.278s 0.072s
Oh-My-Zsh (minimal, 1 plugin) — 0.070s
Antidote (9 plugins) 0.320s 0.315s
Zinit (9 plugins) 0.307s 0.300s
Zim (2 modules) 0.109s 0.028s

Key Observations

  1. OMZ's overhead is the framework, not the plugins: OMZ with 7 plugins (0.072s) barely differs from OMZ with 1 plugin (0.070s). The startup tax comes from oh-my-zsh.sh itself, not what you load on top of it.

  2. Antidote and Zinit trade off: Both are consistently around 0.3s per load because their static file generation and compinit run on every shell start with this configuration. The tradeoff is they give you far more control over what you load and when.

  3. Zim's caching is excellent: After the first run, Zim drops to near-baseline performance (0.008s vs bare 0.017s).

  4. These numbers are best-case: Real-world startup times are dominated by NVM, pyenv init, brew --prefix calls, and other plugin-side overhead — not by the plugin manager itself. See this analysis for a deep dive.

Methodology

The benchmark runs inside a clean Ubuntu 22.04 Docker container with no pre-existing zsh configuration. Each plugin manager is installed fresh and configured with the standard plugin set. The container runs on the host's kernel, so results are comparable across Docker-supported platforms.

Bare zsh is included as a baseline to measure framework overhead.

License

MIT