Sheaf
A functional language for differentiable computation
Sheaf brings Clojure’s code-as-data to machine learning, with models as inspectable, composable, and compiled data structures.
For ML Researchers
- No classes, no boilerplate — Write math, not plumbing
- Runtime Observability — Catch NaN, trace shapes and profile performance without code changes
- Single binary framework — One executable, no dependencies. Train and run on GPU out of the box
For Agentic AI
- Context Density — 60-75% fewer tokens than equivalent Python for the same architecture
- Uniform Syntax — Single syntactic form for all operations reduces ambiguity and generation errors
- Immediate Onboarding — Built-in context generator for Claude Code, Cursor, and Copilot
Neural Networks as Math
In Sheaf, a neural network is a composition of mathematical functions over a parameter tree.
Sheaf's functional core makes differentiation and compilation possible without graph annotations. Supported tensor functions can be differentiated with value-and-grad and compiled automatically for CPU or GPU.
(defn forward [x p]
(as-> x h
(with-params [p :l1] (relu (+ (@ h W) b)))
(with-params [p :l2] (softmax (+ (@ h W) b)))))
(defn transformer-block [x layer-p config]
(as-> x h
(-> h
(layer-norm (get layer-p :ln1) 2)
(multi-head-attention layer-p config)
(first)
(+ h)) ;; residual
(-> h
(layer-norm (get layer-p :ln2) 2)
(mlp (get layer-p :mlp))
(+ h))))
Models as Data
Because models are data, Sheaf requires no module classes, registration, or parameter groups. Even structural operations like pruning, freezing, or weight sharing are expressed as regular data transformations.
Sheaf brings compile-time macros to the computation graph itself, generating architecture variants from a single template.
;; Grow a model: add a layer at runtime
(defn append-layer [params new-layer]
(assoc params :layers
(append (get params :layers) new-layer)))
;; Swap the output head for a different task
(defn hot-swap-head [model task-id heads]
(assoc model :head (get heads task-id)))
Observability
In Sheaf, every function call, tensor shape, and numerical statistic is observable at runtime.
A tracer logs the full call hierarchy with tensor statistics. Guards halt execution on numerical invariants like NaN or range violations. A profiler attributes wall time to each function in the call tree.
├─ [train-step] dict(keys:["l1", "l2"]), f32[4x2] [min:0.00e0 max:1.00e0] (32B), f32[4x1] [min:0.00e0 max:1.00e0] (16B), 0.700000
│ ├─ [forward] f32[4x2] [min:0.00e0 max:1.00e0] (32B), dict(keys:["l1", "l2"])
│ │ ├─ [relu] f32[4x8] [min:-1.37e0 max:2.33e0] (128B)
│ │ └─ ← f32[4x8] [min:0.00e0 max:2.33e0] (128B) (0.8μs)
│ │ ├─ [sigmoid] f32[4x1] [min:-5.48e-2 max:1.18e0] (16B)
│ │ └─ ← f32[4x1] [min:4.86e-1 max:7.66e-1] (16B) (1.8μs)
│ └─ ← f32[4x1] [min:4.86e-1 max:7.66e-1] (16B) (0.0μs)
...
$ sheaf train.shf --guard no-nan
Step 1 | Loss: 0.306990
Step 2 | Loss: 0.500000
/!\ Guard Breached: NoNan
Function: sigmoid
Tensor contains NaN or Inf values: f32[4x1] [min:inf max:-inf]
Backtrace (last 26 operations):
├─ [train-step] dict(keys:["l1", "l2"]), f32[4x2], f32[4x1], 1000.0
│ ├─ [forward] f32[4x2], dict(keys:["l1", "l2"])
│ │ ├─ [relu] f32[4x8] [min:-2.67e0 max:1.73e0]
│ │ └─ ← f32[4x8] [min:0.00e0 max:1.73e0] (0.6μs)
│ │ ├─ [sigmoid] f32[4x1] [min:inf max:-inf] [NaN DETECTED]
...
Profiler: 3.78s wall
Function Calls Total Self Avg/call
------------------------------------------------------------------------
gpt-forward 500 3.17s 3.17s 6.35ms
softmax 500 184.70ms 184.70ms 369.4μs
slide-window 500 142.79ms 142.79ms 285.6μs
reshape 501 109.23ms 109.23ms 218.0μs
choice 500 92.11ms 92.11ms 184.2μs
generate-token 500 3.59s 28.62ms 7.18ms
io 5 26.76ms 26.76ms 5.35ms
<lambda> 502 3.75s 11.83ms 7.46ms
... 21 others 7527 4.86ms
Call tree:
├── generate (3.75s, 1 call)
│ ├── reduce (3.75s, 1 call)
│ │ └── <lambda> (3.75s, 501 calls)
│ │ ├── generate-token (3.59s, 500 calls)
│ │ │ ├── gpt-forward (3.17s, 500 calls)
│ │ │ ├── softmax (184.70ms, 500 calls)
│ │ │ ├── reshape (109.22ms, 500 calls)
│ │ │ ├── choice (92.11ms, 500 calls)
│ │ │ └── ... 7 others (1.69ms, 4000 calls)
│ │ ├── slide-window (142.79ms, 500 calls)
│ │ └── ... 4 others (2.04ms, 3502 calls)
│ └── ... 2 others (1.5μs, 2 calls)
└── ... 7 others (26.86ms, 21 calls)
Compact by Construction
Context usage counts GPT-4 tokens (tiktoken) across model, training, and sampling code. Deploy size is the minimal runtime required to train and run a model on a CUDA GPU.
Native Runtime
Sheaf is written in Rust. The complete runtime with GPU backends ships as a single 4 MB executable.
The compiler toolchain is downloaded on first use and is not required to run a compiled model.
# Standalone, self-contained deployment
$ du -h *
128K __sheaf__ # compiled model
3.2M data
4.0K model.shf
164M out-weights
3.8M sheaf # runtime
4.0K train.shf