Why Sheaf

Sheaf began in 2025 from a simple question: what would differentiable programming look like if program transformation were a property of the language, rather than infrastructure layered around it?

Modern machine learning frameworks solve an enormous practical problem, and Python remains a productive environment for research. But tensor programs often acquire additional machinery to express compilation, automatic differentiation, vectorization, parameter handling, and graph-level transformations. Some systems expose these transformations directly; others trace or capture parts of an imperative program. In either case, the representation being transformed is often distinct from the program a researcher writes.

Sheaf takes a different starting point. A model is a pure, composable program in a Lisp dialect. Its source has a structured representation that macros and other language facilities can inspect and construct. The same language that defines tensor computation can therefore define transformations of model programs. Sheaf then compiles the resulting numerical function to StableHLO and executes it through IREE.

This is the central thesis of Sheaf: the representation used to describe a differentiable computation should remain available to the programmer as a structured value, without separating model construction from model execution into unrelated systems.

Lisp, data, and differentiable programs

Lisp has long been used in both symbolic and connectionist AI, from early systems on DEC PDP-10 machines to neural-network environments such as Lush. Its syntax represents programs as lists and symbols, which makes construction and transformation of program forms a normal part of the language. This property is usually called homoiconicity.

Homoiconicity does not, by itself, make a model able to rewrite or understand itself. It does provide a direct foundation for tools that work on model descriptions: macros can generate a function from a declarative layer specification, introduce instrumentation, or alter an architecture before its tensor operations are compiled. Symbolic structures such as queries, configurations, and parameter dictionaries can also be expressed naturally alongside numerical code.

Sheaf applies this style to differentiable programming. It is not an attempt to make every value a program, nor to replace tensor compilation with interpretation. Rather, it keeps the high-level representation of a computation accessible until it is lowered to a numerical program.

Clojure was the immediate language inspiration. It showed that a modern Lisp can be practical without displacing an existing ecosystem. An early version of Sheaf tested this idea with a Clojure-inspired tensor language backed by JAX/XLA. Sheaf later became a standalone system, retaining the functional and transformation-oriented model while owning its compiler and runtime.

How Sheaf compares

JAX

JAX is Sheaf's closest semantic relative. Both favor pure functions, explicit pseudo-random keys, automatic differentiation, vectorization, and compilation of numerical functions. In both systems, transformations such as differentiation are applied to functions rather than attached to mutable module objects.

The difference is the host language and what it makes available. JAX transforms a constrained, traceable subset of Python numerical programs and relies on Python for abstraction and metaprogramming. Sheaf makes Lisp forms, macros, and code construction part of the language in which the tensor program itself is written. A macro can construct a new Sheaf function, which then follows the same differentiation and compilation path as a handwritten one.

This does not make Sheaf a replacement for JAX. JAX has a broad and mature ecosystem. Sheaf explores a more unified language model for cases where programmable model structure is central.

PyTorch

PyTorch is built around eager, imperative tensor programming and an extensive ecosystem of modules, tools, and integrations. It also has serious compilation and graph-transformation facilities, including torch.compile and FX. Sheaf is not distinguished by merely being able to compile or transform a model.

The distinction is where those operations live. In PyTorch, the ordinary authoring model is Python code and mutable module state; capture and graph rewriting are additional mechanisms with their own representations and constraints. In Sheaf, pure functions and structured program forms are the ordinary model. Architecture-generating macros are not a separate graph API, and the function they produce is directly eligible for automatic differentiation and compilation.

Lush

Lush is an important historical relative rather than a current competitor. It demonstrated that Lisp could serve as a productive environment for neural networks, numerical arrays, and interactive experimentation. Sheaf shares that conviction while targeting a different generation of systems: pure tensor functions, source-level program transformation, reverse-mode automatic differentiation, and ahead-of-time or just-in-time compilation to modern accelerators.

Two examples

One layer specification, several compiled models

The macro example defines a small declarative layer specification and uses it to create several different functions:

(defmodel net (x) [linear :h1 relu] [linear :h2 relu])
(defresidual res-net (x) [linear :h1 relu] [linear :h2 relu])
(definspect inspect-net (x) [linear :h1 relu] [linear :h2 relu])

defmodel creates an ordinary feed-forward network. defresidual expands the same layer descriptions into a network with skip connections. definspect produces an instrumented version that returns per-layer activation statistics. The macros are written in Sheaf and generate ordinary Sheaf functions, so the training function can apply value-and-grad to each generated model without per-architecture module boilerplate.

See examples-dev/macros/run.shf for the complete example.

Differentiable visual reasoning over symbolic queries

The CLEVR example represents a visual question as nested data such as:

["query-color" ["leftmost" ["filter-shape" ":square"]]]

execute-query recursively examines this structure and maps each symbolic operation to a differentiable tensor operation: filtering, soft selection, spatial relations, and attribute classification. The question remains inspectable data, while its execution over a scene is numerical and differentiable.

This example does not claim that arbitrary symbolic interpretation is automatically compiled into one static tensor graph. It shows the more modest and useful property at the core of Sheaf: symbolic structure and numerical computation can be expressed, composed, and transformed in the same language.

Scope

Sheaf does not aim to replace the established AI toolchain. Its purpose is to provide a compact, native environment for people who want functional tensor programming, compiler-backed execution, and language-level control over model structure in one system.