Inventory every node
VRAM, RAM, compute features, bandwidth, latency, availability, and trust signals.
Open-weight inference / ordinary hardware
One large model.Many ordinary machines.
Swarmlet is an orchestration layer that inspects every available node and chooses how to divide an open-weight model across them. It then exposes the result as one system.
First modelQwen3.8 Flash Next
Node mixHeterogeneous
Network targetAnonymous + public
Live planning surface / 01Best-fit route
Macunified memory
16 GB GPUfast compute
4 GB GPUlimited VRAM
CPU + RAMcapacity tier
Swarmlet AImeasure · choose · place
Qwenone served model
01 / Mechanism
Bring hardware. Swarmlet builds the plan.
A 16 GB card can contribute without pretending it can hold the whole model. A slower node can add useful memory without being allowed to stall every token. The assignment changes with the actual mix.
Compare split strategies
Score stages, experts, tensor banks, or whole requests against the model graph and hardware limits.
Keep every host in budget
Assign model parts and routes without relying on a secretly oversized coordinator.
Present one endpoint
Coordinate execution, reduce results deterministically, and remap work when a node leaves.
02 / Adaptive split
Different model. Different hardware. Different answer.
Swarmlet is architecture-neutral by design. The planner may choose one technique or combine several; it can also reject a split when network physics would make it worse.
Stage split
Place sequential blocks on nodes with enough memory and a fast link between them.
Expert split
Route sparse mixture-of-experts work only to the nodes that own the selected experts.
Tensor-bank offload
Move a bounded weight bank to a remote GPU or RAM tier while the main runtime coordinates.
Whole-request routing
Send complete requests between independent cells when the wide-area link is too slow for token-by-token work.
Candidate routes / 02Planner scores all
Profilemodel + nodes
Stages
Experts
Tensors
Requests
Planbest valid fit
03 / Outcomes
Why connect the machines?
The aim is not distribution for its own sake. It is usable model capacity, useful speed, a home for idle hardware, and graceful survival when hardware disappears.
Fit larger models
Pool memory so one model can outgrow every individual card.
Use idle hardware
Let old and new machines contribute according to what they can safely do.
Add parallel capacity
Route independent work concurrently when the topology supports it.
Absorb failures
Detect departed nodes and produce a new placement instead of corrupt output.
04 / Physical proof
A consumer GPU has joined a real Qwen run.
The latest signed artifact is narrow but physical: Qwen3.8 Flash Next generated on an Apple Silicon host while a Legion laptop served a remote layer-0 fused expert bank over an SSH-forwarded GGML RPC path.
21.21
tokens / second
Measured decode speed for the GPU-only remote arm on a Legion GTX 1650 Ti. It matched the 64-token CPU baseline exactly.
1.68 GB
remote GPU memory used
The physical node loaded the bounded remote bank without swap growth; the owned processes were cleaned up afterward.
1 + 2
current physical network rig
One Mac and two Legion hosts are the active network test topology. The one-Legion Qwen path is signed; the separate two-owner protocol exists, while its signed physical result remains pending.
What this does not prove
It is not yet a full model split, true per-expert execution, an autonomous planner, or an anonymous public network. The current coordinator still holds responsibilities the finished cell must distribute.
05 / Full roadmap
From a physical path to a public mesh.
The roadmap exposes the missing work. Dates stay out until the underlying technical gates have evidence.
00
Deterministic foundation
Placement manifests, bounded execution, deterministic reduction, fail-closed behavior, cleanup, and signed evidence formats.
01
Physical remote participation
Real Qwen generation with a bounded tensor bank served by a Legion CPU or CUDA device over authenticated network plumbing.
02
Adaptive split planner
Inventory heterogeneous hardware, profile model graphs, compare valid split techniques, enforce per-node ceilings, and emit the best measured placement.
03
Complete inference cell
Run Qwen across the Mac + two-Legion rig without an oversized coordinator, then add node recovery and plan-identical restart.
04
Product alpha
Package the node agent, capability discovery, model distribution, CLI, OpenAI-compatible API, and web control surface.
05
Anonymous public network
Add enrollment, identity, encrypted transport, attestation, sandboxing, quotas, reputation, incentives, and fault-aware placement.
06
Cell federation + more models
Keep latency-sensitive splits inside complete regional cells, route whole requests between cells, and add compatibility profiles. DeepSeek V4 Flash is the next model target.
06 / Product shape
Four surfaces. One orchestration layer.
Cluster owners, node contributors, and application developers meet the same system from different sides.
Contribute hardware
Install an agent, publish measured capabilities, accept bounded assignments, and leave cleanly.
Operate a cell
Inspect nodes, profile models, generate plans, run proofs, and diagnose placement.
Serve applications
Send normal inference requests to one OpenAI-compatible endpoint.
See the mesh
Watch capacity, assignments, model health, failures, and roadmap gates.
Network physicsFirst Next
A public mesh is a federation of complete cells.
Fast local links carry latency-sensitive model splits. Wide-area links carry whole requests between cells. Swarmlet should never hide a bad network path behind a bigger node count.
Qwen3.8 Flash Next
Current physical evidence and planner target.
DeepSeek V4 Flash
Compatibility target after the Qwen cell closes its gates.
Federated topology / 03LAN split · WAN request
Node A
Node B
Cell 01complete model
Cell 02complete model
Node C
Node D
07 / Wishlist
Follow the proof, not the promise.
Get occasional build notes and an invitation when there is a node or API worth trying. No account, no password, no launch theatre.