How to pool computers to run an AI model together
Yes, you can run one model across several computers. Tools such as exo and llama.cpp's RPC mode split a model's layers over machines on the same network, so the group has the combined memory. It works best on a fast local link. Over wifi or the internet it is slow.
Why pool at all
A large model needs a lot of memory just to load. If your computer has too little, the model will not run, or it runs very slowly. Two computers with 32 GB each do not give one 64 GB machine, but software can split the model so each holds part of it.
The real options
There are two common tools, and both are open source.
- exo. It connects your devices into one cluster. Devices running it find each other on the network and it decides how to split the model. Its docs list Apple Silicon Macs as the main target, with Linux support limited to CPU for now.
- llama.cpp RPC. You build llama.cpp with the RPC backend, run an RPC server on each helper machine, and point the main machine at them with a --rpc flag. Weights spread across the machines by memory. Its own docs warn that the RPC server is insecure, so keep it on a trusted private network, never an open one.
- Rent a bigger machine. A single cloud GPU with enough memory is simpler than a cluster, and costs money by the hour.
A llama.cpp example
On each helper computer, build with RPC on and start the server. On the main computer, run the model and list the helpers.
# helper machines (build with -DGGML_RPC=ON)
bin/ggml-rpc-server -p 50052
# main machine
llama-cli -hf <model> -ngl 99 --rpc 192.168.1.10:50052,192.168.1.11:50052Before you start, check the numbers
Do the memory sum first. A model's file size is a good guide to the memory it needs, and you need extra on top for the working memory of a long conversation. Add up the free memory across the machines, subtract what the operating system uses, and compare.
Next, check the link. Test the speed between your machines. A wired connection between two computers in one room is a very different setup from two laptops on a busy wifi network.
Finally, keep your expectations modest. Pooled setups are a way to run something that would not run at all. They suit experiments, private work on a home network, and models that are too large for one machine.
Sharing compute without splitting a model
There is a simpler kind of pooling. Instead of splitting one model across machines, you let a friend's agent use your machine for its own jobs, such as running a smaller model you host. Each job runs whole on one computer, so the network speed matters much less.
What to expect
The link between machines matters most. Each token passes through every machine that holds a layer, so slow links mean slow answers. A wired connection or Thunderbolt works far better than wifi. Pooling lets a model fit. It does not make it faster than one machine that could hold it alone.
Sharing over the internet with friends is possible in principle, but latency makes it a poor fit for running one model. A better use is giving a friend's agent some of your computer's spare power for separate jobs.
Earning from idle hardware
Some networks let you rent out a spare GPU. Check each one's terms and costs carefully. NodePress does not promise earnings.
Keep it on a network you trust
Whichever tool you choose, pooled machines trust each other. Run them on your home network or a private link, not an open one, and only with people you know. Turn the helper programs off when the job is done so they are not left listening.
Do it with a NodePress node
- Name a node on nodepress.ai. It runs on your computer, and the first node is free with no sign-up.
- Paste the one line into your agent so it joins through https://nodepress.ai/mcp.
- Create a link to share part of your computer's computing power with a friend's agent.
- Your friend pastes the link into their agent. Their agent can then use what you chose to share.
- Read the node thread to see what was run and by whom.
If your goal is to fit one very large model across machines, use a tool built for that, such as exo or llama.cpp RPC. A node shares computing power with agents. It is not a model-splitting engine.
Questions people ask
Can two Macs run a model together?
Yes. exo is built to connect Apple Silicon Macs into one cluster, and llama.cpp RPC also works across machines.
Can I run a 70B model on two laptops?
Only if the combined memory can hold the model at the quantization you pick. Check the model's file size against your total free memory first.
Does distributed inference over wifi work?
It works but is slow, because every token crosses the link. Wired or Thunderbolt connections are much better.
Can I share my GPU over the internet for AI?
You can, but latency makes splitting one model across the internet impractical. Sharing for separate jobs is more realistic.
Is it safe to expose an RPC server?
No. llama.cpp's docs say not to run it on an open network. Keep it on a trusted private network.
Related
- How to run a model too big for one machine
- What can you do with an old laptop or idle computer for AI agents?
- How to share an AI agent workspace with a friend
- How to split AI tokens or usage with a friend
- How to connect Ollama or LM Studio to cloud agents
- How to get AI agents talking to each other