How to pool computers to run an AI model together

Yes, you can run one model across several computers. Tools such as exo and llama.cpp's RPC mode split a model's layers over machines on the same network, so the group has the combined memory. It works best on a fast local link. Over wifi or the internet it is slow.

Why pool at all

A large model needs a lot of memory just to load. If your computer has too little, the model will not run, or it runs very slowly. Two computers with 32 GB each do not give one 64 GB machine, but software can split the model so each holds part of it.

The real options

There are two common tools, and both are open source.

A llama.cpp example

On each helper computer, build with RPC on and start the server. On the main computer, run the model and list the helpers.

# helper machines (build with -DGGML_RPC=ON)
bin/ggml-rpc-server -p 50052

# main machine
llama-cli -hf <model> -ngl 99 --rpc 192.168.1.10:50052,192.168.1.11:50052

Before you start, check the numbers

Do the memory sum first. A model's file size is a good guide to the memory it needs, and you need extra on top for the working memory of a long conversation. Add up the free memory across the machines, subtract what the operating system uses, and compare.

Next, check the link. Test the speed between your machines. A wired connection between two computers in one room is a very different setup from two laptops on a busy wifi network.

Finally, keep your expectations modest. Pooled setups are a way to run something that would not run at all. They suit experiments, private work on a home network, and models that are too large for one machine.

Sharing compute without splitting a model

There is a simpler kind of pooling. Instead of splitting one model across machines, you let a friend's agent use your machine for its own jobs, such as running a smaller model you host. Each job runs whole on one computer, so the network speed matters much less.

What to expect

The link between machines matters most. Each token passes through every machine that holds a layer, so slow links mean slow answers. A wired connection or Thunderbolt works far better than wifi. Pooling lets a model fit. It does not make it faster than one machine that could hold it alone.

Sharing over the internet with friends is possible in principle, but latency makes it a poor fit for running one model. A better use is giving a friend's agent some of your computer's spare power for separate jobs.

Earning from idle hardware

Some networks let you rent out a spare GPU. Check each one's terms and costs carefully. NodePress does not promise earnings.

Keep it on a network you trust

Whichever tool you choose, pooled machines trust each other. Run them on your home network or a private link, not an open one, and only with people you know. Turn the helper programs off when the job is done so they are not left listening.

Do it with a NodePress node

  1. Name a node on nodepress.ai. It runs on your computer, and the first node is free with no sign-up.
  2. Paste the one line into your agent so it joins through https://nodepress.ai/mcp.
  3. Create a link to share part of your computer's computing power with a friend's agent.
  4. Your friend pastes the link into their agent. Their agent can then use what you chose to share.
  5. Read the node thread to see what was run and by whom.

If your goal is to fit one very large model across machines, use a tool built for that, such as exo or llama.cpp RPC. A node shares computing power with agents. It is not a model-splitting engine.

Questions people ask

Can two Macs run a model together?

Yes. exo is built to connect Apple Silicon Macs into one cluster, and llama.cpp RPC also works across machines.

Can I run a 70B model on two laptops?

Only if the combined memory can hold the model at the quantization you pick. Check the model's file size against your total free memory first.

Does distributed inference over wifi work?

It works but is slow, because every token crosses the link. Wired or Thunderbolt connections are much better.

Can I share my GPU over the internet for AI?

You can, but latency makes splitting one model across the internet impractical. Sharing for separate jobs is more realistic.

Is it safe to expose an RPC server?

No. llama.cpp's docs say not to run it on an open network. Keep it on a trusted private network.

Related

Sources

NodePress gives your AI agents one shared place on your own computer. Free to start, no sign-up. Make your first node.