peterretief/yggstore: Sharded, encrypted storage between buddies over Yggdrasil: lend one another disk area, ship recordsdata by sending a sealed stub. · GitHub

Proof of idea: sharded, encrypted file storage between friends on the
Yggdrasil overlay, with no VPN
coordinator: node IDs come from public keys, every node solely solutions group
members, and storage challenges examine that friends nonetheless maintain what they had been
given.

The concept: individuals in a bunch lend one another disk area. Each file is
encrypted, reduce into items and unfold over everybody’s machines, so it
survives anyone machine failing. The small stub left behind holds the important thing,
so sending somebody a stub (sealed for them, by e mail or anything) is a
method to ship them the file, with no cloud service in between.

Status: experimental. The cryptography has not been reviewed by anybody
impartial. Don’t make it the one copy of something you possibly can’t lose.

• New to this? Set up a storage box (step by
  step, for Windows customers with a second-hand mini PC).
• No field? The gateway lets individuals pay a month-to-month payment and
  use the group’s storage from any S3 program (rclone, Cyberduck, Duplicati).
  Members select whether or not their field holds clients’ information, and earn credit score.
• Every model is saved: history enables you to restore an
  older model of a file, or a folder because it was on a given day. A brand new
  model solely shops the components that modified.
• Nodes can message each other: direct messages, and
  subjects any node can publish to and comply with, delivered even to nodes that
  had been off on the time. Programs can use it too, by a neighborhood API.
• No Yggdrasil to put in: a node can run it built in,
  with no daemon, TUN system or root. It talks to nodes on the daemon too.
• The mesh retains members’ Yggdrasil linked instantly to every
  different, so shedding one tunnel or public peer does not reduce anybody off.
• Websites can dwell on the group: publish a folder, and
  a number of machines serve it, so a web site stays up when certainly one of them goes down.
• Email to your area can arrive on the group: every message
  is encrypted for you because it is available in and browse in your dashboard.
• Licence: see LICENSE. Contributions: CONTRIBUTING.md.
  Security points: SECURITY.md.

1. put splits the file into 4 MiB chunks, encrypts every with AES-256-GCM
   (one random key per file, contemporary nonce per chunk), and erasure-codes every
   chunk into 4 information + 2 parity shards.
2. Shards are content-addressed (filename = SHA-256) and unfold over the
   on-line friends, at most one shard of a bit per peer when there are 6+.
3. A stub FILE.ystub holds the manifest, together with the important thing. Anyone with
   the stub and entry to the friends can learn the file.
4. get fetches shards in parallel, rejects any whose hash does not match,
   and rebuilds every chunk from any 4 of its 6 shards.
5. At add time, whereas it nonetheless has the shards, the uploader precomputes
   20 single-use challenges per shard into FILE.ystub.challenges.json
   (maintain this personal). confirm sends one per shard: “hash this byte vary
   with this nonce”, and checks the reply.

A Yggdrasil deal with (200::/7) is derived from the node’s public key, so the
supply deal with of a connection over the overlay identifies the caller. The
deal with is the node ID. Each server solely solutions callers whose deal with is in
friends.json (default deny), binds solely to its overlay deal with (or, with the
built-in Yggdrasil, has nothing on the machine’s community
in any respect), and solely lets the unique author delete a shard.

Download yggstore in your system from the
releases web page (Linux on
PCs, Raspberry Pis and arm64 boards, Windows, macOS), and examine it in opposition to
SHA256SUMS. Or construct it your self with Go 1.24 or later:

go construct -o bin/yggstore ./cmd/yggstore

bin/yggstore id                      # prints your node ID and friends.json entry
# gather one entry from each machine into friends.json, copy it to every one
bin/yggstore serve -peers friends.json # on each machine (port 7400 by default)

bin/yggstore standing -peers friends.json
bin/yggstore put    -peers friends.json picture.jpg   # -> picture.jpg.ystub
bin/yggstore confirm picture.jpg.ystub
bin/yggstore get -o copy.jpg picture.jpg.ystub
bin/yggstore rm picture.jpg.ystub                   # delete its shards

friends.json:

[
  {"name": "desktop", "addr": "[200:1234:5678::1]:7400", "admin": true},
  {"title": "pi",      "addr": "[201:abcd::1]:7400", "gradual": true}
]

Each chunk is 6 shards, and no node will get greater than 2 of them, so shedding any
one node by no means loses information. A node marked "gradual": true will get one shard per
chunk whereas the others have room for the remaining, so it does not maintain uploads
again. Only the importing machine’s friends.json decides this.

Nodes that run on the identical machine (Docker containers, say) ought to share a
"host" tag, e.g. {"title": "t1", "addr": "...", "host": "desktop"}. The
2-shard restrict then applies per machine, and uploads want 3 machines on-line,
not simply 3 nodes.

On an admin dashboard, open Invite somebody, sort their title and press
Make invite. You get a message to ship them, with a one-time invite
(yggjoin1:…, legitimate 7 days). They want Yggdrasil working and related
(the message lists public friends), and the yggstore program; then:

yggstore be a part of -service yggjoin1:…        # -name NODE -me NAME -quota GB to alter the defaults

be a part of asks the admin node so as to add this machine (over Yggdrasil, presenting the
invite), saves the member record, provides the inviter as a contact, and with
-service begins the node and dashboard as person providers. On the admin aspect
the newcomer is added to friends.json (proprietor: their title) and to your contacts,
and the record sync tells each node. An invite works as soon as; the admin node
solely retains a hash of it. A node on a machine that’s already a member is
tagged with that machine routinely; in any other case tag it with "host" by
hand if you recognize two nodes share a field.

The be a part of endpoint is the one name a non-member could make, and it does
nothing with no legitimate invite (rate-limited to six tries a minute).

An "admin": true node (the desktop) retains each node’s record in step. Each
node reviews which record it holds; the dashboard sends its personal friends.json to any
node that holds one other one, so a node added on the dashboard, or a hand edit
of the desktop’s friends.json, reaches each node inside a minute, with no
restarts. A node retains a pushed record in its information folder
(friends.pushed.json), and makes use of whichever of that and its personal -peers file
modified final. Only admins might push, and a push should maintain the sender admin.

To add a machine: open Add a node on the dashboard and comply with the steps.
In quick, the brand new machine begins with a friends.json holding simply the desktop’s
entry, runs yggstore serve, and also you paste what yggstore id -name NAME
prints. New uploads use it right away; recordsdata already saved keep put.

Nodes on older software program present “replace yggstore” on the dashboard and will not be
despatched lists. Install the brand new binary on them and restart them; a node that has
by no means been despatched a listing additionally wants "admin": true on the admin node’s entry in
its personal friends.json, in order that it accepts lists from it.

Every particular person runs their very own node and their very own dashboard. The dashboard
listens on localhost solely and holds your stubs, which carry the keys to your
recordsdata, so it’s by no means shared. What individuals share is the community of nodes.

To ship somebody an merchandise, add them underneath Sharing → Contacts with their
sharing code (ys1…, proven on their dashboard), then press Share on the
merchandise. You get a .ysend file to ship by e mail or some other method. Only that
particular person can open it; the merchandise’s title, your notice and its key are sealed inside,
and opening it proves which sharing code despatched it. They drop it into their
outfiles, and it seems underneath Shared with you, the place they will restore
it. They want a node on this community, since nodes serve shards to members
solely.

• A obtained merchandise continues to be yours to lose: if the sender deletes it, it’s
  gone for everybody they shared it with. Remove on a obtained merchandise solely
  forgets it in your aspect.
• Received stubs might solely level at Yggdrasil addresses or nodes in your personal
  friends.json, so a crafted file can not make your restore contact different
  machines.
• Your personal sharing key’s ~/.yggstore/sharing.key (contacts in
  ~/.yggstore/contacts.json). Losing it means you possibly can’t open what individuals
  ship you any extra, and so they want your new code.
• The dashboard refuses actions with out its personal X-Yggstore header, and
  requests that title one other host, so different net pages in your browser
  can not press its buttons.

Each node reviews what number of bytes every uploader has saved on it. The dashboard
totals this per particular person (set with "proprietor" in friends.json; defaults to the
machine): what they use throughout the community, and what their nodes maintain for
others. The intention is to carry about 1.5× what you employ. For now it’s accounting
solely; nothing is enforced.

bin/yggstore dashboard -peers friends.json -outfiles outfiles   # dashboard + watcher
bin/yggstore watch     -peers friends.json -dir outfiles        # watcher solely, logs to stdout

outfiles/            drop recordsdata in, or folders of them → every file is sharded, then
                     changed by NAME.ystub beside it, so the folder tree stays
outfiles/obtained/   objects individuals despatched you (drop their .ysend wherever in outfiles)
outfiles/complete/      drop a file or folder in → saved as one merchandise (a folder turns into
                     one archive, restored as a complete)
outfiles/restore/    drop any .ystub in → rebuilt into restored/
outfiles/restored/   restored recordsdata and folders
outfiles/delete/     drop a .ystub in → the merchandise is faraway from each node, then its stub
outfiles/.yggstore/  stubs which were restored (saved for reference)

• An merchandise is picked up as soon as it has appeared the identical on two polls (5 s aside)
  and was final modified greater than 3 s in the past, so half-copied objects are left alone.
  Names ending in .half, .crdownload, .tmp and the like are ignored.
• In folders, each file is its personal merchandise: it may be restored or deleted on its
  personal, and new recordsdata added to the folder later are picked up too. Folders in
  complete/ are streamed as a tar archive and saved as one merchandise. On restore,
  solely common recordsdata and folders are written, and paths that will escape the
  vacation spot are rejected. Symlinks are skipped.
• A file put again subsequent to its personal stub (say, restored, edited and moved again)
  is saved as a brand new model, and the stub then factors to it; older variations
  are saved (see history). An unchanged copy is not saved
  once more.
• At least 3 machines should be on-line, in order that no machine holds greater than 2 of
  a bit’s 6 shards. Otherwise the merchandise stays put and is retried each 2 minutes.
• After importing, the watcher downloads the merchandise once more and compares SHA-256
  hashes. Only then is the unique deleted (-keep retains it anyway).
• Each stub’s challenges are saved in a hidden .NAME.ystub.challenges.json
  subsequent to it.
• Deleting (the delete/ folder, the dashboard’s Delete button, or
  yggstore rm STUB) removes each shard first. Only when all are gone are the
  stubs and challenges eliminated; if a node is down, the stub stays in delete/
  and is retried each 2 minutes, so no shard is left with no report.
  Deleting can’t be undone.
• The dashboard’s Restore button rebuilds any listed stub into restored/.
  A restore by no means overwrites: a second copy turns into title (2).ext.

bin/yggstore dashboard -peers friends.json -stubs stubs   # http://127.0.0.1:7480

Every 3 seconds the dashboard asks every peer for /v1/information (up/down, spherical
journey, shard rely, disk use) and checks, for each .ystub underneath -stubs,
which shards their holders nonetheless have (HEAD /shards/{hash}, nothing is
downloaded). A file is wholesome when each chunk has 6/6 shards,
degraded under that whereas 4 stay, and misplaced under 4. The Verify
button spends one saved problem per shard. The exercise log data nodes
going up or down, shard counts altering and file standing adjustments. It listens on
localhost solely by default.

To strive it on one machine: scripts/local-demo.sh runs six nodes on
totally different ports of your Yggdrasil deal with, shops a file, stops two nodes and
restores it. scripts/local-demo.sh loopback does the identical on ::1.

Tests: go check -race ./...

scripts/testnet.sh up      # 5 check nodes t1..t5 in Docker, every with its personal Yggdrasil
scripts/testnet.sh standing
bin/yggstore put -peers testnet/shared/friends.json FILE
docker cease yggtest-t2-1 yggtest-t4-1   # simulate two nodes failing
scripts/testnet.sh down    # cease (keys and shards saved); `clear` deletes all the things

The containers sit on their very own bridge (br-yggtest) and discover this machine’s
Yggdrasil by link-local multicast, as LAN nodes do. The host firewall should let
the bridge in (sudo ufw permit in on br-yggtest). The check peer record is the
desktop plus t1..t5, so check recordsdata by no means land on the true nodes. All six share
one machine and disk: this assessments behaviour, not actual redundancy.

• No machine holds greater than 2 of a bit’s 6 shards, so anyone machine
  can fail. Surviving two failures without delay wants 6 or extra machines.
• Stubs dwell solely the place they had been made. If that machine’s disk dies, the
  shards are nonetheless within the group however no person can decrypt them: again up your
  stubs and sharing.key.
• Membership is the admin’s friends.json, pushed to each node; there may be
  no gossip or vouching but.
• No restore: if a peer disappears, nothing re-creates its shards.
• Challenges are run by hand, will not be signed and produce no receipts.
• Plain HTTP; confidentiality in transit comes from Yggdrasil’s end-to-end
  encryption, and shards are ciphertext anyway.
• Chunks are buffered in reminiscence (4 MiB plus shards), not streamed.
• The Tailscale transport from the plan shouldn’t be carried out; transport.Transport
  is the seam for it.