core/oc-discovery
6
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

saved

Browse Source
This commit is contained in:
mr
2026-03-09 14:57:41 +01:00
parent 3751ec554d
commit 83cef6e6f6
47 changed files with 2704 additions and 1034 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1030
docs/DECENTRALIZED_SYSTEMS_COMPARISON.txt Normal file
View File

File diff suppressed because it is too large Load Diff

362
docs/FUTURE_DHT_ARCHITECTURE.txt Normal file
View File

@@ -0,0 +1,362 @@
================================================================================
OC-DISCOVERY : ARCHITECTURE CIBLE — RÉSEAU DHT SANS NATIFS
Vision d'évolution long terme, issue d'une analyse comparative
================================================================================
Rédigé à partir de l'analyse de l'architecture actuelle et de la discussion
comparative avec Tapestry, Kademlia, EigenTrust et les systèmes de réputation
distribués.
Référence : DECENTRALIZED_SYSTEMS_COMPARISON.txt §9
================================================================================
1. MOTIVATION
================================================================================
L'architecture actuelle (node → indexer → native indexer) est robuste et bien
adaptée à une phase précoce du réseau. Ses limites à l'échelle sont :
- Pool de natives statique au démarrage → dépendance à la configuration
- Cache local des natives = point de défaillance unique (perte = pool vide)
- Consensus inter-natives bloquant (~7s) déclenché à chaque bootstrap node
- État O(N indexers) par native → croît linéairement avec le réseau
- Nœuds privilégiés structurellement → SPOFs relatifs
La cible décrite ici supprime la notion de native indexer en tant que tier
architectural. Le réseau devient plat : indexers et nodes sont des acteurs
de même nature, différenciés uniquement par leur rôle volontaire.
================================================================================
2. PRINCIPES FONDAMENTAUX
================================================================================
P1. Aucun nœud n'est structurellement privilégié.
P2. La confiance est un produit du temps et de la vérification, pas d'un arbitre.
P3. Les claims d'un acteur sont vérifiables indépendamment par tout pair.
P4. La réputation émerge du comportement collectif, pas d'un signalement central.
P5. La DHT est une infrastructure neutre — elle stocke des faits, pas des jugements.
P6. La configuration statique n'existe plus au runtime — seulement au bootstrap.
================================================================================
3. RÔLES
================================================================================
3.1 Node
--------
Consommateur du réseau. Démarre, sélectionne un pool d'indexers via DHT,
heartbeat ses indexers, accumule des scores localement. Ne publie rien en
routine. Participe aux challenges de consensus à la demande.
3.2 Indexer
-----------
Acteur volontaire. S'inscrit dans la DHT à la naissance, maintient son record,
sert le trafic des nodes (heartbeat, Publish, Get). Déclare ses métriques dans
chaque réponse heartbeat. Maintient un score agrégé depuis ses nodes connectés.
Différence avec l'actuel : l'indexer n'a plus de lien avec une native.
Il est autonome. Son existence dans le réseau est prouvée par son record DHT
et par les nodes qui le contactent directement.
3.3 Nœud DHT infrastructure (ex-native)
----------------------------------------
N'importe quel nœud suffisamment stable peut maintenir la DHT sans être un
indexer. C'est une configuration, pas un type architectural : `dht_mode: server`.
Ces nœuds maintiennent les k-buckets Kademlia et stockent les records des
indexers. Ils ne connaissent pas le trafic node↔indexer et ne l'orchestrent pas.
================================================================================
4. BOOTSTRAP D'UN NODE
================================================================================
4.1 Entrée dans le réseau
-------------------------
Le node démarre avec 1 à 3 adresses de nœuds DHT connus (bootstrap peers).
Ce sont les seules informations statiques nécessaires. Ces peers n'ont pas de
rôle sémantique — ils servent uniquement à entrer dans l'overlay DHT.
4.2 Découverte du pool d'indexers
----------------------------------
Node → DHT.FindProviders(hash("/opencloud/indexers"))
→ reçoit une liste de N candidats avec leurs records
Sélection du pool initial :
1. Filtre latence : ping < seuil → proximité réseau réelle
2. Filtre fill rate : préférer les indexers moins chargés
3. Tirage pondéré : probabilité ∝ (1 - fill_rate), courbe w(F) = F×(1-F)
indexer à 20% charge → très probable
indexer à 80% charge → peu probable
4. Filtre diversité : subnet /24 différent pour chaque entrée du pool
Aucun consensus nécessaire à cette étape. Le node démarre avec une tolérance
basse (voir §7) — il accepte des indexers imparfaits et les évalue au fil du temps.
================================================================================
5. REGISTRATION D'UN INDEXER DANS LA DHT
================================================================================
À la naissance, l'indexer publie son record DHT :
clé : hash("/opencloud/indexers") ← clé fixe, connue de tous
valeur: {
multiaddr : <adresse réseau>,
region : <subnet /24>,
capacity : <maxNodesConn>,
fill_rate : <float 0-1>, ← auto-déclaré, vérifiable
peer_count : <int>, ← auto-déclaré, vérifiable
peers : [hash(nodeID1), ...], ← liste hashée des nodes connectés
born_at : <timestamp>,
sig : <signature clé indexer>, ← non-forgeable (PSK context)
}
Le record est rafraîchi toutes les ~60s (avant expiration du TTL).
Si l'indexer tombe : TTL expire → disparaît de la DHT automatiquement.
La peer list est hashée pour la confidentialité mais reste vérifiable :
un challenger peut demander directement à un node s'il est connecté à cet indexer.
================================================================================
6. PROTOCOLE HEARTBEAT — QUESTION ET RÉPONSE
================================================================================
Le heartbeat devient bidirectionnel : le node pose des questions, l'indexer
répond avec ses déclarations courantes.
6.1 Structure
-------------
Node → Indexer :
{
ts : now,
challenge : <optionnel, voir §8>
}
Indexer → Node :
{
ts : now,
fill_rate : 0.42,
peer_count : 87,
cached_score : 0.74, ← score agrégé depuis tous ses nodes connectés
challenge_response : {...} ← si challenge présent dans la requête
}
Le heartbeat normal (sans challenge) est quasi-identique à l'actuel en poids.
Le cached_score indexer est mis à jour progressivement par les feedbacks reçus.
6.2 Le cached_score de l'indexer
---------------------------------
L'indexer agrège les scores que ses nodes connectés lui communiquent
(implicitement via le fait qu'ils restent connectés, ou explicitement lors
d'un consensus). Ce score lui donne une vision de sa propre qualité réseau.
Un node peut comparer son score local de l'indexer avec le cached_score déclaré.
Une forte divergence est un signal d'alerte.
Score local node : 0.40 ← cet indexer est médiocre pour moi
Cached score : 0.91 ← il se prétend excellent globalement
→ déclenche un challenge de vérification
================================================================================
7. MODÈLE DE CONFIANCE PROGRESSIVE
================================================================================
7.1 Cycle de vie d'un node
---------------------------
Naissance
→ tolérance basse : accepte presque n'importe quel indexer du DHT
→ switching cost faible : peu de contexte accumulé
→ minScore ≈ 20% (dynamicMinScore existant, conservé)
Quelques heures
→ uptime s'accumule sur chaque indexer connu
→ scores se stabilisent
→ seuil de remplacement qui monte progressivement
Long terme (jours)
→ pool stable, confiance élevée sur les indexers connus
→ switching coûteux mais déclenché sur déception franche
→ minScore ≈ 80% (maturité)
7.2 Modèle sous-jacent : beta distribution implicite
------------------------------------------------------
α = succès cumulés (heartbeats OK, probes OK, challenges réussis)
β = échecs cumulés (timeouts, probes échoués, challenges ratés)
confiance = α / (α + β)
Nouveau indexer : α=0, β=0 → prior neutre, tolérance basse
Après 10 jours : α élevé → confiance stable, seuil de switch élevé
Déception franche : β monte → confiance chute → switch déclenché
7.3 Ce que "décevoir" signifie
--------------------------------
Heartbeat rate → trop de timeouts → fiabilité en baisse
Bandwidth probe → chute sous déclaré → dégradation ou mensonge
Fill rate réel → supérieur au déclaré → indexer surchargé ou malhonnête
Challenge échoué → peer déclaré absent du réseau → claim invalide
Latence → dérive progressive → qualité réseau dégradée
Cached_score gonflé → divergence forte avec score local → suspicion
================================================================================
8. VÉRIFICATION DES CLAIMS — TROIS COUCHES
================================================================================
8.1 Couche 1 : passive (chaque heartbeat, 60s)
-----------------------------------------------
Mesures automatiques, zéro coût supplémentaire.
- RTT du heartbeat → latence directe
- fill_rate déclaré → tiny payload dans la réponse
- peer_count déclaré → tiny payload
- cached_score indexer → comparé au score local
8.2 Couche 2 : sampling actif (1 heartbeat sur N)
--------------------------------------------------
Vérifications périodiques, asynchrones, légères.
Tous les 5 HB (~5min) : spot-check 1 peer aléatoire (voir §8.4)
Tous les 10 HB (~10min): vérification diversité subnet (lookups DHT légers)
Tous les 15 HB (~15min): bandwidth probe (transfert réel, protocole dédié)
8.3 Couche 3 : consensus (événementiel)
-----------------------------------------
Déclenché sur : admission d'un nouvel indexer dans le pool, ou suspicion détectée.
Node sélectionne une claim vérifiable de l'indexer cible X
Node vérifie lui-même
Node demande à ses indexers de confiance : "vérifiez cette claim sur X"
Chaque indexer vérifie indépendamment
Convergence des résultats → X est honnête → admission
Divergence → X est suspect → rejet ou probation
Le consensus est léger : quelques contacts out-of-band, pas de round bloquant.
Il n'est pas continu — il est événementiel.
8.4 Vérification out-of-band (pas de DHT writes par les nodes)
----------------------------------------------------------------
Les nodes ne publient PAS de contact records continus dans la DHT.
Cela éviterait N×M records à rafraîchir (coût DHT élevé à l'échelle).
À la place, lors d'un challenge :
Challenger sélectionne 2-3 peers dans la peer list déclarée par X
→ contacte ces peers directement : "es-tu connecté à indexer X ?"
→ réponse directe (out-of-band, pas via DHT)
→ vérification sans écriture DHT
L'indexer ne peut pas faire répondre "oui" à des peers qui ne lui sont pas
connectés. La vérification est non-falsifiable et sans coût DHT.
8.5 Pourquoi X ne peut pas tricher
------------------------------------
X ne peut pas coordonner des réponses différentes vers des challengers
simultanés. Chaque challenger contacte indépendamment les mêmes peers.
Si X ment sur sa peer list :
- Challenger A contacte peer P → "non, pas connecté à X"
- Challenger B contacte peer P → "non, pas connecté à X"
- Consensus : X ment → score chute chez tous les challengers
- Effet réseau : progressivement, X perd ses connections
- Peer list DHT se vide → claims futures encore moins crédibles
================================================================================
9. EFFET RÉSEAU SANS SIGNALEMENT CENTRAL
================================================================================
Un node qui pénalise un indexer n'envoie aucun "rapport" à quiconque.
Ses actions locales produisent l'effet réseau par agrégation :
Node baisse le score de X → X reçoit moins de trafic de ce node
Node switche vers Y → X perd un client
Node refuse les challenges X → X ne peut plus participer aux consensus
Si 200 nodes font pareil :
X perd la majorité de ses connections
Sa peer list DHT se vide (peers contactés directement disent "non")
Son cached_score s'effondre (peu de nodes restent)
Les nouveaux nodes qui voient X dans la DHT obtiennent des challenges échoués
X est naturellement exclu sans aucune décision centrale
Inversement, un indexer honnête voit ses scores monter sur tous ses nodes
connectés, sa peer list se densifier, ses challenges réussis systématiquement.
Sa réputation est un produit observable et vérifiable.
================================================================================
10. RÉSUMÉ DE L'ARCHITECTURE
================================================================================
DHT → annuaire neutre, vérité des records indexers
maintenu par tout nœud stable (dht_mode: server)
Indexer → acteur volontaire, s'inscrit, maintient ses claims,
sert le trafic, accumule son propre score agrégé
Node → consommateur, score passif + sampling + consensus léger,
confiance progressive, switching adaptatif
Heartbeat → métronome 60s + vecteur de déclarations légères + challenge optionnel
Consensus → événementiel, multi-challengers indépendants,
vérification out-of-band sur claims DHT
Confiance → beta implicite, progressive, switching cost croissant avec l'âge
Réputation → émerge du comportement collectif, aucun arbitre central
Bootstrap → 1-3 peers DHT connus → seule configuration statique nécessaire
================================================================================
11. TRAJECTOIRE DE MIGRATION
================================================================================
Phase 1 (actuel)
Natives statiques, pool indexers dynamique, consensus inter-natives
→ robuste, adapté à la phase précoce
Phase 2 (intermédiaire)
Pool de natives dynamique via DHT (bootstrap + gossip)
Même protocole natif, juste la découverte devient dynamique
→ supprime la dépendance à la configuration statique des natives
→ voir DECENTRALIZED_SYSTEMS_COMPARISON.txt §9.2
Phase 3 (cible)
Architecture décrite dans ce document
Natives disparaissent en tant que tier architectural
DHT = infrastructure, indexers = acteurs autonomes
Scoring et consensus entièrement côté node
→ aucun nœud privilégié, scalabilité O(log N)
La migration Phase 2 → Phase 3 est une refonte du plan de contrôle.
Le plan de données (heartbeat node↔indexer, Publish, Get) est inchangé.
Les primitives libp2p (Kademlia DHT, GossipSub) sont déjà présentes.
================================================================================
12. PROPRIÉTÉS DU SYSTÈME CIBLE
================================================================================
Scalabilité O(log N) — routage DHT Kademlia
Résilience Pas de SPOF structurel, TTL = seule source de vérité
Confiance Progressive, vérifiable, émergente
Sybil resistance PSK — seuls les nœuds avec la clé peuvent publier
Cold start Tolérance basse initiale, montée progressive (existant)
Honnêteté Claims vérifiables out-of-band, non-falsifiables
Décentralisation Aucun nœud ne connaît l'état global complet
================================================================================

56
docs/diagrams/01_node_init.mmd
View File

@@ -1,56 +0,0 @@
sequenceDiagram
title Node Initialization — Pair A (InitNode)
participant MainA as main (Pair A)
participant NodeA as Node A
participant libp2pA as libp2p (Pair A)
participant DBA as DB Pair A (oc-lib)
participant NATSA as NATS A
participant IndexerA as Indexer (partagé)
participant StreamA as StreamService A
participant PubSubA as PubSubService A
MainA->>NodeA: InitNode(isNode, isIndexer, isNativeIndexer)
NodeA->>NodeA: LoadKeyFromFilePrivate() → priv
NodeA->>NodeA: LoadPSKFromFile() → psk
NodeA->>libp2pA: New(PrivateNetwork(psk), Identity(priv), ListenAddr:4001)
libp2pA-->>NodeA: host A (PeerID_A)
Note over NodeA: isNode == true
NodeA->>libp2pA: NewGossipSub(ctx, host)
libp2pA-->>NodeA: ps (GossipSub)
NodeA->>IndexerA: ConnectToIndexers → SendHeartbeat /opencloud/heartbeat/1.0
Note over IndexerA: Heartbeat long-lived établi<br/>Score qualité calculé (bw + uptime + diversité)
IndexerA-->>NodeA: OK
NodeA->>NodeA: claimInfo(name, hostname)
NodeA->>IndexerA: TempStream /opencloud/record/publish/1.0
NodeA->>IndexerA: json.Encode(PeerRecord A signé)
IndexerA->>IndexerA: DHT.PutValue("/node/"+DID_A, record)
NodeA->>DBA: NewRequestAdmin(PEER).Search(SELF)
DBA-->>NodeA: peer A local (ou UUID généré)
NodeA->>NodeA: StartGC(30s) — GC sur StreamRecords
NodeA->>StreamA: InitStream(ctx, host, PeerID_A, 1000, nodeA)
StreamA->>StreamA: SetStreamHandler(heartbeat/partner, search, planner, ...)
StreamA->>DBA: Search(PEER, PARTNER) → liste partenaires
DBA-->>StreamA: [] (aucun partenaire au démarrage)
StreamA-->>NodeA: StreamService A
NodeA->>PubSubA: InitPubSub(ctx, host, ps, nodeA, streamA)
PubSubA->>PubSubA: subscribeEvents(PB_SEARCH, timeout=-1)
PubSubA-->>NodeA: PubSubService A
NodeA->>NodeA: SubscribeToSearch(ps, callback)
Note over NodeA: callback: GetPeerRecord(evt.From)<br/>→ StreamService.SendResponse
NodeA->>NATSA: ListenNATS(nodeA)
Note over NATSA: Enregistre handlers:<br/>CREATE_RESOURCE, PROPALGATION_EVENT
NodeA-->>MainA: *Node A prêt

28
docs/diagrams/01_node_init.puml
View File

@@ -1,10 +1,10 @@
@startuml
title Node Initialization — Pair A (InitNode)
title Node Initialization — Peer A (InitNode)
participant "main (Pair A)" as MainA
participant "main (Peer A)" as MainA
participant "Node A" as NodeA
participant "libp2p (Pair A)" as libp2pA
participant "DB Pair A (oc-lib)" as DBA
participant "libp2p (Peer A)" as libp2pA
participant "DB Peer A (oc-lib)" as DBA
participant "NATS A" as NATSA
participant "Indexer (partagé)" as IndexerA
participant "StreamService A" as StreamA
@@ -24,35 +24,35 @@ NodeA -> libp2pA: NewGossipSub(ctx, host)
libp2pA --> NodeA: ps (GossipSub)
NodeA -> IndexerA: ConnectToIndexers → SendHeartbeat /opencloud/heartbeat/1.0
note over IndexerA: Heartbeat long-lived établi\nScore qualité calculé (bw + uptime + diversité)
note over IndexerA: Heartbeat long-lived established\nQuality Score evaluated (bw + uptime + diversity)
IndexerA --> NodeA: OK
NodeA -> NodeA: claimInfo(name, hostname)
NodeA -> IndexerA: TempStream /opencloud/record/publish/1.0
NodeA -> IndexerA: json.Encode(PeerRecord A signé)
NodeA -> IndexerA: stream.Encode(PeerRecord A signé)
IndexerA -> IndexerA: DHT.PutValue("/node/"+DID_A, record)
NodeA -> DBA: NewRequestAdmin(PEER).Search(SELF)
DBA --> NodeA: peer A local (ou UUID généré)
NodeA -> DBA: DB(PEER).Search(SELF)
DBA --> NodeA: local peer A (or new generated UUID)
NodeA -> NodeA: StartGC(30s) — GC sur StreamRecords
NodeA -> NodeA: StartGC(30s) — GarbageCollector on StreamRecords
NodeA -> StreamA: InitStream(ctx, host, PeerID_A, 1000, nodeA)
StreamA -> StreamA: SetStreamHandler(heartbeat/partner, search, planner, ...)
StreamA -> DBA: Search(PEER, PARTNER) → liste partenaires
DBA --> StreamA: [] (aucun partenaire au démarrage)
StreamA -> DBA: Search(PEER, PARTNER) → partner list
DBA --> StreamA: Heartbeat long-lived established to partners
StreamA --> NodeA: StreamService A
NodeA -> PubSubA: InitPubSub(ctx, host, ps, nodeA, streamA)
PubSubA -> PubSubA: subscribeEvents(PB_SEARCH, timeout=-1)
PubSubA --> NodeA: PubSubService A
NodeA -> NodeA: SubscribeToSearch(ps, callback)
NodeA -> NodeA: SubscribeToSearch(ps, callback) (search global topic for resources)
note over NodeA: callback: GetPeerRecord(evt.From)\n→ StreamService.SendResponse
NodeA -> NATSA: ListenNATS(nodeA)
note over NATSA: Enregistre handlers:\nCREATE_RESOURCE, PROPALGATION_EVENT
note over NATSA: Subscribes handlers:\nCREATE_RESOURCE, PROPALGATION_EVENT
NodeA --> MainA: *Node A prêt
NodeA --> MainA: *Node A is ready
@enduml

38
docs/diagrams/02_node_claim.mmd
View File

@@ -1,38 +0,0 @@
sequenceDiagram
title Node Claim — Pair A publie son PeerRecord (claimInfo + publishPeerRecord)
participant DBA as DB Pair A (oc-lib)
participant NodeA as Node A
participant IndexerA as Indexer (partagé)
participant DHT as DHT Kademlia
participant NATSA as NATS A
NodeA->>DBA: NewRequestAdmin(PEER).Search(SELF)
DBA-->>NodeA: existing peer (DID_A) ou nouveau UUID
NodeA->>NodeA: LoadKeyFromFilePrivate() → priv A
NodeA->>NodeA: LoadKeyFromFilePublic() → pub A
NodeA->>NodeA: crypto.MarshalPublicKey(pub A) → pubBytes
NodeA->>NodeA: Build PeerRecord A {<br/> Name, DID, PubKey,<br/> PeerID: PeerID_A,<br/> APIUrl: hostname,<br/> StreamAddress: /ip4/.../tcp/4001/p2p/PeerID_A,<br/> NATSAddress, WalletAddress<br/>}
NodeA->>NodeA: sha256(json(rec)) → hash
NodeA->>NodeA: priv.Sign(hash) → signature
NodeA->>NodeA: rec.ExpiryDate = now + 150s
loop Pour chaque StaticIndexer (Indexer A, B, …)
NodeA->>IndexerA: TempStream /opencloud/record/publish/1.0
NodeA->>IndexerA: json.Encode(PeerRecord A signé)
IndexerA->>IndexerA: Verify signature
IndexerA->>IndexerA: Check heartbeat stream actif pour PeerID_A
IndexerA->>DHT: PutValue("/node/"+DID_A, PeerRecord A)
DHT-->>IndexerA: ok
end
NodeA->>NodeA: rec.ExtractPeer(DID_A, DID_A, pub A)
NodeA->>NATSA: SetNATSPub(CREATE_RESOURCE, {PEER, Peer A JSON})
NATSA->>DBA: Upsert Peer A (SearchAttr: peer_id)
DBA-->>NATSA: ok
NodeA-->>NodeA: *peer.Peer A (SELF)

20
docs/diagrams/02_node_claim.puml
View File

@@ -1,31 +1,29 @@
@startuml
title Node Claim — Pair A publie son PeerRecord (claimInfo + publishPeerRecord)
title Node Claim — Peer A publish its PeerRecord (claimInfo + publishPeerRecord)
participant "DB Pair A (oc-lib)" as DBA
participant "DB Peer A (oc-lib)" as DBA
participant "Node A" as NodeA
participant "Indexer (partagé)" as IndexerA
participant "Indexer (shared)" as IndexerA
participant "DHT Kademlia" as DHT
participant "NATS A" as NATSA
NodeA -> DBA: NewRequestAdmin(PEER).Search(SELF)
DBA --> NodeA: existing peer (DID_A) ou nouveau UUID
NodeA -> DBA: DB(PEER).Search(SELF)
DBA --> NodeA: existing peer (DID_A) or new UUID
NodeA -> NodeA: LoadKeyFromFilePrivate() → priv A
NodeA -> NodeA: LoadKeyFromFilePublic() → pub A
NodeA -> NodeA: crypto.MarshalPublicKey(pub A) → pubBytes
NodeA -> NodeA: Build PeerRecord A {\n Name, DID, PubKey,\n PeerID: PeerID_A,\n APIUrl: hostname,\n StreamAddress: /ip4/.../tcp/4001/p2p/PeerID_A,\n NATSAddress, WalletAddress\n}
NodeA -> NodeA: sha256(json(rec)) → hash
NodeA -> NodeA: priv.Sign(hash) → signature
NodeA -> NodeA: priv.Sign(rec) → signature
NodeA -> NodeA: rec.ExpiryDate = now + 150s
loop Pour chaque StaticIndexer (Indexer A, B, ...)
loop For every Node Binded Indexer (Indexer A, B, ...)
NodeA -> IndexerA: TempStream /opencloud/record/publish/1.0
NodeA -> IndexerA: json.Encode(PeerRecord A signé)
NodeA -> IndexerA: strea!.Encode(Signed PeerRecord A)
IndexerA -> IndexerA: Verify signature
IndexerA -> IndexerA: Check heartbeat stream actif pour PeerID_A
IndexerA -> IndexerA: Check PeerID_A heartbeat stream
IndexerA -> DHT: PutValue("/node/"+DID_A, PeerRecord A)
DHT --> IndexerA: ok
end

47
docs/diagrams/03_indexer_heartbeat.mmd
View File

@@ -1,47 +0,0 @@
sequenceDiagram
title Indexer — Heartbeat double (Pair A + Pair B → Indexer partagé)
participant NodeA as Node A
participant NodeB as Node B
participant Indexer as IndexerService (partagé)
Note over NodeA,NodeB: Chaque pair tick toutes les 20s
par Pair A heartbeat
NodeA->>Indexer: NewStream /opencloud/heartbeat/1.0
NodeA->>Indexer: json.Encode(Heartbeat A {Name, DID_A, PeerID_A, IndexersBinded})
Indexer->>Indexer: CheckHeartbeat(host, stream, streams, mu, maxNodes)
Note over Indexer: len(peers) < maxNodes ?
Indexer->>Indexer: getBandwidthChallenge(512–2048 bytes, stream)
Indexer->>NodeA: Write(random payload)
NodeA->>Indexer: Echo(same payload)
Indexer->>Indexer: Mesure round-trip → Mbps A
Indexer->>Indexer: getDiversityRate(host, IndexersBinded_A)
Note over Indexer: /24 subnet diversity des indexeurs liés
Indexer->>Indexer: ComputeIndexerScore(uptimeA%, MbpsA%, diversityA%)
Note over Indexer: Score = 0.4×uptime + 0.4×bpms + 0.2×diversity
alt Score A < 75
Indexer->>NodeA: (close stream)
else Score A ≥ 75
Indexer->>Indexer: StreamRecord[PeerID_A] = {DID_A, Heartbeat, UptimeTracker}
end
and Pair B heartbeat
NodeB->>Indexer: NewStream /opencloud/heartbeat/1.0
NodeB->>Indexer: json.Encode(Heartbeat B {Name, DID_B, PeerID_B, IndexersBinded})
Indexer->>Indexer: CheckHeartbeat → getBandwidthChallenge
Indexer->>NodeB: Write(random payload)
NodeB->>Indexer: Echo(same payload)
Indexer->>Indexer: ComputeIndexerScore(uptimeB%, MbpsB%, diversityB%)
alt Score B ≥ 75
Indexer->>Indexer: StreamRecord[PeerID_B] = {DID_B, Heartbeat, UptimeTracker}
end
end
Note over Indexer: Les deux pairs sont désormais<br/>enregistrés avec leurs streams actifs

58
docs/diagrams/03_indexer_heartbeat.puml
View File

@@ -1,49 +1,49 @@
@startuml
title Indexer — Heartbeat double (Pair A + Pair B → Indexer partagé)
@startuml indexer_heartbeat
title Indexer — Heartbeat node → indexer (score on 5 metrics)
participant "Node A" as NodeA
participant "Node B" as NodeB
participant "IndexerService (partagé)" as Indexer
participant "IndexerService" as Indexer
note over NodeA,NodeB: Chaque pair tick toutes les 20s
note over NodeA,NodeB: Every node tick every 20s (SendHeartbeat)
par Pair A heartbeat
par Node A heartbeat
NodeA -> Indexer: NewStream /opencloud/heartbeat/1.0
NodeA -> Indexer: json.Encode(Heartbeat A {Name, DID_A, PeerID_A, IndexersBinded})
NodeA -> Indexer: stream.Encode(Heartbeat{Name, PeerID_A, IndexersBinded, Record})
Indexer -> Indexer: CheckHeartbeat(host, stream, streams, mu, maxNodes)
note over Indexer: len(peers) < maxNodes ?
Indexer -> Indexer: CheckHeartbeat(host, stream, dec, streams, mu, maxNodes)
note over Indexer: len(h.Network().Peers()) >= maxNodes → reject
Indexer -> Indexer: getBandwidthChallenge(512-2048 bytes, stream)
Indexer -> NodeA: Write(random payload)
NodeA -> Indexer: Echo(same payload)
Indexer -> Indexer: Mesure round-trip → Mbps A
Indexer -> Indexer: getBandwidthChallengeRate(host, remotePeer, 512-2048B)
Indexer -> Indexer: getDiversityRate(host, IndexersBinded_A)
note over Indexer: /24 subnet diversity des indexeurs liés
Indexer -> Indexer: getOwnDiversityRate(host)\\nh.Network().Peers() + Peerstore.Addrs()\\n→ ratio /24 subnets distincts
Indexer -> Indexer: ComputeIndexerScore(uptimeA%, MbpsA%, diversityA%)
note over Indexer: Score = 0.4×uptime + 0.4×bpms + 0.2×diversity
Indexer -> Indexer: fillRate = len(h.Network().Peers()) / maxNodes
alt Score A < 75
Indexer -> NodeA: (close stream)
else Score A >= 75
Indexer -> Indexer: StreamRecord[PeerID_A] = {DID_A, Heartbeat, UptimeTracker}
Indexer -> Indexer: Retrieve existing UptimeTracker\\noldTracker.RecordHeartbeat()\\n→ TotalOnline += gap si gap ≤ 120s\\nuptimeRatio = TotalOnline / time.Since(FirstSeen)
Indexer -> Indexer: ComputeIndexerScore(\\n uptimeRatio, bpms, diversity,\\n latencyScore, fillRate\\n)\\nScore = (0.20×U + 0.20×B + 0.20×D + 0.15×L + 0.25×F) × 100
Indexer -> Indexer: dynamicMinScore(age)\\n= 20 + 60×(hours/24), max 80
alt Score A < dynamicMinScore(age)
Indexer -> NodeA: (close stream — "not enough trusting value")
else Score A >= dynamicMinScore(age)
Indexer -> Indexer: streams[PeerID_A].HeartbeatStream = hb.Stream\\nstreams[PeerID_A].HeartbeatStream.UptimeTracker = oldTracker\\nstreams[PeerID_A].LastScore = hb.Score
note over Indexer: AfterHeartbeat → republish PeerRecord on DHT
end
else Pair B heartbeat
else Node B heartbeat
NodeB -> Indexer: NewStream /opencloud/heartbeat/1.0
NodeB -> Indexer: json.Encode(Heartbeat B {Name, DID_B, PeerID_B, IndexersBinded})
NodeB -> Indexer: stream.Encode(Heartbeat{Name, PeerID_B, IndexersBinded, Record})
Indexer -> Indexer: CheckHeartbeat → getBandwidthChallenge
Indexer -> NodeB: Write(random payload)
NodeB -> Indexer: Echo(same payload)
Indexer -> Indexer: ComputeIndexerScore(uptimeB%, MbpsB%, diversityB%)
Indexer -> Indexer: CheckHeartbeat → getBandwidthChallengeRate\\n→ getOwnDiversityRate → ComputeIndexerScore(5 composants)
alt Score B >= 75
Indexer -> Indexer: StreamRecord[PeerID_B] = {DID_B, Heartbeat, UptimeTracker}
alt Score B >= dynamicMinScore(age)
Indexer -> Indexer: streams[PeerID_B] subscribed + LastScore updated
end
end par
note over Indexer: Les deux pairs sont désormais\nenregistrés avec leurs streams actifs
note over Indexer: GC ticker 30s — gc()\\nnow.After(Expiry) où Expiry = lastHBTime + 2min\\n→ AfterDelete(pid, name, did)
@enduml

41
docs/diagrams/04_indexer_publish.mmd
View File

@@ -1,41 +0,0 @@
sequenceDiagram
title Indexer — Pair A publie, Pair B publie (handleNodePublish → DHT)
participant NodeA as Node A
participant NodeB as Node B
participant Indexer as IndexerService (partagé)
participant DHT as DHT Kademlia
Note over NodeA: Après claimInfo ou refresh TTL
par Pair A publie son PeerRecord
NodeA->>Indexer: TempStream /opencloud/record/publish/1.0
NodeA->>Indexer: json.Encode(PeerRecord A {DID_A, PeerID_A, PubKey_A, Expiry, Sig_A})
Indexer->>Indexer: Verify sig_A (reconstruit rec minimal, pubKey_A.Verify)
Indexer->>Indexer: Check StreamRecords[Heartbeat][PeerID_A] existe
alt Heartbeat actif pour A
Indexer->>Indexer: StreamRecord A → DID_A, Record=PeerRecord A, LastSeen=now
Indexer->>DHT: PutValue("/node/"+DID_A, PeerRecord A JSON)
DHT-->>Indexer: ok
else Pas de heartbeat
Indexer->>NodeA: (erreur "no heartbeat", stream close)
end
and Pair B publie son PeerRecord
NodeB->>Indexer: TempStream /opencloud/record/publish/1.0
NodeB->>Indexer: json.Encode(PeerRecord B {DID_B, PeerID_B, PubKey_B, Expiry, Sig_B})
Indexer->>Indexer: Verify sig_B
Indexer->>Indexer: Check StreamRecords[Heartbeat][PeerID_B] existe
alt Heartbeat actif pour B
Indexer->>Indexer: StreamRecord B → DID_B, Record=PeerRecord B, LastSeen=now
Indexer->>DHT: PutValue("/node/"+DID_B, PeerRecord B JSON)
DHT-->>Indexer: ok
else Pas de heartbeat
Indexer->>NodeB: (erreur "no heartbeat", stream close)
end
end
Note over DHT: DHT contient maintenant<br/>"/node/DID_A" et "/node/DID_B"

24
docs/diagrams/04_indexer_publish.puml
View File

@@ -1,43 +1,47 @@
@startuml
title Indexer — Pair A publie, Pair B publie (handleNodePublish → DHT)
title Indexer — Peer A publishing, Peer B publishing (handleNodePublish → DHT)
participant "Node A" as NodeA
participant "Node B" as NodeB
participant "IndexerService (partagé)" as Indexer
participant "IndexerService (shared)" as Indexer
participant "DHT Kademlia" as DHT
note over NodeA: Après claimInfo ou refresh TTL
note over NodeA: Start after claimInfo or refresh TTL
par Pair A publie son PeerRecord
par Peer A publish its PeerRecord
NodeA -> Indexer: TempStream /opencloud/record/publish/1.0
NodeA -> Indexer: json.Encode(PeerRecord A {DID_A, PeerID_A, PubKey_A, Expiry, Sig_A})
NodeA -> Indexer: stream.Encode(PeerRecord A {DID_A, PeerID_A, PubKey_A, Expiry, Sig_A})
Indexer -> Indexer: Verify sig_A (reconstruit rec minimal, pubKey_A.Verify)
Indexer -> Indexer: Check StreamRecords[Heartbeat][PeerID_A] existe
alt Heartbeat actif pour A
alt A active Heartbeat
Indexer -> Indexer: StreamRecord A → DID_A, Record=PeerRecord A, LastSeen=now
Indexer -> DHT: PutValue("/node/"+DID_A, PeerRecord A JSON)
Indexer -> DHT: PutValue("/name/"+name_A, DID_A)
Indexer -> DHT: PutValue("/peer/"+peer_id_A, DID_A)
DHT --> Indexer: ok
else Pas de heartbeat
Indexer -> NodeA: (erreur "no heartbeat", stream close)
end
else Pair B publie son PeerRecord
else Peer B publish its PeerRecord
NodeB -> Indexer: TempStream /opencloud/record/publish/1.0
NodeB -> Indexer: json.Encode(PeerRecord B {DID_B, PeerID_B, PubKey_B, Expiry, Sig_B})
NodeB -> Indexer: stream.Encode(PeerRecord B {DID_B, PeerID_B, PubKey_B, Expiry, Sig_B})
Indexer -> Indexer: Verify sig_B
Indexer -> Indexer: Check StreamRecords[Heartbeat][PeerID_B] existe
alt Heartbeat actif pour B
alt B Active Heartbeat
Indexer -> Indexer: StreamRecord B → DID_B, Record=PeerRecord B, LastSeen=now
Indexer -> DHT: PutValue("/node/"+DID_B, PeerRecord B JSON)
Indexer -> DHT: PutValue("/name/"+name_B, DID_B)
Indexer -> DHT: PutValue("/peer/"+peer_id_B, DID_B)
DHT --> Indexer: ok
else Pas de heartbeat
Indexer -> NodeB: (erreur "no heartbeat", stream close)
end
end par
note over DHT: DHT contient maintenant\n"/node/DID_A" et "/node/DID_B"
note over DHT: DHT got \n"/node/DID_A" et "/node/DID_B"
@enduml

49
docs/diagrams/05_indexer_get.mmd
View File

@@ -1,49 +0,0 @@
sequenceDiagram
title Indexer — Pair A résout Pair B (GetPeerRecord + handleNodeGet)
participant NATSA as NATS A
participant DBA as DB Pair A (oc-lib)
participant NodeA as Node A
participant Indexer as IndexerService (partagé)
participant DHT as DHT Kademlia
participant NATSA2 as NATS A (retour)
Note over NodeA: Déclenché par : NATS PB_SEARCH PEER<br/>ou callback SubscribeToSearch
NodeA->>DBA: NewRequestAdmin(PEER).Search(DID_B ou PeerID_B)
DBA-->>NodeA: Peer B local (si connu) → résout DID_B + PeerID_B<br/>sinon utilise la valeur brute
loop Pour chaque StaticIndexer
NodeA->>Indexer: TempStream /opencloud/record/get/1.0
NodeA->>Indexer: json.Encode(GetValue{Key: DID_B, PeerID: PeerID_B})
Indexer->>Indexer: key = "/node/" + DID_B
Indexer->>DHT: SearchValue(ctx 10s, "/node/"+DID_B)
DHT-->>Indexer: channel de bytes (PeerRecord B)
loop Pour chaque résultat DHT
Indexer->>Indexer: Unmarshal → PeerRecord B
alt PeerRecord.PeerID == PeerID_B
Indexer->>Indexer: resp.Found=true, resp.Records[PeerID_B]=PeerRecord B
Indexer->>Indexer: StreamRecord B.LastSeen = now (si heartbeat actif)
end
end
Indexer->>NodeA: json.Encode(GetResponse{Found:true, Records:{PeerID_B: PeerRecord B}})
end
loop Pour chaque PeerRecord retourné
NodeA->>NodeA: rec.Verify() → valide signature de B
NodeA->>NodeA: rec.ExtractPeer(ourDID_A, DID_B, pubKey_B)
alt ourDID_A == DID_B (c'est notre propre entrée)
Note over NodeA: Republier pour rafraîchir le TTL
NodeA->>Indexer: publishPeerRecord(rec) [refresh 2 min]
end
NodeA->>NATSA2: SetNATSPub(CREATE_RESOURCE, {PEER, Peer B JSON,<br/>SearchAttr:"peer_id"})
NATSA2->>DBA: Upsert Peer B dans DB A
DBA-->>NATSA2: ok
end
NodeA-->>NodeA: []*peer.Peer → [Peer B]

32
docs/diagrams/05_indexer_get.puml
View File

@@ -1,5 +1,5 @@
@startuml
title Indexer — Pair A résout Pair B (GetPeerRecord + handleNodeGet)
title Indexer — Peer A discover Peer B (GetPeerRecord + handleNodeGet)
participant "NATS A" as NATSA
participant "DB Pair A (oc-lib)" as DBA
@@ -8,41 +8,41 @@ participant "IndexerService (partagé)" as Indexer
participant "DHT Kademlia" as DHT
participant "NATS A (retour)" as NATSA2
note over NodeA: Déclenché par : NATS PB_SEARCH PEER\nou callback SubscribeToSearch
note over NodeA: Trigger : NATS PB_SEARCH PEER\nor callback SubscribeToSearch
NodeA -> DBA: NewRequestAdmin(PEER).Search(DID_B ou PeerID_B)
DBA --> NodeA: Peer B local (si connu) → résout DID_B + PeerID_B\nsinon utilise la valeur brute
NodeA -> DBA: (PEER).Search(DID_B or PeerID_B)
DBA --> NodeA: Local Peer B (if known) → solve DID_B + PeerID_B\nor use search value
loop Pour chaque StaticIndexer
NodeA -> Indexer: TempStream /opencloud/record/get/1.0
NodeA -> Indexer: json.Encode(GetValue{Key: DID_B, PeerID: PeerID_B})
loop For every Peer A Binded Indexer
NodeA -> Indexer: TempStream /opencloud/record/get/1.0 -> streamAI
NodeA -> Indexer: streamAI.Encode(GetValue{Key: DID_B, PeerID: PeerID_B})
Indexer -> Indexer: key = "/node/" + DID_B
Indexer -> DHT: SearchValue(ctx 10s, "/node/"+DID_B)
DHT --> Indexer: channel de bytes (PeerRecord B)
loop Pour chaque résultat DHT
Indexer -> Indexer: Unmarshal → PeerRecord B
loop Pour every results in DHT
Indexer -> Indexer: read → PeerRecord B
alt PeerRecord.PeerID == PeerID_B
Indexer -> Indexer: resp.Found=true, resp.Records[PeerID_B]=PeerRecord B
Indexer -> Indexer: StreamRecord B.LastSeen = now (si heartbeat actif)
Indexer -> Indexer: StreamRecord B.LastSeen = now (if active heartbeat)
end
end
Indexer -> NodeA: json.Encode(GetResponse{Found:true, Records:{PeerID_B: PeerRecord B}})
Indexer -> NodeA: streamAI.Encode(GetResponse{Found:true, Records:{PeerID_B: PeerRecord B}})
end
loop Pour chaque PeerRecord retourné
NodeA -> NodeA: rec.Verify() → valide signature de B
loop For every PeerRecord founded
NodeA -> NodeA: rec.Verify() → valid B signature
NodeA -> NodeA: rec.ExtractPeer(ourDID_A, DID_B, pubKey_B)
alt ourDID_A == DID_B (c'est notre propre entrée)
note over NodeA: Republier pour rafraîchir le TTL
alt ourDID_A == DID_B (it's our proper entry)
note over NodeA: Republish to refresh TTL
NodeA -> Indexer: publishPeerRecord(rec) [refresh 2 min]
end
NodeA -> NATSA2: SetNATSPub(CREATE_RESOURCE, {PEER, Peer B JSON,\nSearchAttr:"peer_id"})
NATSA2 -> DBA: Upsert Peer B dans DB A
NATSA2 -> DBA: Upsert Peer B in DB A
DBA --> NATSA2: ok
end

39
docs/diagrams/06_native_registration.mmd
View File

@@ -1,39 +0,0 @@
sequenceDiagram
title Native Indexer — Enregistrement d'un Indexer auprès du Native
participant IndexerA as Indexer A
participant IndexerB as Indexer B
participant Native as Native Indexer (partagé)
participant DHT as DHT Kademlia
participant PubSub as GossipSub (oc-indexer-registry)
Note over IndexerA,IndexerB: Au démarrage + toutes les 60s (StartNativeRegistration)
par Indexer A s'enregistre
IndexerA->>IndexerA: Build IndexerRegistration{PeerID_A, Addr_A}
IndexerA->>Native: NewStream /opencloud/native/subscribe/1.0
IndexerA->>Native: json.Encode(IndexerRegistration A)
Native->>Native: Decode → liveIndexerEntry{PeerID_A, Addr_A, ExpiresAt=now+66s}
Native->>DHT: PutValue("/indexer/"+PeerID_A, entry A)
DHT-->>Native: ok
Native->>Native: liveIndexers[PeerID_A] = entry A
Native->>Native: knownPeerIDs[PeerID_A] = {}
Native->>PubSub: topic.Publish([]byte(PeerID_A))
Note over PubSub: Gossipé aux autres Natives<br/>→ ils ajoutent PeerID_A à knownPeerIDs<br/>→ refresh DHT au prochain tick 30s
IndexerA->>Native: stream.Close()
and Indexer B s'enregistre
IndexerB->>IndexerB: Build IndexerRegistration{PeerID_B, Addr_B}
IndexerB->>Native: NewStream /opencloud/native/subscribe/1.0
IndexerB->>Native: json.Encode(IndexerRegistration B)
Native->>Native: Decode → liveIndexerEntry{PeerID_B, Addr_B, ExpiresAt=now+66s}
Native->>DHT: PutValue("/indexer/"+PeerID_B, entry B)
DHT-->>Native: ok
Native->>Native: liveIndexers[PeerID_B] = entry B
Native->>PubSub: topic.Publish([]byte(PeerID_B))
IndexerB->>Native: stream.Close()
end
Note over Native: liveIndexers = {PeerID_A: entryA, PeerID_B: entryB}

48
docs/diagrams/06_native_registration.puml
View File

@@ -1,41 +1,49 @@
@startuml
title Native Indexer — Enregistrement d'un Indexer auprès du Native
@startuml native_registration
title Native Indexer — Indexer Subscription (StartNativeRegistration)
participant "Indexer A" as IndexerA
participant "Indexer B" as IndexerB
participant "Native Indexer (partagé)" as Native
participant "Native Indexer" as Native
participant "DHT Kademlia" as DHT
participant "GossipSub (oc-indexer-registry)" as PubSub
note over IndexerA,IndexerB: Au démarrage + toutes les 60s (StartNativeRegistration)
note over IndexerA,IndexerB: At start + every 60s (RecommendedHeartbeatInterval)\\nStartNativeRegistration → RegisterWithNative
par Indexer A subscribe
IndexerA -> IndexerA: fillRateFn()\\n= len(StreamRecords[HB]) / maxNodes
IndexerA -> IndexerA: Build IndexerRegistration{\\n PeerID_A, Addr_A,\\n Timestamp=now.UnixNano(),\\n FillRate=fillRateFn(),\\n PubKey, Signature\\n}\\nreg.Sign(h)
par Indexer A s'enregistre
IndexerA -> IndexerA: Build IndexerRegistration{PeerID_A, Addr_A}
IndexerA -> Native: NewStream /opencloud/native/subscribe/1.0
IndexerA -> Native: json.Encode(IndexerRegistration A)
IndexerA -> Native: stream.Encode(IndexerRegistration A)
Native -> Native: reg.Verify() — verify signature
Native -> Native: liveIndexerEntry{\\n PeerID_A, Addr_A,\\n ExpiresAt = now + IndexerTTL (90s),\\n FillRate = reg.FillRate,\\n PubKey, Signature\\n}
Native -> Native: liveIndexers[PeerID_A] = entry A
Native -> Native: knownPeerIDs[PeerID_A] = Addr_A
Native -> Native: Decode → liveIndexerEntry{PeerID_A, Addr_A, ExpiresAt=now+66s}
Native -> DHT: PutValue("/indexer/"+PeerID_A, entry A)
DHT --> Native: ok
Native -> Native: liveIndexers[PeerID_A] = entry A
Native -> Native: knownPeerIDs[PeerID_A] = {}
Native -> PubSub: topic.Publish([]byte(PeerID_A))
note over PubSub: Gossipé aux autres Natives\n→ ils ajoutent PeerID_A à knownPeerIDs\n→ refresh DHT au prochain tick 30s
IndexerA -> Native: stream.Close()
else Indexer B s'enregistre
IndexerB -> IndexerB: Build IndexerRegistration{PeerID_B, Addr_B}
IndexerB -> Native: NewStream /opencloud/native/subscribe/1.0
IndexerB -> Native: json.Encode(IndexerRegistration B)
note over PubSub: Gossip to other Natives\\n→ it adds PeerID_A to knownPeerIDs\\n→ refresh DHT next tick (30s)
Native -> Native: Decode → liveIndexerEntry{PeerID_B, Addr_B, ExpiresAt=now+66s}
Native -> DHT: PutValue("/indexer/"+PeerID_B, entry B)
DHT --> Native: ok
IndexerA -> Native: stream.Close()
else Indexer B subscribe
IndexerB -> IndexerB: fillRateFn() + reg.Sign(h)
IndexerB -> Native: NewStream /opencloud/native/subscribe/1.0
IndexerB -> Native: stream.Encode(IndexerRegistration B)
Native -> Native: reg.Verify() + liveIndexerEntry{FillRate=reg.FillRate, ExpiresAt=now+90s}
Native -> Native: liveIndexers[PeerID_B] = entry B
Native -> DHT: PutValue("/indexer/"+PeerID_B, entry B)
Native -> PubSub: topic.Publish([]byte(PeerID_B))
IndexerB -> Native: stream.Close()
end par
note over Native: liveIndexers = {PeerID_A: entryA, PeerID_B: entryB}
note over Native: liveIndexers = {PeerID_A: {FillRate:0.3}, PeerID_B: {FillRate:0.6}}\\nTTL 90s — IndexerTTL
note over Native: Explicit unsubcrive on stop :\\nUnregisterFromNative → /opencloud/native/unsubscribe/1.0\\nNative close all now.
@enduml

60
docs/diagrams/07_native_get_consensus.mmd
View File

@@ -1,60 +0,0 @@
sequenceDiagram
title Native — ConnectToNatives + Consensus (Pair A bootstrap)
participant NodeA as Node A
participant Native1 as Native #1 (primary)
participant Native2 as Native #2
participant NativeN as Native #N
participant DHT as DHT Kademlia
Note over NodeA: NativeIndexerAddresses configuré<br/>Appelé pendant InitNode → ConnectToIndexers
NodeA->>NodeA: Parse NativeIndexerAddresses → StaticNatives
NodeA->>Native1: SendHeartbeat /opencloud/heartbeat/1.0 (20s tick)
NodeA->>Native2: SendHeartbeat /opencloud/heartbeat/1.0 (20s tick)
%% Étape 1 : récupérer un pool initial
NodeA->>Native1: Connect + NewStream /opencloud/native/indexers/1.0
NodeA->>Native1: json.Encode(GetIndexersRequest{Count: maxIndexer})
Native1->>Native1: reachableLiveIndexers()
Note over Native1: Filtre liveIndexers par TTL<br/>ping chaque candidat (PeerIsAlive)
alt Aucun indexer connu par Native1
Native1->>Native1: selfDelegate(NodeA.PeerID, resp)
Note over Native1: IsSelfFallback=true<br/>Indexers=[native1 addr]
Native1->>NodeA: GetIndexersResponse{IsSelfFallback:true, Indexers:[native1]}
NodeA->>NodeA: StaticIndexers[native1] = native1
Note over NodeA: Pas de consensus — native1 utilisé directement comme indexeur
else Indexers disponibles
Native1->>NodeA: GetIndexersResponse{Indexers:[Addr_IndexerA, Addr_IndexerB, ...]}
%% Étape 2 : consensus
Note over NodeA: clientSideConsensus(candidates)
par Requêtes consensus parallèles
NodeA->>Native1: NewStream /opencloud/native/consensus/1.0
NodeA->>Native1: ConsensusRequest{Candidates:[Addr_A, Addr_B]}
Native1->>Native1: Croiser avec liveIndexers propres
Native1->>NodeA: ConsensusResponse{Trusted:[Addr_A, Addr_B], Suggestions:[]}
and
NodeA->>Native2: NewStream /opencloud/native/consensus/1.0
NodeA->>Native2: ConsensusRequest{Candidates:[Addr_A, Addr_B]}
Native2->>Native2: Croiser avec liveIndexers propres
Native2->>NodeA: ConsensusResponse{Trusted:[Addr_A], Suggestions:[Addr_C]}
and
NodeA->>NativeN: NewStream /opencloud/native/consensus/1.0
NodeA->>NativeN: ConsensusRequest{Candidates:[Addr_A, Addr_B]}
NativeN->>NativeN: Croiser avec liveIndexers propres
NativeN->>NodeA: ConsensusResponse{Trusted:[Addr_A, Addr_B], Suggestions:[]}
end
Note over NodeA: Aggrège les votes (timeout 4s)<br/>Addr_A → 3/3 votes → confirmé ✓<br/>Addr_B → 2/3 votes → confirmé ✓
alt confirmed < maxIndexer && suggestions disponibles
Note over NodeA: Round 2 — rechallenge avec suggestions
NodeA->>NodeA: clientSideConsensus(confirmed + sample(suggestions))
end
NodeA->>NodeA: StaticIndexers = adresses confirmées à majorité
end

102
docs/diagrams/07_native_get_consensus.puml
View File

@@ -1,62 +1,70 @@
@startuml
title Native — ConnectToNatives + Consensus (Pair A bootstrap)
@startuml native_get_consensus
title Native — ConnectToNatives : fetch pool + Phase 1 + Phase 2
participant "Node A" as NodeA
participant "Native #1 (primary)" as Native1
participant "Native #2" as Native2
participant "Native #N" as NativeN
participant "DHT Kademlia" as DHT
participant "Node / Indexer\\n(appelant)" as Caller
participant "Native A" as NA
participant "Native B" as NB
participant "Indexer A\\n(stable voter)" as IA
note over NodeA: NativeIndexerAddresses configuré\nAppelé pendant InitNode → ConnectToIndexers
note over Caller: NativeIndexerAddresses configured\\nConnectToNatives() called from ConnectToIndexers
NodeA -> NodeA: Parse NativeIndexerAddresses → StaticNatives
NodeA -> Native1: SendHeartbeat /opencloud/heartbeat/1.0 (20s tick)
NodeA -> Native2: SendHeartbeat /opencloud/heartbeat/1.0 (20s tick)
== Step 1 : heartbeat to the native mesh (nativeHeartbeatOnce) ==
Caller -> NA: SendHeartbeat /opencloud/heartbeat/1.0
Caller -> NB: SendHeartbeat /opencloud/heartbeat/1.0
' Étape 1 : récupérer un pool initial
NodeA -> Native1: Connect + NewStream /opencloud/native/indexers/1.0
NodeA -> Native1: json.Encode(GetIndexersRequest{Count: maxIndexer})
== Step 2 : parrallel fetch pool (timeout 6s) ==
par fetchIndexersFromNative — parallel
Caller -> NA: NewStream /opencloud/native/indexers/1.0\\nGetIndexersRequest{Count: maxIndexer, From: PeerID}
NA -> NA: reachableLiveIndexers()\\ntri par w(F) = fillRate×(1−fillRate) desc
NA --> Caller: GetIndexersResponse{Indexers:[IA,IB], FillRates:{IA:0.3,IB:0.6}}
else
Caller -> NB: NewStream /opencloud/native/indexers/1.0
NB -> NB: reachableLiveIndexers()
NB --> Caller: GetIndexersResponse{Indexers:[IA,IB], FillRates:{IA:0.3,IB:0.6}}
end par
Native1 -> Native1: reachableLiveIndexers()
note over Native1: Filtre liveIndexers par TTL\nping chaque candidat (PeerIsAlive)
note over Caller: Fusion → candidates=[IA,IB]\\nisFallback=false
alt Aucun indexer connu par Native1
Native1 -> Native1: selfDelegate(NodeA.PeerID, resp)
note over Native1: IsSelfFallback=true\nIndexers=[native1 addr]
Native1 -> NodeA: GetIndexersResponse{IsSelfFallback:true, Indexers:[native1]}
NodeA -> NodeA: StaticIndexers[native1] = native1
note over NodeA: Pas de consensus — native1 utilisé directement comme indexeur
else Indexers disponibles
Native1 -> NodeA: GetIndexersResponse{Indexers:[Addr_IndexerA, Addr_IndexerB, ...]}
' Étape 2 : consensus
note over NodeA: clientSideConsensus(candidates)
par Requêtes consensus parallèles
NodeA -> Native1: NewStream /opencloud/native/consensus/1.0
NodeA -> Native1: ConsensusRequest{Candidates:[Addr_A, Addr_B]}
Native1 -> Native1: Croiser avec liveIndexers propres
Native1 -> NodeA: ConsensusResponse{Trusted:[Addr_A, Addr_B], Suggestions:[]}
alt isFallback=true (native give themself as Fallback indexer)
note over Caller: resolvePool : avoid consensus\\nadmittedAt = Now (zero)\\nStaticIndexers = {native_addr}
else isFallback=false → Phase 1 + Phase 2
== Phase 1 — clientSideConsensus (timeout 3s/natif, 4s total) ==
par Parralel Consensus
Caller -> NA: NewStream /opencloud/native/consensus/1.0\\nConsensusRequest{Candidates:[IA,IB]}
NA -> NA: compare with clean liveIndexers
NA --> Caller: ConsensusResponse{Trusted:[IA,IB], Suggestions:[]}
else
NodeA -> Native2: NewStream /opencloud/native/consensus/1.0
NodeA -> Native2: ConsensusRequest{Candidates:[Addr_A, Addr_B]}
Native2 -> Native2: Croiser avec liveIndexers propres
Native2 -> NodeA: ConsensusResponse{Trusted:[Addr_A], Suggestions:[Addr_C]}
else
NodeA -> NativeN: NewStream /opencloud/native/consensus/1.0
NodeA -> NativeN: ConsensusRequest{Candidates:[Addr_A, Addr_B]}
NativeN -> NativeN: Croiser avec liveIndexers propres
NativeN -> NodeA: ConsensusResponse{Trusted:[Addr_A, Addr_B], Suggestions:[]}
Caller -> NB: NewStream /opencloud/native/consensus/1.0
NB --> Caller: ConsensusResponse{Trusted:[IA], Suggestions:[IC]}
end par
note over NodeA: Aggrège les votes (timeout 4s)\nAddr_A → 3/3 votes → confirmé ✓\nAddr_B → 2/3 votes → confirmé ✓
note over Caller: IA → 2/2 votes → confirmed ✓\\nIB → 1/2 vote → refusé ✗\\nIC → suggestion → round 2 if confirmed < maxIndexer
alt confirmed < maxIndexer && suggestions disponibles
note over NodeA: Round 2 — rechallenge avec suggestions
NodeA -> NodeA: clientSideConsensus(confirmed + sample(suggestions))
alt confirmed < maxIndexer && available suggestions
note over Caller: Round 2 — rechallenge with confirmed + sample(suggestions)\\nclientSideConsensus([IA, IC])
end
NodeA -> NodeA: StaticIndexers = adresses confirmées à majorité
note over Caller: admittedAt = time.Now()
== Phase 2 — indexerLivenessVote (timeout 3s/votant, 4s total) ==
note over Caller: Search for stable voters in Subscribed Indexers\\nAdmittedAt != zero && age >= MinStableAge (2min)
alt Stable Voters are available
par Phase 2 parrallel
Caller -> IA: NewStream /opencloud/indexer/consensus/1.0\\nIndexerConsensusRequest{Candidates:[IA]}
IA -> IA: StreamRecords[ProtocolHB][candidate]\\ntime.Since(LastSeen) <= 120s && LastScore >= 30.0
IA --> Caller: IndexerConsensusResponse{Alive:[IA]}
end par
note over Caller: alive IA confirmed per quorum > 0.5\\npool = {IA}
else No voters are stable (startup)
note over Caller: Phase 1 keep directly\\n(no indexer reaches MinStableAge)
end
== Replacement pool ==
Caller -> Caller: replaceStaticIndexers(pool, admittedAt)\\nStaticIndexerMeta[IA].AdmittedAt = admittedAt
end
== Étape 3 : heartbeat to indexers pool (ConnectToIndexers) ==
Caller -> Caller: SendHeartbeat /opencloud/heartbeat/1.0\\nvers StaticIndexers
@enduml

49
docs/diagrams/08_nats_create_resource.mmd
View File

@@ -1,49 +0,0 @@
sequenceDiagram
title NATS — CREATE_RESOURCE : Pair A découvre Pair B et établit le stream
participant AppA as App Pair A (oc-api)
participant NATSA as NATS A
participant NodeA as Node A
participant StreamA as StreamService A
participant NodeB as Node B
participant StreamB as StreamService B
participant DBA as DB Pair A (oc-lib)
Note over AppA: Pair B vient d'être découvert<br/>(via indexeur ou manuel)
AppA->>NATSA: Publish(CREATE_RESOURCE, {<br/> FromApp:"oc-api",<br/> Datatype:PEER,<br/> Payload: Peer B {StreamAddress_B, Relation:PARTNER}<br/>})
NATSA->>NodeA: ListenNATS callback → CREATE_RESOURCE
NodeA->>NodeA: resp.FromApp == "oc-discovery" ? → Non, continuer
NodeA->>NodeA: json.Unmarshal(payload) → peer.Peer B
NodeA->>NodeA: pp.AddrInfoFromString(B.StreamAddress)
Note over NodeA: ad_B = {ID: PeerID_B, Addrs: [...]}
NodeA->>StreamA: Mu.Lock()
alt peer B.Relation == PARTNER
NodeA->>StreamA: ConnectToPartner(B.StreamAddress)
StreamA->>StreamA: AddrInfoFromString(B.StreamAddress) → ad_B
StreamA->>NodeB: Connect (libp2p)
StreamA->>NodeB: NewStream /opencloud/resource/heartbeat/partner/1.0
StreamA->>NodeB: json.Encode(Heartbeat{Name_A, DID_A, PeerID_A})
NodeB->>StreamB: HandlePartnerHeartbeat(stream)
StreamB->>StreamB: CheckHeartbeat → bandwidth challenge
StreamB->>StreamA: Echo(payload)
StreamB->>StreamB: streams[ProtocolHeartbeatPartner][PeerID_A] = {DID_A, Expiry=now+10s}
StreamA->>StreamA: streams[ProtocolHeartbeatPartner][PeerID_B] = {DID_B, Expiry=now+10s}
Note over StreamA,StreamB: Stream partner long-lived établi<br/>dans les deux sens
else peer B.Relation != PARTNER (révocation / blacklist)
Note over NodeA: Supprimer tous les streams vers Pair B
loop Pour chaque protocole dans Streams
NodeA->>StreamA: streams[proto][PeerID_B].Stream.Close()
NodeA->>StreamA: delete(streams[proto], PeerID_B)
end
end
NodeA->>StreamA: Mu.Unlock()
NodeA->>DBA: (pas de write direct ici — géré par l'app source)

28
docs/diagrams/08_nats_create_resource.puml → docs/diagrams/08_nats_create_update_peer.puml
View File

@@ -1,50 +1,42 @@
@startuml
title NATS — CREATE_RESOURCE : Pair A découvre Pair B et établit le stream
title NATS — CREATE_RESOURCE : Peer A Create/Update Peer B & establishing stream
participant "App Pair A (oc-api)" as AppA
participant "App Peer A (oc-api)" as AppA
participant "NATS A" as NATSA
participant "Node A" as NodeA
participant "StreamService A" as StreamA
participant "Node B" as NodeB
participant "StreamService B" as StreamB
participant "DB Pair A (oc-lib)" as DBA
participant "DB Peer A (oc-lib)" as DBA
note over AppA: Pair B vient d'être découvert\n(via indexeur ou manuel)
note over AppA: Peer B is discovered\n(per indexer or manually)
AppA -> NATSA: Publish(CREATE_RESOURCE, {\n FromApp:"oc-api",\n Datatype:PEER,\n Payload: Peer B {StreamAddress_B, Relation:PARTNER}\n})
NATSA -> NodeA: ListenNATS callback → CREATE_RESOURCE
NodeA -> NodeA: resp.FromApp == "oc-discovery" ? → Non, continuer
NodeA -> NodeA: if from himself ? → No, continue
NodeA -> NodeA: json.Unmarshal(payload) → peer.Peer B
NodeA -> NodeA: pp.AddrInfoFromString(B.StreamAddress)
note over NodeA: ad_B = {ID: PeerID_B, Addrs: [...]}
NodeA -> StreamA: Mu.Lock()
alt peer B.Relation == PARTNER
NodeA -> StreamA: ConnectToPartner(B.StreamAddress)
StreamA -> StreamA: AddrInfoFromString(B.StreamAddress) → ad_B
StreamA -> NodeB: Connect (libp2p)
StreamA -> NodeB: NewStream /opencloud/resource/heartbeat/partner/1.0
StreamA -> NodeB: json.Encode(Heartbeat{Name_A, DID_A, PeerID_A})
NodeB -> StreamB: HandlePartnerHeartbeat(stream)
StreamB -> StreamB: CheckHeartbeat → bandwidth challenge
StreamB -> StreamA: Echo(payload)
StreamB -> StreamB: streams[ProtocolHeartbeatPartner][PeerID_A] = {DID_A, Expiry=now+10s}
StreamA -> StreamA: streams[ProtocolHeartbeatPartner][PeerID_B] = {DID_B, Expiry=now+10s}
note over StreamA,StreamB: Stream partner long-lived établi\ndans les deux sens
else peer B.Relation != PARTNER (révocation / blacklist)
note over NodeA: Supprimer tous les streams vers Pair B
loop Pour chaque protocole dans Streams
note over StreamA,StreamB: Stream partner long-lived établi\nbi-directionnal
else peer B.Relation != PARTNER (revoke / blacklist)
note over NodeA: Suppress all streams onto Peer B
loop For every Streams
NodeA -> StreamA: streams[proto][PeerID_B].Stream.Close()
NodeA -> StreamA: delete(streams[proto], PeerID_B)
end
end
NodeA -> StreamA: Mu.Unlock()
NodeA -> DBA: (pas de write direct ici — géré par l'app source)
NodeA -> DBA: (no write — only app source manually add peer)
@enduml

66
docs/diagrams/09_nats_propagation.mmd
View File

@@ -1,66 +0,0 @@
sequenceDiagram
title NATS — PROPALGATION_EVENT : Pair A propage vers Pair B
participant AppA as App Pair A
participant NATSA as NATS A
participant NodeA as Node A
participant StreamA as StreamService A
participant NodeB as Node B
participant NATSB as NATS B
participant DBB as DB Pair B (oc-lib)
AppA->>NATSA: Publish(PROPALGATION_EVENT, {Action, DataType, Payload})
NATSA->>NodeA: ListenNATS callback → PROPALGATION_EVENT
NodeA->>NodeA: resp.FromApp != "oc-discovery" ? → continuer
NodeA->>NodeA: json.Unmarshal → PropalgationMessage{Action, DataType, Payload}
alt Action == PB_DELETE
NodeA->>StreamA: ToPartnerPublishEvent(PB_DELETE, dt, user, payload)
StreamA->>StreamA: searchPeer(PARTNER) → [Pair B, ...]
StreamA->>NodeB: write(PeerID_B, addr_B, dt, user, payload, ProtocolDeleteResource)
Note over NodeB: /opencloud/resource/delete/1.0
NodeB->>NodeB: handleEventFromPartner(evt, ProtocolDeleteResource)
NodeB->>NATSB: SetNATSPub(REMOVE_RESOURCE, {DataType, resource JSON})
NATSB->>DBB: Supprimer ressource dans DB B
else Action == PB_UPDATE (via ProtocolUpdateResource)
NodeA->>StreamA: ToPartnerPublishEvent(PB_UPDATE, dt, user, payload)
StreamA->>NodeB: write → /opencloud/resource/update/1.0
NodeB->>NATSB: SetNATSPub(CREATE_RESOURCE, {DataType, resource JSON})
NATSB->>DBB: Upsert ressource dans DB B
else Action == PB_CONSIDERS + WORKFLOW_EXECUTION
NodeA->>NodeA: Unmarshal → executionConsidersPayload{PeerIDs:[PeerID_B, ...]}
loop Pour chaque peer_id cible
NodeA->>StreamA: PublishCommon(dt, user, PeerID_B, ProtocolConsidersResource, payload)
StreamA->>NodeB: write → /opencloud/resource/considers/1.0
NodeB->>NodeB: passConsidering(evt)
NodeB->>NATSB: SetNATSPub(PROPALGATION_EVENT, {PB_CONSIDERS, dt, payload})
NATSB->>DBB: (traité par oc-workflow sur NATS B)
end
else Action == PB_PLANNER (broadcast)
NodeA->>NodeA: Unmarshal → {peer_id: nil, ...payload}
loop Pour chaque stream ProtocolSendPlanner ouvert
NodeA->>StreamA: PublishCommon(nil, user, pid, ProtocolSendPlanner, payload)
StreamA->>NodeB: write → /opencloud/resource/planner/1.0
end
else Action == PB_CLOSE_PLANNER
NodeA->>NodeA: Unmarshal → {peer_id: PeerID_B}
NodeA->>StreamA: Streams[ProtocolSendPlanner][PeerID_B].Stream.Close()
NodeA->>StreamA: delete(Streams[ProtocolSendPlanner], PeerID_B)
else Action == PB_SEARCH + DataType == PEER
NodeA->>NodeA: Unmarshal → {search: "..."}
NodeA->>NodeA: GetPeerRecord(ctx, search)
Note over NodeA: Résolution via DB A + Indexer + DHT
NodeA->>NATSA: SetNATSPub(SEARCH_EVENT, {PEER, PeerRecord JSON})
NATSA->>NATSA: (AppA reçoit le résultat)
else Action == PB_SEARCH + autre DataType
NodeA->>NodeA: Unmarshal → {type:"all"|"known"|"partner", search:"..."}
NodeA->>NodeA: PubSubService.SearchPublishEvent(ctx, dt, type, user, search)
Note over NodeA: Voir diagrammes 10 et 11
end

49
docs/diagrams/09_nats_propagation.puml
View File

@@ -1,50 +1,55 @@
@startuml
title NATS — PROPALGATION_EVENT : Pair A propage vers Pair B
title NATS — PROPALGATION_EVENT : Peer A propalgate to Peer B lookup
participant "App Pair A" as AppA
participant "NATS A" as NATSA
participant "Node A" as NodeA
participant "StreamService A" as StreamA
participant "Node B" as NodeB
participant "Node Partner B" as PeerB
participant "Node C" as PeerC
participant "NATS B" as NATSB
participant "DB Pair B (oc-lib)" as DBB
note over App: only our proper resource (db data) can be propalgate : creator_id==self
AppA -> NATSA: Publish(PROPALGATION_EVENT, {Action, DataType, Payload})
NATSA -> NodeA: ListenNATS callback → PROPALGATION_EVENT
NodeA -> NodeA: resp.FromApp != "oc-discovery" ? → continuer
NodeA -> NodeA: propalgate from himself ? → no, continue
NodeA -> NodeA: json.Unmarshal → PropalgationMessage{Action, DataType, Payload}
alt Action == PB_DELETE
NodeA -> StreamA: ToPartnerPublishEvent(PB_DELETE, dt, user, payload)
StreamA -> StreamA: searchPeer(PARTNER) → [Pair B, ...]
StreamA -> StreamA: searchPeer(PARTNER) → [Peer Partner B, ...]
StreamA -> NodeB: write(PeerID_B, addr_B, dt, user, payload, ProtocolDeleteResource)
note over NodeB: /opencloud/resource/delete/1.0
NodeB -> NodeB: handleEventFromPartner(evt, ProtocolDeleteResource)
NodeB -> NATSB: SetNATSPub(REMOVE_RESOURCE, {DataType, resource JSON})
NATSB -> DBB: Supprimer ressource dans DB B
NATSB -> DBB: Suppress ressource into DB B
else Action == PB_UPDATE (via ProtocolUpdateResource)
else Action == PB_UPDATE (per ProtocolUpdateResource)
NodeA -> StreamA: ToPartnerPublishEvent(PB_UPDATE, dt, user, payload)
StreamA -> StreamA: searchPeer(PARTNER) → [Peer Partner B, ...]
StreamA -> NodeB: write → /opencloud/resource/update/1.0
NodeB -> NATSB: SetNATSPub(CREATE_RESOURCE, {DataType, resource JSON})
NATSB -> DBB: Upsert ressource dans DB B
else Action == PB_CONSIDERS + WORKFLOW_EXECUTION
else Action == PB_CREATE (per ProtocolCreateResource)
NodeA -> StreamA: ToPartnerPublishEvent(PB_UPDATE, dt, user, payload)
StreamA -> StreamA: searchPeer(PARTNER) → [Peer Partner B, ...]
StreamA -> NodeB: write → /opencloud/resource/create/1.0
NodeB -> NATSB: SetNATSPub(CREATE_RESOURCE, {DataType, resource JSON})
NATSB -> DBB: Create ressource dans DB B
else Action == PB_CONSIDERS (is a considering a previous action, such as planning or creating resource)
NodeA -> NodeA: Unmarshal → executionConsidersPayload{PeerIDs:[PeerID_B, ...]}
loop Pour chaque peer_id cible
loop For every peer_id targeted
NodeA -> StreamA: PublishCommon(dt, user, PeerID_B, ProtocolConsidersResource, payload)
StreamA -> NodeB: write → /opencloud/resource/considers/1.0
NodeB -> NodeB: passConsidering(evt)
NodeB -> NATSB: SetNATSPub(PROPALGATION_EVENT, {PB_CONSIDERS, dt, payload})
NATSB -> DBB: (traité par oc-workflow sur NATS B)
end
else Action == PB_PLANNER (broadcast)
NodeA -> NodeA: Unmarshal → {peer_id: nil, ...payload}
loop Pour chaque stream ProtocolSendPlanner ouvert
NodeA -> StreamA: PublishCommon(nil, user, pid, ProtocolSendPlanner, payload)
StreamA -> NodeB: write → /opencloud/resource/planner/1.0
NATSB -> DBB: (treat per emmitters app of a previous action on NATS B)
end
else Action == PB_CLOSE_PLANNER
@@ -53,16 +58,16 @@ else Action == PB_CLOSE_PLANNER
NodeA -> StreamA: delete(Streams[ProtocolSendPlanner], PeerID_B)
else Action == PB_SEARCH + DataType == PEER
NodeA -> NodeA: Unmarshal → {search: "..."}
NodeA -> NodeA: read → {search: "..."}
NodeA -> NodeA: GetPeerRecord(ctx, search)
note over NodeA: Résolution via DB A + Indexer + DHT
note over NodeA: Resolved per DB A or Indexer + DHT
NodeA -> NATSA: SetNATSPub(SEARCH_EVENT, {PEER, PeerRecord JSON})
NATSA -> NATSA: (AppA reçoit le résultat)
NATSA -> NATSA: (AppA retrieve results)
else Action == PB_SEARCH + autre DataType
NodeA -> NodeA: Unmarshal → {type:"all"|"known"|"partner", search:"..."}
else Action == PB_SEARCH + other DataType
NodeA -> NodeA: read → {type:"all"|"known"|"partner", search:"..."}
NodeA -> NodeA: PubSubService.SearchPublishEvent(ctx, dt, type, user, search)
note over NodeA: Voir diagrammes 10 et 11
note over NodeA: Watch after pubsub_search & stream_search diagrams
end
@enduml

52
docs/diagrams/10_pubsub_search.mmd
View File

@@ -1,52 +0,0 @@
sequenceDiagram
title PubSub — Recherche gossip globale (type "all") : Pair A cherche, Pair B répond
participant AppA as App Pair A
participant NATSA as NATS A
participant NodeA as Node A
participant PubSubA as PubSubService A
participant GossipSub as GossipSub libp2p (mesh)
participant NodeB as Node B
participant PubSubB as PubSubService B
participant DBB as DB Pair B (oc-lib)
participant StreamB as StreamService B
participant StreamA as StreamService A
AppA->>NATSA: Publish(PROPALGATION_EVENT, {PB_SEARCH, type:"all", search:"gpu"})
NATSA->>NodeA: ListenNATS → PB_SEARCH (type "all")
NodeA->>PubSubA: SearchPublishEvent(ctx, dt, "all", user, "gpu")
PubSubA->>PubSubA: publishEvent(PB_SEARCH, user, {search:"gpu"})
PubSubA->>PubSubA: GenerateNodeID() → from = DID_A
PubSubA->>PubSubA: priv_A.Sign(event body) → sig
PubSubA->>PubSubA: Build Event{Type:"search", From:DID_A, Payload:{search:"gpu"}, Sig}
PubSubA->>GossipSub: topic.Join("search")
PubSubA->>GossipSub: topic.Publish(ctx, json(Event))
GossipSub-->>NodeB: Message propagé (gossip mesh)
NodeB->>PubSubB: subscribeEvents écoute topic "search#"
PubSubB->>PubSubB: json.Unmarshal → Event{From: DID_A}
PubSubB->>NodeB: GetPeerRecord(ctx, DID_A)
Note over NodeB: Résolution Pair A via DB B ou Indexer
NodeB-->>PubSubB: Peer A {PublicKey_A, Relation, ...}
PubSubB->>PubSubB: event.Verify(Peer A) → valide sig_A
PubSubB->>PubSubB: handleEventSearch(ctx, evt, PB_SEARCH)
PubSubB->>StreamB: SendResponse(Peer A, evt)
StreamB->>DBB: Search(COMPUTE + STORAGE + ..., filters{creator=self, access=PUBLIC OR partnerships[PeerID_A]}, search="gpu")
DBB-->>StreamB: [Resource1, Resource2, ...]
loop Pour chaque ressource matchée
StreamB->>StreamB: write(PeerID_A, addr_A, dt, resource JSON, ProtocolSearchResource)
StreamB->>StreamA: NewStream /opencloud/resource/search/1.0
StreamB->>StreamA: json.Encode(Event{Type:search, From:DID_B, DataType, Payload:resource})
end
StreamA->>StreamA: readLoop → handleEvent(ProtocolSearchResource, evt)
StreamA->>StreamA: retrieveResponse(evt)
StreamA->>NATSA: SetNATSPub(SEARCH_EVENT, {DataType, resource JSON})
NATSA->>AppA: Résultats de recherche de Pair B

40
docs/diagrams/10_pubsub_search.puml
View File

@@ -1,54 +1,58 @@
@startuml
title PubSub — Recherche gossip globale (type "all") : Pair A cherche, Pair B répond
title PubSub — Gossip Global search (type "all") : Peer A searching, Peer B answering
participant "App Pair A" as AppA
participant "App UI A" as UIA
participant "App Peer A" as AppA
participant "NATS A" as NATSA
participant "Node A" as NodeA
participant "StreamService A" as StreamA
participant "PubSubService A" as PubSubA
participant "GossipSub libp2p (mesh)" as GossipSub
participant "Node B" as NodeB
participant "PubSubService B" as PubSubB
participant "DB Pair B (oc-lib)" as DBB
participant "DB Peer B (oc-lib)" as DBB
participant "StreamService B" as StreamB
participant "StreamService A" as StreamA
AppA -> NATSA: Publish(PROPALGATION_EVENT, {PB_SEARCH, type:"all", search:"gpu"})
UIA -> AppA: websocket subscription, sending {type:"all", search:"search"} in query
AppA -> NATSA: Publish(PROPALGATION_EVENT, {PB_SEARCH, type:"all", search:"search"})
NATSA -> NodeA: ListenNATS → PB_SEARCH (type "all")
NodeA -> PubSubA: SearchPublishEvent(ctx, dt, "all", user, "gpu")
PubSubA -> PubSubA: publishEvent(PB_SEARCH, user, {search:"gpu"})
PubSubA -> PubSubA: GenerateNodeID() → from = DID_A
NodeA -> PubSubA: SearchPublishEvent(ctx, dt, "all", user, "search")
PubSubA -> PubSubA: publishEvent(PB_SEARCH, user, {search:"search"})
PubSubA -> PubSubA: priv_A.Sign(event body) → sig
PubSubA -> PubSubA: Build Event{Type:"search", From:DID_A, Payload:{search:"gpu"}, Sig}
PubSubA -> PubSubA: Build Event{Type:"search", From:DID_A, Payload:{search:"search"}, Sig}
PubSubA -> GossipSub: topic.Join("search")
PubSubA -> GossipSub: topic.Publish(ctx, json(Event))
GossipSub --> NodeB: Message propagé (gossip mesh)
GossipSub --> NodeB: Propalgate message (gossip mesh)
NodeB -> PubSubB: subscribeEvents écoute topic "search#"
PubSubB -> PubSubB: json.Unmarshal → Event{From: DID_A}
NodeB -> PubSubB: subscribeEvents listen to topic "search#"
PubSubB -> PubSubB: read → Event{From: DID_A}
PubSubB -> NodeB: GetPeerRecord(ctx, DID_A)
note over NodeB: Résolution Pair A via DB B ou Indexer
note over NodeB: Resolve Peer A per DB B or ask to Indexer
NodeB --> PubSubB: Peer A {PublicKey_A, Relation, ...}
PubSubB -> PubSubB: event.Verify(Peer A) → valide sig_A
PubSubB -> PubSubB: event.Verify(Peer A) → valid sig_A
PubSubB -> PubSubB: handleEventSearch(ctx, evt, PB_SEARCH)
PubSubB -> StreamB: SendResponse(Peer A, evt)
StreamB -> DBB: Search(COMPUTE + STORAGE + ..., filters{creator=self, access=PUBLIC OR partnerships[PeerID_A]}, search="gpu")
StreamB -> DBB: Search(COMPUTE + STORAGE + ..., filters{creator=self, access=PUBLIC OR partnerships[PeerID_A]}, search="search")
DBB --> StreamB: [Resource1, Resource2, ...]
loop Pour chaque ressource matchée
loop For every matching resource, only match our own resource creator_id=self_did
StreamB -> StreamB: write(PeerID_A, addr_A, dt, resource JSON, ProtocolSearchResource)
StreamB -> StreamA: NewStream /opencloud/resource/search/1.0
StreamB -> StreamA: json.Encode(Event{Type:search, From:DID_B, DataType, Payload:resource})
StreamB -> StreamA: stream.Encode(Event{Type:search, From:DID_B, DataType, Payload:resource})
end
StreamA -> StreamA: readLoop → handleEvent(ProtocolSearchResource, evt)
StreamA -> StreamA: retrieveResponse(evt)
StreamA -> NATSA: SetNATSPub(SEARCH_EVENT, {DataType, resource JSON})
NATSA -> AppA: Résultats de recherche de Pair B
NATSA -> AppA: Search results from Peer B
AppA -> UIA: emit on websocket
@enduml

52
docs/diagrams/11_stream_search.mmd
View File

@@ -1,52 +0,0 @@
sequenceDiagram
title Stream — Recherche directe (type "known"/"partner") : Pair A → Pair B
participant AppA as App Pair A
participant NATSA as NATS A
participant NodeA as Node A
participant PubSubA as PubSubService A
participant StreamA as StreamService A
participant DBA as DB Pair A (oc-lib)
participant NodeB as Node B
participant StreamB as StreamService B
participant DBB as DB Pair B (oc-lib)
AppA->>NATSA: Publish(PROPALGATION_EVENT, {PB_SEARCH, type:"partner", search:"gpu"})
NATSA->>NodeA: ListenNATS → PB_SEARCH (type "partner")
NodeA->>PubSubA: SearchPublishEvent(ctx, dt, "partner", user, "gpu")
PubSubA->>StreamA: SearchPartnersPublishEvent(dt, user, "gpu")
StreamA->>DBA: Search(PEER, PARTNER) + PeerIDS config
DBA-->>StreamA: [Peer B, ...]
loop Pour chaque pair partenaire (Pair B)
StreamA->>StreamA: json.Marshal({search:"gpu"}) → payload
StreamA->>StreamA: write(PeerID_B, addr_B, dt, user, payload, ProtocolSearchResource)
StreamA->>NodeB: TempStream /opencloud/resource/search/1.0
StreamA->>NodeB: json.Encode(Event{Type:search, From:DID_A, DataType, Payload:{search:"gpu"}})
NodeB->>StreamB: HandleResponse(stream) → readLoop
StreamB->>StreamB: handleEvent(ProtocolSearchResource, evt)
StreamB->>StreamB: handleEventFromPartner(evt, ProtocolSearchResource)
alt evt.DataType == -1 (toutes ressources)
StreamB->>DBA: Search(PEER, evt.From=DID_A)
Note over StreamB: Résolution locale ou via GetPeerRecord
StreamB->>StreamB: SendResponse(Peer A, evt)
StreamB->>DBB: Search(ALL_RESOURCES, filter{creator=B + public OR partner A + search:"gpu"})
DBB-->>StreamB: [Resource1, Resource2, ...]
else evt.DataType spécifié
StreamB->>DBB: Search(DataType, filter{creator=B + access + search:"gpu"})
DBB-->>StreamB: [Resource1, ...]
end
loop Pour chaque ressource
StreamB->>StreamA: write(PeerID_A, addr_A, dt, resource JSON, ProtocolSearchResource)
StreamA->>StreamA: readLoop → handleEvent(ProtocolSearchResource, evt)
StreamA->>StreamA: retrieveResponse(evt)
StreamA->>NATSA: SetNATSPub(SEARCH_EVENT, {DataType, resource JSON})
NATSA->>AppA: Résultat de Pair B
end
end
Note over NATSA,DBA: Optionnel: App A persiste<br/>les ressources découvertes dans DB A

18
docs/diagrams/11_stream_search.puml
View File

@@ -1,6 +1,7 @@
@startuml
title Stream — Recherche directe (type "known"/"partner") : Pair A → Pair B
title Stream — Direct search (type "known"/"partner") : Peer A → Peer B
participant "App UI A" as UIA
participant "App Pair A" as AppA
participant "NATS A" as NATSA
participant "Node A" as NodeA
@@ -11,6 +12,8 @@ participant "Node B" as NodeB
participant "StreamService B" as StreamB
participant "DB Pair B (oc-lib)" as DBB
UIA -> AppA: websocket subscription, sending {type:"all", search:"search"} in query
AppA -> NATSA: Publish(PROPALGATION_EVENT, {PB_SEARCH, type:"partner", search:"gpu"})
NATSA -> NodeA: ListenNATS → PB_SEARCH (type "partner")
NodeA -> PubSubA: SearchPublishEvent(ctx, dt, "partner", user, "gpu")
@@ -20,10 +23,9 @@ StreamA -> DBA: Search(PEER, PARTNER) + PeerIDS config
DBA --> StreamA: [Peer B, ...]
loop Pour chaque pair partenaire (Pair B)
StreamA -> StreamA: json.Marshal({search:"gpu"}) → payload
StreamA -> StreamA: write(PeerID_B, addr_B, dt, user, payload, ProtocolSearchResource)
StreamA -> NodeB: TempStream /opencloud/resource/search/1.0
StreamA -> NodeB: json.Encode(Event{Type:search, From:DID_A, DataType, Payload:{search:"gpu"}})
StreamA -> NodeB: stream.Encode(Event{Type:search, From:DID_A, DataType, Payload:{search:"gpu"}})
NodeB -> StreamB: HandleResponse(stream) → readLoop
StreamB -> StreamB: handleEvent(ProtocolSearchResource, evt)
@@ -31,11 +33,11 @@ loop Pour chaque pair partenaire (Pair B)
alt evt.DataType == -1 (toutes ressources)
StreamB -> DBA: Search(PEER, evt.From=DID_A)
note over StreamB: Résolution locale ou via GetPeerRecord
note over StreamB: Local Resolving (DB) or GetPeerRecord (Indexer Way)
StreamB -> StreamB: SendResponse(Peer A, evt)
StreamB -> DBB: Search(ALL_RESOURCES, filter{creator=B + public OR partner A + search:"gpu"})
DBB --> StreamB: [Resource1, Resource2, ...]
else evt.DataType spécifié
else evt.DataType specified
StreamB -> DBB: Search(DataType, filter{creator=B + access + search:"gpu"})
DBB --> StreamB: [Resource1, ...]
end
@@ -45,10 +47,8 @@ loop Pour chaque pair partenaire (Pair B)
StreamA -> StreamA: readLoop → handleEvent(ProtocolSearchResource, evt)
StreamA -> StreamA: retrieveResponse(evt)
StreamA -> NATSA: SetNATSPub(SEARCH_EVENT, {DataType, resource JSON})
NATSA -> AppA: Résultat de Pair B
NATSA -> AppA: Peer B results
AppA -> UIA: emit on websocket
end
end
note over NATSA,DBA: Optionnel: App A persiste\nles ressources découvertes dans DB A
@enduml

58
docs/diagrams/12_partner_heartbeat.mmd
View File

@@ -1,58 +0,0 @@
sequenceDiagram
title Stream — Partner Heartbeat et propagation CRUD Pair A ↔ Pair B
participant DBA as DB Pair A (oc-lib)
participant StreamA as StreamService A
participant NodeA as Node A
participant NodeB as Node B
participant StreamB as StreamService B
participant NATSB as NATS B
participant DBB as DB Pair B (oc-lib)
participant NATSA as NATS A
Note over StreamA: Démarrage → connectToPartners()
StreamA->>DBA: Search(PEER, PARTNER) + PeerIDS config
DBA-->>StreamA: [Peer B, ...]
StreamA->>NodeB: Connect (libp2p)
StreamA->>NodeB: NewStream /opencloud/resource/heartbeat/partner/1.0
StreamA->>NodeB: json.Encode(Heartbeat{Name_A, DID_A, PeerID_A, IndexersBinded_A})
NodeB->>StreamB: HandlePartnerHeartbeat(stream)
StreamB->>StreamB: CheckHeartbeat → bandwidth challenge
StreamB->>StreamA: Echo(payload)
StreamB->>StreamB: streams[ProtocolHeartbeatPartner][PeerID_A] = {DID_A, Expiry=now+10s}
StreamA->>StreamA: streams[ProtocolHeartbeatPartner][PeerID_B] = {DID_B, Expiry=now+10s}
Note over StreamA,StreamB: Stream partner long-lived établi<br/>GC toutes les 8s (StreamService A)<br/>GC toutes les 30s (StreamService B)
Note over NATSA: Pair A reçoit PROPALGATION_EVENT{PB_DELETE, dt:"storage", payload:res}
NATSA->>NodeA: ListenNATS → ToPartnerPublishEvent(PB_DELETE, dt, user, payload)
NodeA->>StreamA: ToPartnerPublishEvent(ctx, PB_DELETE, dt_storage, user, payload)
alt dt == PEER (mise à jour relation partenaire)
StreamA->>StreamA: json.Unmarshal → peer.Peer B updated
alt B.Relation == PARTNER
StreamA->>NodeB: ConnectToPartner(B.StreamAddress)
Note over StreamA,NodeB: Reconnexion heartbeat si relation upgrade
else B.Relation != PARTNER
loop Tous les protocoles
StreamA->>StreamA: delete(streams[proto][PeerID_B])
StreamA->>NodeB: (streams fermés)
end
end
else dt != PEER (ressource ordinaire)
StreamA->>DBA: Search(PEER, PARTNER) → [Pair B, ...]
loop Pour chaque protocole partner (Create/Update/Delete)
StreamA->>NodeB: write(PeerID_B, addr_B, dt, user, payload, ProtocolDeleteResource)
Note over NodeB: /opencloud/resource/delete/1.0
NodeB->>StreamB: HandleResponse → readLoop
StreamB->>StreamB: handleEventFromPartner(evt, ProtocolDeleteResource)
StreamB->>NATSB: SetNATSPub(REMOVE_RESOURCE, {DataType, resource JSON})
NATSB->>DBB: Supprimer ressource dans DB B
end
end

49
docs/diagrams/13_planner_flow.mmd
View File

@@ -1,49 +0,0 @@
sequenceDiagram
title Stream — Session Planner : Pair A demande le plan de Pair B
participant AppA as App Pair A (oc-booking)
participant NATSA as NATS A
participant NodeA as Node A
participant StreamA as StreamService A
participant NodeB as Node B
participant StreamB as StreamService B
participant DBB as DB Pair B (oc-lib)
participant NATSB as NATS B
%% Ouverture session planner
AppA->>NATSA: Publish(PROPALGATION_EVENT, {PB_PLANNER, peer_id:PeerID_B, payload:{}})
NATSA->>NodeA: ListenNATS → PB_PLANNER
NodeA->>NodeA: Unmarshal → {peer_id: PeerID_B, payload: {}}
NodeA->>StreamA: PublishCommon(nil, user, PeerID_B, ProtocolSendPlanner, {})
Note over StreamA: WaitResponse=true, TTL=24h<br/>Stream long-lived vers Pair B
StreamA->>NodeB: TempStream /opencloud/resource/planner/1.0
StreamA->>NodeB: json.Encode(Event{Type:planner, From:DID_A, Payload:{}})
NodeB->>StreamB: HandleResponse → readLoop(ProtocolSendPlanner)
StreamB->>StreamB: handleEvent(ProtocolSendPlanner, evt)
StreamB->>StreamB: sendPlanner(evt)
alt evt.Payload vide (requête initiale)
StreamB->>DBB: planner.GenerateShallow(AdminRequest)
DBB-->>StreamB: plan (shallow booking plan de Pair B)
StreamB->>StreamA: PublishCommon(nil, user, DID_A, ProtocolSendPlanner, planJSON)
StreamA->>NodeA: json.Encode(Event{plan de B})
NodeA->>NATSA: (forwardé à AppA via SEARCH_EVENT ou PLANNER event)
NATSA->>AppA: Plan de Pair B
else evt.Payload non vide (mise à jour planner)
StreamB->>StreamB: m["peer_id"] = evt.From (DID_A)
StreamB->>NATSB: SetNATSPub(PROPALGATION_EVENT, {PB_PLANNER, peer_id:DID_A, payload:plan})
NATSB->>DBB: (oc-booking traite le plan sur NATS B)
end
%% Fermeture session planner
AppA->>NATSA: Publish(PROPALGATION_EVENT, {PB_CLOSE_PLANNER, peer_id:PeerID_B})
NATSA->>NodeA: ListenNATS → PB_CLOSE_PLANNER
NodeA->>NodeA: Unmarshal → {peer_id: PeerID_B}
NodeA->>StreamA: Mu.Lock()
NodeA->>StreamA: Streams[ProtocolSendPlanner][PeerID_B].Stream.Close()
NodeA->>StreamA: delete(Streams[ProtocolSendPlanner], PeerID_B)
NodeA->>StreamA: Mu.Unlock()
Note over StreamA,NodeB: Stream planner fermé — session terminée

59
docs/diagrams/14_native_offload_gc.mmd
View File

@@ -1,59 +0,0 @@
sequenceDiagram
title Native Indexer — Boucles background (offload, DHT refresh, GC streams)
participant IndexerA as Indexer A (enregistré)
participant IndexerB as Indexer B (enregistré)
participant Native as Native Indexer
participant DHT as DHT Kademlia
participant NodeA as Node A (responsible peer)
Note over Native: runOffloadLoop — toutes les 30s
loop Toutes les 30s
Native->>Native: len(responsiblePeers) > 0 ?
Note over Native: responsiblePeers = peers pour lesquels<br/>le native a fait selfDelegate (aucun indexer dispo)
alt Des responsible peers existent (ex: Node A)
Native->>Native: reachableLiveIndexers()
Note over Native: Filtre liveIndexers par TTL<br/>ping PeerIsAlive pour chaque candidat
alt Indexers A et B maintenant joignables
Native->>Native: responsiblePeers = {} (libère Node A et autres)
Note over Native: Node A se reconnectera<br/>au prochain ConnectToNatives
else Toujours aucun indexer
Note over Native: Node A reste sous la responsabilité du native
end
end
end
Note over Native: refreshIndexersFromDHT — toutes les 30s
loop Toutes les 30s
Native->>Native: Collecter tous les knownPeerIDs<br/>= {PeerID_A, PeerID_B, ...}
loop Pour chaque PeerID connu
Native->>Native: liveIndexers[PeerID] encore frais ?
alt Entrée manquante ou expirée
Native->>DHT: SearchValue(ctx 5s, "/indexer/"+PeerID)
DHT-->>Native: channel de bytes
loop Pour chaque résultat DHT
Native->>Native: Unmarshal → liveIndexerEntry
Native->>Native: Garder le meilleur (ExpiresAt le plus récent, valide)
end
Native->>Native: liveIndexers[PeerID] = best entry
Note over Native: "native: refreshed indexer from DHT"
end
end
end
Note over Native: LongLivedStreamRecordedService GC — toutes les 30s
loop Toutes les 30s
Native->>Native: gc() — lock StreamRecords[Heartbeat]
loop Pour chaque StreamRecord (Indexer A, B, ...)
Native->>Native: now > rec.Expiry ?<br/>OU timeSince(LastSeen) > 2×TTL restant ?
alt Pair périmé (ex: Indexer B disparu)
Native->>Native: Supprimer Indexer B de TOUS les maps de protocoles
Note over Native: Stream heartbeat fermé<br/>liveIndexers[PeerID_B] expirera naturellement
end
end
end
Note over IndexerA: Indexer A continue à heartbeater normalement<br/>et reste dans StreamRecords + liveIndexers

49
docs/diagrams/15_archi_config_nominale.puml Normal file
View File

@@ -0,0 +1,49 @@
@startuml 15_archi_config_nominale
skinparam componentStyle rectangle
skinparam backgroundColor white
skinparam defaultTextAlignment center
title C1 — Topologie nominale\n2 natifs · 2 indexeurs · 2 nœuds
package "Couche 1 — Mesh natif" #E8F4FD {
component "Native A\n(hub autoritaire)" as NA #AED6F1
component "Native B\n(hub autoritaire)" as NB #AED6F1
NA <--> NB : heartbeat /opencloud/heartbeat/1.0 (20s)\n+ gossip PubSub oc-indexer-registry
}
package "Couche 2 — Indexeurs" #E9F7EF {
component "Indexer A\n(DHT server)" as IA #A9DFBF
component "Indexer B\n(DHT server)" as IB #A9DFBF
}
package "Couche 3 — Nœuds" #FEFBD8 {
component "Node 1" as N1 #FAF0BE
component "Node 2" as N2 #FAF0BE
}
' Enregistrements (one-shot, 60s)
IA -[#117A65]--> NA : subscribe signé (60s)\n/opencloud/native/subscribe/1.0
IA -[#117A65]--> NB : subscribe signé (60s)
IB -[#117A65]--> NA : subscribe signé (60s)
IB -[#117A65]--> NB : subscribe signé (60s)
' Heartbeats indexeurs → natifs (long-lived, 20s)
IA -[#27AE60]..> NA : heartbeat (20s)
IA -[#27AE60]..> NB : heartbeat (20s)
IB -[#27AE60]..> NA : heartbeat (20s)
IB -[#27AE60]..> NB : heartbeat (20s)
' Heartbeats nœuds → indexeurs (long-lived, 20s)
N1 -[#E67E22]--> IA : heartbeat long-lived (20s)\n/opencloud/heartbeat/1.0
N1 -[#E67E22]--> IB : heartbeat long-lived (20s)
N2 -[#E67E22]--> IA : heartbeat long-lived (20s)
N2 -[#E67E22]--> IB : heartbeat long-lived (20s)
note as Legend
Légende :
──► enregistrement one-shot (signé)
···► heartbeat long-lived (20s)
──► heartbeat nœud → indexeur (20s)
end note
@enduml

38
docs/diagrams/16_archi_config_seed.puml Normal file
View File

@@ -0,0 +1,38 @@
@startuml 16_archi_config_seed
skinparam componentStyle rectangle
skinparam backgroundColor white
skinparam defaultTextAlignment center
title C2 — Mode seed (sans natif)\nIndexerAddresses seuls · AdmittedAt = zero
package "Couche 2 — Indexeurs seeds" #E9F7EF {
component "Indexer A\n(seed, AdmittedAt=0)" as IA #A9DFBF
component "Indexer B\n(seed, AdmittedAt=0)" as IB #A9DFBF
}
package "Couche 3 — Nœuds" #FEFBD8 {
component "Node 1" as N1 #FAF0BE
component "Node 2" as N2 #FAF0BE
}
note as NNative #FFDDDD
Aucun natif configuré.
AdmittedAt = zero → IsStableVoter() = false
Phase 2 sans votants : Phase 1 conservée directement.
Risque D20 : circularité du trust (seeds se valident mutuellement).
end note
' Heartbeats nœuds → indexeurs seeds
N1 -[#E67E22]--> IA : heartbeat long-lived (20s)
N1 -[#E67E22]--> IB : heartbeat long-lived (20s)
N2 -[#E67E22]--> IA : heartbeat long-lived (20s)
N2 -[#E67E22]--> IB : heartbeat long-lived (20s)
note bottom of IA
Après 2s : goroutine async
fetchNativeFromIndexers → ?
Si natif trouvé → ConnectToNatives (upgrade vers C1)
Si non → mode indexeur pur (D20 actif)
end note
@enduml

63
docs/diagrams/17_startup_consensus_phase1_phase2.puml Normal file
View File

@@ -0,0 +1,63 @@
@startuml 17_startup_consensus_phase1_phase2
title Démarrage avec natifs — Phase 1 (admission) + Phase 2 (vivacité)
participant "Node / Indexer\n(appelant)" as Caller
participant "Native A" as NA
participant "Native B" as NB
participant "Indexer A" as IA
participant "Indexer B" as IB
note over Caller: ConnectToNatives()\nNativeIndexerAddresses configuré
== Étape 0 : heartbeat vers le mesh natif ==
Caller -> NA: SendHeartbeat /opencloud/heartbeat/1.0 (goroutine longue durée)
Caller -> NB: SendHeartbeat /opencloud/heartbeat/1.0 (goroutine longue durée)
== Étape 1 : fetch pool en parallèle ==
par Fetch parallèle (timeout 6s)
Caller -> NA: GET /opencloud/native/indexers/1.0\nGetIndexersRequest{Count: max, FillRates demandés}
NA -> NA: reachableLiveIndexers()\ntri par w(F) = fillRate×(1−fillRate)
NA --> Caller: GetIndexersResponse{Indexers:[IA,IB], FillRates:{IA:0.3, IB:0.6}}
else
Caller -> NB: GET /opencloud/native/indexers/1.0
NB -> NB: reachableLiveIndexers()
NB --> Caller: GetIndexersResponse{Indexers:[IA,IB], FillRates:{IA:0.3, IB:0.6}}
end par
note over Caller: Fusion + dédup → candidates = [IA, IB]\nisFallback = false
== Étape 2a : Phase 1 — Admission native (clientSideConsensus) ==
par Consensus parallèle (timeout 3s par natif, 4s total)
Caller -> NA: /opencloud/native/consensus/1.0\nConsensusRequest{Candidates:[IA,IB]}
NA -> NA: croiser avec liveIndexers
NA --> Caller: ConsensusResponse{Trusted:[IA,IB], Suggestions:[]}
else
Caller -> NB: /opencloud/native/consensus/1.0\nConsensusRequest{Candidates:[IA,IB]}
NB -> NB: croiser avec liveIndexers
NB --> Caller: ConsensusResponse{Trusted:[IA], Suggestions:[IC]}
end par
note over Caller: IA → 2/2 votes → confirmé ✓\nIB → 1/2 vote → refusé ✗\nadmittedAt = time.Now()
== Étape 2b : Phase 2 — Liveness vote (indexerLivenessVote) ==
note over Caller: Cherche votants stables dans StaticIndexerMeta\n(AdmittedAt != zero, age >= MinStableAge=2min)
alt Votants stables disponibles
par Phase 2 parallèle (timeout 3s)
Caller -> IA: /opencloud/indexer/consensus/1.0\nIndexerConsensusRequest{Candidates:[IA]}
IA -> IA: vérifier StreamRecords[ProtocolHB][candidate]\nLastSeen ≤ 2×60s && LastScore ≥ 30
IA --> Caller: IndexerConsensusResponse{Alive:[IA]}
end par
note over Caller: IA confirmé vivant par quorum > 0.5
else Aucun votant stable (premier démarrage)
note over Caller: Phase 1 conservée directement\n(aucun votant MinStableAge atteint)
end
== Étape 3 : remplacement StaticIndexers ==
Caller -> Caller: replaceStaticIndexers(pool={IA}, admittedAt)\nStaticIndexerMeta[IA].AdmittedAt = time.Now()
== Étape 4 : heartbeat long-lived vers pool ==
Caller -> IA: SendHeartbeat /opencloud/heartbeat/1.0 (goroutine longue durée)
note over Caller: Pool actif. NudgeIndexerHeartbeat()
@enduml

51
docs/diagrams/18_startup_seed_discovers_native.puml Normal file
View File

@@ -0,0 +1,51 @@
@startuml 18_startup_seed_discovers_native
title C2 → C1 — Seed découvre un natif (upgrade async)
participant "Node / Indexer\\n(seed mode)" as Caller
participant "Indexer A\\n(seed)" as IA
participant "Indexer B\\n(seed)" as IB
participant "Native A\\n(découvert)" as NA
note over Caller: Démarrage sans NativeIndexerAddresses\\nStaticIndexers = [IA, IB] (AdmittedAt=0)
== Phase initiale seed ==
Caller -> IA: SendHeartbeat /opencloud/heartbeat/1.0 (goroutine longue durée)
Caller -> IB: SendHeartbeat /opencloud/heartbeat/1.0 (goroutine longue durée)
note over Caller: Pool actif en mode seed.\\nIsStableVoter() = false (AdmittedAt=0)\\nPhase 2 sans votants → Phase 1 conservée.
== Goroutine async après 2s ==
note over Caller: time.Sleep(2s)\\nfetchNativeFromIndexers()
Caller -> IA: GET /opencloud/indexer/natives/1.0
IA --> Caller: GetNativesResponse{Natives:[NA]}
note over Caller: Natif découvert : NA\\nAppel ConnectToNatives([NA])
== Upgrade vers mode nominal (ConnectToNatives) ==
Caller -> NA: SendHeartbeat /opencloud/heartbeat/1.0 (goroutine longue durée)
par Fetch pool depuis natif (timeout 6s)
Caller -> NA: GET /opencloud/native/indexers/1.0\\nGetIndexersRequest{Count: max}
NA -> NA: reachableLiveIndexers()\\ntri par w(F) = fillRate×(1−fillRate)
NA --> Caller: GetIndexersResponse{Indexers:[IA,IB], FillRates:{IA:0.4, IB:0.6}}
end par
note over Caller: candidates = [IA, IB], isFallback = false
par Consensus Phase 1 (timeout 3s)
Caller -> NA: /opencloud/native/consensus/1.0\\nConsensusRequest{Candidates:[IA,IB]}
NA -> NA: croiser avec liveIndexers
NA --> Caller: ConsensusResponse{Trusted:[IA,IB], Suggestions:[]}
end par
note over Caller: IA ✓ IB ✓ (1/1 vote)\\nadmittedAt = time.Now()
note over Caller: Aucun votant stable (AdmittedAt vient d'être posé)\\nPhase 2 sautée → Phase 1 conservée directement
== Remplacement pool ==
Caller -> Caller: replaceStaticIndexers(pool={IA,IB}, admittedAt)\\nStaticIndexerMeta[IA].AdmittedAt = time.Now()\\nStaticIndexerMeta[IB].AdmittedAt = time.Now()
note over Caller: Pool upgradé dans la map partagée StaticIndexers.\\nLa goroutine heartbeat existante (démarrée en mode seed)\\ndétecte les nouveaux membres sur le prochain tick (20s).\\nAucune nouvelle goroutine créée.\\nIsStableVoter() deviendra true après MinStableAge (2min).\\nD20 (circularité seeds) éliminé.
@enduml

55
docs/diagrams/19_failure_indexer_crash.puml Normal file
View File

@@ -0,0 +1,55 @@
@startuml failure_indexer_crash
title Indexer Failure → replenish from a Native
participant "Node" as N
participant "Indexer A (alive)" as IA
participant "Indexer B (crashed)" as IB
participant "Native A" as NA
participant "Native B" as NB
note over N: Active Pool : Indexers = [IA, IB]\\nActive Heartbeat long-lived from IA & IB
== IB Failure ==
IB ->x N: heartbeat fails (sendHeartbeat err)
note over N: doTick() dans SendHeartbeat triggers failure\\n→ delete(Indexers[IB])\\n→ delete(IndexerMeta[IB])\\nUnique heartbeat goroutine continue
N -> N: go replenishIndexersFromNative(need=1)
note over N: Reduced Pool to 1 indexers.\\nReplenish triggers with goroutine.
== Replenish from natives ==
par Fetch pool (timeout 6s)
N -> NA: GET /opencloud/native/indexers/1.0\\nGetIndexersRequest{Count: max}
NA -> NA: reachableLiveIndexers()\\n(IB absent because of a expired heartbeat)
NA --> N: GetIndexersResponse{Indexers:[IA,IC], FillRates:{IA:0.4,IC:0.2}}
else
N -> NB: GET /opencloud/native/indexers/1.0
NB --> N: GetIndexersResponse{Indexers:[IA,IC]}
end par
note over N: Fusion + duplication → candidates = [IA, IC]\\n(IA already in pool → IC new candidate)
par Consensus Phase 1 (timeout 4s)
N -> NA: /opencloud/native/consensus/1.0\\nConsensusRequest{Candidates:[IA,IC]}
NA --> N: ConsensusResponse{Trusted:[IA,IC]}
else
N -> NB: /opencloud/native/consensus/1.0
NB --> N: ConsensusResponse{Trusted:[IA,IC]}
end par
note over N: IC → 2/2 votes → admit\\nadmittedAt = time.Now()
par Phase 2 — liveness vote (if stable voters )
N -> IA: /opencloud/indexer/consensus/1.0\\nIndexerConsensusRequest{Candidates:[IC]}
IA -> IA: StreamRecords[ProtocolHB][IC]\\nLastSeen ≤ 120s && LastScore ≥ 30
IA --> N: IndexerConsensusResponse{Alive:[IC]}
end par
note over N: IC confirmed alive → add to pool
N -> N: replaceStaticIndexers(pool={IA,IC})
N -> IC: SendHeartbeat /opencloud/heartbeat/1.0 (goroutine long-live)
note over N: Pool restaured to 2 indexers.
@enduml

51
docs/diagrams/20_failure_indexers_native_falback.puml Normal file
View File

@@ -0,0 +1,51 @@
@startuml failure_indexers_native_falback
title indexers failures → native IsSelfFallback
participant "Node" as N
participant "Indexer A (crashed)" as IA
participant "Indexer B (crashed)" as IB
participant "Native A" as NA
participant "Native B" as NB
note over N: Active Pool : Indexers = [IA, IB]
== Successive Failures on IA & IB ==
IA ->x N: heartbeat failure (sendHeartbeat err)
IB ->x N: heartbeat failure (sendHeartbeat err)
note over N: doTick() in SendHeartbeat triggers failures\\n→ delete(StaticIndexers[IA]), delete(StaticIndexers[IB])\\n→ delete(StaticIndexerMeta[IA/IB])\\n unique heartbeat goroutine continue.
N -> N: go replenishIndexersFromNative(need=2)
== Replenish attempt — natives switches to self-fallback mode ==
par Fetch from natives (timeout 6s)
N -> NA: GET /opencloud/native/indexers/1.0
NA -> NA: reachableLiveIndexers() → 0 alive indexer\\nFallback : included as himself(IsSelfFallback=true)
NA --> N: GetIndexersResponse{Indexers:[NA_addr], IsSelfFallback:true}
else
N -> NB: GET /opencloud/native/indexers/1.0
NB --> N: GetIndexersResponse{Indexers:[NB_addr], IsSelfFallback:true}
end par
note over N: isFallback=true → resolvePool avoids consensus\\nadmittedAt = time.Time{} (zero)\\nStaticIndexers = {NA_addr} (native as fallback)
N -> NA: SendHeartbeat /opencloud/heartbeat/1.0\\n(native as temporary fallback indexers)
note over NA: responsiblePeers[N] registered.\\nrunOffloadLoop look after real indexers.
== Reprise IA → runOffloadLoop native side ==
IA -> NA: /opencloud/native/subscribe/1.0\\nIndexerRegistration{FillRate: 0}
note over NA: liveIndexers[IA] updated.\\nrunOffloadLoop triggers a real available indexer\\migrate from N to IA.
== Replenish on next heartbeat tick ==
N -> NA: GET /opencloud/native/indexers/1.0
NA --> N: GetIndexersResponse{Indexers:[IA], IsSelfFallback:false}
note over N: isFallback=false → Classic Phase 1 + Phase 2
N -> N: replaceStaticIndexers(pool={IA}, admittedAt)
N -> IA: SendHeartbeat /opencloud/heartbeat/1.0
note over N: Pool restaured. Native self extracted as indexer.
@enduml

46
docs/diagrams/21_failure_native_one_down.puml Normal file
View File

@@ -0,0 +1,46 @@
@startuml failure_native_one_down
title Native failure, with one still alive
participant "Indexer A" as IA
participant "Indexer B" as IB
participant "Native A (crashed)" as NA
participant "Native B (alive)" as NB
participant "Node" as N
note over IA, NB: Native State : IA, IB heartbeats to NA & NB
== Native A Failure ==
NA ->x IA: stream reset
NA ->x IB: stream reset
NA ->x N: stream reset (heartbeat Node → NA)
== Indexers side : replenishNativesFromPeers ==
note over IA: SendHeartbeat(NA) détecte reset\\nAfterDelete(NA)\\nStaticNatives = [NB] (still 1)
IA -> IA: replenishNativesFromPeers()\\nphase 1 : fetchNativeFromNatives
IA -> NB: GET /opencloud/native/peers/1.0
NB --> IA: GetPeersResponse{Peers:[NC]} /' new native if one known '/
alt NC disponible
IA -> NC: SendHeartbeat /opencloud/heartbeat/1.0\\nSubscribe /opencloud/native/subscribe/1.0
note over IA: StaticNatives = [NB, NC]\\nNative Pool restored.
else Aucun peer natif
IA -> IA: fetchNativeFromIndexers()\\nAsk to any indexers their natives
IB --> IA: GetNativesResponse{Natives:[]} /' IB also only got NB '/
note over IA: Impossible to find a 2e native.\\nStaticNatives = [NB] (degraded but alive).
end
== Node side : alive indexers pool ==
note over N: Node heartbeats to IA & IB.\\nNA Failure does not affect indexers pool.\\nFuture Consensus did not use NB (1/1 vote = quorum OK).
N -> NB: /opencloud/native/consensus/1.0\\nConsensusRequest{Candidates:[IA,IB]}
NB --> N: ConsensusResponse{Trusted:[IA,IB]}
note over N: Consensus 1/1 alive natif → admit.\\nAuto downgrade of the consensus floor (alive majority).
== NB side : heartbeat to NA fails ==
note over NB: EnsureNativePeers / SendHeartbeat to NA\\nfail (sendHeartbeat err)\\n→ delete(StaticNatives[NA])\\nreplenishNativesFromPeers(NA) triggers
note over NB: Mesh natif downgraded to NB alone.\\Downgraded but functionnal.
@enduml

60
docs/diagrams/22_failure_both_natives.puml Normal file
View File

@@ -0,0 +1,60 @@
@startuml 22_failure_both_natives
title F4 — Panne des 2 natifs → fallback pool pré-validé
participant "Node" as N
participant "Indexer A\\n(vivant)" as IA
participant "Indexer B\\n(vivant)" as IB
participant "Native A\\n(crashé)" as NA
participant "Native B\\n(crashé)" as NB
note over N: Pool actif : StaticIndexers = [IA, IB]\\nStaticNatives = [NA, NB]\\nAdmittedAt[IA] et AdmittedAt[IB] posés (stables)
== Panne simultanée NA et NB ==
NA ->x N: stream reset
NB ->x N: stream reset
N -> N: AfterDelete(NA) + AfterDelete(NB)\\nStaticNatives = {} (vide)
== replenishNativesFromPeers (sans résultat) ==
N -> N: fetchNativeFromNatives() → aucun natif vivant
N -> IA: GET /opencloud/indexer/natives/1.0
IA --> N: GetNativesResponse{Natives:[NA,NB]}
note over N: NA et NB connus mais non joignables.\\nAucun nouveau natif trouvé.
== Fallback : pool d'indexeurs conservé ==
note over N: isFallback = true\\nStaticIndexers conservé tel quel [IA, IB]\\n(dernier pool validé avec AdmittedAt != zero)\\nRisque D19 atténué : quorum natif = 0 → fallback accepté
note over N: Heartbeats IA et IB continuent normalement.\\nPool d'indexeurs opérationnel sans natifs.
N -> IA: SendHeartbeat /opencloud/heartbeat/1.0 (continue)
N -> IB: SendHeartbeat /opencloud/heartbeat/1.0 (continue)
== retryLostNative (30s ticker) ==
loop toutes les 30s
N -> N: retryLostNative()\\ntente reconnexion NA et NB
N -> NA: dial (échec)
N -> NB: dial (échec)
note over N: Retry sans résultat.\\nPool indexeurs maintenu en fallback.
end
== Reprise natifs ==
NA -> NA: redémarrage
NB -> NB: redémarrage
N -> NA: dial (succès)
N -> NA: SendHeartbeat /opencloud/heartbeat/1.0
N -> NB: SendHeartbeat /opencloud/heartbeat/1.0
note over N: StaticNatives = [NA, NB] restauré\\nisFallback = false
== Re-consensus pool indexeurs (optionnel) ==
par Consensus Phase 1
N -> NA: /opencloud/native/consensus/1.0\\nConsensusRequest{Candidates:[IA,IB]}
NA --> N: ConsensusResponse{Trusted:[IA,IB]}
else
N -> NB: /opencloud/native/consensus/1.0
NB --> N: ConsensusResponse{Trusted:[IA,IB]}
end par
note over N: Pool [IA,IB] reconfirmé.\\nisFallback = false. AdmittedAt[IA,IB] rafraîchi.
@enduml

63
docs/diagrams/23_failure_native_plus_indexer.puml Normal file
View File

@@ -0,0 +1,63 @@
@startuml 23_failure_native_plus_indexer
title F5 — Panne combinée : 1 natif + 1 indexeur
participant "Node" as N
participant "Indexer A\\n(vivant)" as IA
participant "Indexer B\\n(crashé)" as IB
participant "Native A\\n(vivant)" as NA
participant "Native B\\n(crashé)" as NB
note over N: Pool nominal : StaticIndexers=[IA,IB], StaticNatives=[NA,NB]
== Pannes simultanées NB + IB ==
NB ->x N: stream reset
IB ->x N: stream reset
N -> N: AfterDelete(NB) — StaticNatives = [NA]
N -> N: AfterDelete(IB) — StaticIndexers = [IA]
== Replenish natif (1 vivant) ==
N -> N: replenishNativesFromPeers()
N -> NA: GET /opencloud/native/peers/1.0
NA --> N: GetPeersResponse{Peers:[]} /' NB seul pair, disparu '/
note over N: Aucun natif alternatif.\\nStaticNatives = [NA] — dégradé.
== Replenish indexeur depuis NA ==
par Fetch pool (timeout 6s)
N -> NA: GET /opencloud/native/indexers/1.0
NA -> NA: reachableLiveIndexers()\\n(IB absent — heartbeat expiré)
NA --> N: GetIndexersResponse{Indexers:[IA,IC], FillRates:{IA:0.5,IC:0.3}}
end par
note over N: candidates = [IA, IC]
par Consensus Phase 1 — 1 seul natif vivant (timeout 3s)
N -> NA: /opencloud/native/consensus/1.0\\nConsensusRequest{Candidates:[IA,IC]}
NA --> N: ConsensusResponse{Trusted:[IA,IC]}
end par
note over N: IC → 1/1 vote → admis (quorum sur vivants)\\nadmittedAt = time.Now()
par Phase 2 liveness vote
N -> IA: /opencloud/indexer/consensus/1.0\\nIndexerConsensusRequest{Candidates:[IC]}
IA -> IA: StreamRecords[ProtocolHB][IC]\\nLastSeen ≤ 120s && LastScore ≥ 30
IA --> N: IndexerConsensusResponse{Alive:[IC]}
end par
N -> N: replaceStaticIndexers(pool={IA,IC})
N -> IC: SendHeartbeat /opencloud/heartbeat/1.0
note over N: Pool restauré à [IA,IC].\\nMode dégradé : 1 natif seulement.\\nretryLostNative(NB) actif (30s ticker).
== retryLostNative pour NB ==
loop toutes les 30s
N -> NB: dial (échec)
end
NB -> NB: redémarrage
NB -> NA: heartbeat (mesh natif reconstruit)
N -> NB: dial (succès)
N -> NB: SendHeartbeat /opencloud/heartbeat/1.0
note over N: StaticNatives = [NA,NB] restauré.\\nMode nominal retrouvé.
@enduml

45
docs/diagrams/24_failure_retry_lost_native.puml Normal file
View File

@@ -0,0 +1,45 @@
@startuml 24_failure_retry_lost_native
title F6 — retryLostNative : reconnexion natif après panne réseau
participant "Node / Indexer" as Caller
participant "Native A\\n(vivant)" as NA
participant "Native B\\n(réseau instable)" as NB
note over Caller: StaticNatives = [NA, NB]\\nHeartbeats actifs vers NA et NB
== Panne réseau transitoire vers NB ==
NB ->x Caller: stream reset (timeout réseau)
Caller -> Caller: AfterDelete(NB)\\nStaticNatives = [NA]\\nlostNatives.Store(NB.addr)
== replenishNativesFromPeers — phase 1 ==
Caller -> NA: GET /opencloud/native/peers/1.0
NA --> Caller: GetPeersResponse{Peers:[NB]}
note over Caller: NB connu de NA, tentative de reconnexion directe
Caller -> NB: dial (échec — réseau toujours coupé)
note over Caller: Connexion impossible.\\nPassage en retryLostNative()
== retryLostNative : ticker 30s ==
loop toutes les 30s tant que NB absent
Caller -> Caller: retryLostNative()\\nParcourt lostNatives
Caller -> NB: StartNativeRegistration (dial + heartbeat + subscribe)
NB --> Caller: dial échoue
note over Caller: Retry loggé. Prochain essai dans 30s.
end
== Réseau rétabli ==
note over NB: Réseau rétabli\\nNB de nouveau joignable
Caller -> NB: StartNativeRegistration\\ndial (succès)
Caller -> NB: SendHeartbeat /opencloud/heartbeat/1.0 (goroutine longue durée)
Caller -> NB: /opencloud/native/subscribe/1.0\\nIndexerRegistration{FillRate: fillRateFn()}
NB --> Caller: subscribe ack
Caller -> Caller: lostNatives.Delete(NB.addr)\\nStaticNatives = [NA, NB] restauré
note over Caller: Mode nominal retrouvé.\\nnativeHeartbeatOnce non utilisé (goroutine déjà active pour NA).\\nNouvelle goroutine SendHeartbeat pour NB uniquement.
@enduml

42
docs/diagrams/25_failure_node_gc.puml Normal file
View File

@@ -0,0 +1,42 @@
@startuml 25_failure_node_gc
title F7 — Crash nœud → GC indexeur + AfterDelete
participant "Node\\n(crashé)" as N
participant "Indexer A" as IA
participant "Indexer B" as IB
participant "Native A" as NA
note over N, NA: État nominal : N heartbeatait vers IA et IB
== Crash Node ==
N ->x IA: stream reset (heartbeat coupé)
N ->x IB: stream reset (heartbeat coupé)
== GC côté Indexer A ==
note over IA: HandleHeartbeat : stream reset détecté\\nStreamRecords[ProtocolHB][N].LastSeen figé
loop ticker GC (30s) — StartGC(30*time.Second)
IA -> IA: gc()\\nnow.After(Expiry) où Expiry = lastHBTime + 2min\\n→ si 2min sans heartbeat → éviction
IA -> IA: delete(StreamRecords[ProtocolHB][N])\\nAfterDelete(N, name, did) appelé hors lock
note over IA: N retiré du registre vivant.\\nFillRate recalculé (n-1 / maxNodes).
end
== Impact sur le scoring / fill rate ==
note over IA: FillRate diminue\\nProchain subscribe vers NA inclura FillRate mis à jour
IA -> NA: /opencloud/native/subscribe/1.0\\nIndexerRegistration{FillRate: 0.3} /' était 0.5 '/
NA -> NA: liveIndexerEntry[IA].FillRate = 0.3\\nPriorité de routage recalculée : w(0.3) = 0.21
== Impact sur la Phase 2 (indexerLivenessVote) ==
note over IA: Si un autre nœud demande consensus,\\nN n'est plus dans StreamRecords.\\nN absent de la réponse Alive[].
note over IB: Même GC effectué côté IB.\\nN retiré de StreamRecords[ProtocolHB].
== Reconnexion éventuelle du nœud ==
N -> N: redémarrage
N -> IA: SendHeartbeat /opencloud/heartbeat/1.0\\nHeartbeat{Score: X, IndexersBinded: 2}
IA -> IA: HandleHeartbeat → nouveau UptimeTracker(FirstSeen=now)\\nStreamRecords[ProtocolHB][N] recréé
note over IA: N de retour avec FirstSeen frais.\\ndynamicMinScore élevé tant que age < 24h.
@enduml

80
docs/diagrams/README.md
View File

@@ -1,39 +1,71 @@
# OC-Discovery — Diagrammes de séquence
# OC-Discovery — Diagrammes d'architecture et de séquence
Tous les fichiers `.mmd` sont au format [Mermaid](https://mermaid.js.org/).
Rendu possible via VS Code (extension Mermaid Preview), IntelliJ, ou [mermaid.live](https://mermaid.live).
Tous les fichiers sont au format [PlantUML](https://plantuml.com/).
Rendu possible via VS Code (extension PlantUML), IntelliJ, ou [plantuml.com/plantuml](https://www.plantuml.com/plantuml/uml/).
## Vue d'ensemble des diagrammes
## Diagrammes de séquence (flux internes)
| Fichier | Description |
|---------|-------------|
| `01_node_init.mmd` | Initialisation complète d'un Node (libp2p host, GossipSub, indexers, StreamService, PubSubService, NATS) |
| `02_node_claim.mmd` | Enregistrement du nœud auprès des indexeurs (`claimInfo` + `publishPeerRecord`) |
| `03_indexer_heartbeat.mmd` | Protocole heartbeat avec calcul du score qualité (bande passante, uptime, diversité) |
| `04_indexer_publish.mmd` | Publication d'un `PeerRecord` vers l'indexeur → DHT |
| `05_indexer_get.mmd` | Résolution d'un pair via l'indexeur (`GetPeerRecord` + `handleNodeGet` + DHT) |
| `06_native_registration.mmd` | Enregistrement d'un indexeur auprès d'un Native Indexer + gossip PubSub |
| `07_native_get_consensus.mmd` | `ConnectToNatives` : pool d'indexeurs + protocole de consensus (vote majoritaire) |
| `08_nats_create_resource.mmd` | Handler NATS `CREATE_RESOURCE` : connexion/déconnexion d'un partner |
| `09_nats_propagation.mmd` | Handler NATS `PROPALGATION_EVENT` : delete, considers, planner, search |
| `10_pubsub_search.mmd` | Recherche gossip globale (type `"all"`) via GossipSub |
| `11_stream_search.mmd` | Recherche directe par stream (type `"known"` ou `"partner"`) |
| `12_partner_heartbeat.mmd` | Heartbeat partner + propagation CRUD vers les partenaires |
| `13_planner_flow.mmd` | Session planner (ouverture, échange, fermeture) |
| `14_native_offload_gc.mmd` | Boucles background du Native Indexer (offload, DHT refresh, GC) |
| `01_node_init.puml` | Initialisation complète d'un Node (libp2p host, GossipSub, indexers, StreamService, PubSubService, NATS) |
| `02_node_claim.puml` | Enregistrement du nœud auprès des indexeurs (`claimInfo` + `publishPeerRecord`) |
| `03_indexer_heartbeat.puml` | Protocole heartbeat avec score 5 composants (U/B/D/L/F), UptimeTracker, dynamicMinScore |
| `04_indexer_publish.puml` | Publication d'un `PeerRecord` vers l'indexeur → DHT |
| `05_indexer_get.puml` | Résolution d'un pair via l'indexeur (`GetPeerRecord` + `handleNodeGet` + DHT) |
| `06_native_registration.puml` | Enregistrement d'un indexeur auprès du Native (FillRate, signature, TTL 90s, unsubscribe) |
| `07_native_get_consensus.puml` | `ConnectToNatives` : fetch pool + Phase 1 (clientSideConsensus) + Phase 2 (indexerLivenessVote) |
| `08_nats_create_resource.puml` | Handler NATS `CREATE_RESOURCE` : connexion/déconnexion d'un partner |
| `09_nats_propagation.puml` | Handler NATS `PROPALGATION_EVENT` : delete, considers, planner, search |
| `10_pubsub_search.puml` | Recherche gossip globale (type `"all"`) via GossipSub |
| `11_stream_search.puml` | Recherche directe par stream (type `"known"` ou `"partner"`) |
| `12_partner_heartbeat.puml` | Heartbeat partner + propagation CRUD vers les partenaires |
| `13_planner_flow.puml` | Session planner (ouverture, échange, fermeture) |
| `14_native_offload_gc.puml` | Boucles background du Native Indexer (offload, DHT refresh, GC) |
## Protocoles libp2p utilisés
## Diagrammes de topologie et flux de panne
### Configurations réseau
| Fichier | Description |
|---------|-------------|
| `15_archi_config_nominale.puml` | C1 — Topologie nominale : 2 natifs · 2 indexeurs · 2 nœuds, tous flux |
| `16_archi_config_seed.puml` | C2 — Mode seed sans natif : indexeurs à AdmittedAt=0, risque D20 actif |
### Flux de démarrage
| Fichier | Description |
|---------|-------------|
| `17_startup_consensus_phase1_phase2.puml` | Démarrage nominal : Phase 1 (admission native) + Phase 2 (liveness vote) |
| `18_startup_seed_discovers_native.puml` | Upgrade seed → nominal : goroutine async découvre un natif via l'indexeur |
### Flux de panne
| Fichier | Code | Description |
|---------|------|-------------|
| `19_failure_indexer_crash.puml` | F1 | Panne 1 indexeur → replenish depuis natif → IC admis |
| `20_failure_both_indexers_selfdelegate.puml` | F2 | Panne 2 indexeurs → natif `IsSelfFallback=true`, runOffloadLoop |
| `21_failure_native_one_down.puml` | F3 | Panne 1 natif → quorum 1/1 suffisant, mode dégradé |
| `22_failure_both_natives.puml` | F4 | Panne 2 natifs → fallback pool pré-validé, retryLostNative |
| `23_failure_native_plus_indexer.puml` | F5 | Panne combinée : 1 natif + 1 indexeur → double replenish |
| `24_failure_retry_lost_native.puml` | F6 | Panne réseau transitoire → retryLostNative (30s ticker) |
| `25_failure_node_gc.puml` | F7 | Crash nœud → GC indexeur (120s), AfterDelete, fill rate recalculé |
## Protocoles libp2p utilisés (référence complète)
| Protocole | Description |
|-----------|-------------|
| `/opencloud/heartbeat/1.0` | Heartbeat node → indexeur (long-lived) |
| `/opencloud/heartbeat/indexer/1.0` | Heartbeat indexeur → native (long-lived) |
| `/opencloud/heartbeat/1.0` | Heartbeat universel : node→indexeur, indexeur→native, native→native (long-lived) |
| `/opencloud/probe/1.0` | Sonde de bande passante (echo, mesure latence + débit) |
| `/opencloud/resource/heartbeat/partner/1.0` | Heartbeat node ↔ partner (long-lived) |
| `/opencloud/record/publish/1.0` | Publication `PeerRecord` vers indexeur |
| `/opencloud/record/get/1.0` | Requête `GetPeerRecord` vers indexeur |
| `/opencloud/native/subscribe/1.0` | Enregistrement indexeur auprès du native |
| `/opencloud/native/indexers/1.0` | Requête de pool d'indexeurs au native |
| `/opencloud/native/consensus/1.0` | Validation de pool d'indexeurs (consensus) |
| `/opencloud/native/subscribe/1.0` | Enregistrement indexeur auprès du native (+ FillRate) |
| `/opencloud/native/unsubscribe/1.0` | Désenregistrement explicite indexeur → native |
| `/opencloud/native/indexers/1.0` | Requête de pool d'indexeurs au native (tri par w(F)=F×(1-F)) |
| `/opencloud/native/consensus/1.0` | Phase 1 : validation de pool d'indexeurs (vote majoritaire natifs) |
| `/opencloud/native/peers/1.0` | Demande de pairs natifs connus (replenish mesh natif) |
| `/opencloud/indexer/natives/1.0` | Demande d'adresses de natifs connus par un indexeur |
| `/opencloud/indexer/consensus/1.0` | Phase 2 : liveness vote (LastSeen ≤ 120s && LastScore ≥ 30) |
| `/opencloud/resource/search/1.0` | Recherche de ressources entre peers |
| `/opencloud/resource/create/1.0` | Propagation création ressource vers partner |
| `/opencloud/resource/update/1.0` | Propagation mise à jour ressource vers partner |