openid & other

This commit is contained in:
ycc 2024-12-10 18:01:58 +01:00
parent 33bfe79f66
commit 91f5f44cea
18 changed files with 688 additions and 20 deletions

View File

@ -1,16 +0,0 @@
# General architecture
Each OpenCloud instance will provide an OpenId interface. This interface may be connected to an existing LDAP Server or a dedicated one.
# User rights definition
Each OpenCloud instance will manage it's users and their permissions :
* a user has permition to start a distributed workflow in using remote peers
* a user has administrative rights and may change the service exchenge rates
* a user is limited to view financial information on the instance
* a user belongs to a group (that may represent a project, a department,...)
# Authentication process
Each OpenCloud peer will accept a company as a whole.
Upon user connection, it will receive user rights form the origninating OpenId connect server and apply them. ex: specific pricing for a group (company agreement, project agreement, ...)

View File

@ -0,0 +1,27 @@
# General architecture
Each OpenCloud instance will provide an OpenId interface. This interface may be connected to an existing LDAP Server or a dedicated one.
The main advanytage of this distributed solution is that each partner will manage it's own iusers and profiles. It simplifies access control management as each peer does not have to be aware of other peers users, but will only define access rules globally for the peers.
# Users / roles / groups
# User permissions definition
Each OpenCloud instance will manage it's users and their permissions :
On a local instance :
* a user has permission to start a distributed workflow in using remote peers
* a user has administrative rights and may change the service exchenge rates
* a user is limited to view financial information on the instance
* a user belongs to a group (that may represent a project, a department,...)
# Authentication process
Each OpenCloud peer will accept a company as a whole.
Upon user connection, it will receive user rights form the origninating OpenId connect server and apply them. ex: specific pricing for a group (company agreement, project agreement, ...)
A collaborative workspace
# Resources don't have an url
They will map to an internal url of the service
Once a workflow is initialized and ready for launch temporary urls proxying to the real service will be provided to the wokflow at booking time

69
docs/catalog_metadata.md Normal file
View File

@ -0,0 +1,69 @@
# Metadata
``` json
{
"id" : "string"
"name" : "string"
"short description" : "string"
"description" : "string"
"logo" : "string"
"creator" : "peer_id"
"owner(s)" {
"name" : "string"
"logo" : "string"
}
"instances":
[
{
"location" : "geo coord"
"country" : "string"
"url" : "string"
allowed_groups : peers_group
<specific data> see below
}
]
}
```
## Common
* locations: location list
## Data
* personal_data: bool
* anonymized_personal_data: bool
* type: string
* license
* open_data: bool
* quality: string
## ComputeUnit
* infrastructure : Kubernetes, Docker, HW, Slurm, Condor
* architecture : X86_64
* access_protocol : enum(KubeAPI(https) [over SSH], DirectSSH, Slurm [over SSH], Docker [over SSH], over Opencloud, over VPN) or string
* security_level: Secnumcloud, HDS, ... Gaia1/2/3...
* countries
* investors:
* power_source:
* usage_restrictions: string
## Processing
* url: string
* type: container, exe
* license: string
* open_source: bool
* marutity: (stable, alpha,...)
## Storage
* type: File, S3, Database
* url: string
* access_protocol : enum(KubeAPI(https) [over SSH], DirectSSH, Slurm [over SSH], Docker [over SSH], over Opencloud, over VPN) or string
* security_level: Secnumcloud, HDS, ... Gaia1/2/3...
* country:
* investors:
* usage_restrictions: string

259
docs/catalog_metadata.puml Normal file
View File

@ -0,0 +1,259 @@
@startuml
class Resource
{
"id" : "string"
"name" : "string"
"short description" : "string"
"description" : "string"
"logo" : "string"
"creator" : "peer_id"
"usage_restrictions" : "string"
}
class StoragePartnerResource
{
"namespace" : "string"
"sizing_indicator" : "string"
"peer_group"="string"
}
class ProcessingPartnerResource
{
"namespace" : "string"
"sizing_indicator" : "string"
"peer_group"="string"
}
class ComputeUnitPartnerResource
{
"namespace" : "string"
"sizing_indicator" : "string"
"peer_group"="string"
}
class DataPartnerResource
{
"namespace" : "string"
"sizing_indicator" : "string"
"peer_group"="string"
}
class ResourceInstance
{
"location" : "geo coord"
"country" : "string"
"url" : "string"
}
class StorageInstance
{
}
class DataInstance
{
}
class ComputeUnitInstance
{
"cpus":
"gpus":
"ram":
"security_level" : "string"
"power_source" : "string"
}
class cpu
{
"model" : "string"
"cores" : "int"
"frequency" : "float"
"architecture" : "string"
}
class gpu
{
"model" : "string"
"memory" : "float"
}
class ram
{
"size" : "int"
}
class bandwidth
{
"up" : "float"
"down" : "float"
}
class ProcessingInstance
{
}
class Owner
{
"name" : "string"
"logo" : "string"
}
class DataPricingStrategy {
"unlimited"
"subscription"
"pay per use"
}
class DataPricing
{
"price" : "float"
"price_per_gb" : "float"
"price_per_request" : "float"
"price_per_api_call" : "float"
"price_per_data_transfer" : "float"
"price_per_data_download" : "float"
}
class ProcessingPricing
{
"price" : "float"
"price_per_request" : "float"
"price_per_api_call" : "float"
"price_per_data_transfer" : "float"
"price_per_data_processing" : "float"
"price_per_data_storage" : "float"
"price_per_data_download" : "float"
}
class ComputeUnitPricingStrategy {
"overflow" : "booked, allowed, garanted"
}
class ComputeUnitPricing
{
"cpu_price" : "float"
"gpu_price" : "float"
"ram_price" : "float"
"refund" : "bool"
}
class StoragePricing
{
"price" : "float"
"price_per_request" : "float"
"price_per_api_call" : "float"
"price_per_data_transfer" : "float"
"price_per_data_processing" : "float"
"price_per_data_storage" : "float"
"price_per_data_download" : "float"
}
class Data
{
"personal_data" : "bool"
"anonymized_personal_data" : "bool"
"type" : "string"
"license" : "string"
"open_data" : "bool"
"quality" : "string"
"static" : bool
"update_period" : "string"
}
class ComputeUnit
{
"type" : "string"
"infrastructure" : "string"
"architecture" : "string"
"investors" : "string"
}
class Processing {
"type" : "string"
"license" : "string"
"open_source" : "bool"
"maturity" : "string"
"service" : "bool"
}
class Container
{
"image" : "string"
"command" : "string"
"args" : "string"
"env" : "string"
"volumes" : "string"
}
Processing "0" *-- "1" Container
Container "0" *-- "*" Expose
class Expose
{
Port:int
Reverse: "string"
PAT:int
}
class ProcessingUsage
{
"hypothesis" : "string"
"cpu":
"gpu":
"ram":
"storage": "float"
"scalingmodel": "string"
}
class Storage {
"type" : "string"
"security_level" : "string"
"investors" : "string"
"support" : "string"
}
Resource -- Owner
Resource <|- Data
Resource <|- ComputeUnit
Resource <|- Processing
Resource <|- Storage
Resource <|- Workflow
StoragePartnerResource -- StoragePricingStrategy
ProcessingPartnerResource -- ProcessingPricingStrategy
ComputeUnitPartnerResource -- ComputeUnitPricingStrategy
DataPartnerResource -- DataPricingStrategy
StorageInstance "1" *-- "*" StoragePartnerResource
DataInstance "1" *-- "*" DataPartnerResource
ProcessingInstance "1" *-- "*" ProcessingPartnerResource
ComputeUnitInstance "1" *-- "*" ComputeUnitPartnerResource
DataPricingStrategy <|-- DataPricing
ProcessingPricingStrategy <|-- ProcessingPricing
ComputeUnitPricingStrategy <|-- ComputeUnitPricing
StoragePricingStrategy <|-- StoragePricing
ResourceInstance <|-- StorageInstance
ResourceInstance <|-- DataInstance
ResourceInstance <|-- ComputeUnitInstance
ResourceInstance <|-- ProcessingInstance
Storage "1" *-- "*" StorageInstance
Processing "1" *-- "*" ProcessingInstance
ComputeUnit "1" *-- "*" ComputeUnitInstance
Data "1" *-- "*" DataInstance
@enduml

28
docs/cnes-cloud.md Normal file
View File

@ -0,0 +1,28 @@
K8s Trex OK avec cloisonnement alpha
Vm rebond pour attaquer le HPC
2 managers au lieu dans le cluster
Déploiement Helm
Pas de helm,
1 seul namespace
pas d'accès au control plane
=> VCluster
Argo workflow
Contrat à rompre avec ATOS
=> à terme cluster dédié OpenShift
====================
Accès au cluster et tester
========
Compte opérateur de service qui possède les secrets dans le namespace

View File

@ -0,0 +1,13 @@
Rulebook integrated to OpenCloud
Dynamic rulebook update
Intégration d'Ekitia par défaut pour contôle éventuel
Ok from all to start workflow ?
Possibilité de révoquer un traitement par n'importe quel membre ?
blocage workflow si modif critères workspace
attributs sur le workflow ?

View File

@ -32,9 +32,9 @@ In order to build the software :
bee run -downdoc=true -gendoc=true
The -downdoc=true -gendoc=true will automatically generate swagger documentation in teh /swagger path
The -downdoc=true -gendoc=true will automatically generate swagger documentation in the /swagger path
If default Swagger page is displayed instead of tyour api, change url in swagger/index.html file to :
If default Swagger page is displayed instead of your api, change url in swagger/index.html file to :
url: "swagger.json"
@ -54,7 +54,7 @@ The GUI are developped using Flutter framework
* In "Tools"->"SDK Manager"->"Apparenace & Behaviour/System Settings/Android SDK", go to "SDK tools" and tick the "Android SDK command line tools"
* Run <code>flutter doctor</code> commmand and follow instructions to accept SDK licenses
* Add Vscode flutter plugin and use Vscode Command palette to create a Flutter project
* Also set the target Device uring command Palette
* Also set the target Device using command Palette
### Project build

0
docs/discovery.md Normal file
View File

11
docs/opencloud_intro.md Normal file
View File

@ -0,0 +1,11 @@
OpenCloud is an open source distributed cloud solution.
It allows selectively sharing/selling/renting your infrastrucure resources (data, algorithm, compute, storage) with other OpenCloud peers.
It allows distributed workflow execution between partners.
Distributed execution in that peer to peer network can be organized depending on your own priorites :
- maximal sovereingty
- speed up calculation
- minimising production cost
- optimizing your infrasturcutre investments
OpenCloud provides an OpenId based distributed authentication system.
OpenCloud is a fully distributed solution, without any central organization or SPOF.
OpenCloud provides trnasaction tracking, for every partner to be aware of it's distributed resource consumption and ensure peer to peer billing.

104
docs/openid/glossary.md Normal file
View File

@ -0,0 +1,104 @@
# Glossary
# Oauth
## Ressource owner
The user that will allow the app to read ressources that he/she will grant access for
ex: the person that has a mail account
## Client
The application that is requesting the ressources to use them on the behalf of the user
ex : a mass mailing list service to all your contacts
## Authorization server
The application that knows the resource owner because it has an account there
ex: the mail server authentication service
## Resource server
The API that the client will use on behalf of the user
ex : the contact list API
## Redirect uri
Url that will be used by the authorization server to send back the ressource owner to the client app after consenting to ressources access
ex : mass mailing list "contact retrieve success/failure" page
## Response type
Response type expeted by the client, usually "code" for an authorization code
## Scope
Granular permission that the client wants
ex: read contacts, read profile
## Consent
The auhorization server takes the scopes that the clients requests and let the ressource owner choose to acccept them or not
ex: access to your contacts ?
## Client Id
To identify the client with the authorization server
## Client secret
Shared between authorization server and client
## Authorization code
Temporary code sent by authorization server to client
The client then privately sends the authorization code along with the client secret to tha authorization server, in exchange for an access token
## Access token
Key the client will use to communicate withe the ressource server
## Refresh token
Token to get a new access token
# OIDC
## Oauth vs Oidc
Oauth provides only a token for application access without any info on the user. OpenId adds information on the user.
* Oauth enables an app to access ressources
* Oidc enables an app to establish a login session and to access info about the user
## End user
Oauth Resource Owner
## Relaying party
Oauth client
## Identity provider
OIDC enabled Oauth authorization server
## IdToken
JWT token added to access token by OIDC with your identity info.
## Claims
Attributes of the Id Token
* Subject : uid for the user
* Issuing Authority : url of identity provider
* Audience : irdentifies the relying party that can use this token
* Issue Date
* Expiration Date
* [Authentication Time]
* [Nonce] : prevent replay attacks
* [Name]
* [Email]
## Scopes
openid is a mandatory scope
There a are 4 openid predefined scopes :
* profile : access to the default profile claims
* email
* address
* phone
## Identity provider Endpoints
Several predefined endpoints exist on the Identity provider
* Authorization endpoint
* Token endpoint
* UserInfo endpoint
## Recommended authorization flows
* Authorization code
* Authorization code with PKCE (Proof Key for Code Exchange) : for devices
## PKCE

View File

@ -0,0 +1,19 @@
@startuml
"User(ressource owner)"->"RequestingApp(client)": Select mail provider
"RequestingApp(client)"->"User(ressource owner)": Redirect to mail provider with clientid,redirect_uri,response_type,scope
"User(ressource owner)"->"MailProvider(authorization provider)": clientid,redirect_uri,response_type,scope
"MailProvider(authorization provider)"->"MailProvider(authorization provider)": Active session ?
"MailProvider(authorization provider)"-->"User(ressource owner)" : Login if no active session
"User(ressource owner)"-->"MailProvider(authorization provider)" : Logs in
"MailProvider(authorization provider)"->"User(ressource owner)": Asks for consent for each scope
"User(ressource owner)"->"MailProvider(authorization provider)" : Grant or deny permission for each scope
"MailProvider(authorization provider)"->"User(ressource owner)": Redirect to redirect_uri with authorization code
"User(ressource owner)"->"RequestingApp(client)": Redirect to redirect_uri with authorization code
"RequestingApp(client)"->"MailProvider(authorization provider)": Send authorization code, clientid, client_secret
"MailProvider(authorization provider)"->"RequestingApp(client)": Send access token
"RequestingApp(client)"->"MailProvider(resource server)": asks for contacts with access token
"MailProvider(resource server)"->"RequestingApp(client)": Return contacts
"RequestingApp(client)"->"User(ressource owner)": Display contacts
@enduml

View File

@ -0,0 +1,19 @@
@startuml
"User(ressource owner)"->"RequestingApp(client)": Select mail provider
"RequestingApp(client)"->"User(ressource owner)": Redirect to mail provider with clientid,redirect_uri,response_type,scope<font color=red>+"openid"
"User(ressource owner)"->"MailProvider(authorization provider)": clientid,redirect_uri,response_type,scope
"MailProvider(authorization provider)"->"MailProvider(authorization provider)": Active session ?
"MailProvider(authorization provider)"-->"User(ressource owner)" : Login if no active session
"User(ressource owner)"-->"MailProvider(authorization provider)" : Logs in
"MailProvider(authorization provider)"->"User(ressource owner)": Asks for consent for each scope
"User(ressource owner)"->"MailProvider(authorization provider)" : Grant or deny permission for each scope
"MailProvider(authorization provider)"->"User(ressource owner)": Redirect to redirect_uri with authorization code
"User(ressource owner)"->"RequestingApp(client)": Redirect to redirect_uri with authorization code
"RequestingApp(client)"->"MailProvider(authorization provider)": Send authorization code, clientid, client_secret
"MailProvider(authorization provider)"->"RequestingApp(client)": Send access token<font color=red>+"idtoken"
"RequestingApp(client)"->"MailProvider(resource server)": asks for contacts with access token
"MailProvider(resource server)"->"RequestingApp(client)": Return contacts
"RequestingApp(client)"->"User(ressource owner)": Display contacts
@enduml

View File

@ -0,0 +1,25 @@
@startuml
title "OpenID Connect Authorization Code Flow"
actor "End User"
boundary "Browser"
"Relaying party"->"Browser": Identity providers list
"End User"->"Browser": Select identity provider
"Browser"->"Relaying party": Identity provider clicked
"Relaying party"->"Browser": Redirect to identity provider with clientid, state,redirect_uri,response_type,scope<font color=red>+"openid"
"Browser"->"Authorization endpoint": clientid,state,redirect_uri,response_type,scope
"Authorization endpoint"->"Authorization endpoint": Active session ?
"Authorization endpoint"-->"Browser" : Login if no active session
"End User"-->"Browser" : Fills credentials
"Browser"-->"Authorization endpoint" : Logs in
"Authorization endpoint"->"Browser": Form for consent for each scope
"End User"->"Browser": Grant or deny permission for each scope
"Browser"->"Authorization endpoint" :Selected scopes
"Authorization endpoint"->"Browser": Redirect to redirect_uri with authorization code+state provided earlier
"Browser"->"Relaying party": Redirect to redirect_uri with authorization code
"Relaying party"->"Token endpoint": Send authorization code, clientid, client_secret, redirect uri (for validation)
"Token endpoint"->"Relaying party": Send access token<font color=red>+"idtoken"
"Relaying party"->"UserInfo endpoint": Asks for profile with access token
"UserInfo endpoint"->"Relaying party": Return profile
"Relaying party"->"Browser": Display profile
@enduml

View File

@ -0,0 +1,25 @@
@startuml
title "OpenID Connect Authorization Code Flow with PKCE"
actor "End User"
boundary "App"
"App"->"App": Identity providers list
"End User"->"App": Select identity provider
"App"->"App": Identity provider clicked
"App"->"App": Generate code verifier and challenge
"App"->"Authorization endpoint": clientid,state,redirect_uri,response_type,scope
"Authorization endpoint"->"Authorization endpoint": Active session ?
"Authorization endpoint"-->"App" : Login if no active session
"End User"-->"App" : Fills credentials
"App"-->"Authorization endpoint" : Logs in
"Authorization endpoint"->"App": Form for consent for each scope
"End User"->"App": Grant or deny permission for each scope
"App"->"Authorization endpoint" :Selected scopes
"Authorization endpoint"->"App": Redirect to redirect_uri with authorization code+state provided earlier
"App"->"App": Redirect to redirect_uri with authorization code
"App"->"Token endpoint": Send authorization code, clientid, --client_secret--,<font color=blue>+"code verifier"</font> , redirect uri (for validation)
"Token endpoint"->"App": Send access token<font color=red>+"idtoken"
"App"->"UserInfo endpoint": Asks for profile with access token
"UserInfo endpoint"->"App": Return profile
"App"->"App": Display profile
@enduml

View File

@ -0,0 +1,46 @@
@startuml
Actor User
Node "OpenCloud 1" as OC1 {
Agent Traefik as tfk1
Agent Catalog as cat1
Agent Scheduler as shed1
Collections "OC Services" as svcs1
Component "Auth Service" as auth1
Component OIDC as OIDC1
Component "Keto?" as keto1
Component "LDAP" as ldap1
}
User -> tfk1:sessionId
tfk1 ---> cat1:IdToken+AccessToken
tfk1 ---> shed1:IdToken+AccessToken
tfk1 ---> svcs1:IdToken+AccessToken
tfk1 ---> auth1
auth1 -down-> OIDC1
auth1 -down-> keto1
OIDC1 -down-> ldap1
Node "OpenCloud 2" as OC2 {
Agent Traefik as tfk2
Agent Catalog as cat2
Agent Scheduler as shed2
Collections "OC Services" as svcs2
Component "Auth Service" as auth2
Component OIDC as OIDC2
Component "Keto?" as keto2
Component "LDAP" as ldap2
}
cat1 --> tfk2:IdToken+AccessToken
tfk2 ---> cat2:IdToken+AccessToken
tfk2 ---> shed2:IdToken+AccessToken
tfk2 ---> svcs2:IdToken+AccessToken
tfk2 -down-> auth2
auth2 -down-> OIDC2
auth2 -down-> keto2
OIDC2 -down-> ldap2
auth2 -> auth1: validate id & access user groups
auth2 -> tfk2: moderated scopes
@enduml

39
mft.svg Normal file
View File

@ -0,0 +1,39 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"
width="780mm" height="390mm" viewBox="0 0 780 390" displayInline="False">
<defs>
</defs>
<circle cx="48.0" cy="48.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="48.0" cy="144.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="48.0" cy="240.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="48.0" cy="336.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="144.0" cy="48.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="144.0" cy="144.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="144.0" cy="240.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="144.0" cy="336.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="240.0" cy="48.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="240.0" cy="144.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="240.0" cy="240.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="240.0" cy="336.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="336.0" cy="48.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="336.0" cy="144.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="336.0" cy="240.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="336.0" cy="336.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="432.0" cy="48.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="432.0" cy="144.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="432.0" cy="240.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="432.0" cy="336.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="528.0" cy="48.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="528.0" cy="144.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="528.0" cy="240.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="528.0" cy="336.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="624.0" cy="48.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="624.0" cy="144.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="624.0" cy="240.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="624.0" cy="336.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="720.0" cy="48.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="720.0" cy="144.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="720.0" cy="240.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<circle cx="720.0" cy="336.0" r="15" fill="none" stroke="red" stroke-width="0.0275" />
<rect x="0" y="0" width="780" height="390" fill="none" stroke="red" stroke-width="0.0275" />
</svg>

After

Width:  |  Height:  |  Size: 3.0 KiB

View File

@ -8,6 +8,6 @@ Peers/groups (project) having a specific agreement may benefit of custom rates
# Requirements
* An authorized user (specific permission) will be able to define default rates and specific peers rates.
* The default rates shall be accessible to everlonging to they user internal and external.
* The default rates shall be accessible to every internal and external user.
* The custom rates shall be only accessible to users belonging to the relevant peer
*