66 lines
2.2 KiB
Markdown
66 lines
2.2 KiB
Markdown
# Goal
|
|
|
|
We want to be able to instantiate a service that allows to store file located on a `processing` pod onto it.
|
|
|
|
We have already tested it with a static `Argo` yaml file, a MinIO running on the same kubernetes node, the minio service is reached because it is the only associated to the `serviceAccount`.
|
|
|
|
We have established three otpions that need to be available to the user for the feature to be implemented:
|
|
|
|
- Use a MinIO running constantly on the node that executes the argo workflow
|
|
- Use a MinIO
|
|
- A MinIO is instanciated when a new workflow is launched
|
|
|
|
# Requirements
|
|
|
|
- Helm : `https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3`
|
|
- Helm GO client : `$ go get github.com/mittwald/go-helm-client`
|
|
- MinIO chart : `https://charts.min.io/`
|
|
|
|
|
|
# Ressources
|
|
|
|
We need to create several ressources in order for the pods to communicate with the MinIO
|
|
|
|
## MinIO Auth Secrets
|
|
|
|
## Bucket ConfigMap
|
|
|
|
With the name `artifact-repositories` this configMap will be used by default. It contains the URL to the MinIO server and the key to the authentication data held in a `secret` ressource.
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: ConfigMap
|
|
metadata:
|
|
# If you want to use this config map by default, name it "artifact-repositories".
|
|
name: artifact-repositories
|
|
# annotations:
|
|
# # v3.0 and after - if you want to use a specific key, put that key into this annotation.
|
|
# workflows.argoproj.io/default-artifact-repository: oc-s3-artifact-repository
|
|
data:
|
|
oc-s3-artifact-repository: |
|
|
s3:
|
|
bucket: oc-bucket
|
|
endpoint: [ retrieve cluster with kubectl get service argo-artifacts -o jsonpath="{.spec.clusterIP}" ]:9000
|
|
insecure: true
|
|
accessKeySecret:
|
|
name: argo-artifact-secret
|
|
key: access-key
|
|
secretKeySecret:
|
|
name: argo-artifact-secret
|
|
key: secret-key
|
|
|
|
```
|
|
|
|
|
|
# Code modifications
|
|
|
|
Rajouter un attribut "isDataLink"
|
|
- true/false
|
|
|
|
Rajouter un attribut DataPath ou un truc comme ca
|
|
|
|
- liste de map[string]string permet de n'avoir qu'une copie par fichier)
|
|
- éditable uniquement a travers la méthode addDataPath
|
|
- clé : path du fichier / value : nom de la copie dans minio
|
|
|
|
===> on a besoin du meme attribut pour Processing -> Data et Data -> Processing |