Goal

We want to be able to instantiate a service that allows to store file located on a processing pod onto it.

We have already tested it with a static Argo yaml file, a MinIO running on the same kubernetes node, the minio service is reached because it is the only associated to the serviceAccount.

We have established three otpions that need to be available to the user for the feature to be implemented:

Use a MinIO running constantly on the node that executes the argo workflow
Use a MinIO
A MinIO is instanciated when a new workflow is launched

Requirements

Helm : https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
Helm GO client : $ go get github.com/mittwald/go-helm-client
MinIO chart : https://charts.min.io/

Ressources

We need to create several ressources in order for the pods to communicate with the MinIO

MinIO Auth Secrets

Bucket ConfigMap

With the name artifact-repositories this configMap will be used by default. It contains the URL to the MinIO server and the key to the authentication data held in a secret ressource.

apiVersion: v1
kind: ConfigMap
metadata:
  # If you want to use this config map by default, name it "artifact-repositories".
  name: artifact-repositories
  # annotations:
  #   # v3.0 and after - if you want to use a specific key, put that key into this annotation.
  #   workflows.argoproj.io/default-artifact-repository: oc-s3-artifact-repository
data:
  oc-s3-artifact-repository: |
    s3:
      bucket: oc-bucket
      endpoint: [ retrieve cluster with kubectl get service argo-artifacts -o jsonpath="{.spec.clusterIP}" ]:9000
      insecure: true
      accessKeySecret: 
        name: argo-artifact-secret
        key: access-key
      secretKeySecret:
        name: argo-artifact-secret
        key: secret-key

Code modifications

Rajouter un attribut "isDataLink"

true/false

Rajouter un attribut DataPath ou un truc comme ca

liste de map[string]string permet de n'avoir qu'une copie par fichier)
éditable uniquement a travers la méthode addDataPath
clé : path du fichier / value : nom de la copie dans minio

===> on a besoin du meme attribut pour Processing -> Data et Data -> Processing

2.2 KiB Raw Blame History