Make terminationGracePeriodSeconds configurable for Harness Delegate (Helm)
M
Moonstone Vole
Currently we can't configure terminationGracePeriodSeconds in our delegates when installed using Harness Delegate helm chart. We need a longer configurable shutdown time (some of our jobs last up to 1 hour). See https://github.com/harness/delegate-helm-chart/blob/main/harness-delegate-ng/templates/deployment.yaml#L71
In addition to that, there should be an option to set a custom lifecycle hook.
We use linkerd and need to call its shutdown endpoint using an API call once the delegate finished handling its tasks.
lifecycle:
preStop:
exec:
command:
- /bin/sh
- -c
- |
echo "Starting preStop hook: waiting for harness delegate to finish its tasks..."
So to sum it up:
- Configureable terminationGracePeriodSeconds
- Allow configuring a lifecycle hook.
- Make sure http://127.0.0.1:3460/api/health is still responsive after the pod gets a TERM signal and return something like:
{"status":"SUCCESS","data":"task-in-progress","metaData":null,"correlationId":null}
I've seen this metric: io_harness_custom_metric_tasks_currently_executing under /api/metrics but it won't be as convenient to consume.
Or any other solution that will tell us that a task is still running post TERM signal.
Log In