When pipelines become stuck, the recommendation from support is currently to "ABORT" the pipeline. The consequence of this is that the deployment is stopped midway (e.g. canaries which have not been taken down, cleanup which is not performed, etc.). At a small(er) scale, this might be possible, but it is not feasible for either large workloads or cases where multiple services are affected.
Please see corresponding support request: https://support.harness.io/hc/en-us/requests/121220
In this feature request, we ask for a better recovery solution for cases where pipelines become stuck. The pipeline should be able to resume from the stuck stage, either through a one-off stage retry, or auto-resume.