Coming from jenkins, we had the ability to queue certain jobs. We want the ability to queue jobs more efficiently at the Org and Project level.

User need an option in Harness to configure the max threshold. Here are the changes we are looking for in Harness

Need the ability to configure Maximum threshold in Queue Step.

Option to skip the First-IN Executions of more than the max threshold pipeline executions Queued. Example: Max threshold is 3, the moment any pipeline execution beyond 3 reaches the first one (First -In execution, the request must be ejected, and the latest one needs to be queued.

Hi Team,

1000 concurrent executions per account isn't sufficient for us as per project we are going to run more than 1000 concurrent executions.

Hello everyone,

I would like to suggest an enhancement that allows the "Concurrent Active Pipeline Executions" option to be project-specific, with the ability to inherit the value from the account while still being modifiable at the project level. This enhancement would provide us with increased flexibility in configuration, enabling diverse projects to have distinct infrastructure requirements.

By incorporating this feature, we could manage concurrency differently across various projects, addressing their unique needs and enhancing overall system versatility.

I appreciate your consideration of this proposal and look forward to any feedback or further discussion on this matter.

Best regards,

Miraldo Ramos

Use case:

You are deploying 20 services in a single pipeline to multiple environment/IS definition. Say Prod and DR.

The max concurrency is hardcoded to Env X services = 40

Parallelism is good but with a concurrency control option.

Pls provide a concurrency option either in UI or as property in yaml edit directly.

We would like the ability to queue build executions.

We are using the harness pipeline queue to ensure we lock pipelines while deploying/testing.

Our tests can run up to 1hr, and multiple PRs can come in during that time.

We would like the ability to set a maximum jobs allowed in the queue, and cancel jobs if there are multiple in the queue.

Example

1(processing) <- 2(canceled) <- 3(canceled) <- 4(waiting)

If another one comes in while 1 is processing it would become

1(processing) <- 2(canceled) <- 3(canceled) <- 4(canceled) <- 5(waiting)

As far as I can tell this isn't possible with any harness features?

I'd like to request adding a "Count" or "Size" option to the Queue step. I have a few pipelines where I want “no more than 3 running simultaneously,” but not a strict single-item queue. The pipelines are very complex and expensive, so I want to limit their concurrency to reduce our risks of "Common mode failure, all pipelines will fail". If we limit it to 3 at a time, we reduce our exposure and spend less money by cancelling the next N number of pipelines that are guaranteed to fail