Support for Sparse Checkout in Harness CD
long-term
D
Dandelion Salamander
Hello,
In our organization we have various teams with exceptionally large repositories. This can cause longer deploy times when a Harness Delegate must re-fetch an entire repository. In some cases, it can cause unexpected failures if the dev team has not set the timeout for their deployment high enough.
In most cases, a full repository checkout is not necessary (i.e. pulling helm overrides). It would be faster to perform a sparse checkout. We understand a sparse checkout is possible by creating a pre-deployment step and then executing the deploy on the same delegate. However, this is not preferred as this creates a single point of failure on the single Harness Delegate. We consider it best practice to not use delegate selectors wherever possible such that any delegate can be selected for a job.
With that background, we would like to request that Harness support a sparse checkout directly in the deploy steps such that the sparse checkout does not limit the deployment to a single specific Harness delegate.
Log In
Autopilot
Merged in a post:
Deploy stage should have some option to download git repo to utilize custom scripts
C
Coral Swift
We have some pipelines that need to execute a large number of PowerShell scripts—around hundreds of files. We attempted to manage this using a Harnes Service with Git config options, but due to the high file count, cloning the repository failed. Ideally, we need a solution that allows us to either clone specific folders or defer cloning the full repository until the deployment stage.
Rohan Gupta
marked this post as
long-term
S
Sole Chimpanzee
We want to do this on CI pipelines without disabling the cloning and without this caveat from the suggested work-around:
Disabling the built-in clone step removes access to some associated functionality, such as PR status updates or resolution of <+codebase.
> expressions.
Rohan Gupta
marked this post as
pending feedback
N
Novel Cod
Rohan Gupta this sparse checkout and recursive features are very useful to be included as part of codebase . Using a seperate clone or git step will not give the PR status update as well as not allow codebase variables to be used elsewhere in pipeline. Pls consider a RFE to allow sparse checkout,depth and recursive feature in codebase setting. The optimized git fetch will not give the same capability as sparse checkout.
Rohan Gupta
Hi @Dandelion Salamander has the team tried the optimized git fetch files capability?
D
Dandelion Salamander
Rohan Gupta - Hi Rohan, do you have any details or documentation I can review on this functionality.
Rohan Gupta
Dandelion Salamander https://developer.harness.io/docs/platform/connectors/code-repositories/connect-to-code-repo/#network-connection-times-out-when-fetching-large-repos can you check this out? Will this help?