We have completed our investigation into the recent issue that only affected customers using high-network-rate sources (e.g., Azure DevOps, on-prem build servers, etc.). We have worked with the cloud vendor to implement a solution.
If you experience any further trouble, our recommendations are:
The Telerik NuGet Server is back to full availability, even for affected environments. We will share a postmortem as soon as possible.
Posted Apr 08, 2024 - 19:57 UTC
Update
The network protection is still in place to ensure server availability throughout the incident. We are actively working with the datacenter's networking team and are continuing to monitor the situation.
We are still continuing to monitor the situation. The service itself is fully operational, however the servers are still behind additional networking protection services and a connection failure may be experienced. This is temporary, but make sure you're using a backoff interval that grows in duration upon each retry attempt.
It is still recommended that you add a packageSourceMapping to any nuget.config(s) in your project(s). This is a long term approach that will dramatically reduce the number of network requests your build is performing against the Telerik servers. See the previous update for a link to a code example.
Posted Apr 05, 2024 - 22:18 UTC
Monitoring
We have widened the network pipeline to the Telerik NuGet servers significantly, this should unblock many folks in those regions. Please try your builds again,
Tips: - MAJOR TIP - Use a packageSourceMapping in your nuget.config to only use the Telerik NuGet Server for Telerik packages. A cross-product example can be found here => https://github.com/LanceMcCarthy/DevOpsExamples/blob/main/src/NuGet.Config (lines 31-47). This can significantly speed up your builds as a bonus.
- If you are triggering it immediately after this update notice and it fails, please consider that others teams may be trying at the same time as well. Add a short delay and try it again, avoid immediately retry it to let the post-update congestion tail off. This is more relevant if you have many parallel builds.
Posted Apr 04, 2024 - 21:34 UTC
Identified
Network protection is currently active to protect the servers from becoming unavailable for all customers.
[Edit] Removed request for individual build machine IPs, this is no longer a valid mitigation approach.
Posted Apr 04, 2024 - 15:53 UTC
This incident affected: Telerik NuGet v3 and Telerik NuGet v2.