tag:status.telerik.com,2005:/historyTelerik Status - Incident History2024-03-29T05:22:46ZTeleriktag:status.telerik.com,2005:Incident/202692802024-03-18T17:57:12Z2024-03-18T17:57:12ZInvestigating Cloud Networking Interruptions<p><small>Mar <var data-var='date'>18</var>, <var data-var='time'>17:57</var> UTC</small><br><strong>Resolved</strong> - After several days of monitoring and investigation, the cloud-to-cloud networking problem from South Central US and West US appears to be resolved.<br /><br />As we were not able to determine a specific root cause, we ask that you please open a support ticket if you experience the problem again. In the ticket, include the results of a ping and tracert for nuget.telerik.com, along with the machine's public IP address. This will allow us to directly investigate that specific connection.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>20:42</var> UTC</small><br><strong>Monitoring</strong> - We have investigated this with the datacenter vendor and they are unable to determine any possible cause with the available information. This status has been moved to Monitoring, both the datacenter and Progress teams will be monitoring closely over the next several days. <br /><br />If you experience this again, please provide us with all of the following information from the affected machine: ping, traceroute, and public IP address.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>18:09</var> UTC</small><br><strong>Update</strong> - We have confirmed there are no issues with the Telerik NuGet services or their firewalls, so the components statuses will revert to Operational for the remainder of this investigation.<br /><br />If you are affected by this issue, please open a support ticket (or reply to your existing case) with your build agent's public IP address. This will allow us to speed up the investigation with the vendor and help restore their connection to your build machines.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>14:42</var> UTC</small><br><strong>Investigating</strong> - The Telerik NuGet servers are operating nominally. However, we are investigating reports of networking issues that are only affecting some customers who are using hosted Azure DevOps agents and AWS build hosts that are located in the West US region. This problem results in a timeout, but only in that environment's networking.<br /><br />At this time, we have not been able to replicate the problem, so we are working with affected customers to try and isolate the issue and determine a solution. This status item will be updated as we learn more information.</p>tag:status.telerik.com,2005:Incident/190353232023-11-06T14:49:33Z2023-11-06T14:49:33ZTelerik.com Your Account - Access Issues<p><small>Nov <var data-var='date'> 6</var>, <var data-var='time'>14:49</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved. If you are still experiencing a problem, please perform a hard refresh of the page. Most browsers perform this if you A) hold the Shift key while clicking the browser's refresh button, or B) use Shift + Ctrl + R key combination.</p><p><small>Nov <var data-var='date'> 6</var>, <var data-var='time'>14:40</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating an issue where attempting to load any page under Your Account (licenses, downloads, support tickets) are unavailable.</p>tag:status.telerik.com,2005:Incident/187555952023-10-11T16:57:48Z2023-10-11T16:59:33ZDegraded Telerik NuGet Services<p><small>Oct <var data-var='date'>11</var>, <var data-var='time'>16:57</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'>11</var>, <var data-var='time'>15:41</var> UTC</small><br><strong>Monitoring</strong> - Some customers may experience issues connectivity to the Telerik NuGet server when searching, restoring or downloading packages. This has been resolved, but we are closely monitoring performance throughout the day.</p>tag:status.telerik.com,2005:Incident/187545552023-10-11T08:00:00Z2023-10-11T14:06:01ZNuGet | Short-Period Degraded Performance<p><small>Oct <var data-var='date'>11</var>, <var data-var='time'>08:00</var> UTC</small><br><strong>Resolved</strong> - The NuGet feed experienced service degradation between 8:18 and 9:20 UTC today. The service is stable now and we are monitoring the performance.</p>tag:status.telerik.com,2005:Incident/185569502023-09-19T17:00:00Z2023-09-20T15:22:29ZAutomatic DDoS Mitigation Triggered<p><small>Sep <var data-var='date'>19</var>, <var data-var='time'>17:00</var> UTC</small><br><strong>Resolved</strong> - On Sept 19th, 2023 there was a small period inaccessibility to the Telerik NuGet servers due to a DDoS attack. Our DDoS protection was triggered and mitigated the issue:<br /><br />Mitigation Start Time (UTC): 2023-09-15T05:36:14.351560+00:00<br />Mitigation Stop Time (UTC): 2023-09-15T05:41:43.805230+00:00<br /><br />All services have returned to normal operation.</p>tag:status.telerik.com,2005:Incident/184911822023-09-13T16:33:24Z2023-09-13T16:36:43ZKendo Dojo - Runner Issue Investigation<p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>16:33</var> UTC</small><br><strong>Resolved</strong> - Public use of Kendo dojo is verified available and stable.</p><p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>15:23</var> UTC</small><br><strong>Identified</strong> - The conditions that cause the problem have been identified; this issue only affects a very small segment of users and only under a very specific circumstance. The issue is still in effect and we are continuing to investigate potential solutions.</p><p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>15:05</var> UTC</small><br><strong>Investigating</strong> - We are current investigating an issue where the Kendo Dojo (dojo.telerik.com) runner is not executing code from the editor.</p>tag:status.telerik.com,2005:Incident/165288332023-03-16T14:38:01Z2023-03-16T14:38:01ZTelerik NuGet v3 Server Degraded Performance<p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>14:38</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>14:16</var> UTC</small><br><strong>Monitoring</strong> - All nodes appear to be healthy. We are monitoring the performance.</p><p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>14:09</var> UTC</small><br><strong>Identified</strong> - We have identified an issue with the NuGet v3 server and are working to resolve it.<br /><br />Possible workarounds are:<br />- Use the NuGet v2 server https://nuget.telerik.com/nuget<br />- Download the required NuGet packages from http://telerik.com/account/downloads/ and use them from a local NuGet feed.</p>tag:status.telerik.com,2005:Incident/165119982023-03-15T21:59:22Z2023-03-15T21:59:22ZTelerik NuGet<p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>21:59</var> UTC</small><br><strong>Resolved</strong> - The service feed has returned to normal operation. Package listing, search and restore are back at full capacity.<br /><br />Important Note: If you are using UI for Blazor, the Telerik.DataSource package requires an additional update. This will occur approximately ~9AM UTC.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>19:54</var> UTC</small><br><strong>Update</strong> - Upgrading to Operational. We are continuing to closely monitor systems so that we can act quickly in the event of any unstable behavior.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>17:43</var> UTC</small><br><strong>Update</strong> - Downgrading to Degraded Performance => When authenticating with or listing packages from the NuGet Server you may receive a timeout or an error stating "unable to load service index".<br /><br />Tips: <br />- If you are experiencing this, wait a few minutes before attempting again.<br />- If you're using automatic retry, try adding a backoff delay.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>15:29</var> UTC</small><br><strong>Monitoring</strong> - The issue has been resolved. We'll be closely monitoring the performance for the next few hours to ensure stability.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>14:12</var> UTC</small><br><strong>Investigating</strong> - You may occasionally get an error "unable to load the service index" when restoring NuGet packages from the Telerik NuGet server. We are currently investigating and will keep you informed.</p>tag:status.telerik.com,2005:Incident/123136812022-10-20T16:05:30Z2022-10-20T16:05:30ZTelerik NuGet Degraded Performance<p><small>Oct <var data-var='date'>20</var>, <var data-var='time'>16:05</var> UTC</small><br><strong>Resolved</strong> - The service has been operating as expected throughout the monitoring period.</p><p><small>Oct <var data-var='date'>19</var>, <var data-var='time'>16:01</var> UTC</small><br><strong>Monitoring</strong> - The servers are operational. We are monitoring the traffic to ensure stability over the next 24 hours.</p><p><small>Oct <var data-var='date'>19</var>, <var data-var='time'>15:40</var> UTC</small><br><strong>Investigating</strong> - We are investigating a newly observed degraded performance issue with the Telerik NuGet servers.</p>tag:status.telerik.com,2005:Incident/114319192022-10-19T15:39:07Z2022-10-19T15:39:07ZTelerik NuGet Servers - Transient Unavailability<p><small>Oct <var data-var='date'>19</var>, <var data-var='time'>15:39</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'>19</var>, <var data-var='time'>15:37</var> UTC</small><br><strong>Update</strong> - We are observing some degraded performance when searching and restoring packages. We are continuing to monitor the situation and taking appropriate actions.</p><p><small>Oct <var data-var='date'>10</var>, <var data-var='time'>13:51</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Oct <var data-var='date'> 5</var>, <var data-var='time'>17:09</var> UTC</small><br><strong>Identified</strong> - We have identified some issues and are working on fixes for them now. Over the next 24 hours, we will be internally testing and validating the fixes before a potential production roll-out tomorrow afternoon.</p><p><small>Oct <var data-var='date'> 4</var>, <var data-var='time'>21:25</var> UTC</small><br><strong>Update</strong> - Notices:<br />1) The v2 server is operational, we will continue to monitor this endpoint.<br />2) We are continuing to investigate the v3 service index issue.</p><p><small>Oct <var data-var='date'> 4</var>, <var data-var='time'>21:05</var> UTC</small><br><strong>Investigating</strong> - We are investigating transient issues that affect a small fraction of users whom are receiving either service index errors from the v3 server; or are getting a 404 from the v2 server.<br /><br />Note: For mission-critical builds, you can always download the .nupkg file of any product from https://www.telerik.com/account/downloads and restore offline.</p>tag:status.telerik.com,2005:Incident/111829702022-09-19T20:42:08Z2022-09-19T20:42:08ZDNS Issues with Telerik Reporting CDN<p><small>Sep <var data-var='date'>19</var>, <var data-var='time'>20:42</var> UTC</small><br><strong>Resolved</strong> - The DNS issue has been fixed, all requests to reporting.cdn.telerik.com will resolve.</p><p><small>Sep <var data-var='date'>19</var>, <var data-var='time'>19:46</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Sep <var data-var='date'>19</var>, <var data-var='time'>19:26</var> UTC</small><br><strong>Investigating</strong> - There is a DNS resolution problem for the new Telerik Reporting CDN (introduced in R3 2022). We are investigating the cause and will update this status as soon as possible.<br /><br />Note: This does not affect the Unpkg or Kendo CDNs, it only affects new implementations of the Telerik ReportViewer that are leveraging CDN endpoints at reporting.cdn.telerik.com.</p>tag:status.telerik.com,2005:Incident/98666852022-04-28T08:09:12Z2022-04-28T08:09:12ZIntermittent NuGet Server Access<p><small>Apr <var data-var='date'>28</var>, <var data-var='time'>08:09</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Apr <var data-var='date'>28</var>, <var data-var='time'>08:08</var> UTC</small><br><strong>Update</strong> - The issue is now resolved and we are monitoring the NuGet servers' performance.</p><p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>20:09</var> UTC</small><br><strong>Update</strong> - Downgraded impact from Major to Minor. The v2 server has been responsive for 2+ hours and the v3 endpoint has been responsive for 6+ hours (at the time of this update).<br /><br />However, the underlying cause of the previous outages has not been identified. We will continue to investigate and monitor over the next 12-24 hours and update this incident as necessary.</p><p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>18:51</var> UTC</small><br><strong>Investigating</strong> - In the past 24 hours there were several 2 minute-duration periods where v2 and v3 NuGet servers did not respond to requests.<br /><br />- If you experienced a timeout error, please wait about 10-15 minutes before trying again to avoid being affected by retry congestion.<br />- If you're using an automated workflow that allows you to configure a retry back-off timer, this is highly recommended.<br /><br />We are investigating the issue and will update the status as the situation evolves. thank you for your patience and understanding.</p>tag:status.telerik.com,2005:Incident/88546842021-12-16T17:00:38Z2021-12-16T17:00:38ZTelerik NuGet Server Availability<p><small>Dec <var data-var='date'>16</var>, <var data-var='time'>17:00</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'>15</var>, <var data-var='time'>21:04</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and services are fully operational. We'll closely monitor the services for a couple hours before closing this incident.</p><p><small>Dec <var data-var='date'>15</var>, <var data-var='time'>16:30</var> UTC</small><br><strong>Identified</strong> - The problem has been identified and the teams are working on a solution. During this time, you may still experience intermittent trouble with any of the following items:<br /><br />- Downloading installers or packages from telerik.com<br />- Visual Studio extensions that rely on downloading product installers or packages from *.telerik.com<br />- Telerik NuGet server package searches and downloads.</p><p><small>Dec <var data-var='date'>15</var>, <var data-var='time'>15:29</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating the cause of intermittent 503 errors from both v2 and v3 Telerik NuGet server endpoints.</p>tag:status.telerik.com,2005:Incident/84689862021-11-11T18:37:05Z2021-11-11T18:42:51ZNuGet Service Disruption (deprecated v2 server only)<p><small>Nov <var data-var='date'>11</var>, <var data-var='time'>18:37</var> UTC</small><br><strong>Resolved</strong> - All Telerik NuGet servers are back at full operation.</p><p><small>Nov <var data-var='date'>11</var>, <var data-var='time'>17:06</var> UTC</small><br><strong>Update</strong> - The cause of disruption was identified and repaired. The servers are currently fulfilling queued requests in the backlog. We will update this status again when operations return to 100%.<br /><br />Note: some new request responses may be slow, which might cause a timeout depending on your client. In such a case, rerun the restore command again after waiting several minutes.</p><p><small>Nov <var data-var='date'>11</var>, <var data-var='time'>17:01</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Nov <var data-var='date'>11</var>, <var data-var='time'>16:08</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating service disruption reports for the deprecated v2 NuGet server (https://nuget.telerik.com/nuget). <br /><br />Note: There are no currently known issues with the new v3 NuGet server (https://nuget.telerik.com/v3/index.json). You can do an in-place swap of the server URLs to regain access to the feed.</p>tag:status.telerik.com,2005:Incident/37687122020-03-23T20:49:50Z2020-03-23T20:49:52ZDojo cannot load saved snippets, nor create new snippets.<p><small>Mar <var data-var='date'>23</var>, <var data-var='time'>20:49</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>23</var>, <var data-var='time'>20:09</var> UTC</small><br><strong>Monitoring</strong> - We have taken additional steps to mitigate the issue and the Core Platform Services are now operational. We will continue to monitor the performance for some time before resolving the incident.</p><p><small>Mar <var data-var='date'>23</var>, <var data-var='time'>16:14</var> UTC</small><br><strong>Identified</strong> - We have taken steps to mitigate the issue, but we are still seeing some instability and performance degradation. We are continuing with our investigation.</p><p><small>Mar <var data-var='date'>23</var>, <var data-var='time'>14:54</var> UTC</small><br><strong>Investigating</strong> - We are experiencing an internal issue and currently, creating and loading code snippets in our dojo (https://dojo.telerik.com/) is not available. We are working on resolving the issue.</p>tag:status.telerik.com,2005:Incident/36991882020-03-17T19:09:09Z2020-03-17T19:09:09ZDojo cannot load saved snippets, nor create new snippets.<p><small>Mar <var data-var='date'>17</var>, <var data-var='time'>19:09</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>17</var>, <var data-var='time'>17:39</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented for the underlying service and Kendo Dojo snippets are successfully being saved and loaded. We are continuing to monitor operation to confirm resolution.</p><p><small>Mar <var data-var='date'>17</var>, <var data-var='time'>16:28</var> UTC</small><br><strong>Investigating</strong> - We are experiencing an internal issue and currently, code snippets creation and loading in our dojo (https://dojo.telerik.com/) are not available. We are working on resolving the issue.</p>tag:status.telerik.com,2005:Incident/36689472020-03-10T17:36:18Z2020-03-10T17:36:18ZTelerik NuGet - Connection Attempt Failed<p><small>Mar <var data-var='date'>10</var>, <var data-var='time'>17:36</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>10</var>, <var data-var='time'>16:03</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Mar <var data-var='date'>10</var>, <var data-var='time'>16:00</var> UTC</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Mar <var data-var='date'>10</var>, <var data-var='time'>15:20</var> UTC</small><br><strong>Identified</strong> - An issue has been identified that prevents certain users from connecting to our NuGet server. <br /><br />Error: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond<br /><br />We are currently working on a fix that should resolve the problem without any additional actions required from your side.</p>