tag:status.revcascade.com,2005:/historyRevCascade Status - Incident History2024-03-13T02:23:49ZRevCascadetag:status.revcascade.com,2005:Incident/87838162021-12-07T23:59:08Z2021-12-07T23:59:08ZCloud Services Outage<p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>23:59</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>23:59</var> UTC</small><br><strong>Update</strong> - Services have recovered; will continue to monitor.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>23:40</var> UTC</small><br><strong>Update</strong> - Services are continuing to improve and we will be monitoring through the evening.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>22:07</var> UTC</small><br><strong>Monitoring</strong> - We are starting to see recovery; many requests are returning a positive response although we are still seeing some degradation. We will continue to monitor.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>18:56</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>17:48</var> UTC</small><br><strong>Investigating</strong> - RevCascade is experiencing issues connecting to our APIs due to a critical cloud services provider outage. We are monitoring the issue closely.</p>tag:status.revcascade.com,2005:Incident/77047892021-08-07T10:00:47Z2021-08-07T17:24:49ZIntermittent API Request Failures<p><small>Aug <var data-var='date'> 7</var>, <var data-var='time'>10:00</var> UTC</small><br><strong>Resolved</strong> - Between August 6, 5:04 PM and August 7, 12:02 AM PDT, our cloud infrastructure provider saw elevated error rates in the service responsible for handling API requests. This issue is now resolved as of 3AM PDT and invoke success rates are back to normal.</p><p><small>Aug <var data-var='date'> 7</var>, <var data-var='time'>08:15</var> UTC</small><br><strong>Update</strong> - Our cloud infrastructure provider is continuing to investigate higher error rates. We are upgrading from partial outage to degraded performance due to the reported impact we are seeing in our reporting.</p><p><small>Aug <var data-var='date'> 7</var>, <var data-var='time'>00:00</var> UTC</small><br><strong>Identified</strong> - Our cloud infrastructure is experiencing issues handling API invocations and are investigating. RevCascade engineering is aware of increased error rates some users may be seeing when using the API.</p>tag:status.revcascade.com,2005:Incident/59181712020-12-31T08:35:32Z2020-12-31T08:35:32ZDatabase Maintenance / Upgrades<p><small>Dec <var data-var='date'>31</var>, <var data-var='time'>08:35</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Dec <var data-var='date'>31</var>, <var data-var='time'>08:33</var> UTC</small><br><strong>Verifying</strong> - Verification is currently underway for the maintenance items.</p><p><small>Dec <var data-var='date'>31</var>, <var data-var='time'>07:31</var> UTC</small><br><strong>Update</strong> - We are beginning the period where service may become interrupted/degraded.</p><p><small>Dec <var data-var='date'>31</var>, <var data-var='time'>03:35</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Dec <var data-var='date'>31</var>, <var data-var='time'>03:33</var> UTC</small><br><strong>Scheduled</strong> - We will be upgrading our Sandbox and Production databases during this window. There may be periodic interruptions in API and Console service.</p>tag:status.revcascade.com,2005:Incident/58905682020-12-26T23:43:50Z2020-12-26T23:43:50ZProduction Server Upgrade<p><small>Dec <var data-var='date'>26</var>, <var data-var='time'>23:43</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Dec <var data-var='date'>26</var>, <var data-var='time'>23:31</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Dec <var data-var='date'>26</var>, <var data-var='time'>23:30</var> UTC</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.revcascade.com,2005:Incident/56537442020-11-26T07:35:45Z2020-11-26T07:35:45ZIntermittent API Request Failures<p><small>Nov <var data-var='date'>26</var>, <var data-var='time'>07:35</var> UTC</small><br><strong>Resolved</strong> - Our cloud provider has resolved this issue. We have been seeing few-to-no errors in the past several hours and are now seeing services ramp up to normal levels of throughput.</p><p><small>Nov <var data-var='date'>25</var>, <var data-var='time'>19:17</var> UTC</small><br><strong>Update</strong> - Our cloud provider has identified the root cause affecting this service and are working towards a resolution. We will continue to monitor for updates.</p><p><small>Nov <var data-var='date'>25</var>, <var data-var='time'>17:17</var> UTC</small><br><strong>Update</strong> - We have downgraded our status to "Degraded Performance" based on the intermittent errors we are seeing. The root cause is a service outage with our cloud provider. Automated processes such as feed processing may not run as scheduled. API requests may unexpectedly return a 500 error. We are continuing to monitor for updates.</p><p><small>Nov <var data-var='date'>25</var>, <var data-var='time'>15:48</var> UTC</small><br><strong>Monitoring</strong> - After investigation, it appears that underlying services with our cloud provider experienced degraded performance which caused RevCascade's API issues. We will continue to monitor our performance as well as the status of the issues with the cloud provider.</p><p><small>Nov <var data-var='date'>25</var>, <var data-var='time'>15:14</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.revcascade.com,2005:Incident/54457602020-10-30T00:29:07Z2020-10-30T00:29:07ZProduction Server Upgrade<p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>00:29</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>00:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'>29</var>, <var data-var='time'>22:24</var> UTC</small><br><strong>Scheduled</strong> - We will be upgrading our production server this evening. During the upgrade process, the console may be intermittently unreachable. API requests will be unaffected.</p>tag:status.revcascade.com,2005:Incident/33137642019-12-12T13:24:06Z2020-02-27T23:17:09ZCritical Outage<p><small>Dec <var data-var='date'>12</var>, <var data-var='time'>13:24</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'>12</var>, <var data-var='time'>13:18</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Dec <var data-var='date'>12</var>, <var data-var='time'>13:14</var> UTC</small><br><strong>Identified</strong> - We have identified an issue causing our production website and API to be unresponsive and are currently deploying a fix.</p>tag:status.revcascade.com,2005:Incident/32454422019-11-22T19:50:07Z2020-01-29T05:14:59ZIntermittent Timeouts<p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>19:50</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>18:35</var> UTC</small><br><strong>Monitoring</strong> - The fix has been implemented. We will continue to monitor to ensure the issue is resolved.</p><p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>18:30</var> UTC</small><br><strong>Identified</strong> - We have identified an issue that is causing intermittent timeouts and will be addressing it shortly. It may result in a brief outage of console and API requests.</p>tag:status.revcascade.com,2005:Incident/31629992019-10-31T06:08:37Z2019-10-31T06:08:58ZDatabase Maintenance<p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>06:08</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>06:08</var> UTC</small><br><strong>Update</strong> - We completed our maintenance with an estimated 9 second downtime on our database server.</p><p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>05:53</var> UTC</small><br><strong>Update</strong> - Scheduled maintenance is still ongoing. There has not yet been downtime for any affected components.</p><p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>05:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>01:11</var> UTC</small><br><strong>Scheduled</strong> - We will be undergoing some routine database maintenance which may result in a brief API outage and potentially degraded service before and after the outage.</p>tag:status.revcascade.com,2005:Incident/30908402019-10-11T00:26:17Z2019-10-11T00:26:17ZConsole Down for Routine Maintenance<p><small>Oct <var data-var='date'>11</var>, <var data-var='time'>00:26</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Oct <var data-var='date'>11</var>, <var data-var='time'>00:15</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'>11</var>, <var data-var='time'>00:14</var> UTC</small><br><strong>Scheduled</strong> - Console will be unreachable while we perform routine maintenance on the underlying server.</p>tag:status.revcascade.com,2005:Incident/30628982019-10-03T09:48:59Z2020-01-29T05:12:29ZAPI Unreachable<p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>09:48</var> UTC</small><br><strong>Resolved</strong> - All components are fully operational.</p><p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>09:15</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and the API should be fully functional. The console and other ancillary scheduled processes will be ramping up shortly.</p><p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>04:07</var> UTC</small><br><strong>Investigating</strong> - Complications resulting from our database upgrade has caused our database to be unreachable. In the interest of protecting data integrity while we repair the database, we have disabled all API requests.</p>tag:status.revcascade.com,2005:Incident/30623042019-10-03T04:01:00Z2019-10-03T04:01:01ZDatabase Upgrade Window<p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>04:01</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>01:01</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>22:57</var> UTC</small><br><strong>Scheduled</strong> - We will be performing database upgrades that will result in degraded performance.</p>tag:status.revcascade.com,2005:Incident/25108632019-06-08T05:32:53Z2019-06-08T05:32:53ZScheduled Maintenance<p><small>Jun <var data-var='date'> 8</var>, <var data-var='time'>05:32</var> UTC</small><br><strong>Completed</strong> - Servers have been upgraded and are back to fully operational.</p><p><small>Jun <var data-var='date'> 8</var>, <var data-var='time'>05:01</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'> 7</var>, <var data-var='time'>22:55</var> UTC</small><br><strong>Scheduled</strong> - We will be performing scheduled server upgrades. Users may experience issues with Console responsiveness during this time.</p>tag:status.revcascade.com,2005:Incident/23012492019-04-01T20:44:11Z2020-01-29T04:50:36ZFTP Accounts Unreachable<p><small>Apr <var data-var='date'> 1</var>, <var data-var='time'>20:44</var> UTC</small><br><strong>Resolved</strong> - All queued files have been uploaded to customer FTPs.</p><p><small>Apr <var data-var='date'> 1</var>, <var data-var='time'>20:25</var> UTC</small><br><strong>Monitoring</strong> - FTP connection issues should be resolved, we are currently working through a backlog of files queued to be uploaded to FTP accounts.</p><p><small>Apr <var data-var='date'> 1</var>, <var data-var='time'>19:35</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating an issue with FTP accounts being unreachable by customers. We will provide updates as we have them.</p>tag:status.revcascade.com,2005:Incident/22082232019-02-16T06:00:57Z2019-02-16T06:01:47ZScheduled Maintenance Window<p><small>Feb <var data-var='date'>16</var>, <var data-var='time'>06:01</var> UTC</small><br><strong>Update</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'>16</var>, <var data-var='time'>06:00</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'>16</var>, <var data-var='time'>04:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>21:23</var> UTC</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance from 2/15/2019 8:00PM PST (2/16/2019 4:00AM UTC) to 2/15/2019 10:00PM PST (2/16/2019 6:00AM UTC). Customers may experience periodic outages of Production API and Console.</p>