I wanted to personally reach out and keep you informed on the status of what has transpired over the past few days.
First and foremost, I want to recognize that this is a significant inconvenience, and the timing is less than ideal for all parties involved. Once our Technology Management team is able to complete a full-system restore, I can begin to coordinate discussions with them to get more in-depth detailed information about this issue. I will then be issuing a root cause notification to explain the cause and the steps we will be taking to mitigate potential future situations, improvements and remediation plans moving forward. Fortunately, we believe this to be an incredibly rare occurrence, as it should be.
WHAT IS HAPPENING?
DETAILS ABOUT WHAT IS OCCURRING
At the moment, I believe the most pressing matter is the loss of data between 11-SEP-2018 ~8 PM MT (9 PM CST) thru 12-SEP-2018 ~4:34 PM CST. After several hours and attempts to recover the data from the corrupt drives, our Technology Management & Engineering and Data Center teams were unsuccessful. This log data loss occurred due in part to the normal established SIS backup process. Transaction log backups are paused between 8 PM and 1130 PM and a backup copy of the database file is then set to kick-off during that window. It just so happened that the failure occurred at the exact moment the backup was set to run. Regrettably, any data your users removed, entered or manipulated between these dates/times can’t be recovered; there is nothing more that we can do.
Furthermore, we still have the ongoing issue with the reporting server. As you may have already noticed, the SIS Home screen (e.g. dashboards screen) may display the message - "The destination in the SIS you are trying to access cannot be reached". This is due to the report server that is still offline and the dashboard widgets run off that reporting server. Please let your users know should they inquire about the issue. Additionally, the reporting server is the engine that drives the Reports module in the SIS. What this means is that all TopView reports, TopView Custom Reports, TopView Scheduled Reports, the Ad Hoc reporting tool to build TopView reports, and the use of Quick Query (with custom fields) services will not be available until the reporting server is restored and back online. I am working in tandem with the Technology Management & Engineering teams to gather updates every few hours. I will make all attempts to keep you all informed.
Please know that we are actively continuing to monitor the SIS environment accordingly. However, as stated before, monitoring all potential issues may not be possible, and it is in those instances that we rely on our Student Solution Community to contact us should you experience a connectivity problem or issue. Users always have the option to submit a support ticket via Freshdesk or by sending an email directly to: firstname.lastname@example.org with a detailed description of the issue at-hand.
HOW YOU CAN GET HELP
As always, you can always reach out to me directly via email or via phone. Also, I am actively updating the SIS community forums with regular updates as they come available, so be diligent and stay informed by visiting the announcements and known issues forums.
Thank you for your attention and understanding.
All the best,
Product Manager, Student Solution
Campus Management Corp.
5201 Congress Ave., Boca Raton, Florida 33487