Penn Computing
Computing Menu Computing A-Z
Computing Home Information Systems & Computing Penn

Microsoft's Incident Report for PennO365

Following is a data extract from the PennO365 administrative console of incident reports sent by Microsoft within the last 30 days. We pull out incidents that are relevant to the current PennO365 service offerings and we refresh this data every 15 minutes. The last extract was taken on June 19, 2018 18:30.

--------------------------------------------------------------------------------------------------
Title TM142152
ServiceAffected Microsoft Teams
Status Investigating
Time 6/19/2018 6:00:00 PM UTC
LastUpdated 6/19/2018 9:33:03 PM UTC
Message Title: Unable to load messages or conversations

TM142152 Details

==========
User Impact: Users may be unable to load messages or conversations within Microsoft Teams.

Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes., Title: Unable to load messages or conversations

==========
User Impact: Users may be unable to load messages or conversations within Microsoft Teams.

Current status: We're analyzing diagnostic data to determine the source of the issue and identify a remediation plan.

Scope of impact: This issue could affect any users hosted within the European region, who are attempting to load messages or conversations.

Start time: Tuesday, June 19, 2018, at 6:00 PM UTC

Next update by: Tuesday, June 19, 2018, at 9:30 PM UTC, Title: Unable to load messages or conversations

==========
User Impact: Users may be unable to load messages or conversations within Microsoft Teams.

Current status: We've determined that connectivity loss to a subset of database infrastructure has resulted in impact. We're working to restore connectivity and service functionality.

Scope of impact: This issue could affect any users hosted within the European region, who are attempting to load messages or conversations.

Start time: Tuesday, June 19, 2018, at 6:00 PM UTC

Preliminary root cause: Connectivity loss to a subset of database infrastructure has impacted some Microsoft Teams functionality.

Next update by: Tuesday, June 19, 2018, at 11:30 PM UTC}
--------------------------------------------------------------------------------------------------
Title SH141994
ServiceAffected Microsoft StaffHub
Status Service restored
Time 6/16/2018 12:13:00 AM UTC
LastUpdated 6/16/2018 1:57:44 AM UTC
Message Title: Service is degraded

SH141994 Details

==========
User Impact: Users may have experienced degraded StaffHub functionality.

Final status: We determined that a portion of the infrastructure that provides StaffHub functionality became degraded. Our automated recovery system repaired this problem, and we confirmed that service was restored after monitoring the environment.

Scope of impact: This issue could have potentially affected any of your users intermittently if they were routed through the affected infrastructure.

Start time: Saturday, June 16, 2018, at 12:13 AM UTC

End time: Saturday, June 16, 2018, at 12:40 AM UTC

This is the final update for the event.
--------------------------------------------------------------------------------------------------
Title MO141957
ServiceAffected Office 365 Portal
Status Service restored
Time 6/15/2018 12:40:00 PM UTC
LastUpdated 6/15/2018 5:26:40 PM UTC
Message Title: Office 365 application connectivity issues

MO141957 Details

==========
User Impact: Users may experience intermittent connectivity issues when accessing the service with the Outlook client and other apps.

More info: Our monitoring indicates that web clients and mobile applications are not impacted by this issue. Word, Excel, and Power Point documents may be affected by this issue if they're hosted on OneDrive for Business and are being opened from the associated client.

Current status: We've identified a configuration issue with the way active service usage is balanced across the infrastructure and are implementing a fix remediate impact.

Scope of impact: This issue could potentially affect any of your users intermittently if they are routed through the affected infrastructure.

Start time: Friday, June 15, 2018, at 12:40 PM UTC

Preliminary root cause: A configuration issue caused active service usage to become unbalanced within the infrastructure. We're reconfiguring the way active service usage is balanced across the infrastructure.

Next update by: Friday, June 15, 2018, at 7:30 PM UTC, Title: Office 365 application connectivity issues

==========
User Impact: Users experienced intermittent connectivity issues when accessing the service with the Outlook client and other apps.

More info: Our monitoring indicates that web clients and mobile applications were not impacted by this issue. Word, Excel, and Power Point documents may have been affected by this issue if they're hosted on OneDrive for Business and were being opened from the associated client.

Final status: We reconfigured the way active service usage is balanced across the infrastructure and monitored service health to confirm impact remediation.

Scope of impact: This issue could have potentially affected any of your users intermittently if they were routed through the affected infrastructure.

Start time: Friday, June 15, 2018, at 12:40 PM UTC

End time: Friday, June 15, 2018, at 3:50 PM UTC

Preliminary root cause: A configuration issue caused active service usage to become unbalanced within the infrastructure.

Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again.

This is the final update for the event.}
--------------------------------------------------------------------------------------------------
Title MO141928
ServiceAffected Office 365 Portal
Status False positive
Time 6/15/2018 10:30:00 AM UTC
LastUpdated 6/15/2018 2:27:39 PM UTC
Message Title: Access issues

MO141928 Details

==========
User Impact: Users may experience intermittent issues accessing Office 365 services or features.

Current status: We're investigating reports of slowness accessing Office 365 services or features. We'll provide an update once we have more information.

Scope of impact: Impact is specific to a subset of users based in the United Kingdom.

Next update by: Friday, June 15, 2018, at 12:00 PM UTC, Title: Access issues

==========
User Impact: Users may experience intermittent issues accessing Office 365 services or features.

Current status: We're analyzing trace route data to isolate the cause of the issue.

Scope of impact: Impact is specific to a subset of users based in the United Kingdom.

Next update by: Friday, June 15, 2018, at 2:00 PM UTC, Title: Access issues

==========
User Impact: Users may experience intermittent issues accessing Office 365 services or features.

Current status: We're continuing to gather forensic data to help isolate what's causing the intermittent connectivity issues. Additionally, due to the nature of the issue, we're running periodic network tests in parallel with our analysis.

Scope of impact: Impact is specific to a subset of users based in the United Kingdom.

Next update by: Friday, June 15, 2018, at 4:00 PM UTC, Title: Access issues

==========
User Impact: Users may experience intermittent issues accessing Office 365 services or features.

Final status: We've completed our analysis of the network, and can confirm that we were unable to identify a significant network event on our side. We received reports during the investigation that the issue was no longer reproducible.}
--------------------------------------------------------------------------------------------------
Title EX141935
ServiceAffected Exchange Online
Status Service restored
Time 6/15/2018 12:00:00 AM UTC
LastUpdated 6/16/2018 3:09:12 PM UTC
Message Title: Administrators are unable to use the import page feature for SCC government protection.

EX141935 Details

==========
User Impact: Administrators may be unable to use the import page feature for SCC (Security and Compliance Center) data governance.

Currentstatus:We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes. , Title: Administrators are unable to use the import page feature for SCC governance protection.

==========
User Impact: Administrators may be unable to use the import page feature for SCC (Security and Compliance Center) data governance.

Current status: We're reviewing diagnostic data to isolate the root cause of the issue and identify a fix.

Scope of impact: Any administrator using the import page feature for SCC may be impacted.

Next update by: Friday, June 15, 2018, at 4:30 PM UTC, Title: Administrators are unable to import PST files

==========
User Impact: Administrators may be unable to import PST files via network upload.

More info: After navigating to Admin center > Setup > Data Migration > Upload PST files, two errors are displayed after being redirected to the Security and Compliance Center (SCC). First a pop-up with a general "Your request couldn't be completed" message, and after clicking through that, an error message is displayed on the page itself that states: "The Import service for your region has not been rolled out yet. Please check back after rollout is announced." Please note that this issue does not affect import jobs that are already in progress. Admins can check status of in-progress jobs via the following PowerShell commands while we work to restore service fully: Get-MailboxImportRequest Get-MailboxImportRequestStatistics

Current status: We've determined that this issue was caused by an expired certificate. We're working to create a new certificate and deploy it to the affected infrastructure to restore service as soon as possible.

Scope of impact: This issue may affect any administrator attempting to import PST files via the Network Upload option within SCC.

Start time: Friday, June 15, 2018, at 12:00 AM UTC

Preliminary root cause: This issue was caused by an expired certificate that prevented access to the PST import feature within Security and Compliance Center.

Next update by: Friday, June 15, 2018, at 7:00 PM UTC, Title: Administrators are unable to import PST files

==========
User Impact: Administrators may be unable to import PST files via network upload.

More info: After navigating to Admin center > Setup > Data Migration > Upload PST files, two errors are displayed after being redirected to the Security and Compliance Center (SCC). First a pop-up with a general "Your request couldn't be completed" message, and after clicking through that, an error message is displayed on the page itself that states: "The Import service for your region has not been rolled out yet. Please check back after rollout is announced." Please note that this issue does not affect import jobs that are already in progress. Admins can check status of in-progress jobs via the following PowerShell commands while we work to restore service fully: Get-MailboxImportRequest Get-MailboxImportRequestStatistics

Current status: The renewed certificate is ready, and we're now working to deploy it to resolve the issue. We anticipate this will take a few hours to complete.

Scope of impact: This issue may affect any administrator attempting to import PST files via the Network Upload option within SCC.

Start time: Friday, June 15, 2018, at 12:00 AM UTC

Estimated time to resolve: Within the next two to three hours.

Preliminary root cause: This issue was caused by an expired certificate that prevented access to the PST import feature within Security and Compliance Center.

Next update by: Friday, June 15, 2018, at 9:30 PM UTC...}
--------------------------------------------------------------------------------------------------
Title MO142135
ServiceAffected Office 365 Portal
Status False positive
Time 6/15/2018 12:00:00 AM UTC
LastUpdated 6/19/2018 3:59:51 PM UTC
Message Title: Issue with Office 365 add-ins

MO142135 Details

==========
User Impact: Users may experience issues with Office 365 add-ins that tenant admins give permissions to.

Current status: We're investigating an issue where users may be unable to access web client add-ins that were granted to them via the tenant admin.

Scope of impact: This issue may potentially affect any of your users attempting to use Office 365 add-ins.

Start time: Friday, June 15, 2018, at 12:00 AM UTC

Next update by: Tuesday, June 19, 2018, at 5:00 PM UTC, Title: Issue with Office 365 add-ins

==========
User Impact: Users may experience issues with Office 365 web client add-ins.

Final status: The investigation is complete and we've determined the service is healthy. A service incident did not actually occur.}
--------------------------------------------------------------------------------------------------
Title TM141955
ServiceAffected Microsoft Teams
Status Service restored
Time 6/14/2018 5:30:00 PM UTC
LastUpdated 6/18/2018 7:12:21 AM UTC
Message Title: Can't sign in to Microsoft Teams

TM141955 Details

==========
User Impact: Users are unable to sign in to Microsoft Teams through the desktop client and web app.

Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Scope of impact: Your organization is affected by this event, and impact is specific to a subset of your users., Title: Can't sign in to Microsoft Teams

==========
User Impact: Users are unable to sign in to Microsoft Teams through the desktop client and web app.

Current status: We're analyzing client logs to determine the source of the issue.

Scope of impact: Your organization is affected by this event, and impact is likely specific to users attempting to sign in to Microsoft Teams when using your internal network.

Next update by: Friday, June 15, 2018, at 6:30 PM UTC, Title: Can't sign in to Microsoft Teams

==========
User Impact: Users may be intermittently unable to sign in to Microsoft Teams through the desktop client and web app.

Current status: We've determined that a recent service update has caused the issue. We're currently in the process of reverting the affected infrastructure to an unaffected build to resolve the problem.

Scope of impact: Your organization is affected by this event, and impact is likely specific to users attempting to sign in to Microsoft Teams when using your internal network.

Preliminary root cause: A recent service update has caused intermittent sign in problems.

Next update by: Friday, June 15, 2018, at 8:30 PM UTC, Title: Can't sign in to Microsoft Teams

==========
User Impact: Users may be intermittently unable to sign in to Microsoft Teams through the desktop client and web app.

More info: This issue only affects tenants using Express Routing.

Current status: We've determined that a recent service update has caused the issue. We're currently in the process of reverting the update on the affected infrastructure to resolve the problem.

Scope of impact: Impact is likely specific to users attempting to sign in to Microsoft Teams when using your internal network.

Start time: Thursday, June 14, 2018, at 5:30 PM UTC

Preliminary root cause: A recent service update has caused intermittent sign in problems.

Next update by: Friday, June 15, 2018, at 8:30 PM UTC...}
--------------------------------------------------------------------------------------------------
Title TM141797
ServiceAffected Microsoft Teams
Status Service restored
Time 6/14/2018 5:27:32 AM UTC
LastUpdated 6/14/2018 8:15:47 AM UTC
Message Title: Unable to create Teams

TM141797 Details

==========
User Impact: Users may be unable to create a new Team within Microsoft Teams

Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes., Title: Unable to create Teams

==========
User Impact: Users may be unable to create a new Team within Microsoft Teams

Current status: We're analyzing system logs to determine the source of the issue.

Scope of impact: Your organization is affected by this event. This issue may potentially affect any of your users attempting to create a new Team in Teams.

Next update by: Thursday, June 14, 2018, at 8:00 AM UTC, Title: Unable to create Teams

==========
User Impact: Users may have been unable to create a new Team within Microsoft Teams.

Final status: We identified that a subset of infrastructure which facilitates team creation requests, became degraded. We removed the affected subset and routed traffic to alternate, healthy infrastructure, which remediated impact.

Scope of impact: Your organization was affected by this event. This issue may have potentially affected any of your users attempting to create a new Team in Teams.

Start time: Thursday, June 14, 2018, at 1:46 AM UTC

End time: Thursday, June 14, 2018, at 7:44 AM UTC

Preliminary root cause: A subset of infrastructure which facilitates team creation requests, became degraded.

Next steps: - We're reviewing our monitoring services to look for ways to reduce detection time and more quickly restore service. - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again.

This is the final update for the event.}
--------------------------------------------------------------------------------------------------
Title LY141764
ServiceAffected Skype for Business
Status Service restored
Time 6/13/2018 3:00:00 PM UTC
LastUpdated 6/13/2018 7:59:05 PM UTC
Message Title: Can't start or delayed start to Skype Broadcast Meetings

LY141764 Details

==========
User Impact: Users may be unable to start newor experience delays of up to 12 minutes when creating new Skype Broadcast Meetings.

More info: User may experience delays of up to 12 minutes.

Current status: We're reviewing system logs from the affected environment to determine the cause of the problem.

Scope of impact: This issue may affect any user attempting to create SkypeBroadcast Meetings.

Start time: Friday, June 1, 2018, at 4:52 PM UTC

Next update by: Wednesday, June 13, 2018, at 7:00 PM UTC, Title: Can't start or delayed start to Skype Broadcast Meetings

==========
User Impact: Users may be unable to start newor experience delays of up to 12 minutes when creating new Skype Broadcast Meetings.

Current status: We're continuing to analyze system logs to troubleshoot the problem.

Scope of impact: This issue may affect any user attempting to create Skype Broadcast Meetings.

Start time: Friday, June 1, 2018, at 4:52 PM UTC

Next update by: Wednesday, June 13, 2018, at 9:00 PM UTC, Title: Can't start or delayed start to Skype Broadcast Meetings

==========
User Impact: Users may have experienced delays of up to 12 minutes when creating new Skype Broadcast Meetings.

Final status: We've determined that a portion of storage infrastructure that hosts Skype Broadcast Meetings was degraded, which delayed creation requests. We've rerouted the impacted systems to alternate infrastructure and our monitoring indicates that the issue is resolved.

Scope of impact: This issue may have affected any user attempting to create Skype Broadcast Meetings.

Start time: Wednesday, June 13, 2018, at 3:00 PM UTC

End time: Wednesday, June 13, 2018, at 6:20 PM UTC

Preliminary root cause: A portion of storage infrastructure responsible for Skype Broadcast Meetings was degraded, which delayed creation requests.

Next steps: We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again.

This is the final update for the event.}
--------------------------------------------------------------------------------------------------
Title MO141760
ServiceAffected Office 365 Portal
Status Service restored
Time 6/12/2018 5:00:00 PM UTC
LastUpdated 6/15/2018 9:55:16 PM UTC
Message Title: Access and rendering issues with the admin portal

MO141760 Details

==========
User Impact: Tenant administrators may be unable to access or render the Office 365 administration portal.

More Info: Our initial findings indicate that impact may be limited to admins using the Microsoft Edge internet browser. While we work to remediate impact, affected admins may be able to successfully access the admin portal with other internet browsers.

Current status: We're investigating a potential issue related to Office 365 portal access and site rendering issues.

Next update by: Wednesday, June 13, 2018, at 7:30 PM UTC, Title: Access and rendering issues with the admin portal

==========
User Impact: Tenant admins may be unable to access or render the Office 365 admin portal.

More info: Our initial findings indicate that impact may be limited to admins using the Microsoft Edge browser. While we work to remediate impact, affected admins may be able to successfully access the admin portal with other internet browsers.

Current status: We're performing tests to better understand the circumstances under which the issue occurs as part of our work to develop a viable remediation plan.

Scope of impact: Impact is specific to admins accessing the Office 365 admin portal while using the Microsoft Edge browser.

Next update by: Wednesday, June 13, 2018, at 11:00 PM UTC, Title: Access and rendering issues with the admin portal

==========
User Impact: Tenant admins may be unable to access or render the Office 365 admin portal.

More info: Our initial findings indicate that impact may be limited to admins using the Microsoft Edge browser. While we work to remediate impact, affected admins may be able to successfully access the admin portal with other internet browsers.

Current status: We've identified a compatibility issue between Microsoft Edge and the Office 365 admin portal as the probable cause. We've reproduced the issue and are formulating a remediation plan.

Scope of impact: Impact is specific to admins accessing the Office 365 admin portal while using the Microsoft Edge browser.

Next update by: Thursday, June 14, 2018, at 7:30 PM UTC, Title: Access and rendering issues with the admin portal

==========
User Impact: Tenant admins may be unable to access or render the Office 365 admin portal.

More info: While we work to remediate impact, affected admins may be able to successfully access the admin portal with other internet browsers.

Current status: We're developing a fix that will resolve the incompatibility issue for the Office 365 admin portal and Microsoft Edge. Once this process is complete, we'll validate that the fix resolves the problem and deploy it into the affected environment to remediate impact.

Scope of impact: Impact is specific to admins accessing the Office 365 admin portal while using the Microsoft Edge browser.

Start time: Tuesday, June 12, 2018, at 5:00 PM UTC

Preliminary root cause: An update to the Edge browser may have caused incompatibility issues when the browser is utilized to access or render the O365 admin portal.

Next update by: Friday, June 15, 2018, at 6:45 PM UTC...}
--------------------------------------------------------------------------------------------------
Title MO141774
ServiceAffected Office 365 Portal
Status Service restored
Time 6/12/2018 4:00:00 PM UTC
LastUpdated 6/14/2018 3:15:03 PM UTC
Message Title: Can't load apps on Mac devices

MO141774 Details

==========
User Impact: Users may be unable to load Office 365 apps on their Mac device.

More info: While we're working to resolve the issue, users can download and re-install the application packages listed below as an alternate solution.

Word: https://officecdn-microsoft-com.akamaized.net/pr/C1297A 47-86C4-4C1F-97FA-950631F94777/OfficeMac/Microsoft_Word_16.14 .18061000_Updater.pkg

Excel: https://officecdn-microsoft-com.akamaized.net/pr/C1297 A47-86C4-4C1F-97FA-950631F94777/OfficeMac/Microsoft_Excel_16. 14.18061000_Updater.pkg

PowerPoint: https://officecdn-microsoft-com.akamaized.net/pr/ C1297A47-86C4-4C1F-97FA-950631F94777/OfficeMac/Microsoft_Powe rPoint_16.14.18061000_Updater.pkg

OneNote: https://officecdn-microsoft-com.akamaized.net/pr/C12 97A47-86C4-4C1F-97FA-950631F94777/OfficeMac/Microsoft_OneNote _16.14.18061000_Updater.pkg

Outlook: https://officecdn-microsoft-com.akamaized.net/pr/C12 97A47-86C4-4C1F-97FA-950631F94777/OfficeMac/Microsoft_Outlook _16.14.18061000_Updater.pkg

Current status: We've determined that a routine service update contained a code issue that may have caused a problem to occur during installation of the update for some users. Subsequently, this may be preventing Office 365 apps from loading properly. We're developing a fix to address the code issue which we anticipate will be ready for release within 24 hours.

Scope of impact: A very limited number of customers and users appear to be impacted.

Start time: Tuesday, June 12, 2018, at 4:00 PM UTC

Preliminary root cause: A routine service update contained a code issue that may have caused a problem to occur during installation of the update.

Next update by: Thursday, June 14, 2018, at 6:00 PM UTC, Title: Can't load apps on Mac devices

==========
User Impact: Users may have been unable to load Office 365 apps on their Mac device.

More info: While we were working to resolve the issue, users could have downloaded and re-installed the application packages listed below as an alternate solution.

Word: https://officecdn-microsoft-com.akamaized.net/pr/C1297A 47-86C4-4C1F-97FA-950631F94777/OfficeMac/Microsoft_Word_16.14 .18061000_Updater.pkg

Excel: https://officecdn-microsoft-com.akamaized.net/pr/C1297 A47-86C4-4C1F-97FA-950631F94777/OfficeMac/Microsoft_Excel_16. 14.18061000_Updater.pkg

PowerPoint: https://officecdn-microsoft-com.akamaized.net/pr/ C1297A47-86C4-4C1F-97FA-950631F94777/OfficeMac/Microsoft_Powe rPoint_16.14.18061000_Updater.pkg

OneNote: https://officecdn-microsoft-com.akamaized.net/pr/C12 97A47-86C4-4C1F-97FA-950631F94777/OfficeMac/Microsoft_OneNote _16.14.18061000_Updater.pkg

Outlook: https://officecdn-microsoft-com.akamaized.net/pr/C12 97A47-86C4-4C1F-97FA-950631F94777/OfficeMac/Microsoft_Outlook _16.14.18061000_Updater.pkg

Final status: We completed development of the fix which has been released to the affected environment. Further testing has confirmed that the issue is resolved.

Scope of impact: A very limited number of customers and users appeared to have been impacted.

Start time: Tuesday, June 12, 2018, at 4:00 PM UTC

End time: Thursday, June 14, 2018, at 4:00 AM UTC

Preliminary root cause: A routine service update contained a code issue that may have caused a problem to occur during installation of the update.

Next steps: - We're reviewing our update procedures to better identify similar issues during our development and testing cycles.

This is the final update for the event.}
--------------------------------------------------------------------------------------------------
Title MO141461
ServiceAffected Office 365 Portal
Status Service restored
Time 6/12/2018 1:37:00 AM UTC
LastUpdated 6/12/2018 4:22:01 AM UTC
Message Title: Can't sign in

MO141461 Details

User impact: Users may have been unable to sign in to StaffHub service.



More info: This service incident number is a duplicate of an existing issue.
--------------------------------------------------------------------------------------------------
Title SH141466
ServiceAffected Microsoft StaffHub
Status Service restored
Time 6/12/2018 1:37:00 AM UTC
LastUpdated 6/12/2018 4:18:21 AM UTC
Message Title: Can't sign in

SH141466 Details

User impact: Users may have been unable to sign in to StaffHub.

More info: Impact was limited to users in the Eastern US region.

Final status: We've confirmed that restarting a portion of infrastructure that facilitates the ability to sign in to the service has remediated this issue.

Scope of impact: Impact was specific to a subset of users who were served through the affected infrastructure.

Start time: Tuesday, June 12, 2018, at 1:37 AM UTC

End time: Tuesday, June 12, 2018, at 2:10 AM UTC

Preliminary root cause: A portion of infrastructure that facilitates the ability to sign in to the service became degraded.

Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again.

This is the final update for the event.
--------------------------------------------------------------------------------------------------
Title TM141394
ServiceAffected Microsoft Teams
Status Service restored
Time 6/11/2018 5:35:00 AM UTC
LastUpdated 6/11/2018 10:53:42 AM UTC
Message Title: Microsoft Teams access issue

TM141394 Details

==========
User Impact: Users may be unable to access Microsoft Teams.

Current status: We're analyzing diagnostic logs to determine the next troubleshooting steps.

Scope of impact: This issue could potentially affect any of your users if they are routed through the affected infrastructure. Impact is specific to a subset of your users in the Europe region.

Start time: Monday, June 11, 2018, at 5:35 AM UTC

Next update by: Monday, June 11, 2018, at 11:00 AM UTC, Title: Microsoft Teams access issue

==========
User Impact: Users may have been unable to access Microsoft Teams.

More info: Users may have seen an "Oops" page or may have not been able to sign in to the Teams application. Additionally, logged in users may have been logged out.

Final status: We've determined that a subset of network infrastructure that hosts the Microsoft Teams service became degraded. We've routed traffic to an alternate, healthy infrastructure, which resolved the issue.

Scope of impact: This issue could have potentially affected any of your users if they were routed through the affected infrastructure. Impact was specific to a subset of your users in the Europe region.

Start time: Monday, June 11, 2018, at 5:35 AM UTC

End time: Monday, June 11, 2018, at 9:07 AM UTC

Preliminary root cause: A subset of network infrastructure that hosts the Microsoft Teams service became degraded.

Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again.

This is the final update for the event.}
--------------------------------------------------------------------------------------------------
Title SH141353
ServiceAffected Microsoft StaffHub
Status Service restored
Time 6/10/2018 2:50:00 AM UTC
LastUpdated 6/10/2018 5:44:01 AM UTC
Message Title: Can't sign in

SH141353 Details

==========
User Impact: Users may have been unable to sign in or access calendars when using StaffHub services.

Final status: We've received alerts indicating that the StaffHub service was operating below normal thresholds. After monitoring the event, we've determined that our automated recovery systems have restored service functionality.

Scope of impact: Any user may have been unable to sign in or access calendars when using StaffHub services.

Start time: Sunday, June 10, 2018, at 2:50 AM UTC

End time: Sunday, June 10, 2018, at 3:38 AM UTC

Preliminary root cause: Connectivity to the infrastructure that facilitates StaffHub service became degraded, resulting in impact.

Next steps: We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again.

This is the final update for the event.
--------------------------------------------------------------------------------------------------
Title TM141341
ServiceAffected Microsoft Teams
Status Service restored
Time 6/9/2018 9:30:00 PM UTC
LastUpdated 6/10/2018 2:41:26 AM UTC
Message Title: Can't sign in

TM141341 Details

==========
User Impact: Users may be unable to access Microsoft Teams.

More info: Users may see an "Oops" page and/or may not be able to sign in to the Teams application. Additionally, logged in users may get logged out.

Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes., Title: Can't sign in

==========
User Impact: Users may be unable to access Microsoft Teams.

More info: Users may see an "Oops" page and/or may not be able to sign in to the Teams application. Additionally, logged in users may get logged out.

Current status: We've determined that a portion of the network infrastructure that hosts the Microsoft Teams service has become degraded. We're rerouting network traffic to an alternate infrastructure to fix the issue.

Scope of impact: This issue could potentially affect any of your users intermittently if they are routed through the affected infrastructure.

Start time: Saturday, June 9, 2018, at 10:11 PM UTC

Next update by: Sunday, June 10, 2018, at 1:30 AM UTC, Title: Can't sign in

==========
User Impact: Users may be unable to access Microsoft Teams.

More info: Users may see an "Oops" page and/or may not be able to sign in to the Teams application. Additionally, logged in users may get logged out.

Current status: We've rerouted traffic to an alternate infrastructure and we're working to confirm that impact has been mitigated.

Scope of impact: This issue could potentially affect any of your users intermittently if they are routed through the affected infrastructure.

Start time: Saturday, June 9, 2018, at 10:11 PM UTC

Preliminary root cause: A portion of the network infrastructure that hosts the Microsoft Teams service has become degraded.

Next update by: Sunday, June 10, 2018, at 3:00 AM UTC, Title: Can't sign in

==========
User Impact: Users may have been unable to access Microsoft Teams.

More info: Users may have seen an "Oops" page and/or may have not been able to sign in to the Teams application. Additionally, logged in users may have gotten logged out.

Final status: We've rerouted traffic to an alternate infrastructure, which has fixed the issue.

Scope of impact: This issue could have potentially affected any of your users intermittently if they were routed through the affected infrastructure.

Start time: Saturday, June 9, 2018, at 9:30 PM UTC

End time: Sunday, June 10, 2018, at 12:00 AM UTC

Preliminary root cause: A portion of the network infrastructure that hosts the Microsoft Teams service had become degraded.

Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again.

This is the final update for the event.}
--------------------------------------------------------------------------------------------------
Title LY141274
ServiceAffected Skype for Business
Status Service restored
Time 6/8/2018 9:00:00 AM UTC
LastUpdated 6/8/2018 6:32:43 PM UTC
Message Title: Can't make outbound calls through Direct Routing

LY141274 Details

==========
User Impact: Users may be unable to make outbound calls through Direct Routing.

Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes., Title: Can't make outbound calls through Direct Routing

==========
User Impact: User may be unable to make outbound calls through Direct Routing.

Current status: We're analyzing system logs to determine the source of the issue.

Scope of impact: This issue could potentially affect any of your users intermittently if they utilize Direct Routing to make outbound calls.

Start time: Friday, June 8, 2018, at 9:00 AM UTC

Next update by: Friday, June 8, 2018, at 5:00 PM UTC, Title: Can't make outbound calls through Direct Routing

==========
User Impact: User may be unable to make outbound calls through Direct Routing.

Current status: We've identified a potential issue with a network component responsible for routing the affected call requests. We're investigating further to determine the next troubleshooting steps and possible mitigation actions.

Scope of impact: This issue could potentially affect any of your users intermittently if they utilize Direct Routing to make outbound calls.

Start time: Friday, June 8, 2018, at 9:00 AM UTC

Next update by: Friday, June 8, 2018, at 7:00 PM UTC, Title: Can't make outbound calls through Direct Routing

==========
User Impact: Users may have been unable to make outbound calls through Direct Routing.

Final status: Upon further review, we identified that an unusually large influx of requests resulted in the impact. We've redistributed load across the affected environment to mitigate impact and have confirmed that the issue is resolved.

Scope of impact: This issue could have potentially affected any of your users intermittently if they utilize Direct Routing to make outbound calls.

Start time: Friday, June 8, 2018, at 9:00 AM UTC

End time: Friday, June 8, 2018, at 4:00 PM UTC

Preliminary root cause: An unusually large influx of requests resulted in failures when attempting to make outbound calls through Direct Routing.

Next steps: - We're continuing to investigate the underlying source of the problem to prevent his issue from happening again.

This is the final update for the event.}
--------------------------------------------------------------------------------------------------
Title TM141278
ServiceAffected Microsoft Teams
Status Service restored
Time 6/8/2018 9:00:00 AM UTC
LastUpdated 6/8/2018 6:31:29 PM UTC
Message Title: Can't make outbound calls through Direct Routing

TM141278 Details

==========
User Impact: Users may be unable to make outbound calls through Direct Routing.

Current status: We're analyzing system logs to determine the source of the issue.

Scope of impact: This issue could potentially affect any of your users intermittently if they utilize Direct Routing to make outbound calls.

Start time: Friday, June 8, 2018, at 9:00 AM UTC

Next update by: Friday, June 8, 2018, at 5:00 PM UTC, Title: Can't make outbound calls through Direct Routing

==========
User Impact: Users may be unable to make outbound calls through Direct Routing.

Current status: We've identified a potential issue with a network component responsible for routing the affected call requests. We're investigating further to determine the next troubleshooting steps and possible mitigation actions.

Scope of impact: This issue could potentially affect any of your users intermittently if they utilize Direct Routing to make outbound calls.

Start time: Friday, June 8, 2018, at 9:00 AM UTC

Next update by: Friday, June 8, 2018, at 7:00 PM UTC, Title: Can't make outbound calls through Direct Routing

==========
User Impact: Users may have been unable to make outbound calls through Direct Routing.

Final status: Upon further review, we identified that an unusually large influx of requests resulted in the impact. We've redistributed load across the affected environment to mitigate impact and have confirmed that the issue is resolved.

Scope of impact: This issue could have potentially affected any of your users intermittently if they utilize Direct Routing to make outbound calls.

Start time: Friday, June 8, 2018, at 9:00 AM UTC

End time: Friday, June 8, 2018, at 4:00 PM UTC

Preliminary root cause: An unusually large influx of requests resulted in failures when attempting to make outbound calls through Direct Routing.

Next steps: - We're continuing to investigate the underlying source of the problem to prevent his issue from happening again.

This is the final update for the event.}
--------------------------------------------------------------------------------------------------
Title TM141194
ServiceAffected Microsoft Teams
Status Service restored
Time 6/7/2018 3:32:00 PM UTC
LastUpdated 6/7/2018 8:07:34 PM UTC
Message Title: Can't dial out over PSTN from conference calls

TM141194 Details

==========
User Impact: Users may be intermittently unable to dial out over the Public Switched Telephone Network (PSTN) from conference calls.

Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes., Title: Delays when dialing out over PSTN from conference calls

==========
User Impact: Users may experience delays when dialing out over the Public Switched Telephone Network (PSTN) from conference calls.

More info: Previously, we mentioned that these calls may fail; however, after further investigation, we determined that they were only being delayed.

Current status: We're analyzing system logs to determine the source of the issue.

Scope of impact: This issue could potentially affect any of your users intermittently if they are attempting to dial out of conference calls over the PSTN.

Start time: Thursday, June 7, 2018, at 3:32 PM UTC

Next update by: Thursday, June 7, 2018, at 6:30 PM UTC, Title: Delays when dialing out over PSTN from conference calls

==========
User Impact: Users may experience delays when dialing out over the Public Switched Telephone Network (PSTN) from conference calls.

Final status: We determined that a portion of infrastructure from which dial out policies are referenced by the service became degraded, resulting in the delays. We've rerouted affected user traffic to alternate infrastructure to resolve the issue.

Scope of impact: This issue could have potentially affected any of your users intermittently if they were attempting to dial out of conference calls over the PSTN.

Start time: Thursday, June 7, 2018, at 2:18 PM UTC

End time: Thursday, June 7, 2018, at 4:05 PM UTC

Preliminary root cause: A portion of infrastructure from which dial out policies are referenced by the service became degraded, resulting in delays when attempting to dial out from conference calls over the PSTN.

Next steps: - We're analyzing performance data and trends on the affected infrastructure to help prevent this problem from happening again.

This is the final update for the event.}
--------------------------------------------------------------------------------------------------
Title LY141222
ServiceAffected Skype for Business
Status Service restored
Time 6/7/2018 2:18:00 PM UTC
LastUpdated 6/7/2018 7:40:49 PM UTC
Message Title: Delays when dialing out over PSTN from conference calls

LY141222 Details

==========
User Impact: Users may have experienced delays dialing out over the Public Switched Telephone Network (PSTN) from conference calls.

More info: Users may have experienced delays up to one minute.

Final status: We determined that the impact associated with Service Incident LY141192 reoccurred. On further investigation, we determined that a conferencing feature setting was not working as expected. We've disabled this feature and haveconfirmed that the impact has been mitigated.

Scope of impact: This issue could have potentially affected any of your users intermittently if they were attempting to dial out of conference calls over the PSTN.

Start time: Thursday, June 7, 2018, at 2:18 PM UTC

End time: Thursday, June 7, 2018, at 7:14 PM UTC

Preliminary root cause: A conferencing feature setting was not working as expected, resulting in delays of up to one minute when attempting to dial out from conference calls through the PSTN.

Next steps: - We're investigating why the feature setting was not working as expectedwhich resulted in the delays.

This is the final update for the event.
--------------------------------------------------------------------------------------------------
Title LY141192
ServiceAffected Skype for Business
Status Service restored
Time 6/7/2018 2:18:00 PM UTC
LastUpdated 6/7/2018 8:08:15 PM UTC
Message Title: Can't dial out over PSTN from conference calls

LY141192 Details

==========
User Impact: Users may be intermittently unable to dial out over the Public Switched Telephone Network (PSTN) from conference calls.

More info: Previously, we mentioned that these calls may fail; however, after further investigation, we determined that they were only being delayed.

Current status: We're analyzing system logs to determine the source of the issue.

Scope of impact: This issue could potentially affect any of your users intermittently if they are attempting to dial out of conference calls over the PSTN.

Start time: Thursday, June 7, 2018, at 3:32 PM UTC

Next update by: Thursday, June 7, 2018, at 6:30 PM UTC, Title: Can't dial out over PSTN from conference calls

==========
User Impact: Users may be intermittently unable to dial out over the Public Switched Telephone Network (PSTN) from conference calls.

Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes., Title: Can't dial out over PSTN from conference calls

==========
User Impact: Users may be intermittently unable to dial out over the Public Switched Telephone Network (PSTN) from conference calls.

Current status: We're analyzing system logs to determine the source of the issue.

Scope of impact: This issue could potentially affect any of your users intermittently if they are attempting to dial out of conference calls over the PSTN.

Start time: Thursday, June 7, 2018, at 3:32 PM UTC

Next update by: Thursday, June 7, 2018, at 6:30 PM UTC, Title: Delays when dialing out over PSTN from conference calls

==========
User Impact: Users may have experienced delays when dialing out over the PSTN from conference calls.

Final status: We determined that a portion of infrastructure from which dial out policies are referenced by the service became degraded, resulting in the delays. We've rerouted affected user traffic to alternate infrastructure to resolve the issue.

Scope of impact: This issue could have potentially affected any of your users intermittently if they were attempting to dial out of conference calls over the PSTN.

Start time: Thursday, June 7, 2018, at 2:18 PM UTC

End time: Thursday, June 7, 2018, at 4:05 PM UTC

Preliminary root cause: A portion of infrastructure from which dial out policies are referenced by the service became degraded, resulting in delays when attempting to dial out from conference calls over the PSTN.

Next steps: - We're analyzing performance data and trends on the affected infrastructure to help prevent this problem from happening again.

This is the final update for the event.}
--------------------------------------------------------------------------------------------------
Title TM141223
ServiceAffected Microsoft Teams
Status Service restored
Time 6/7/2018 2:18:00 PM UTC
LastUpdated 6/7/2018 7:45:37 PM UTC
Message Title: Delays when dialing out over PSTN from conference calls

TM141223 Details

==========
User Impact: Users may have experienced delays dialing out over the Public Switched Telephone Network (PSTN) from conference calls.

More info: Users may have experienced delays up to one minute.

Final status: We determined that the impact associated with Service Incident TM141194 reoccurred. On further investigation, we determined that a conferencing feature setting was not working as expected. We've disabled this feature and have confirmed that the impact has been mitigated.

Scope of impact: This issue could have potentially affected any of your users intermittently if they were attempting to dial out of conference calls over the PSTN.

Start time: Thursday, June 7, 2018, at 2:18 PM UTC

End time: Thursday, June 7, 2018, at 7:14 PM UTC

Preliminary root cause: A conferencing feature setting was not working as expected, resulting in delays of up to one minute when attempting to dial out from conference calls through the PSTN.

Next steps: - We're investigating why the feature setting was not working as expectedwhich resulted in the delays.

This is the final update for the event.
--------------------------------------------------------------------------------------------------
Title MO141475
ServiceAffected Office 365 Portal
Status Service restored
Time 6/7/2018 12:00:00 AM UTC
LastUpdated 6/19/2018 12:17:15 PM UTC
Message Title: Admin usage reports processing delays

MO141475 Details

==========
User Impact: Admins may notice that usage reports do not contain data beyond Thursday, June 7, 2018, at 12:00 AM UTC.

Current status: We discovered an issue where Skype usage data was unavailable, impacting our reporting systems ability to produce usage reports. We've mitigated the issue and are investigating the root cause, and our systems are now processing the data backlog to restore full service. This is estimated to complete on Friday, June 22, 2018.

Scope of impact: Impact is specific to Admins who are attempting to view usage reports that rely on Skype usage data. This issue may affect any report that uses Skype usage data after Thursday, June 7, 2018, at 12:00 AM UTC.

Start time: Thursday, June 7, 2018, at 12:00 AM UTC

Estimated time to resolve: Based on current rate of processing, we expect this problem to be fully resolved by Friday, June 22, 2018 UTC.

Next update by: Friday, June 15, 2018, at 12:00 PM UTC, Title: Skype for Business usage reports processing delays

==========
User Impact: Admins may notice that usage reports do not contain data beyond Thursday, June 7, 2018, at 12:00 AM UTC.

More info: Please note that this issue is specific to Skype for Business usage reporting.

Current status: We determined that this problem was caused by a separate issue within the database infrastructure that collects and stores service usage data. That issue was already resolved and at this point we're processing the backlog of data, which is likely to take several days.

Scope of impact: Impact is specific to Admins who are attempting to view usage reports that rely on Skype usage data. This issue may affect any report that uses Skype usage data after Thursday, June 7, 2018, at 12:00 AM UTC.

Start time: Thursday, June 7, 2018, at 12:00 AM UTC

Estimated time to resolve: Based on current rate of processing, we expect this problem to be fully resolved by Friday, June 22, 2018 at 8:00 PM UTC.

Preliminary root cause: A separate issue within the database infrastructure that collects and stores service usage data caused a delay in the availability of Skype for Business service usage data.

Next update by: Friday, June 15, 2018, at 8:00 PM UTC, Title: Skype for Business usage reports processing delays

==========
User Impact: Admins may notice that usage reports do not contain data beyond Thursday, June 7, 2018, at 12:00 AM UTC.

More info: Please note that this issue is specific to Skype for Business usage reporting.

Current status: We're continuing to monitor the backlog of data as it processes through our systems.

Scope of impact: Impact is specific to Admins who are attempting to view usage reports that rely on Skype usage data. This issue may affect any report that uses Skype usage data after Thursday, June 7, 2018, at 12:00 AM UTC.

Start time: Thursday, June 7, 2018, at 12:00 AM UTC

Estimated time to resolve: Based on current rate of processing, we expect this problem to be fully resolved by Saturday, June 23, 2018, at 12:00 AM UTC.

Preliminary root cause: A separate issue within the database infrastructure that collects and stores service usage data caused a delay in the availability of Skype for Business service usage data.

Next update by: Saturday, June 16, 2018, at 12:00 AM UTC, Title: Skype for Business usage reports processing delays

==========
User Impact: Admins may notice that usage reports do not contain data beyond Thursday, June 7, 2018, at 12:00 AM UTC.

More info: Please note that this issue is specific to Skype for Business usage reporting.

Current status: We're continuing to monitor the backlog of data as it processes through our systems; however, this process is taking longer than expected. We'll continue to monitor the backlog as it processes over the weekend.

Scope of impact: Impact is specific to Admins who are attempting to view usage reports that rely on Skype usage data. This issue may affect any report that uses Skype usage data after Thursday, June 7, 2018, at 12:00 AM UTC.

Start time: Thursday, June 7, 2018, at 12:00 AM UTC

Preliminary root cause: A separate issue within the database infrastructure that collects and stores service usage data caused a delay in the availability of Skype for Business service usage data.

Next update by: Wednesday, June 20, 2018, at 2:00 AM UTC...}
--------------------------------------------------------------------------------------------------
Title EX141294
ServiceAffected Exchange Online
Status Service restored
Time 6/6/2018 12:00:00 AM UTC
LastUpdated 6/18/2018 3:20:23 AM UTC
Message Title: Unable to update group SMTP addresses

EX141294 Details

==========
User Impact: Admins are unable to update the primary SMTP address associated with Office 365 groups.

Current status: We've identified an issue in which admins are unable to update the primary Simple Mail Transfer Protocol (SMTP) address associated with Office 365 groups. We've developed a fix for this issue and are preparing to deploy it across the affected environments to remediate the problem. We expect to begin the deployment of the fix in the next 24-48 hours.

Scope of impact: This issue may potentially affect any of your admins attempting to update the primary SMTP address for an Office 365 group.

Start time: Wednesday, June 6, 2018, at 12:00 AM UTC

Preliminary root cause: A code regression occurred while we were building out a new service feature.

Next update by: Wednesday, June 13, 2018, at 8:00 PM UTC, Title: Unable to update group SMTP addresses

==========
User Impact: Admins are unable to update the primary SMTP address associated with Office 365 groups.

Current status: We've begun the deployment of the fix and are monitoring it's progression throughout the affected environment. We expect that the fix will be fully saturated and the issue resolved by the end of the week.

Scope of impact: This issue may potentially affect any of your admins attempting to update the primary SMTP address for an Office 365 group.

Start time: Wednesday, June 6, 2018, at 12:00 AM UTC

Preliminary root cause: A code regression occurred while we were building out a new service feature.

Next update by: Friday, June 15, 2018, at 11:00 PM UTC, Title: Unable to update group SMTP addresses

==========
User Impact: Admins were unable to update the primary SMTP address associated with Office 365 groups.

Final status: Our fix has propagated across the affected environment and we've confirmed that the issue is resolved.

Scope of impact: This issue may have potentially affected any of your admins attempting to update the primary SMTP address for an Office 365 group.

Start time: Wednesday, June 6, 2018, at 12:00 AM UTC

End time: Thursday, June 14, 2018, at 10:00 AM UTC

Preliminary root cause: A code regression occurred while we were building out a new service feature.

Next steps: - We're reviewing our update procedures to better identify similar issues during our development and testing cycles.

This is the final update for the event.}
--------------------------------------------------------------------------------------------------
Title LY141267
ServiceAffected Skype for Business
Status Service restored
Time 6/5/2018 1:40:00 PM UTC
LastUpdated 6/8/2018 3:05:39 PM UTC
Message Title: Conference dial-in dial-out service provisioning issue

LY141267 Details

==========
User Impact: Admins are unable to provision new users with conference dial-in dial-out service.

Current status: We've determined that a recent service update is preventing admins from being able to provision new users with conference dial-in dial-out service. We're reverting the update to remediate impact.

Scope of impact: Your organization is affected by this event, and impact is specific to admins trying to provision new users.

Start time: Tuesday, June 5, 2018, at 1:40 PM UTC

Preliminary root cause: A recent build is preventing admins from being able to provision new users within the service.

Next update by: Friday, June 8, 2018, at 7:00 PM UTC, Title: Conference dial-in/dial-out service provisioning issue

==========
User Impact: Admins were unable to provision new users with conference dial-in/dial-out service.

Final status: We've reverted the update and have confirmed that the issue is resolved.

Scope of impact: Your organization was affected by this event, and impact was specific to admins trying to provision new users.

Start time: Tuesday, June 5, 2018, at 1:40 PM UTC

End time: Friday, June 8, 2018, at 1:15 PM UTC

Preliminary root cause: A recent service update prevented admins from being able to provision new users with dial-in/dial-out service.

Next steps: - We're reviewing our standard service update procedures to avoid similar impact in the future.

This is the final update for the event.}
--------------------------------------------------------------------------------------------------
Title TM141041
ServiceAffected Microsoft Teams
Status Service restored
Time 6/5/2018 1:41:00 AM UTC
LastUpdated 6/5/2018 3:03:02 AM UTC
Message Title: Access issue

TM141041 Details

==========
User Impact: Users may be unable to access Microsoft Teams.

More information: Impact is specific to users in Europe.

Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes., Title: Access issue

==========
User Impact: Users may be unable to access Microsoft Teams.

Current status: We're analyzing system logs to determine the source of the issue.

Scope of impact: This issue may potentially affect any European users attempting to access Microsoft Teams.

Start time: Tuesday, June 5, 2018, at 1:41 AM UTC

Next update by: Tuesday, June 5, 2018, at 5:00 AM UTC, Title: Access issue

==========
User Impact: Users may have been unable to access Microsoft Teams.

Final status: After analyzing system logs, we've determined that a certificate issue had been preventing access to Microsoft Teams. We've removed the related certificate which has mitigated impact.

Scope of impact: This issue may have potentially affected any European users attempting to access Microsoft Teams.

Start time: Tuesday, June 5, 2018, at 1:41 AM UTC

End time: Tuesday, June 5, 2018, at 2:37 AM UTC

Preliminary root cause: A certificate issue may have prevented access to Microsoft Teams for European users.

Next steps: - We're reviewing our certificate procedures to find ways to prevent this problem from happening again.

This is the final update for the event.}
--------------------------------------------------------------------------------------------------
Title SH141042
ServiceAffected Microsoft StaffHub
Status Service restored
Time 6/5/2018 1:13:00 AM UTC
LastUpdated 6/5/2018 3:06:55 AM UTC
Message Title: Can't sign in

SH141042 Details

==========
User Impact: Users may be unable to sign in to StaffHub.

More info: Users that are signed in may experience intermittent failures when performing specific actions.

Current status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes., Title: Can't sign in

==========
User Impact: Users may have been unable to sign in to StaffHub.

More info: Users that were signed in may have experienced intermittent failures when performing specific actions.

Final status: We've identified that a certificate issue had resulted in impact. We've removed the related certificate to mitigate impact and after monitoring the environment, we have confirmed that the service has recovered.

Scope of impact: A very limited number of customers and users appeared to have been impacted.

Start time: Tuesday, June 5, 2018, at 1:13 AM UTC

End time: Tuesday, June 5, 2018, at 2:29 AM UTC

Preliminary root cause: A certificate issue prevented a limited number of users from accessing the StaffHub service.

Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again.

This is the final update for the event.}
top

Information Systems and Computing
University of Pennsylvania
Comments & Questions


University of Pennsylvania Penn Computing University of Pennsylvania Information Systems & Computing (ISC)
Information Systems and Computing, University of Pennsylvania