Penn Computing
Computing Menu Computing A-Z
Computing Home Information Systems & Computing Penn

Microsoft's Incident Report for PennO365

Following is a data extract from the PennO365 administrative console of incident reports sent by Microsoft within the last 30 days. We pull out incidents that are relevant to the current PennO365 service offerings and we refresh this data every 15 minutes. The last extract was taken on October 25, 2021 02:45.

--------------------------------------------------------------------------------------------------
Title Some admins may be unable to access M365 Usage Reports within Office 365 Suite
ID MO293125
Service Microsoft 365 suite
Feature Portal
Status Service restored
Impact Admins may have been unable to access M365 Usage Reports within Office 365 Suite.
Severity Sev2
Start time 2021-10-22T08:00:00Z
End time 2021-10-23T03:00:00Z
Last update 2021-10-23T03:25:39.123Z
More info Impacted reports included: - M365 Admin Center Usage Reports. - Graph API Usage Reports. - Power BI Usage Reports. Final status: We've confirmed via system telemetry that our fix has resynced data for affected admins, resolving impact. Scope of impact: Impact was specific to admins who were served through the impacted infrastructure hosted out of Europe, Middle East and Africa. Start time: Friday, October 22, 2021, at 8:00 AM'UTC End time: Saturday, October 23, 2021, at 3:00 AM'UTC Root cause: A processor designed to sync and facilitate M365 Usage Reports was performing below acceptable performance thresholds. Next steps: - We're reviewing our M365 Usage Reports sync components and functionality to explore options to more quickly identify and prevent similar impact in the future. This is the final update for the event.; PublishedTime=2021-10-23T03:25:39.123Z
Root cause A processor designed to sync and facilitate M365 Usage Reports was performing below acceptable performance thresholds. Next steps: - We're reviewing our M365 Usage Reports sync components and functionality to explore options to more quickly identify and prevent similar impact in the future. This is the final update for the event.; PublishedTime=2021-10-23T03:25:39.123Z

MO293125 Details

Status: We're analyzing system logs to determine the source of the issue. Next update by: Friday, October 22, 2021, at 2:30 PM UTC; PublishedTime=2021-10-22T12:05:58.373Z

Status: We've identified that a processer designed to facilitate usage reports is performing below acceptable performance thresholds. We're developing a fix, to remediate impact. Next update by: Friday, October 22, 2021, at 4:30 PM UTC; PublishedTime=2021-10-22T14:22:38.343Z

Status: We've identified a processor designed to sync and facilitate M365 Usage Reports is performing below acceptable performance thresholds. We're applying a fix to resync the data and we're monitoring the service to ensure that this action fully remediates the problem. Next update by: Saturday, October 23, 2021, at 10:00 PMUTC; PublishedTime=2021-10-22T16:16:37.417Z

Final status: We've confirmed via system telemetry that our fix has resynced data for affected admins, resolving impact. Next update by: Saturday, October 23, 2021, at 10:00 PMUTC; PublishedTime=2021-10-22T16:16:37.417Z

--------------------------------------------------------------------------------------------------
Title Some admins may have been unable to generate meeting attendance reports within Microsoft Teams
ID TM293105
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Admins may have been unable to generate meeting attendance reports within Microsoft Teams.
Severity Sev2
Start time 2021-10-22T01:16:00Z
End time 2021-10-22T13:00:00Z
Last update 2021-10-22T14:32:21.19Z
More info
Root cause A recent standard service update contained a code regression. Next steps: - We're reviewing our deployment procedures to help prevent similar problems in the future. This is the final update for the event.; PublishedTime=2021-10-22T14:29:18.48Z

TM293105 Details

Status: We've determined a recent deployment contained a code regression, which is resulting in impact. We're reverting this change and monitoring to ensure it resolves the issue. Next update by: Friday, October 22, 2021, at 11:30 PM UTC; PublishedTime=2021-10-22T09:34:07.26Z

Status: We're continuing to monitor the rollback of the change to the affected infrastructure, which is currently 20% complete. Next update by: Friday, October 22, 2021, at 6:30 PM UTC; PublishedTime=2021-10-22T10:32:40.967Z

Final status: We've confirmed that the rollback has completed and impact has been mitigated. Next update by: Friday, October 22, 2021, at 6:30 PM UTC; PublishedTime=2021-10-22T10:32:40.967Z

--------------------------------------------------------------------------------------------------
Title Users are unable to launch the OneDrive for Business iOS app
ID OD292873
Service OneDrive for Business
Feature OneDrive for Business
Status Service degradation
Impact Users are unable to launch the OneDrive for Business iOS app.
Severity Sev2
Start time 2021-10-21T21:05:00Z
End time
Last update 2021-10-23T03:41:17.663Z
More info Users that have not recently updated their OneDrive for Business App shouldn't see this impact, and shouldn't update their app until further notice. Current status: The latest version of the OneDrive for Business iOS app - version 12.53.21 - is now available in the company portal. Affected users will need to download the latest version(12.53.21) to resolve the issue. In parallel, we're reaching out to some affected users that have downloaded the fix version to confirm that it effectively to validate it's success. Users who haven't previously updated their app version and were unaffected will not need to take any action. Scope of impact: Impact is specific to users who have updated the OneDrive for Business iOS app to the latest version. Start time: Thursday, October 21, 2021, at 9:05 PM'UTC Root cause: A recent update for the OneDrive for Business iOS app contained a code regression, resulting in impact. Next update by: Monday, October 25, 2021, at 9:00 PMUTC; PublishedTime=2021-10-22T19:46:37.337Z
Root cause A recent update for the OneDrive for Business iOS app contained a code regression, resulting in impact. Next update by: Monday, October 25, 2021, at 9:00 PMUTC; PublishedTime=2021-10-22T19:46:37.337Z

OD292873 Details

Status: We're reviewing system diagnostics to isolate the source of the issue. Next update by: Thursday, October 21, 2021, at 12:00 PM UTC; PublishedTime=2021-10-21T09:21:36.713Z

Status: We've identified that a recent update for the OneDrive for Business App contained a code regression, resulting in impact. We're developing a fix which we estimate to have completed by Thursday, October 21, 2021, at 7:00 PM UTC. Next update by: Thursday, October 21, 2021, at 7:00 PM UTC; PublishedTime=2021-10-21T10:39:15.253Z

Status: We're continuing the development and validation process to prepare the fix for deployment. We expect the fix development will complete by the end of Thursday, October 21, 2021, after which we'll submit it to Apple for approval and subsequently release it for users to download. We'll provide continuous updates as this process progresses. Next update by: Friday, October 22, 2021, at 7:00 PMUTC; PublishedTime=2021-10-21T17:57:29.14Z

Status: We've completed our fix and it's been approved by Apple. We're in the final phases of testing and expect to have an estimation on the deployment schedule by our next update. Next update by: Friday, October 22, 2021, at 9:00 PMUTC; PublishedTime=2021-10-22T17:03:30.56Z

Status: The latest version of the OneDrive for Business iOS app - version 12.53.21 - is now available in the company portal. Affected users will need to download the latest version(12.53.21) to resolve the issue. In parallel, we're reaching out to some affected users that have downloaded the fix version to confirm that it effectively to validate it's success. Users who haven't previously updated their app version and were unaffected will not need to take any action. Next update by: Monday, October 25, 2021, at 9:00 PMUTC; PublishedTime=2021-10-22T19:46:37.337Z

--------------------------------------------------------------------------------------------------
Title Some users intermittently see only profile pictures or avatars instead of video during Microsoft Teams meetings
ID TM292912
Service Microsoft Teams
Feature Teams Components
Status Service degradation
Impact Users are intermittently seeing only profile pictures or avatars instead of video during Microsoft Teams meetings.
Severity Sev2
Start time 2021-10-21T15:31:34Z
End time
Last update 2021-10-22T20:27:24.173Z
More info
Root cause Video rendering logic within Microsoft Teams isn't performing as expected due to a possible code regression. Next update by: Monday, October 25, 2021, at 9:30 PMUTC; PublishedTime=2021-10-22T20:26:26.99Z

TM292912 Details

Status: We've determined that the video rendering logic within Microsoft Teams isn't performing as expected due to a possible code regression. We're developing a code fix for the rendering logic that may remediate impact. Next update by: Thursday, October 21, 2021, at 5:30 PMUTC; PublishedTime=2021-10-21T15:31:53.79Z

Status: We're continuing to develop a code fix for the rendering logic that may remediate impact. We'll monitor the progress of the fix development process and provide a deployment timeline as soon as one becomes available. Next update by: Friday, October 22, 2021, at 9:30 PMUTC; PublishedTime=2021-10-21T16:43:57.557Z

Status: We've validated and initiated the incremental deployment of a fix for the rendering logic, which we anticipate should complete early next week. We'll have a better estimation on the impact resolution by our next update. Next update by: Monday, October 25, 2021, at 9:30 PMUTC; PublishedTime=2021-10-22T20:26:26.99Z

--------------------------------------------------------------------------------------------------
Title Users may be unable to execute flows in Microsoft Power Automate
ID MF292932
Service Microsoft Power Automate in Microsoft 365
Feature Service and web access issues
Status Service restored
Impact Users may have been unable to execute flows in Microsoft Power Automate.
Severity Sev2
Start time 2021-10-21T14:05:00Z
End time 2021-10-21T19:21:30Z
Last update 2021-10-21T21:39:04.047Z
More info Affected flow operations included create, delete, clone, edit, design, run, get, and update, and users may have receive an error message which stated "HTTP 503 Service unavailable." Final status: We've identified that the high resource utilization issue occurred as the result of an update process which took longer than expected, and automated recovery functions did not take action as expected. We've completed our resource utilization configuration adjustments manually, and have confirmed that impact is resolved. Scope of impact: Your organization was affected by this event, and all users may have been unable to execute flows in Microsoft Power Automate. Start time: Thursday, October 21, 2021, at 2:05 PM UTC End time: Thursday, October 21, 2021, at 6:03 PM UTC Root cause: A portion of infrastructure responsible for processing requests for the Microsoft 365 services connectors wasn't performing as expected due to an update process taking longer than expected, resulting in impact. Next steps: - We're reviewing our automated recovery functionality to determine why this failed to recover prior to manual intervention. - We're investigating the update process to determine why it was taking longer than usual to better prevent similar future impact. This is the final update for the event.; PublishedTime=2021-10-21T19:21:48.01Z
Root cause A portion of infrastructure responsible for processing requests for the Microsoft 365 services connectors wasn't performing as expected due to an update process taking longer than expected, resulting in impact. Next steps: - We're reviewing our automated recovery functionality to determine why this failed to recover prior to manual intervention. - We're investigating the update process to determine why it was taking longer than usual to better prevent similar future impact. This is the final update for the event.; PublishedTime=2021-10-21T19:21:48.01Z

MF292932 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're reviewing recent changes to the service to determine if they may be responsible for impact and formulate our next troubleshooting steps. Next update by: Thursday, October 21, 2021, at 7:00 PMUTC; PublishedTime=2021-10-21T17:42:05.22Z

Status: Our investigation has determined that a portion of infrastructure responsible for processing requests for the Microsoft 365 services connectors isn't performing as expected, resulting in impact. We're reviewing additional service health telemetry information to better understand why the infrastructure is underperforming, while in parallel, we're adjusting resource utilization configurations on the affected infrastructure to remediate impact. Next update by: Thursday, October 21, 2021, at 9:00 PMUTC; PublishedTime=2021-10-21T18:55:44.337Z

Final status: We've identified that the high resource utilization issue occurred as the result of an update process which took longer than expected, and automated recovery functions did not take action as expected. We've completed our resource utilization configuration adjustments manually, and have confirmed that impact is resolved. Next update by: Thursday, October 21, 2021, at 9:00 PMUTC; PublishedTime=2021-10-21T18:55:44.337Z

--------------------------------------------------------------------------------------------------
Title Users may be unable to execute flows in Microsoft Power Automate
ID MF292932
Service Microsoft Power Automate in Microsoft 365
Feature Service and web access issues
Status Service restored
Impact Users may have been unable to execute flows in Microsoft Power Automate.
Severity Sev2
Start time 2021-10-21T14:05:00Z
End time 2021-10-21T19:21:30Z
Last update 2021-10-21T21:39:04.047Z
More info Affected flow operations included create, delete, clone, edit, design, run, get, and update, and users may have receive an error message which stated "HTTP 503 Service unavailable." Final status: We've identified that the high resource utilization issue occurred as the result of an update process which took longer than expected, and automated recovery functions did not take action as expected. We've completed our resource utilization configuration adjustments manually, and have confirmed that impact is resolved. Scope of impact: Your organization was affected by this event, and all users may have been unable to execute flows in Microsoft Power Automate. Start time: Thursday, October 21, 2021, at 2:05 PM UTC End time: Thursday, October 21, 2021, at 6:03 PM UTC Root cause: A portion of infrastructure responsible for processing requests for the Microsoft 365 services connectors wasn't performing as expected due to an update process taking longer than expected, resulting in impact. Next steps: - We're reviewing our automated recovery functionality to determine why this failed to recover prior to manual intervention. - We're investigating the update process to determine why it was taking longer than usual to better prevent similar future impact. This is the final update for the event.; PublishedTime=2021-10-21T19:21:48.01Z
Root cause A portion of infrastructure responsible for processing requests for the Microsoft 365 services connectors wasn't performing as expected due to an update process taking longer than expected, resulting in impact. Next steps: - We're reviewing our automated recovery functionality to determine why this failed to recover prior to manual intervention. - We're investigating the update process to determine why it was taking longer than usual to better prevent similar future impact. This is the final update for the event.; PublishedTime=2021-10-21T19:21:48.01Z

MF292932 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're reviewing recent changes to the service to determine if they may be responsible for impact and formulate our next troubleshooting steps. Next update by: Thursday, October 21, 2021, at 7:00 PMUTC; PublishedTime=2021-10-21T17:42:05.22Z

Status: Our investigation has determined that a portion of infrastructure responsible for processing requests for the Microsoft 365 services connectors isn't performing as expected, resulting in impact. We're reviewing additional service health telemetry information to better understand why the infrastructure is underperforming, while in parallel, we're adjusting resource utilization configurations on the affected infrastructure to remediate impact. Next update by: Thursday, October 21, 2021, at 9:00 PMUTC; PublishedTime=2021-10-21T18:55:44.337Z

Final status: We've identified that the high resource utilization issue occurred as the result of an update process which took longer than expected, and automated recovery functions did not take action as expected. We've completed our resource utilization configuration adjustments manually, and have confirmed that impact is resolved. Next update by: Thursday, October 21, 2021, at 9:00 PMUTC; PublishedTime=2021-10-21T18:55:44.337Z

--------------------------------------------------------------------------------------------------
Title Users may be unable to execute flows in Microsoft Power Automate
ID MF292932
Service Microsoft Power Automate in Microsoft 365
Feature Service and web access issues
Status Service restored
Impact Users may have been unable to execute flows in Microsoft Power Automate.
Severity Sev2
Start time 2021-10-21T14:05:00Z
End time 2021-10-21T19:21:30Z
Last update 2021-10-21T21:39:04.047Z
More info Affected flow operations included create, delete, clone, edit, design, run, get, and update, and users may have receive an error message which stated "HTTP 503 Service unavailable." Final status: We've identified that the high resource utilization issue occurred as the result of an update process which took longer than expected, and automated recovery functions did not take action as expected. We've completed our resource utilization configuration adjustments manually, and have confirmed that impact is resolved. Scope of impact: Your organization was affected by this event, and all users may have been unable to execute flows in Microsoft Power Automate. Start time: Thursday, October 21, 2021, at 2:05 PM UTC End time: Thursday, October 21, 2021, at 6:03 PM UTC Root cause: A portion of infrastructure responsible for processing requests for the Microsoft 365 services connectors wasn't performing as expected due to an update process taking longer than expected, resulting in impact. Next steps: - We're reviewing our automated recovery functionality to determine why this failed to recover prior to manual intervention. - We're investigating the update process to determine why it was taking longer than usual to better prevent similar future impact. This is the final update for the event.; PublishedTime=2021-10-21T19:21:48.01Z
Root cause A portion of infrastructure responsible for processing requests for the Microsoft 365 services connectors wasn't performing as expected due to an update process taking longer than expected, resulting in impact. Next steps: - We're reviewing our automated recovery functionality to determine why this failed to recover prior to manual intervention. - We're investigating the update process to determine why it was taking longer than usual to better prevent similar future impact. This is the final update for the event.; PublishedTime=2021-10-21T19:21:48.01Z

MF292932 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're reviewing recent changes to the service to determine if they may be responsible for impact and formulate our next troubleshooting steps. Next update by: Thursday, October 21, 2021, at 7:00 PMUTC; PublishedTime=2021-10-21T17:42:05.22Z

Status: Our investigation has determined that a portion of infrastructure responsible for processing requests for the Microsoft 365 services connectors isn't performing as expected, resulting in impact. We're reviewing additional service health telemetry information to better understand why the infrastructure is underperforming, while in parallel, we're adjusting resource utilization configurations on the affected infrastructure to remediate impact. Next update by: Thursday, October 21, 2021, at 9:00 PMUTC; PublishedTime=2021-10-21T18:55:44.337Z

Final status: We've identified that the high resource utilization issue occurred as the result of an update process which took longer than expected, and automated recovery functions did not take action as expected. We've completed our resource utilization configuration adjustments manually, and have confirmed that impact is resolved. Next update by: Thursday, October 21, 2021, at 9:00 PMUTC; PublishedTime=2021-10-21T18:55:44.337Z

--------------------------------------------------------------------------------------------------
Title Users may experience intermittent issues with updates from communities, groups, and creating groups in Yammer
ID YA292943
Service Yammer Enterprise
Feature Yammer Components
Status Service restored
Impact Users may have experienced intermittent issues with updates from communities, groups, and creating groups in Yammer.
Severity Sev2
Start time 2021-10-20T19:00:00Z
End time 2021-10-21T18:31:00Z
Last update 2021-10-21T19:42:04.76Z
More info
Root cause A Yammer authentication component was misconfigured and caused connection issues with Azure Active Directory. Next steps: - We'll continue to monitor service health and look into ways to notify us of authentication component misconfigurations more promptly in the future. This is the final update for the event.; PublishedTime=2021-10-21T19:42:04.76Z

YA292943 Details

Status: We've investigated a problem with the Yammer service in which users may have experienced intermittent issues with updates from communities, groups, and creating groups and confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-10-21T18:52:17.267

Final status: During our investigation we found that an authentication component configuration was misconfigured. Engineers corrected the configuration and impact was remediated shortly after. Users are now able to see and create updates from their communities and groups in Yammer.

--------------------------------------------------------------------------------------------------
Title Some Power Automate audit logs may be unavailable through the Security & Compliance Center (SCC)
ID EX292764
Service Exchange Online
Feature Management and Provisioning
Status Service restored
Impact Admins' Power Automate audit log searches through the SCC may not return data return all expected data.
Severity Sev2
Start time 2021-10-20T18:30:00Z
End time 2021-10-20T23:39:00Z
Last update 2021-10-21T19:21:02.873Z
More info Audit log data since October 19, 2021, at 4:40 PM UTC may have been unavailable. Final status: We've confirmed that our changes have taken affect via internal monitoring and impact is remediated. Scope of impact: Your organization was affected by this event and any admin searching Power Automate audit log data since October 19, 2021, at 4:30 PM UTC through the SCC may have been affected. Start time: Wednesday, October 20, 2021, at 6:30 PM'UTC End time: Wednesday, October 20, 2021, at 11:39 PM'UTC Root cause: A recent policy update meant to improve data transfer between the Power Automate services and the SCC for audit reports introduced a misconfiguration, resulting in the impact. Next steps: - We're reviewing our update procedures to better identify similar misconfiguration issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-21T19:21:02.873Z
Root cause A recent policy update meant to improve data transfer between the Power Automate services and the SCC for audit reports introduced a misconfiguration, resulting in the impact. Next steps: - We're reviewing our update procedures to better identify similar misconfiguration issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-21T19:21:02.873Z

EX292764 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-10-20T19:53:51.147

Status: We've taken corrective action to mitigate this issue and will provide additional details on the root cause and resolution steps on our next update. Next update by: Wednesday, October 20, 2021, at 9:30 PM UTC; PublishedTime=2021-10-20T20:10:01.6Z

Status: We've confirmed that a recent policy update meant to improve data transfer between the Power Automate services and the SCC for audit reports introduced a misconfiguration, resulting in the impact. We've redirected traffic away from the problematic policy and infrastructure where the configuration issue is present to allow updated data to become available again. We've also corrected the configuration setting to allow data from the impact period to become available; however, it may take up to 24 hours for this to take effect. Next update by: Thursday, October 21, 2021, at 9:00 PMUTC; PublishedTime=2021-10-20T21:28:06.087Z

Final status: We've confirmed that our changes have taken affect via internal monitoring and impact is remediated. Next update by: Thursday, October 21, 2021, at 9:00 PMUTC; PublishedTime=2021-10-20T21:28:06.087Z

--------------------------------------------------------------------------------------------------
Title Some users may notice external email messages aren't automatically flagged as external
ID EX292874
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users may have noticed external email messages aren't automatically flagged as external.
Severity Sev2
Start time 2021-10-20T14:35:00Z
End time 2021-10-21T16:33:00Z
Last update 2021-10-21T19:56:17.837Z
More info Affected users would have seen that external messages didn't include the banner that states: "This sender was from outside your organization". Final status: We've completed deploying the override with the correct settings. After checking telemetry and reaching out to affected users, we've confirmed that impact has been successfully remediated. It may take up to 24 hours for the override to fully propagate, and although our data supports that this issue is resolved for all users, should users encounter this issue again then they may need to wait until Friday, October 22, 2021, at 4:33 PM UTC for the issue to be resolved. Scope of impact: Impact was specific to users who were served through the affected infrastructure and utilize this feature. Start time: Wednesday, October 20, 2021, at 2:35 PM UTC End time: Thursday, October 21, 2021, at 4:33 PM UTC Root cause: An authentication configuration issue within a recent service update was causing a communication issue for external email flagging, resulting in impact. Next steps: - We're reviewing our update procedures to understand why this update contained an authentication configuration issue and better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-21T19:35:32.75Z
Root cause An authentication configuration issue within a recent service update was causing a communication issue for external email flagging, resulting in impact. Next steps: - We're reviewing our update procedures to understand why this update contained an authentication configuration issue and better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-21T19:35:32.75Z

EX292874 Details

Status: We're reviewing error logs and details provided by affected users to determine our next troubleshooting steps. Next update by: Thursday, October 21, 2021, at 11:30 AM UTC; PublishedTime=2021-10-21T09:32:10.923Z

Status: Our initial analysis of error logs proved inconclusive. We're revising all recent deployments to help isolate the root cause of the issue. Next update by: Thursday, October 21, 2021, at 1:30 PM UTC; PublishedTime=2021-10-21T11:08:32.83Z

Status: From our investigation of recent deployments, we've identified a new configuration deployment is impacting the automated rules that tag external emails. We're reviewing this configuration to identify why this occurred and develop a mitigation plan. Next update by: Thursday, October 21, 2021, at 3:30 PM UTC; PublishedTime=2021-10-21T13:14:41.16Z

Status: We're continuing to review the suspected configuration, along with Exchange diagnostics logs, to understand why this new configuration deployment is impacting the automated rules that tag external email messages. Next update by: Thursday, October 21, 2021, at 6:00 PMUTC; PublishedTime=2021-10-21T15:23:31.587Z

Status: We've determined that an authentication configuration issue within a recent service update is causing a communication issue for external email flagging, resulting in impact. We're deploying an override with the correct settings, which we anticipate will resolve the issue. Next update by: Thursday, October 21, 2021, at 8:00 PMUTC; PublishedTime=2021-10-21T17:26:12.5Z

Final status: We've completed deploying the override with the correct settings. After checking telemetry and reaching out to affected users, we've confirmed that impact has been successfully remediated. It may take up to 24 hours for the override to fully propagate, and although our data supports that this issue is resolved for all users, should users encounter this issue again then they may need to wait until Friday, October 22, 2021, at 4:33 PM UTC for the issue to be resolved. Next update by: Thursday, October 21, 2021, at 8:00 PMUTC; PublishedTime=2021-10-21T17:26:12.5Z

--------------------------------------------------------------------------------------------------
Title Users wouldn't have had their locations sent when making an emergency call
ID TM292728
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users wouldn't have had their locations sent when making an emergency call.
Severity Sev2
Start time 2021-10-20T13:14:00Z
End time 2021-10-20T14:00:00Z
Last update 2021-10-20T14:13:54.903Z
More info
Root cause A section of infrastructure, responsible for sending emergency locations, was seeing higher levels of user traffic than expected. Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-10-20T14:13:46.69Z

TM292728 Details

Status: We're reviewing diagnostic data from the impacted service to determine our next troubleshooting steps. Next update by: Wednesday, October 20, 2021, at 3:30 PM UTC; PublishedTime=2021-10-20T13:35:01.28Z

Final status: We've identified a section of infrastructure, responsible for sending emergency locations, was seeing higher levels of user traffic than expected. We've optimised the environment to handle requests more efficiently and confirmed that this has resolved the issue. Next update by: Wednesday, October 20, 2021, at 3:30 PM UTC; PublishedTime=2021-10-20T13:35:01.28Z

--------------------------------------------------------------------------------------------------
Title Users may be unable to update their Microsoft Teams presence
ID TM292721
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have been unable to update their Microsoft Teams presence.
Severity Sev2
Start time 2021-10-20T10:00:00Z
End time 2021-10-20T17:00:00Z
Last update 2021-10-20T17:51:16.277Z
More info
Root cause A networking issue was causing some infrastructure that facilitates updates to user presence to perform below acceptable performance thresholds, resulting in the impact. Next steps: We're continuing our investigation to determine what caused the offending networking problems to prevent issues like this from occurring in the future. This is the final update for the event.; PublishedTime=2021-10-20T17:29:11.443Z

TM292721 Details

Status: We're analyzing system logs to determine the source of the issue. Next update by: Wednesday, October 20, 2021, at 2:30 PM UTC; PublishedTime=2021-10-20T12:16:43.643Z

Status: We've identified a potential networking issue which may be causing some infrastructure, that facilitates updating user presence, to perform below acceptable performance thresholds. We're further investigating the potential networking issue, to confirm that this is the cause of impact. Next update by: Wednesday, October 20, 2021, at 5:30 PM UTC; PublishedTime=2021-10-20T14:08:20.663Z

Final status: We've confirmed that a networking issue was causing some infrastructure that facilitates updating user presence to perform below acceptable performance thresholds, resulting in the impact. We've redirected the affected traffic to healthy infrastructure to mitigate the impact and confirmed via telemetry that this issue is resolved. Next update by: Wednesday, October 20, 2021, at 5:30 PM UTC; PublishedTime=2021-10-20T14:08:20.663Z

--------------------------------------------------------------------------------------------------
Title Users can't utilize the Tasks app in Microsoft Teams
ID TM291906
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were unable to utilize the Tasks app in Microsoft Teams.
Severity Sev2
Start time 2021-10-15T21:52:00Z
End time 2021-10-15T23:35:00Z
Last update 2021-10-16T00:10:21.62Z
More info Desktop and web app users were able to access cached content in the Tasks app, but wouldn't see new content. Additionally, new users without cached content weren't able to open the app. Mobile users would have been able to open the app but would have been unable to utilize any features. Final status: Our investigation determined that a service authentication component that Microsoft Teams relies on for Tasks app integration wasn't configured correctly, resulting in impact. We've developed and deployed a fix to update the misconfiguration and confirmed via system monitoring that this has mitigated impact. Scope of impact: This issue may have potentially affected any of your users attempting to utilize the Tasks app in Microsoft Teams. Start time: Friday, October 15, 2021, at 9:52 PM UTC End time: Friday, October 15, 2021, at 11:35 PM UTC Root cause: A service authentication component that Microsoft Teams relies on for Tasks app integration wasn't configured correctly, resulting in impact. Next steps: - We're reviewing our authentication component configuration update and review processes to prevent similar impact in the future. This is the final update for the event.; PublishedTime=2021-10-16T00:10:21.62Z
Root cause A service authentication component that Microsoft Teams relies on for Tasks app integration wasn't configured correctly, resulting in impact. Next steps: - We're reviewing our authentication component configuration update and review processes to prevent similar impact in the future. This is the final update for the event.; PublishedTime=2021-10-16T00:10:21.62Z

TM291906 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-10-15T23:00:14.313

Status: We're reviewing available system data to determine possible underlying causes of this issue. Next update by: Saturday, October 16, 2021, at 12:30 AMUTC; PublishedTime=2021-10-15T23:18:03.957Z

Final status: Our investigation determined that a service authentication component that Microsoft Teams relies on for Tasks app integration wasn't configured correctly, resulting in impact. We've developed and deployed a fix to update the misconfiguration and confirmed via system monitoring that this has mitigated impact. Next update by: Saturday, October 16, 2021, at 12:30 AMUTC; PublishedTime=2021-10-15T23:18:03.957Z

--------------------------------------------------------------------------------------------------
Title Some users are intermittently unable to update their Microsoft Teams presence
ID TM291520
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were intermittently unable to update their Microsoft Teams presence.
Severity Sev2
Start time 2021-10-14T12:50:00Z
End time 2021-10-14T17:20:00Z
Last update 2021-10-14T18:59:46.9Z
More info Additionally, users may have seen the stale or unknown presence of other users who were affected by this issue. Final status: We re-directed affected traffic away from the problematic portion of infrastructure and have confirmed via telemetry that this has mitigated impact. Scope of impact: Impact was specific to some users who were served through the affected infrastructure. Start time: Thursday, October 14, 2021, at 12:50 PM'UTC End time: Thursday, October 14, 2021, at 5:20 PM'UTC Root cause: A portion of infrastructure on which Microsoft Teams relies for updating user presence information was operating in a sub-optimal state, which resulted in impact. Next steps: - We're reviewing our Unified Presence Service (UPS) code for improved performance and potential automated recovery options to reduce or avoid similar impact in the future. - We're analyzing the infrastructure on which this service relies to better understand how a portion of it fell into a sub-optimal state. This is the final update for the event.; PublishedTime=2021-10-14T18:59:46.9Z
Root cause A portion of infrastructure on which Microsoft Teams relies for updating user presence information was operating in a sub-optimal state, which resulted in impact. Next steps: - We're reviewing our Unified Presence Service (UPS) code for improved performance and potential automated recovery options to reduce or avoid similar impact in the future. - We're analyzing the infrastructure on which this service relies to better understand how a portion of it fell into a sub-optimal state. This is the final update for the event.; PublishedTime=2021-10-14T18:59:46.9Z

TM291520 Details

Status: We're analyzing diagnostic data to determine the source of the issue and develop a mitigation plan. Next update by: Thursday, October 14, 2021, at 4:30 PM UTC; PublishedTime=2021-10-14T14:21:39.377Z

Status: We believe that a portion of infrastructure which Microsoft Teams relies on for updating user presence information is operating in a sub-optimal state, resulting in impact. We're continuing to investigate service logs and infrastructure data to confirm our next troubleshooting steps. Next update by: Thursday, October 14, 2021, at 7:30 PMUTC; PublishedTime=2021-10-14T16:22:50.447Z

Final status: We re-directed affected traffic away from the problematic portion of infrastructure and have confirmed via telemetry that this has mitigated impact. Next update by: Thursday, October 14, 2021, at 7:30 PMUTC; PublishedTime=2021-10-14T16:22:50.447Z

--------------------------------------------------------------------------------------------------
Title Users are intermittently unable to update their Microsoft Teams presence
ID TM291482
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were intermittently unable to update their Microsoft Teams presence.
Severity Sev2
Start time 2021-10-14T06:40:00Z
End time 2021-10-14T07:55:00Z
Last update 2021-10-14T22:30:23.02Z
More info
Root cause A section of database infrastructure, that facilitates updates to Microsoft Teams presence, was performing below acceptable performance thresholds. Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-10-14T08:58:01.86Z

TM291482 Details

Status: We're analyzing diagnostic data to determine the source of the issue and develop a mitigation plan. Next update by: Thursday, October 14, 2021, at 10:00 AM UTC; PublishedTime=2021-10-14T07:51:40.2Z

Final status: We've identified that a section of database infrastructure, that facilitates updates to Microsoft Teams presence, was performing below acceptable performance thresholds. We've removed the problematic infrastructure from service, and confirmed from monitoring that the issue has been mitigated. Next update by: Thursday, October 14, 2021, at 10:00 AM UTC; PublishedTime=2021-10-14T07:51:40.2Z

--------------------------------------------------------------------------------------------------
Title Users can only see a portion of image thumbnails when attached in the Microsoft Teams web client
ID TM291450
Service Microsoft Teams
Feature Teams Components
Status Service degradation
Impact Users can only see a portion of image thumbnails when attached in the Microsoft Teams web client.
Severity Sev2
Start time 2021-10-14T03:01:39Z
End time
Last update 2021-10-20T17:35:28.997Z
More info The thumbnail images are likely zoomed into a specific part of the image. Clicking the image shows the image correctly. To bypass the impact in the Microsoft Teams web client, users can copy/paste the image instead of attaching it. Alternatively, users can view the attachment thumbnail in the Microsoft Teams desktop client. Current status: We've determined that the feature setting enablement will require a code change deployment to remediate impact. We're developing a code fix to remeidate impact and expect that testing should be complete by our next scheduled update. Scope of impact: Your organization is affected by this event, and any user attempting to attach images may be impacted. Root cause: A recent update to an image preview feature for the Microsoft Teams desktop client created a compatibility and rendering issue for attached thumbnails is responsible for the impact. Next update by: Monday, October 25, 2021, at 7:00 PMUTC; PublishedTime=2021-10-20T17:35:28.997Z
Root cause A recent update to an image preview feature for the Microsoft Teams desktop client created a compatibility and rendering issue for attached thumbnails is responsible for the impact. Next update by: Monday, October 25, 2021, at 7:00 PMUTC; PublishedTime=2021-10-20T17:35:28.997Z

TM291450 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're analyzing impact information provided by some affected users to determine the source of the issue. Next update by: Thursday, October 14, 2021, at 5:30 AMUTC; PublishedTime=2021-10-14T03:22:25.61Z

Status: We're analyzing network trace logs and reproducing the impact to isolate the problem. Next update by: Thursday, October 14, 2021, at 7:30 AMUTC; PublishedTime=2021-10-14T04:45:32.98Z

Status: We've identified that a recent change to a feature, responsible for image previews, may be causing impact. We're investigating this change further, to confirm our findings and to develop a mitigation plan. Next update by: Thursday, October 14, 2021, at 7:00 PM UTC; PublishedTime=2021-10-14T07:23:08.92Z

Status: We've confirmed that a recent update to an image preview feature for the Microsoft Teams desktop client created a compatibility and rendering issue for attached thumbnails is responsible for the impact. We've developed a fix for this issue in the form of enabling additional feature settings. While we expect that this feature setting enable action will need to progress through our safe deployment management process; we're investigating methods for expediting the process without causing additional impact. Next update by: Wednesday, October 20, 2021, at 7:00 PM UTC; PublishedTime=2021-10-14T18:21:51.307Z

Status: We've determined that the feature setting enablement will require a code change deployment to remediate impact. We're developing a code fix to remeidate impact and expect that testing should be complete by our next scheduled update. Next update by: Monday, October 25, 2021, at 7:00 PMUTC; PublishedTime=2021-10-20T17:35:28.997Z

--------------------------------------------------------------------------------------------------
Title Some admins may be unable to access the SharePoint admin center from the left pane in the classic experience
ID SP291527
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Admins may have been unable to access the SharePoint admin center from the left pane in the classic experience.
Severity Sev2
Start time 2021-10-14T02:47:00Z
End time 2021-10-14T17:20:00Z
Last update 2021-10-14T21:50:15.743Z
More info Admins could have accessed the SharePoint admin center from the left pane in the modern experience. Alternatively, admins could have accessed the SharePoint admin center directly through this link, "https://tenantURL-admin/_layouts/15/online/AdminHome.aspx#/settings/NewPortalAsDefault" by substituting their own tenant URL for the "tenantURL" section of the link. Final status: We've discovered that impact was specific to admins whose organizations have opted to receive Private Preview features. We've confirmed with affected admins that disabling the offending code portion has mitigated the issue. Scope of impact: This issue may have affected admins whose organizations have opted to receive Private Preview features and was specifically felt when admins attempted to access the SharePoint admin center via the left pane in the classic experience. Start time: Thursday, October 14, 2021, at 2:47 AM UTC End time: Thursday, October 14, 2021, at 5:20 PM UTC Root cause: A recent change introduced a code issue that resulted in impact. Next steps: - We're reviewing our SharePoint admin center update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-14T21:00:03.39Z
Root cause A recent change introduced a code issue that resulted in impact. Next steps: - We're reviewing our SharePoint admin center update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-14T21:00:03.39Z

SP291527 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-10-14T15:24:23.867

Status: We're reviewing user reports and attempting to reproduce the issue in our internal environment to gain a better understanding of root cause and identify our next troubleshooting steps. Next update by: Thursday, October 14, 2021, at 5:30 PMUTC; PublishedTime=2021-10-14T15:49:04.367Z

Status: We successfully reproduced the issue in our internal environment and are analyzing this outcome as well as reviewing recent changes, correlation IDs, and Unified Logging Service (ULS) logs to gain a deeper understanding of root cause and inform our next troubleshooting actions. Next update by: Thursday, October 14, 2021, at 8:30 PMUTC; PublishedTime=2021-10-14T17:14:06.787Z

Status: We've identified a recent change meant to enhance the service may have inadvertently resulted in impact. We've disabled the offending portion of code and initial tests prove this was effective in mitigating impact. We're reaching out to some affected users to confirm that service is restored. Next update by: Thursday, October 14, 2021, at 10:30 PMUTC; PublishedTime=2021-10-14T17:38:40.477Z

Final status: We've discovered that impact was specific to admins whose organizations have opted to receive Private Preview features. We've confirmed with affected admins that disabling the offending code portion has mitigated the issue. Next update by: Thursday, October 14, 2021, at 10:30 PMUTC; PublishedTime=2021-10-14T17:38:40.477Z

--------------------------------------------------------------------------------------------------
Title Users with SafetyNet enforced policies may experience compliance failures and lose access to corporate resources
ID IT291049
Service Microsoft Intune
Feature Microsoft Intune
Status Service restored
Impact Users with SafetyNet enforced policies may have experienced compliance failures and lost access to corporate resources.
Severity Sev2
Start time 2021-10-13T16:50:00Z
End time 2021-10-13T23:30:00Z
Last update 2021-10-14T00:14:32.407Z
More info This issue affected Android Enterprise Dedicated, Android Enterprise Fully-Managed, or Corporate Owned with a Work Profile user. Devices were updated once they checked into the service. To manually force the update, a manual sync could have been triggered by going to Settings (Settings > Privacy > Your Work Policy info > Top menu > Sync policies). Final status: We've confirmed that over 99 percent of the affected devices have recovered and the issue is now mitigated. Scope of impact: Impact was specific to some Android MDM devices that met the criteria outlined in the More info section that fail the SafetyNet attestation check. Start time: Wednesday, October 13, 2021, at 4:50 PM UTC End time: Wednesday, October 13, 2021, at 11:30 PM UTC Root cause: A recent third-party partner update was causing users with SafetyNet enforced policies to experience compliance failures and lose access to corporate resources. Next steps: - We're working with our third-party partner to determine ways to prevent future updates from resulting in similar impact. This is the final update for the event.; PublishedTime=2021-10-13T23:56:59.6Z
Root cause A recent third-party partner update was causing users with SafetyNet enforced policies to experience compliance failures and lose access to corporate resources. Next steps: - We're working with our third-party partner to determine ways to prevent future updates from resulting in similar impact. This is the final update for the event.; PublishedTime=2021-10-13T23:56:59.6Z

IT291049 Details

Status: We're reviewing available diagnostic data and third party API logging to isolate the source of impact and identify mitigations actions. Next update by: Tuesday, October 12, 2021, at 11:00 PMUTC; PublishedTime=2021-10-12T21:59:58.04Z

Status: We're continuing our review of available diagnostic data and third party API logging to isolate the source of impact. Additionally, we're working to gather additional reproduction data from our third party to assist in our investigation. Next update by: Wednesday, October 13, 2021, at 2:00 AMUTC; PublishedTime=2021-10-12T22:53:21.027Z

Status: Our third party partner has confirmed that they are able to reproduce the issue. We're actively collaborating to troubleshoot and remediate the impact. Next update by: Wednesday, October 13, 2021, at 5:00 AM UTC; PublishedTime=2021-10-13T00:37:03.117Z

Status: Our third party partner, has identified a potential issue with a recent update and are in the process of releasing a fix. We are actively collaborating to remediate the issue. Next update by: Wednesday, October 13, 2021, at 3:00 PMUTC; PublishedTime=2021-10-13T03:28:40.023Z

Status: Our third party partner has confirmed that impact is caused by a recent update. They've completed a roll back of the update and we have had confirmation that impact has been resolved on the majority of affected devices. We're monitoring the remainder of devices until impact has been successfully mitigated. Next update by: Wednesday, October 13, 2021, at 7:30 PM UTC; PublishedTime=2021-10-13T14:00:16.01Z

Status: We're continuing to monitor a small group of devices that remain affected following our third-party partner's update reversion. We're expecting for the remainder of impacted devices to see the remediation of impact by our next scheduled update, and if not, we'll reassess our estimate for the process completing and the remediation of impact. Next update by: Thursday, October 14, 2021, at 1:30 AM UTC; PublishedTime=2021-10-13T18:16:01.073Z

Final status: We've confirmed that over 99 percent of the affected devices have recovered and the issue is now mitigated. Next update by: Thursday, October 14, 2021, at 1:30 AM UTC; PublishedTime=2021-10-13T18:16:01.073Z

--------------------------------------------------------------------------------------------------
Title Users are intermittently unable to update their Microsoft Teams presence
ID TM291000
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were intermittently unable to update their Microsoft Teams presence.
Severity Sev2
Start time 2021-10-12T14:20:00Z
End time 2021-10-12T18:15:00Z
Last update 2021-10-12T21:20:20.337Z
More info
Root cause A subsection of Microsoft Teams database infrastructure that updates user presence information dropped below our controllable service performance thresholds, resulting in impact. Next steps: - We're analyzing performance data and trends on the affected databases to better understand why performance levels dropped, and to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-10-12T21:20:02.233Z

TM291000 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-10-12T15:40:57.163

Status: We've determined that a subsection of Microsoft Teams service infrastructure that updates user presence information has dropped below our manageable service performance thresholds, resulting in impact. We're reviewing service logs and infrastructure data to determine the reasoning for this drop in performance and identify our next troubleshooting steps. Next update by: Tuesday, October 12, 2021, at 6:00 PMUTC; PublishedTime=2021-10-12T16:05:33.6Z

Status: We've redistributed traffic to an alternate subsection of database infrastructure and we're monitoring the environment to ensure that impact has been resolved. Next update by: Tuesday, October 12, 2021, at 8:00 PMUTC; PublishedTime=2021-10-12T17:43:06.353Z

Status: Our internal telemetry indicates that users are no longer experiencing problems when updating their presence within Microsoft Teams. During our monitoring of the environment, we've determined that additional service traffic throughput will be required to ensure that the problem does not return. We're conducting our additional traffic distribution to ensure that impact remains remediated. Next update by: Tuesday, October 12, 2021, at 10:00 PMUTC; PublishedTime=2021-10-12T19:48:13.147Z

Final status: We've internally validated that our redistribution of traffic to an alternate subsection of database infrastructure has successfully remediated impact. Next update by: Tuesday, October 12, 2021, at 10:00 PMUTC; PublishedTime=2021-10-12T19:48:13.147Z

--------------------------------------------------------------------------------------------------
Title Some users may be intermittently unable to update their presence in Microsoft Teams
ID TM290799
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have been intermittently unable to update their presence in Microsoft Teams.
Severity Sev2
Start time 2021-10-11T14:00:00Z
End time 2021-10-11T19:00:00Z
Last update 2021-10-11T20:28:47.233Z
More info Users may have additionally seen outdated presence information for affected users. Final status: We've re-directed affected traffic away from the problematic portion of infrastructure and have confirmed via telemetry that this has mitigated impact. Scope of impact: This issue could have potentially affected any user intermittently if they were routed through the affected infrastructure. Start time: Monday, October 11, 2021, at 2:00 PM'UTC End time: Monday, October 11, 2021, at 7:00 PM'UTC Root cause: A portion of infrastructure which Microsoft Teams relies on for updating user presence information was operating in a sub-optimal state, which resulted in impact. Next steps: - We're reviewing our Unified Presence Service (UPS) code for improved performance and potential automated recovery options to reduce or avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-10-11T20:28:47.233Z
Root cause A portion of infrastructure which Microsoft Teams relies on for updating user presence information was operating in a sub-optimal state, which resulted in impact. Next steps: - We're reviewing our Unified Presence Service (UPS) code for improved performance and potential automated recovery options to reduce or avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-10-11T20:28:47.233Z

TM290799 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-10-11T15:53:39.28

Status: We're reviewing recent service updates and telemetry to better understand the root cause and identify our next troubleshooting steps. Next update by: Monday, October 11, 2021, at 6:00 PMUTC; PublishedTime=2021-10-11T16:15:28.953Z

Status: We believe that a portion of infrastructure which Microsoft Teams relies on for updating user presence information is operating in a sub-optimal state, resulting in impact. We're continuing to investigate service logs and infrastructure data to confirm and identify our next troubleshooting steps. Next update by: Monday, October 11, 2021, at 8:30 PMUTC; PublishedTime=2021-10-11T18:00:11.347Z

Final status: We've re-directed affected traffic away from the problematic portion of infrastructure and have confirmed via telemetry that this has mitigated impact. Next update by: Monday, October 11, 2021, at 8:30 PMUTC; PublishedTime=2021-10-11T18:00:11.347Z

--------------------------------------------------------------------------------------------------
Title Users see a blank chat window when messaging users via a third-party virtual desktop
ID TM290727
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users see a blank chat window when messaging users via a third-party virtual desktop.
Severity Sev2
Start time 2021-10-11T07:00:00Z
End time 2021-10-11T13:00:00Z
Last update 2021-10-11T23:03:50.347Z
More info We recommended that users who had access to the Microsoft Teams web client use that connection method as an alternative method to access the service. Final status: We've determined that a recent service update intended to optimize the chat app infrastructure contained a compatibility issue, causing impact. We've reverted this change and after reaching out to your representatives and reviewing telemetry, have confirmed that impact has been fully remediated. Scope of impact: Your organization was affected by this event, and impacted a limited number of your third-party virtual desktop users. Start time: Friday, October 8, 2021, at 6:00 AM'UTC End time: Monday, October 11, 2021, at 1:00 PM'UTC Root cause: A recent service update intended to optimize the chat app infrastructure contained a compatibility issue, causing impact. Next steps: - We're reviewing this recent update to understand why it had a compatibility issue which will help us to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-11T22:55:06.507Z
Root cause A recent service update intended to optimize the chat app infrastructure contained a compatibility issue, causing impact. Next steps: - We're reviewing this recent update to understand why it had a compatibility issue which will help us to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-11T22:55:06.507Z

TM290727 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-10-11T05:53:40.557

Status: We're analyzing Microsoft Teams diagnostics logs provided by some affected users to determine the source of the issue. Next update by: Monday, October 11, 2021, at 8:30 AMUTC; PublishedTime=2021-10-11T06:22:46.26Z

Status: We've identified a recently enabled change introduced a new feature flag, which we believe is resulting in impact. We're disabling the new feature flag to mitigate the issue. Next update by: Monday, October 11, 2021, at 10:30 AM UTC; PublishedTime=2021-10-11T08:27:53.707Z

Status: We've disabled the feature flag and are verifying with representatives from your organization that the issue has been mitigated. Next update by: Tuesday, October 12, 2021, at 11:00 AM UTC; PublishedTime=2021-10-11T09:57:49.277Z

Status: While we continue to verify mitigation with previously affected users, we're reviewing options for a long-term mitigation plan for this scenario. Next update by: Tuesday, October 12, 2021, at 12:00 AM UTC; PublishedTime=2021-10-11T12:41:55.333Z

Final status: We've determined that a recent service update intended to optimize the chat app infrastructure contained a compatibility issue, causing impact. We've reverted this change and after reaching out to your representatives and reviewing telemetry, have confirmed that impact has been fully remediated. Next update by: Tuesday, October 12, 2021, at 12:00 AM UTC; PublishedTime=2021-10-11T12:41:55.333Z

--------------------------------------------------------------------------------------------------
Title Users see a blank chat window when messaging users via a third-party virtual desktop
ID TM290727
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users see a blank chat window when messaging users via a third-party virtual desktop.
Severity Sev2
Start time 2021-10-11T07:00:00Z
End time 2021-10-11T13:00:00Z
Last update 2021-10-11T23:03:50.347Z
More info We recommended that users who had access to the Microsoft Teams web client use that connection method as an alternative method to access the service. Final status: We've determined that a recent service update intended to optimize the chat app infrastructure contained a compatibility issue, causing impact. We've reverted this change and after reaching out to your representatives and reviewing telemetry, have confirmed that impact has been fully remediated. Scope of impact: Your organization was affected by this event, and impacted a limited number of your third-party virtual desktop users. Start time: Friday, October 8, 2021, at 6:00 AM'UTC End time: Monday, October 11, 2021, at 1:00 PM'UTC Root cause: A recent service update intended to optimize the chat app infrastructure contained a compatibility issue, causing impact. Next steps: - We're reviewing this recent update to understand why it had a compatibility issue which will help us to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-11T22:55:06.507Z
Root cause A recent service update intended to optimize the chat app infrastructure contained a compatibility issue, causing impact. Next steps: - We're reviewing this recent update to understand why it had a compatibility issue which will help us to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-11T22:55:06.507Z

TM290727 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-10-11T05:53:40.557

Status: We're analyzing Microsoft Teams diagnostics logs provided by some affected users to determine the source of the issue. Next update by: Monday, October 11, 2021, at 8:30 AMUTC; PublishedTime=2021-10-11T06:22:46.26Z

Status: We've identified a recently enabled change introduced a new feature flag, which we believe is resulting in impact. We're disabling the new feature flag to mitigate the issue. Next update by: Monday, October 11, 2021, at 10:30 AM UTC; PublishedTime=2021-10-11T08:27:53.707Z

Status: We've disabled the feature flag and are verifying with representatives from your organization that the issue has been mitigated. Next update by: Tuesday, October 12, 2021, at 11:00 AM UTC; PublishedTime=2021-10-11T09:57:49.277Z

Status: While we continue to verify mitigation with previously affected users, we're reviewing options for a long-term mitigation plan for this scenario. Next update by: Tuesday, October 12, 2021, at 12:00 AM UTC; PublishedTime=2021-10-11T12:41:55.333Z

Final status: We've determined that a recent service update intended to optimize the chat app infrastructure contained a compatibility issue, causing impact. We've reverted this change and after reaching out to your representatives and reviewing telemetry, have confirmed that impact has been fully remediated. Next update by: Tuesday, October 12, 2021, at 12:00 AM UTC; PublishedTime=2021-10-11T12:41:55.333Z

--------------------------------------------------------------------------------------------------
Title Some users of Graph Change Notifications APIs may be unable to access or update data for services using the APIs
ID MO290590
Service Microsoft 365 suite
Feature Portal
Status Service restored
Impact Users of the Graph Change Notifications APIs may have been unable to access or update data for services using the APIs.
Severity Sev2
Start time 2021-10-10T10:43:00Z
End time 2021-10-10T16:30:00Z
Last update 2021-10-10T17:39:17.147Z
More info Examples of some impact scenarios that may have been affected included but weren't limited to: - Azure Active Directory objects - Office 365 and Microsoft Teams data Final status: We've confirmed via telemetry that performing an infrastructure restart and software query optimizations has succeeded in remediating impact. Scope of impact: Your organization was affected by this event, and impact was specific to some users who were served through the affected infrastructure in Europe. Start time: Sunday, October 10, 2021, at 10:43 AM'UTC End time: Sunday, October 10, 2021, at 4:30 PM'UTC Root cause: A subset of infrastructure responsible for directing user requests became degraded, which was being exacerbated by queries containing large page sizes, causing it to perform below acceptable thresholds and resulting impact. Next steps: - We're reviewing this subset of infrastructure to understand why it became degraded and prevent similar impact from reoccurring in the future. This is the final update for the event.; PublishedTime=2021-10-10T17:39:17.147Z
Root cause A subset of infrastructure responsible for directing user requests became degraded, which was being exacerbated by queries containing large page sizes, causing it to perform below acceptable thresholds and resulting impact. Next steps: - We're reviewing this subset of infrastructure to understand why it became degraded and prevent similar impact from reoccurring in the future. This is the final update for the event.; PublishedTime=2021-10-10T17:39:17.147Z

MO290590 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-10-10T12:19:05.727

Status: We're analyzing system trace logs to determine the cause of the issue, and establish our next troubleshooting steps. Next update by: Sunday, October 10, 2021, at 3:00 PM UTC; PublishedTime=2021-10-10T12:48:17.71Z

Status: We've identified that a subset of infrastructure, responsible for directing user requests, may be performing below acceptable thresholds. We've restarted the affected infrastructure and are awaiting confirmation by affected users that impact is being mitigated. Next update by: Sunday, October 10, 2021, at 5:00 PM UTC; PublishedTime=2021-10-10T14:48:20.06Z

Status: We've determined that a subset of infrastructure responsible for directing user requests became degraded, which is being exacerbated by queries containing large page sizes, causing it to perform below acceptable thresholds and resulting impact. We also determined that restarting that subset of infrastructure succeeded in remediating a large portion of the impact, and we're currently performing infrastructure optimizations which we believe will succeed in fully resolving the issue. Next update by: Sunday, October 10, 2021, at 7:30 PM UTC; PublishedTime=2021-10-10T17:02:20.813Z

Final status: We've confirmed via telemetry that performing an infrastructure restart and software query optimizations has succeeded in remediating impact. Next update by: Sunday, October 10, 2021, at 7:30 PM UTC; PublishedTime=2021-10-10T17:02:20.813Z

--------------------------------------------------------------------------------------------------
Title Admins see delayed Microsoft 365 Apps usage reports in the Microsoft 365 admin center since October 5, 2021
ID MO290334
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Admins saw delayed usage reports for Microsoft 365 apps in the Microsoft 365 admin center.
Severity Sev2
Start time 2021-10-09T00:00:00Z
End time 2021-10-11T00:00:00Z
Last update 2021-10-11T03:05:51.57Z
More info
Root cause A recent code change introduced a Dynamic Link Library (DLL) conflict that resulted in an unexpected delay in our upstream data source, preventing admins from seeing updated data in Microsoft 365 Apps usage reports in the Microsoft 365 admin center. Next steps: - To help prevent similar impact in the future, we're further reviewing why the change introduced the DLL conflict and how we can improve our validation procedures to catch such issues before deployment. This is the final update for the event.; PublishedTime=2021-10-11T03:05:51.57Z

MO290334 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-10-09T04:33:32.53

Status: We've determined that an unexpected delay in our upstream data source is preventing admins from seeing updated data in Apps usage. We're formulating the best course of action to resolve the impact. Next update by: Sunday, October 10, 2021, at 5:00 AMUTC; PublishedTime=2021-10-09T04:59:09.21Z

Status: We've determined a recent code change introduced a Dynamic Link Library (DLL) conflict which resulted in an unexpected delay in our upstream data source and caused impact. We've corrected the DLL conflict to restore the upstream data source and we anticipate the delayed reports will be resolved by the next scheduled update. Next update by: Monday, October 11, 2021, at 7:00 AMUTC; PublishedTime=2021-10-10T04:19:50.537Z

Final status: We've confirmed via telemetry that correcting the DLL conflict has restored the upstream data source and resolved the usage reports delay. Next update by: Monday, October 11, 2021, at 7:00 AMUTC; PublishedTime=2021-10-10T04:19:50.537Z

--------------------------------------------------------------------------------------------------
Title Admins can't access the Microsoft Teams admin center via any connection method
ID TM290310
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Admins were unable to access the Microsoft Teams admin center via any connection method.
Severity Sev2
Start time 2021-10-08T23:25:00Z
End time 2021-10-09T02:21:00Z
Last update 2021-10-09T02:58:56.033Z
More info
Root cause A recent service routing change contained a misconfiguration, resulting in impact. Next steps: - We're reviewing the impacting change to better understand how this wasn't discovered before deployment to prevent similar impact in the future. This is the final update for the event.; PublishedTime=2021-10-09T02:58:56.033Z

TM290310 Details

Status: We're looking into your reported issue and checking for impact to your organization. We'll provide an update within one hour.; PublishedTime=2021-10-09T00:47:46.68

Status: We're reviewing reports of a possible issue with the Microsoft Teams admin center which is preventing admin access. We're reviewing the associated service telemetry and are internally reproducing the impact scenario to isolate the source of this issue and determine the necessary remediation actions. Next update by: Saturday, October 9, 2021, at 3:30 AMUTC; PublishedTime=2021-10-09T01:13:10.533Z

Status: We're reviewing diagnostic data from our internal reproduction of the impact scenario to help us isolate any recent changes which may be contributing to this issue and determine our next troubleshooting steps. Next update by: Saturday, October 9, 2021, at 4:30 AMUTC; PublishedTime=2021-10-09T02:00:55.477Z

Final status: We've found that a recent service routing change contained a misconfiguration, resulting in impact. We've reverted that change and confirmed via service monitoring that this has remediated impact. Next update by: Saturday, October 9, 2021, at 4:30 AMUTC; PublishedTime=2021-10-09T02:00:55.477Z

--------------------------------------------------------------------------------------------------
Title Users see a blank chat window when messaging users via a third-party virtual desktop
ID TM291011
Service Microsoft Teams
Feature Teams Components
Status Investigation suspended
Impact Users saw a blank chat window when messaging users via a third-party virtual desktop.
Severity Sev2
Start time 2021-10-08T06:00:00Z
End time 2021-10-12T19:20:00Z
Last update 2021-10-13T02:25:28.153Z
More info During the impact window, we recommended that users who had access to the Microsoft Teams web client use that connection method as an alternative method to access the service. Final status: Our internal testing shows that the configuration changes have mitigated impact. Users will need to restart their Microsoft Teams desktop client to alleviate impact in full. Once you've confirmed that our mitigation actions, along with a client restart have fixed the issue, please contact support to provide confirmation that the issue has been resolved. Scope of impact: Impact was specific to a limited number of third-party virtual desktop users. Start time: Friday, October 8, 2021, at 6:00 AM'UTC End time: Tuesday, October 12, 2021, at 7:20 PM'UTC Root cause: A recent service update intended to optimize the chat app infrastructure contained a compatibility issue, causing impact. Next steps: - We're reviewing our service update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-13T02:25:28.153Z
Root cause A recent service update intended to optimize the chat app infrastructure contained a compatibility issue, causing impact. Next steps: - We're reviewing our service update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-13T02:25:28.153Z

TM291011 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 60 minutes.; PublishedTime=2021-10-12T17:14:36.36

Status: We've identified that a recent service update intended to optimize the chat app infrastructure contained a compatibility issue, causing impact. We've reverted this change to mitigate impact, but we've confirmed this has not fully alleviated impact. We suggest that users manually restart their Microsoft Teams desktop client to potentially resolve impact while we continue investigating available diagnostic logs to determine if additional mitigations actions are necessary. Next update by: Tuesday, October 12, 2021, at 8:00 PMUTC; PublishedTime=2021-10-12T18:04:02.123Z

Status: We've identified that reverting the offending change didn't target Microsoft Teams desktop client filters, and as an additional mitigation action, we've applied a client configuration change to disable the chat app infrastructure optimization for affected Microsoft Teams desktop versions. We're monitoring the impacted service and testing with affected users to verify that this additional mitigation action has successfully remediated impact. Additionally, users will need to manually restart their Microsoft Teams desktop client to alleviate impact in full. Next update by: Tuesday, October 12, 2021, at 11:30 PM UTC; PublishedTime=2021-10-12T20:01:03.573Z

Status: We're awaiting word from your representatives to verify that the client configuration change has successfully remediated impact. Next update by: Wednesday, October 13, 2021, at 4:00 AMUTC; PublishedTime=2021-10-12T22:13:54.133Z

Final status: Our internal testing shows that the configuration changes have mitigated impact. Users will need to restart their Microsoft Teams desktop client to alleviate impact in full. Once you've confirmed that our mitigation actions, along with a client restart have fixed the issue, please contact support to provide confirmation that the issue has been resolved. Next update by: Wednesday, October 13, 2021, at 4:00 AMUTC; PublishedTime=2021-10-12T22:13:54.133Z

--------------------------------------------------------------------------------------------------
Title Signing in issues
ID CS290079
Service Cloud App Security
Feature Cloud App Security
Status Service restored
Impact Users may have experienced issues when signing in to specific Cloud Apps within Microsoft Cloud App Security.
Severity Sev2
Start time 2021-10-07T23:00:00Z
End time 2021-10-08T10:32:00Z
Last update 2021-10-08T12:49:58.66Z
More info Users who already signed in may have experienced issues with downloads. Final status: The applied targeted configuration has been applied and and we've confirmed after monitoring the environment that the impact is mitigated. Scope of impact: Impact was specific to users who were served through the affected infrastructure. Start time: Thursday, October 7, 2021, at 11:00 PM UTC End time: Friday, October 8, 2021, at 10:32 AM UTC Root cause: Impact was caused by an authentication component configuration problem. Next steps: - We're reviewing our authentication configuration procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-10-08T12:48:38.763Z
Root cause Impact was caused by an authentication component configuration problem. Next steps: - We're reviewing our authentication configuration procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-10-08T12:48:38.763Z

CS290079 Details

Status: We're analyzing system logs to determine the source of the issue. Next update by: Friday, October 8, 2021, at 11:30 AM UTC; PublishedTime=2021-10-08T09:05:29.733Z

Status: Our analysis has identified that impact is caused by an authentication component configuration problem. We're applying a targeted configuration change to repair the issue. Next update by: Friday, October 8, 2021, at 1:30 PM UTC; PublishedTime=2021-10-08T11:25:20.2Z

Final status: The applied targeted configuration has been applied and and we've confirmed after monitoring the environment that the impact is mitigated. Next update by: Friday, October 8, 2021, at 1:30 PM UTC; PublishedTime=2021-10-08T11:25:20.2Z

--------------------------------------------------------------------------------------------------
Title Unable to make or receive PSTN calls through Microsoft Teams
ID TM289868
Service Microsoft Teams
Feature Teams Components
Status Post-incident report published
Impact Users may have experienced issues making or receiving PSTN calls through Microsoft Teams.
Severity Sev2
Start time 2021-10-07T12:06:00Z
End time 2021-10-07T15:00:00Z
Last update 2021-10-14T14:57:08.52Z
More info Users dialing into audio conference meetings were also affected. Additionally, users attempting to call Auto Attendant and Call Queue features may have experienced disconnects. Some organizations which leverage Direct Routing may see a delay in recovery due to SBC (Session Border Controller) configurations. They may have marked connections to Microsoft services as down. These organizations will need to clear their cache by restarting the SBC or allow the cache to clear naturally. Direct Routing is a way to provide a PSTN (Public Switched Telephone Network) connection to Microsoft Teams users, allowing them to place and receive external phone calls on any device using Teams. Final status: After an extended period of monitoring, we've confirmed this issue is resolved Scope of impact: This issue may have potentially affected any user routed through the affected infrastructure. Start time: Thursday, October 7, 2021, at 12:06 PM UTC End time: Thursday, October 7, 2021, at 3:00 PM UTC Root cause: We identified that some internal monitoring probes for PSTN calls were not correctly configured in specific environments. We decided to use the Microsoft Teams central repository of configurations to enable the monitoring probes. However, due to an error when selecting the filtering logic, all incoming PSTN requests across the service were inadvertently routed to a specific environment for processing. As this environment was not configured to handle this volume of requests, this resulted in impact to some incoming PSTN requests. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. We'll publish a post-incident report within five business days.; PublishedTime=2021-10-07T16:30:37.843Z
Root cause We identified that some internal monitoring probes for PSTN calls were not correctly configured in specific environments. We decided to use the Microsoft Teams central repository of configurations to enable the monitoring probes. However, due to an error when selecting the filtering logic, all incoming PSTN requests across the service were inadvertently routed to a specific environment for processing. As this environment was not configured to handle this volume of requests, this resulted in impact to some incoming PSTN requests. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. We'll publish a post-incident report within five business days.; PublishedTime=2021-10-07T16:30:37.843Z

TM289868 Details

Status: We're looking into your reported issue and checking for impact to your organization. We'll provide an update within one hour.; PublishedTime=2021-10-07T13:03:12.3

Status: We're reviewing diagnostic data and error logs to isolate the source of the issue. Next update by: Thursday, October 7, 2021, at 3:00 PM UTC; PublishedTime=2021-10-07T13:43:12.99Z

Status: We've determined that a recent update contained a network routing misconfiguration for incoming PSTN requests. We're reverting the update to remediate impact. Next update by: Thursday, October 7, 2021, at 4:00 PM UTC; PublishedTime=2021-10-07T14:53:33.847Z

Status: We've reverted the update and we've received multiple reports that impact has been remediated. We're continuing to monitor the service to confirm recovery. Next update by: Thursday, October 7, 2021, at 6:00 PM UTC; PublishedTime=2021-10-07T15:58:22.597Z

Final status: After an extended period of monitoring, we've confirmed this issue is resolved Next update by: Thursday, October 7, 2021, at 6:00 PM UTC; PublishedTime=2021-10-07T15:58:22.597Z

--------------------------------------------------------------------------------------------------
Title Users were unable to print from within Microsoft Bookings
ID MB291020
Service Microsoft Bookings
Feature Microsoft Bookings
Status Service restored
Impact Users were unable to print from within Microsoft Bookings.
Severity Sev2
Start time 2021-10-07T07:00:00Z
End time 2021-10-15T15:00:00Z
Last update 2021-10-18T10:26:43.143Z
More info Users would have seen the following error when attempting to print: "An error has occurred. There has been an error in the application. Please refresh your browser window to continue." Users could successfully print when using Bookings via the Internet Explorer web browser. Final status: The fix has been successfully deployed across the affected environment. We've confirmed from our internal monitoring that the issue has been resolved. Scope of impact: Any user attempting to print from Microsoft Bookings may have experienced impact. Start time: Thursday, October 7, 2021, at 7:00 AM UTC End time: Friday, October 15, 2021, at 3:00 PM UTC Root cause: A recent calendaring update exposed a code regression that prevented users from printing within Microsoft Bookings. Next steps: - We're reviewing our calendaring service update procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-10-18T10:26:43.143Z
Root cause A recent calendaring update exposed a code regression that prevented users from printing within Microsoft Bookings. Next steps: - We're reviewing our calendaring service update procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-10-18T10:26:43.143Z

MB291020 Details

Status: We're investigating a potential issue and we'll provide an update within 30 minutes.; PublishedTime=2021-10-12T18:03:46.207

Status: We're reviewing HTTP Archive format (HAR) trace files and Fiddler trace logs from affected users so we can better determine the origin of the issue and develop a subsequent mitigation plan. Next update by: Tuesday, October 12, 2021, at 8:00 PMUTC; PublishedTime=2021-10-12T18:38:41.777Z

Status: Our analysis of the Fiddler and HAR files revealed that a code regression, initially exposed by a recent calendaring update on October 7, 2021, is inhibiting some users from printing from within Microsoft Bookings. To mitigate, we've developed and deployed a fix which is currently propagating throughout the affected environment and may already be providing alleviation to some affected users. We'll provide additional information regarding the status of the fix and its projected time to completion via our next scheduled update. Next update by: Wednesday, October 13, 2021, at 11:30 AMUTC; PublishedTime=2021-10-12T19:47:12.027Z

Status: We've confirmed that the fix is approximately 10 percent deployed across all affected infrastructures. We'll continue to monitor as it progresses and we estimate completion by Monday, October 18, 2021. Next update by: Monday, October 18, 2021, at 11:30 AM UTC; PublishedTime=2021-10-13T11:24:45.197Z

Final status: The fix has been successfully deployed across the affected environment. We've confirmed from our internal monitoring that the issue has been resolved. Next update by: Monday, October 18, 2021, at 11:30 AM UTC; PublishedTime=2021-10-13T11:24:45.197Z

--------------------------------------------------------------------------------------------------
Title Admins may see a delay of over 12 hours in discovery pillar events and alerts
ID CS289836
Service Cloud App Security
Feature Cloud App Security Discovery
Status Service degradation
Impact Admins may see a delay of over 12 hours in discovery pillar events and alerts.
Severity Sev2
Start time 2021-10-07T06:59:12Z
End time
Last update 2021-10-18T12:00:20.94Z
More info
Root cause A section of the impacted infrastructure was performing below acceptable performance requests following an unexpected spike in user requests. Next update by: Monday, October 25, 2021, at 11:00 AM UTC; PublishedTime=2021-10-18T12:00:20.94Z

CS289836 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-10-07T07:16:01.38

Status: We've identified that the impacted infrastructure was performing below acceptable performance requests following an unexpected spike in user requests. We've optimized the environment to handle these requests more efficiently, and are processing the backlog of delayed alerts to mitigate impact. Next update by: Monday, October 11, 2021, at 9:00 AM UTC; PublishedTime=2021-10-07T07:40:00.267Z

Status: We're continuing to process the backlog of delayed alerts to fully resolve the issue. Next update by: Thursday, October 14, 2021, at 1:00 PM UTC; PublishedTime=2021-10-11T08:28:42.46Z

Status: We've determined that the processing of the backlog of delayed alerts is taking longer than expected to complete. We're monitoring it's progress until full mitigation. Next update by: Monday, October 18, 2021, at 1:00 PM UTC; PublishedTime=2021-10-14T12:36:49.133Z

Status: We're encountering further delays in processing the backlog of delayed alerts. We're monitoring the current progress of the fix to establish a more accurate completion date. Next update by: Monday, October 25, 2021, at 11:00 AM UTC; PublishedTime=2021-10-18T12:00:20.94Z

--------------------------------------------------------------------------------------------------
Title Admins experiencing errors and timeouts when performing actions within the Power Platform admin center
ID MO289655
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Admins were experiencing errors and timeouts when performing actions within the Power Platform admin center.
Severity Sev2
Start time 2021-10-06T16:49:00Z
End time 2021-10-06T17:21:00Z
Last update 2021-10-06T19:17:40.263Z
More info
Root cause The deployment process of a recent service update caused a traffic processing issues on a subsection of service infrastructure responsible for facilitating Power Platform admin center requests, resulting in impact. Next steps: - We're reviewing our update procedures to better understand why it caused service performance levels to drop and to help identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-06T19:17:05.08Z

MO289655 Details

Status: We've investigated a problem with the PowerApps and the Power Automate services where admins were experiencing errors and timeouts when performing actions within the Power Platform admin center. We've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-10-06T18:40:01.867

Final status: We've determined that a recent service update had caused a subsection of service infrastructure to fall below our manageable service performance thresholds, resulting in impact. We've rerouted traffic to an alternate subsection of service infrastructure to remediate impact.

--------------------------------------------------------------------------------------------------
Title Admins experiencing errors and timeouts when performing actions within the Power Platform admin center
ID MO289655
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Admins were experiencing errors and timeouts when performing actions within the Power Platform admin center.
Severity Sev2
Start time 2021-10-06T16:49:00Z
End time 2021-10-06T17:21:00Z
Last update 2021-10-06T19:17:40.263Z
More info
Root cause The deployment process of a recent service update caused a traffic processing issues on a subsection of service infrastructure responsible for facilitating Power Platform admin center requests, resulting in impact. Next steps: - We're reviewing our update procedures to better understand why it caused service performance levels to drop and to help identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-06T19:17:05.08Z

MO289655 Details

Status: We've investigated a problem with the PowerApps and the Power Automate services where admins were experiencing errors and timeouts when performing actions within the Power Platform admin center. We've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-10-06T18:40:01.867

Final status: We've determined that a recent service update had caused a subsection of service infrastructure to fall below our manageable service performance thresholds, resulting in impact. We've rerouted traffic to an alternate subsection of service infrastructure to remediate impact.

--------------------------------------------------------------------------------------------------
Title Users may be unable to enroll new devices or receive device updates
ID IT289535
Service Microsoft Intune
Feature Microsoft Intune
Status Service restored
Impact Users may have been unable to enroll new devices or receive device updates.
Severity Sev2
Start time 2021-10-05T23:58:00Z
End time 2021-10-06T02:46:00Z
Last update 2021-10-06T03:27:37.88Z
More info
Root cause A configuration change for optimizing a portion of Microsoft Intune infrastructure inadvertently resulted in impact. Next steps: - To help prevent similar impact in the future, we're further reviewing why the change affected users. This is the final update for the event.; PublishedTime=2021-10-06T03:26:42.94Z

IT289535 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Final status: We've determined that a configuration change for optimizing a portion of Microsoft Intune infrastructure inadvertently resulted in impact. We've reverted the configuration change to mitigate impact and we've confirmed via telemetry that service is restored.

--------------------------------------------------------------------------------------------------
Title Users may be unable to access the Microsoft Teams service via any connection method
ID TM289412
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have been unable to access the Microsoft Teams service via any connection method.
Severity Sev2
Start time 2021-10-05T16:30:00Z
End time 2021-10-05T19:30:00Z
Last update 2021-10-06T01:42:55.573Z
More info This was an incremental update to provide the latest status on the issue. Final status: Our investigation determined that a recent update intended to make general improvements to the Microsoft Teams environment inadvertently referenced an obsolete code pathway, resulting in impact. This reference was removed, and we've monitored the environment to confirm that service is restored. Scope of impact: Your organization was affected by this event, and all users attempting to access Microsoft Teams may have been affected. Start time: Tuesday, October 5, 2021, at 4:30 PM'UTC End time: Tuesday, October 5, 2021, at 7:30 PM'UTC Root cause: A recent update intended to make general improvements to the Microsoft Teams environment inadvertently referenced an obsolete code pathway, resulting in connection loss. Next steps: - We're reviewing our update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-05T20:15:49.94Z
Root cause A recent update intended to make general improvements to the Microsoft Teams environment inadvertently referenced an obsolete code pathway, resulting in connection loss. Next steps: - We're reviewing our update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-05T20:15:49.94Z

TM289412 Details

Status: We've investigated a problem with the Microsoft Teams service in which users may have been unable to access the service via any connection method, and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-10-05T20:12:56.377

Final status: Our investigation determined that a recent update intended to make general improvements to the Microsoft Teams environment inadvertently referenced an obsolete code pathway, resulting in impact. This reference was removed, and we've monitored the environment to confirm that service is restored.

--------------------------------------------------------------------------------------------------
Title Users' devices managed by Microsoft Intune may be failing to install Win32 applications
ID IT289426
Service Microsoft Intune
Feature Microsoft Intune
Status Service restored
Impact Users' Windows devices managed by Microsoft Intune may have been failing to install Win32 applications.
Severity Sev2
Start time 2021-10-05T05:30:00Z
End time 2021-10-08T01:00:00Z
Last update 2021-10-12T00:09:40.253Z
More info Users may have received notifications that Win32 app installations have failed on their Microsoft Intune managed Windows devices. The issue occurred intermittently and users were able to retry the installation. Final status: We previously indicated that this issue was resolved on October 5, 2021, after we reverted the recent update to the Content Delivery Network (CDN); however, after receiving additional user reports, we determined that some residual impact persisted until Friday, October 8, 2021, at 1:00 AM UTC, due to stale cached data on some infrastructure. We've validated through monitoring that the issue is fully resolved. Scope of impact: Any user attempting to install Win32 applications on Windows devices managed through Microsoft Intune may have intermittently experience impact. Start time: Tuesday, October 5, 2021, at 5:30 PM UTC End time: Friday, October 8, 2021, at 1:00 AM UTC Root cause: A code issue affecting Win32 application installation for Microsoft Intune managed Windows devices, resulting in impact. Next steps: - We're reviewing our update procedures to better identify similar code issues during our development and testing cycles.; PublishedTime=2021-10-12T00:08:32.83Z
Root cause A code issue affecting Win32 application installation for Microsoft Intune managed Windows devices, resulting in impact. Next steps: - We're reviewing our update procedures to better identify similar code issues during our development and testing cycles.; PublishedTime=2021-10-12T00:08:32.83Z

IT289426 Details

Status: We've identified a recent update to the Content Delivery Network (CDN) contains a code bug affecting Win32 application installation for Microsoft Intune managed Windows devices, resulting in impact. We're rolling back this change to determine if impact has been remediated. Next update by: Tuesday, October 5, 2021, at 11:30 PM UTC; PublishedTime=2021-10-05T22:07:03.603Z

Final status: We previously indicated that this issue was resolved on October 5, 2021, after we reverted the recent update to the Content Delivery Network (CDN); however, after receiving additional user reports, we determined that some residual impact persisted until Friday, October 8, 2021, at 1:00 AM UTC, due to stale cached data on some infrastructure. We've validated through monitoring that the issue is fully resolved. Next update by: Tuesday, October 5, 2021, at 11:30 PM UTC; PublishedTime=2021-10-05T22:07:03.603Z

--------------------------------------------------------------------------------------------------
Title Users may have been unable to access Outlook on the web
ID EX289328
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users may have been unable to access Outlook on the web.
Severity Sev2
Start time 2021-10-04T22:42:00Z
End time 2021-10-05T05:30:00Z
Last update 2021-10-05T18:32:29.03Z
More info
Root cause A subsection of POP3 infrastructure was performing below acceptable performance thresholds, preventing access to Outlook on the web. Next steps: - We're analyzing performance data and trends for the subsection of affected POP3 service infrastructure to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-10-05T18:26:11.637Z

EX289328 Details

Status: We've identified that some impact associated with MO289250 persists. We're reviewing Client Access Front End (CAFE) infrastructure, to help determine the next troubleshooting steps. Next update by: Tuesday, October 5, 2021 at 9:00 AM UTC; PublishedTime=2021-10-05T07:08:24.287Z

Status: We've identified that some of our Post Office Protocol (POP3) infrastructure is performing below acceptable performance thresholds, preventing access. We're restarting the affected infrastructures to remediate impact. Next update by: Tuesday, October 5, 2021, at 11:00 AM UTC; PublishedTime=2021-10-05T08:56:41.243Z

Status: We've restarted the affected infrastructure, which seems to have mitigated the issue. We're performing an extended period of monitoring across multiple environments to ensure the service has fully recovery. Next update by: Tuesday, October 5, 2021, at 8:00 PM UTC; PublishedTime=2021-10-05T10:39:45.18Z

Final status: Following our restarting of a subsection of POP3 service infrastructure, we've internally confirmed that impact has been resolved. Next update by: Tuesday, October 5, 2021, at 8:00 PM UTC; PublishedTime=2021-10-05T10:39:45.18Z

--------------------------------------------------------------------------------------------------
Title Users may have been unable to access Outlook on the web
ID EX289328
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users may have been unable to access Outlook on the web.
Severity Sev2
Start time 2021-10-04T22:42:00Z
End time 2021-10-05T05:30:00Z
Last update 2021-10-05T18:32:29.03Z
More info
Root cause A subsection of POP3 infrastructure was performing below acceptable performance thresholds, preventing access to Outlook on the web. Next steps: - We're analyzing performance data and trends for the subsection of affected POP3 service infrastructure to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-10-05T18:26:11.637Z

EX289328 Details

Status: We've identified that some impact associated with MO289250 persists. We're reviewing Client Access Front End (CAFE) infrastructure, to help determine the next troubleshooting steps. Next update by: Tuesday, October 5, 2021 at 9:00 AM UTC; PublishedTime=2021-10-05T07:08:24.287Z

Status: We've identified that some of our Post Office Protocol (POP3) infrastructure is performing below acceptable performance thresholds, preventing access. We're restarting the affected infrastructures to remediate impact. Next update by: Tuesday, October 5, 2021, at 11:00 AM UTC; PublishedTime=2021-10-05T08:56:41.243Z

Status: We've restarted the affected infrastructure, which seems to have mitigated the issue. We're performing an extended period of monitoring across multiple environments to ensure the service has fully recovery. Next update by: Tuesday, October 5, 2021, at 8:00 PM UTC; PublishedTime=2021-10-05T10:39:45.18Z

Final status: Following our restarting of a subsection of POP3 service infrastructure, we've internally confirmed that impact has been resolved. Next update by: Tuesday, October 5, 2021, at 8:00 PM UTC; PublishedTime=2021-10-05T10:39:45.18Z

--------------------------------------------------------------------------------------------------
Title Users were unable to download meeting attendance reports in Microsoft Teams
ID TM290928
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were unable to download meeting attendance reports in Microsoft Teams.
Severity Sev2
Start time 2021-10-04T08:00:00Z
End time 2021-10-12T05:00:00Z
Last update 2021-10-18T09:43:26.833Z
More info As a workaround, affected user could go to the same thread with the missing report, start a new meeting and leave after 10 seconds. Then refresh the channel (go to another tab and back) to see the new just-created-report and the reports that were missing before that. Final status: We've finished replaying the data from previous Microsoft Teams meetings and confirmed that impact associated with this event has been fully mitigated. Scope of impact: This issue may have impacted any user attempting to download Microsoft Teams meeting attendance reports. Start time: Monday, October 4, 2021, at 8:00 AM UTC End time: Tuesday, October 12, 2021, at 5:00 AM UTC Root cause: A recent service update was unexpectedly interfering with the ability to download meeting attendance reports in Microsoft Teams. Next steps: - We're reviewing our standard service update procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-10-18T09:43:26.833Z
Root cause A recent service update was unexpectedly interfering with the ability to download meeting attendance reports in Microsoft Teams. Next steps: - We're reviewing our standard service update procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-10-18T09:43:26.833Z

TM290928 Details

Status: We've received a report from your organization that impact associated with TM289833 persists. We've resumed our investigation of the issue and are currently analyzing the report data. Next update by: Tuesday, October 12, 2021, at 3:00 AMUTC; PublishedTime=2021-10-12T01:41:26.903Z

Status: We're reviewing recent changes to Microsoft Teams to help identify the source of the issue. Next update by: Tuesday, October 12, 2021, at 5:00 AMUTC; PublishedTime=2021-10-12T02:47:19.453Z

Status: After further review, we identified that a recent service update is responsible for the issues with downloading meeting attendance reports. We're working to revert the change to remediate the impact. Next update by: Tuesday, October 12, 2021, at 7:30 AMUTC; PublishedTime=2021-10-12T03:26:45.017Z

Status: We reverted the recent service update and can see that the ability to download meeting attendance reports is working correctly. We're now working to replay the data from any previous Microsoft Teams meetings to ensure that users are able to download those meeting attendance reports. Next update by: Wednesday, October 13, 2021, at 10:00 AMUTC; PublishedTime=2021-10-12T05:56:13.603Z

Status: We're continuing to replay the data from any previous Microsoft Teams meetings, so that users are able to download the meeting attendance reports. Next update by: Monday, October 18, 2021, at 10:00 AM UTC; PublishedTime=2021-10-13T09:19:50.77Z

Final status: We've finished replaying the data from previous Microsoft Teams meetings and confirmed that impact associated with this event has been fully mitigated. Next update by: Monday, October 18, 2021, at 10:00 AM UTC; PublishedTime=2021-10-13T09:19:50.77Z

--------------------------------------------------------------------------------------------------
Title Admins' Microsoft Teams, email storage, and group activity reports haven't refreshed since October 1, 2021
ID MO289071
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Admins' Microsoft Teams, email storage, and group activity reports hadn't refreshed since October 1, 2021.
Severity Sev2
Start time 2021-10-04T00:00:00Z
End time 2021-10-06T00:00:00Z
Last update 2021-10-06T03:25:54.23Z
More info
Root cause An unexpected delay in our upstream data source is preventing admins from seeing updated data in Microsoft Teams, email storage, and group activity reports in the Microsoft 365 admin center since October 1, 2021. Next steps: - To help prevent similar impact in the future, we're further reviewing the underlying cause of the unexpected delay in our upstream data source. This is the final update for the event.; PublishedTime=2021-10-06T03:25:54.23Z

MO289071 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We've determined that an unexpected delay in our upstream data source is preventing admins from seeing updated data in Microsoft Teams, email storage, and group activity reports in the Microsoft 365 admin center. We're determining the best course of action to resolve the impact. Next update by: Tuesday, October 5, 2021, at 5:00 AMUTC; PublishedTime=2021-10-04T04:20:13.003Z

Status: We've repaired the upstream data source and portions of the affected infrastructure are seeing relief. We are monitoring our telemetry to determine whether this has provided complete mitigation. Next update by: Wednesday, October 6, 2021, at 5:00 AMUTC; PublishedTime=2021-10-05T03:24:32.663Z

Final status: The repairs to the upstream data source have completed. We have confirmed through our telemetry that impact has been mitigated. Next update by: Wednesday, October 6, 2021, at 5:00 AMUTC; PublishedTime=2021-10-05T03:24:32.663Z

--------------------------------------------------------------------------------------------------
Title Users unable to see some channels within Microsoft Teams
ID TM288247
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were unable to access some private or standard channels within Microsoft Teams.
Severity Sev2
Start time 2021-09-29T13:57:00Z
End time 2021-10-09T03:00:00Z
Last update 2021-10-14T19:08:33.47Z
More info Users may have also noticed the following issues: - Users may have been unable to view a subset of channels within the affected Teams. - Users may have been unable to view tabs within the affected channels. - Some channel names, memberships, and other channel properties may have been reverted to a previous state. Admins could still view the affected data using the eDiscovery feature while the issue was ongoing. More information on eDiscovery can be found here: https://docs.microsoft.com/en-us/microsoftteams/ediscovery-investigation. The majority of users experienced remediation automatically. However, if your organization had users reporting that they could only see the affected channels reappear in some Teams clients but not all, the users could have performed a logout/login action on any client that was not displaying the restored channels. This action could have been performed by users for desktop, mobile, and web clients. Final status: After monitoring the service we've confirmed that all impact scenarios have been fully remediated. Scope of impact: Your organization was affected by this event, and the issue impacted any user that was attempting to access or view the affected channels and other impacted scenarios. Start time: Wednesday, September 29, 2021, at 1:57 PM UTC End time: Saturday, October 9, 2021, at 3:00 AM UTC Preliminary root cause: A service deployment as part of a scheduled upgrade to our North American infrastructure contained a misconfiguration which led to the identified impact scenarios for Teams and users. Next steps: - We're reviewing our update procedures to better identify similar issues during our development and testing cycles. - We're reviewing our deployment and provisioning procedures to help prevent similar problems in the future. - We're reviewing our monitoring services to look for ways to reduce detection time and more quickly restore service. This is the final update for this event.; PublishedTime=2021-10-14T19:08:33.47Z
Root cause

TM288247 Details

Status: We've identified an issue affecting Teams channel visibility for users when attempting to access a subset of channels within the Teams client or web app. We've completed a service configuration change that has restored access for a majority of the identified channels. We're continuing our investigation to determine next steps to address the remaining channels and we'll provide a timeline for remediation when it becomes available. Next update by: Thursday, September 30, 2021, at 4:00 AMUTC; PublishedTime=2021-09-29T23:53:54.34Z

Status: Our investigation of visibility issues for the remaining affected channels continues. If your users are still unable to see some channels, please open a ticket with support and reference this service health incident. Additionally, we're examining the service for secondary effects that may be impacting customers and we'll provide status updates as available. Next update by: Thursday, September 30, 2021, at 10:00 PMUTC; PublishedTime=2021-09-30T03:46:47.21Z

Status: We have deployed a mitigation to restore visibility for a large portion of the affected channels. We are continuing our work to restore the visibility of the remaining affected channels and to address the additional impacted scenarios listed under the 'more info' section. We'll provide an updated ETA as soon as it becomes available. Next update by: Friday, October 1, 2021, at 9:00 PM UTC; PublishedTime=2021-09-30T21:59:25.38Z

Status: We've confirmed via internal telemetry that we've restored visibility to well over 99 percent of all identified affected channels and that the vast majority of customers are no longer impacted by this problem. We anticipate that visibility will be restored for the remaining affected channels in the next 12 hours. In parallel, we're continuing work to address the additional impacted scenarios listed in the 'more info' section, and we'll provide an updated ETA for those efforts as soon as it becomes available. Next update by: Saturday, October 2, 2021, at 11:00 PM UTC; PublishedTime=2021-10-01T20:59:21.48Z

Status: We've confirmed via internal telemetry that we've restored 100 percent of all identified channel visibility issues. If your users are still unable to see some channels, please open a ticket with support and reference this service health incident. We're continuing work to address the additional impacted scenarios listed in the 'more info' section, and we'll provide an updated ETA for those efforts as soon as it becomes available. Next update by: Monday, October 4, 2021, at 11:00 PM UTC; PublishedTime=2021-10-01T23:36:40.023Z

Status: We've confirmed restoration of standard channel visibility via telemetry and with reporting customers. We're continuing to remediate impact for the additional impact scenarios which include private channel visibility, tab views, channel names, memberships, and other channel properties. We'll provide an updated resolution timeline as soon as it becomes available. Next update by: Wednesday, October 6, 2021, at 11:00 PM UTC; PublishedTime=2021-10-04T22:46:19.333Z

Status: We expect to begin deploying fixes over the next 72-hours to address all remaining impacted scenarios including private channel visibility, tab views, channel names, memberships, and other channel properties. As the remediation progresses some customers will begin to see recovery for some of these additional impacted scenarios. Next update by: Saturday, October 9, 2021, at 02:00 AM UTC; PublishedTime=2021-10-06T19:46:34.38Z

Status: Our mitigations efforts are ongoing and we expect users to see full resolution by our next update. As the remediation progresses some customers will begin to see recovery for some of these additional impacted scenarios. Next update by: Monday, October 11, 2021, at 10:00 PM UTC; PublishedTime=2021-10-09T01:02:41.927Z

Status: We've completed our mitigation efforts and we expect all users to see full resolution at this time for all impacted scenarios. To further validate that our changes have been successful, we're monitoring the environment for 72 hours and working with customers to confirm that service has been restored. Next update by: Thursday, October 14, 2021, at 11:00 PM UTC; PublishedTime=2021-10-11T21:41:29.473Z

Final status: After monitoring the service we've confirmed that all impact scenarios have been fully remediated. Next update by: Thursday, October 14, 2021, at 11:00 PM UTC; PublishedTime=2021-10-11T21:41:29.473Z

--------------------------------------------------------------------------------------------------
Title Users may be unable to check-in devices within Microsoft Intune
ID IT288139
Service Microsoft Intune
Feature Microsoft Intune
Status Service restored
Impact Users may have been unable to check-in devices within Microsoft Intune.
Severity Sev2
Start time 2021-09-29T12:15:00Z
End time 2021-09-29T16:07:00Z
Last update 2021-09-29T19:25:07.96Z
More info Users would have also been unable to do the following in the company portal since they were unable to check in their devices: - Apps deployed after Wednesday, September 29, 2021, at 12:00 PM UTC. - Check device Status. Final status: We've confirmed after monitoring telemetry that rerouting traffic has succeeded in mitigating the impact. Scope of impact: Impact was specific to users who were served through the affected infrastructure. Start time: Wednesday, September 29, 2021, at 12:15 PM'UTC End time: Wednesday, September 29, 2021, at 4:07 PM'UTC Root cause: A portion of infrastructure that facilitates device check-ins was performing below acceptable thresholds, resulting in impact. Next steps: - We're reviewing this portion of affected infrastructure to understand why it was performing below acceptable thresholds and prevent similar impact from reoccurring in the future. This is the final update for the event.; PublishedTime=2021-09-29T19:25:07.96Z
Root cause A portion of infrastructure that facilitates device check-ins was performing below acceptable thresholds, resulting in impact. Next steps: - We're reviewing this portion of affected infrastructure to understand why it was performing below acceptable thresholds and prevent similar impact from reoccurring in the future. This is the final update for the event.; PublishedTime=2021-09-29T19:25:07.96Z

IT288139 Details

Status: We're analyzing diagnostic data to determine the source of the issue and develop a mitigation plan. Next update by: Wednesday, September 29, 2021, at 3:30 PM UTC; PublishedTime=2021-09-29T13:15:29.413Z

Status: We've identified that a portion of infrastructure that facilitates device check-ins is performing below acceptable thresholds, likely resulting in impact. We're reviewing possible mitigation actions, including rerouting traffic to an alternate infrastructure, while also continuing our investigation into the infrastructure performance problem. Next update by: Wednesday, September 29, 2021, at 6:00 PMUTC; PublishedTime=2021-09-29T15:28:39.16Z

Status: We've rerouted traffic to an alternate portion of infrastructure and we're seeing the service trending toward recovery. We're now monitoring telemetry to ensure that impact is fully remediated. Next update by: Wednesday, September 29, 2021, at 8:30 PMUTC; PublishedTime=2021-09-29T17:41:14.853Z

Final status: We've confirmed after monitoring telemetry that rerouting traffic has succeeded in mitigating the impact. Next update by: Wednesday, September 29, 2021, at 8:30 PMUTC; PublishedTime=2021-09-29T17:41:14.853Z

--------------------------------------------------------------------------------------------------
Title Delays or problems connecting to Exchange Online
ID EX288106
Service Exchange Online
Feature Networking Issues
Status Service restored
Impact Users may have been unable to connect to the Exchange Online service.
Severity Sev1
Start time 2021-09-29T06:20:00Z
End time 2021-09-29T09:30:00Z
Last update 2021-09-29T10:44:55.103Z
More info
Root cause A section of our Edge networking infrastructure, responsible for facilitating user traffic through the service, was performing below acceptable performance thresholds. Next steps: - We're analyzing performance data and trends on the affected networking systems to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-29T10:20:18.2Z

EX288106 Details

Status: We've identified that a networking issue is causing impact. We're gathering diagnostic data to help us determine the best steps for remediation. Next update by: Wednesday, September 29, 2021, at 10:30 AM UTC; PublishedTime=2021-09-29T08:31:34.383Z

Final status: We've confirmed that some of our Edge networking infrastructure was performing below acceptable performance thresholds, resulting in impact. We rerouted user traffic to alternate infrastructure, which remediated impact. Next update by: Wednesday, September 29, 2021, at 10:30 AM UTC; PublishedTime=2021-09-29T08:31:34.383Z

--------------------------------------------------------------------------------------------------
Title Users sync requests may fail in OneNote desktop Win32 clients
ID OP288060
Service Microsoft 365 Apps
Feature Office Client issues
Status Service restored
Impact Users sync requests may have failed in OneNote desktop Win32 clients.
Severity Sev2
Start time 2021-09-29T00:33:31Z
End time 2021-09-29T01:28:51Z
Last update 2021-09-29T01:29:18.38Z
More info
Root cause A recent build deployment was causing a communications issue with OneNote sync requests, resulting in sync failures. Next steps: -We're continuing our deployment of the build to correct the configuration. -We're performing additional analysis on the recent build deployment to better understand what caused the configuration issue to occur which will help us to develop ways to avoid similar situations in future deployments. This is the final update for the event.; PublishedTime=2021-09-29T01:29:18.38Z

OP288060 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-09-29T00:37:08.18

Status: We identified a recent build deployment is causing a communications issue with OneNote sync requests. We're redeploying the build to correct the impacting configuration to remediate the impact. Next update by: Wednesday, September 29, 2021, at 3:30 AM UTC; PublishedTime=2021-09-29T00:58:36.533Z

Final status: While we're continuing the redeployment of the build to correct the impacting configuration, we remediated the impact by rerouting traffic to alternate infrastructure to process sync requests. We monitored the environment to verify that sync requests were being processed as expected. Next update by: Wednesday, September 29, 2021, at 3:30 AM UTC; PublishedTime=2021-09-29T00:58:36.533Z

--------------------------------------------------------------------------------------------------
Title Users may have experienced multiple issues in Microsoft PowerApps services due to intermittent connection failures
ID MM288000
Service Power Apps in Microsoft 365
Feature Service and web access issues
Status Service restored
Impact Users may have experienced multiple issues in Microsoft PowerApps services due to intermittent connection failures.
Severity Sev2
Start time 2021-09-28T21:00:00Z
End time 2021-09-28T22:02:00Z
Last update 2021-09-28T22:57:14.203Z
More info Services where users may have experienced impact include: Makers Portal Power Apps Studio Impact may have affected any user attempting to access these services, or execute actions on any app. Final status: We've identified that a latent code issue was inadvertently triggered after a recent service update, which caused transient impact to the service. While reviewing options to mitigate impact, we've confirmed that the transient issue resolved itself, and no further impact is occurring. Scope of impact: Your organization was affected by this event, and all users may have experienced impact. Start time: Tuesday, September 28, 2021, at 9:00 PM UTC End time: Tuesday, September 28, 2021, at 10:02 PM UTC Root cause: A latent code issue was inadvertently triggered after a recent service update, which caused transient impact to the service. Next steps: - We're continuing to review the circumstances resulting in the latent code issue. - We're monitoring service health to ensure the problem doesn't reoccur. This is the final update for the event.; PublishedTime=2021-09-28T22:57:14.203Z
Root cause A latent code issue was inadvertently triggered after a recent service update, which caused transient impact to the service. Next steps: - We're continuing to review the circumstances resulting in the latent code issue. - We're monitoring service health to ensure the problem doesn't reoccur. This is the final update for the event.; PublishedTime=2021-09-28T22:57:14.203Z

MM288000 Details

Status: We've investigated a problem with the Microsoft PowerApps service in which users may have experienced multiple issues, and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.

Final status: We've identified that a latent code issue was inadvertently triggered after a recent service update, which caused transient impact to the service. While reviewing options to mitigate impact, we've confirmed that the transient issue resolved itself, and no further impact is occurring.

--------------------------------------------------------------------------------------------------
Title Users accessing third party or custom OneDrive for Business apps can't use features related to file handling
ID OD288137
Service OneDrive for Business
Feature OneDrive for Business
Status Service restored
Impact Users accessing third party or custom OneDrive for Business apps couldn't use features related to file handling.
Severity Sev2
Start time 2021-09-28T19:35:00Z
End time 2021-09-29T14:25:00Z
Last update 2021-09-29T15:57:40.29Z
More info
Root cause A network authorization component misconfiguration caused a OneDrive for Business file handling issue. Next steps: - We're reviewing our network authorization component configuration procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-29T14:50:22.79Z

OD288137 Details

Status: We're reviewing details provided in the support case to determine the next steps needed to resolve this issue. Next update by: Wednesday, September 29, 2021, at 3:00 PM UTC; PublishedTime=2021-09-29T12:34:20.04Z

Status: We've identified that a network authorization component misconfiguration, is causing the OneDrive for Business file handling issue. We're updating the network authorization component configuration to mitigate impact. Next update by: Wednesday, September 29, 2021, at 4:00 PM UTC; PublishedTime=2021-09-29T13:59:22.513Z

Final status: We've successfully updated the network authorization component configuration and confirmed impact is remediated. Next update by: Wednesday, September 29, 2021, at 4:00 PM UTC; PublishedTime=2021-09-29T13:59:22.513Z

--------------------------------------------------------------------------------------------------
Title On-premises users who utilize Multi-Factor Authentication (MFA) may be unable to access Microsoft 365 services
ID MO287933
Service Microsoft 365 suite
Feature Portal
Status Post-incident report published
Impact On-premises users who utilize MFA have been unable to access Microsoft 365 services.
Severity Sev1
Start time 2021-09-28T13:08:00Z
End time 2021-09-28T17:20:00Z
Last update 2021-10-11T18:40:13.637Z
More info
Root cause

MO287933 Details

Status: We're investigating a potential issue with Multi-Factor Authentication and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-09-28T14:29:57.34

Status: We've identified 503 errors from specific processing components and we're reviewing these errors to identify the source. Next update by: Tuesday, September 28, 2021, at 4:30 PM UTC; PublishedTime=2021-09-28T15:05:57.18Z

Status: We're performing multiple recovery work streams to provide relief. In parallel, we're continuing to investigate the source of the 503 errors on the underlying infrastructure and reviewing recent changes as a potential root cause. Next update by: Tuesday, September 28, 2021, at 6:00 PM UTC; PublishedTime=2021-09-28T16:34:34Z

Status: We've performed scale up activities on the underlying infrastructure and we've received confirmation from some previously impacted users that the issue is now resolved for them. We also are seeing the service returning to a healthy state. In parallel, we continue to investigate the underlying root cause so further actions can be taken to ensure the issue does not reoccur. Engineering remains engaged to ensure that the service remains in a healthy state. Next update by: Tuesday, September 28, 2021, at 7:00 PM UTC; PublishedTime=2021-09-28T17:47:31.127Z

Final status: We performed scale up activities on the underlying infrastructure to mitigate the issue. We've also confirmed with impacted customers that the issue is now resolved for them. Next update by: Tuesday, September 28, 2021, at 7:00 PM UTC; PublishedTime=2021-09-28T17:47:31.127Z

--------------------------------------------------------------------------------------------------
Title Some users may experience multiple Public Switched Telephone Network (PSTN) issues in Microsoft Teams
ID TM287738
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have experienced multiple issues with the PSTN service in Microsoft Teams.
Severity Sev2
Start time 2021-09-27T16:30:00Z
End time 2021-10-08T00:00:00Z
Last update 2021-10-08T19:41:52.047Z
More info Users may have encountered multiple impacts, including but not limited to: Users may have been unable to acquire new phone numbers within the service or via Azure Access Control Service (AACS) while impact was ongoing. Users may have been unable to receive inbound PSTN calls or outbound calls may have failed to connect. Users may have been unable to input menu options utilizing DTMF. This impacted auto attendant, call queues and conferencing join. Users may have been unable to pull other users into conferences. Users may have experienced audio issues such as mid-call drops, disconnects, or being unable to hear users on the call. Users may have been unable to receive number assignments. This also impacted port requests as well as port requests status updates. Final status: We've verified after extended monitoring of the service that rerouting affected traffic to alternate service infrastructure successfully resolved the issue. Scope of impact: This issue affected enterprise voice customers and some users making inbound PSTN, auto attendant, and conference calls. This didn't affect users that use Direct Routing. Start time: Monday, September 27, 2021, at 4:30 PM UTC End time: Friday, October 8, 2021, at 12:00 AM UTC Root cause: An issue affecting our PSTN carrier service had resulted in intermittent impact to users' ability to utilize certain functions in the Skype for Business calling service. Next steps: - We're investigating why the issue affecting our PSTN carrier service had resulted in intermittent impact to prevent this problem from happening again in the future. This is the final update for the event.; PublishedTime=2021-10-08T19:38:02.1Z
Root cause An issue affecting our PSTN carrier service had resulted in intermittent impact to users' ability to utilize certain functions in the Skype for Business calling service. Next steps: - We're investigating why the issue affecting our PSTN carrier service had resulted in intermittent impact to prevent this problem from happening again in the future. This is the final update for the event.; PublishedTime=2021-10-08T19:38:02.1Z

TM287738 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We've identified that an issue affecting our PSTN carrier service has resulted in intermittent impact to users ability to utilize certain functions in the Microsoft Teams calling service. We're working alongside our carrier service team to identify sources of impact and determine mitigation strategies. Next update by: Monday, September 27, 2021, at 8:30 PM UTC; PublishedTime=2021-09-27T19:08:19.203Z

Status: We're continuing to work with our PSTN carrier service to identify the source of impact and formulate mitigation steps to resolve the issue. Next update by: Monday, September 27, 2021, at 10:30 PM UTC; PublishedTime=2021-09-27T20:21:29.523Z

Status: We've confirmed with our PSTN carrier service that their mitigation has been applied, and users should begin experiencing relief from the multiple points of impact. Next update by: Tuesday, September 28, 2021, at 12:30 AM UTC; PublishedTime=2021-09-27T21:53:24.09Z

Status: We've received reports from previously affected users that impact has mitigated and our telemetry reporting indicates the same. We're continuing to monitor the affected services for a period of time to ensure impact does not reoccur to confirm complete remediation. Next update by: Tuesday, September 28, 2021, at 7:30 PM UTC; PublishedTime=2021-09-27T23:36:05.777Z

Status: We've received reports that some users are still encountering intermittent impact. Our PSTN carrier service has taken additional mitigation actions targeting the remaining impact scenarios, and we're continuing to monitor the environment until the issue is fully resolved. Next update by: Wednesday, September 29, 2021, at 7:30 PM UTC; PublishedTime=2021-09-28T18:18:46.577Z

Status: After confirming that users are still intermittently experiencing impact, we're continuing to work with our PSTN carrier service to troubleshoot the problem and identify the actions necessary for completely remediating impact. Next update by: Wednesday, September 29, 2021, at 7:30 PM UTC; PublishedTime=2021-09-29T16:19:14.697Z

Status: We've determined that our PSTN carrier service experienced a period of service interruption after a subsection of service infrastructure dropped below their manageable service performance thresholds. After rerouting traffic to an alternate subsection of service infrastructure, we're internally monitoring the service to ensure that impact has been resolved. Next update by: Wednesday, September 29, 2021, at 9:30 PM UTC; PublishedTime=2021-09-29T19:00:18.697Z

Status: We've confirmed that our applied mitigation measures are working successfully and we're conducting an extended period of service monitoring to ensure that impact has been resolved. Next update by: Thursday, September 30, 2021, at 2:30 AM UTC; PublishedTime=2021-09-29T20:18:30.83Z

Status: Our service monitoring indicates that impact is intermittently recurring for some users. We're working to determine the cause of resumed impact and prepare potential mitigation actions. Next update by: Thursday, September 30, 2021, at 4:30 AM UTC; PublishedTime=2021-09-30T00:35:49.84Z

Status: After monitoring the affected environments, we've determined that no additional mitigation actions are needed and impact should be fully remediated at this time. We're continuing to monitor our telemetry to ensure the service remains stable and no additional scenarios are inadvertently affected. Next update by: Thursday, September 30, 2021, at 7:30 PM UTC; PublishedTime=2021-09-30T02:48:32.313Z

Status: Internal telemetry continues to indicate that impact has been resolved. We're persistently monitoring system telemetry to ensure the service remains stable and that no additional scenarios become inadvertently affected. Next update by: Thursday, September 30, 2021, at 11:00 PM UTC; PublishedTime=2021-09-30T17:58:20.507Z

Status: Our monitoring of the system is ongoing and indicates our mitigation efforts haven't produced any subsequent impact scenarios. We expect to be able to confirm full mitigation via our next scheduled update. Next update by: Friday, October 1, 2021, at 8:30 PM UTC; PublishedTime=2021-09-30T21:33:15.417Z

Status: Our monitoring processes continue, and we've not observed any residual impact thus far. We're continuing our extended monitoring to validate that all impact scenarios have been remediated and anticipate full mitigation by our next scheduled update. Next update by: Monday, October 4, 2021, at 8:30 PM UTC; PublishedTime=2021-10-01T19:02:45.31Z

Status: While we've not seen any new instances of the issue occurring, we're continuing our extended monitoring process to ensure the issue is fully resolved. Next update by: Wednesday, October 6, 2021, at 8:30 PM UTC; PublishedTime=2021-10-04T19:32:19.62Z

Status: We've confirmed we haven't seen any new occurrences of this issue, and will continue our extended monitoring process to ensure that the problem has fully resolved. Next update by: Friday, October 8, 2021, at 8:30 PM UTC; PublishedTime=2021-10-06T20:24:40.453Z

Final status: We've verified after extended monitoring of the service that rerouting affected traffic to alternate service infrastructure successfully resolved the issue. Next update by: Friday, October 8, 2021, at 8:30 PM UTC; PublishedTime=2021-10-06T20:24:40.453Z

--------------------------------------------------------------------------------------------------
Title Some users may experience multiple Public Switched Telephone Network (PSTN) issues in Microsoft Teams
ID TM287738
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have experienced multiple issues with the PSTN service in Microsoft Teams.
Severity Sev2
Start time 2021-09-27T16:30:00Z
End time 2021-10-08T00:00:00Z
Last update 2021-10-08T19:41:52.047Z
More info Users may have encountered multiple impacts, including but not limited to: Users may have been unable to acquire new phone numbers within the service or via Azure Access Control Service (AACS) while impact was ongoing. Users may have been unable to receive inbound PSTN calls or outbound calls may have failed to connect. Users may have been unable to input menu options utilizing DTMF. This impacted auto attendant, call queues and conferencing join. Users may have been unable to pull other users into conferences. Users may have experienced audio issues such as mid-call drops, disconnects, or being unable to hear users on the call. Users may have been unable to receive number assignments. This also impacted port requests as well as port requests status updates. Final status: We've verified after extended monitoring of the service that rerouting affected traffic to alternate service infrastructure successfully resolved the issue. Scope of impact: This issue affected enterprise voice customers and some users making inbound PSTN, auto attendant, and conference calls. This didn't affect users that use Direct Routing. Start time: Monday, September 27, 2021, at 4:30 PM UTC End time: Friday, October 8, 2021, at 12:00 AM UTC Root cause: An issue affecting our PSTN carrier service had resulted in intermittent impact to users' ability to utilize certain functions in the Skype for Business calling service. Next steps: - We're investigating why the issue affecting our PSTN carrier service had resulted in intermittent impact to prevent this problem from happening again in the future. This is the final update for the event.; PublishedTime=2021-10-08T19:38:02.1Z
Root cause An issue affecting our PSTN carrier service had resulted in intermittent impact to users' ability to utilize certain functions in the Skype for Business calling service. Next steps: - We're investigating why the issue affecting our PSTN carrier service had resulted in intermittent impact to prevent this problem from happening again in the future. This is the final update for the event.; PublishedTime=2021-10-08T19:38:02.1Z

TM287738 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We've identified that an issue affecting our PSTN carrier service has resulted in intermittent impact to users ability to utilize certain functions in the Microsoft Teams calling service. We're working alongside our carrier service team to identify sources of impact and determine mitigation strategies. Next update by: Monday, September 27, 2021, at 8:30 PM UTC; PublishedTime=2021-09-27T19:08:19.203Z

Status: We're continuing to work with our PSTN carrier service to identify the source of impact and formulate mitigation steps to resolve the issue. Next update by: Monday, September 27, 2021, at 10:30 PM UTC; PublishedTime=2021-09-27T20:21:29.523Z

Status: We've confirmed with our PSTN carrier service that their mitigation has been applied, and users should begin experiencing relief from the multiple points of impact. Next update by: Tuesday, September 28, 2021, at 12:30 AM UTC; PublishedTime=2021-09-27T21:53:24.09Z

Status: We've received reports from previously affected users that impact has mitigated and our telemetry reporting indicates the same. We're continuing to monitor the affected services for a period of time to ensure impact does not reoccur to confirm complete remediation. Next update by: Tuesday, September 28, 2021, at 7:30 PM UTC; PublishedTime=2021-09-27T23:36:05.777Z

Status: We've received reports that some users are still encountering intermittent impact. Our PSTN carrier service has taken additional mitigation actions targeting the remaining impact scenarios, and we're continuing to monitor the environment until the issue is fully resolved. Next update by: Wednesday, September 29, 2021, at 7:30 PM UTC; PublishedTime=2021-09-28T18:18:46.577Z

Status: After confirming that users are still intermittently experiencing impact, we're continuing to work with our PSTN carrier service to troubleshoot the problem and identify the actions necessary for completely remediating impact. Next update by: Wednesday, September 29, 2021, at 7:30 PM UTC; PublishedTime=2021-09-29T16:19:14.697Z

Status: We've determined that our PSTN carrier service experienced a period of service interruption after a subsection of service infrastructure dropped below their manageable service performance thresholds. After rerouting traffic to an alternate subsection of service infrastructure, we're internally monitoring the service to ensure that impact has been resolved. Next update by: Wednesday, September 29, 2021, at 9:30 PM UTC; PublishedTime=2021-09-29T19:00:18.697Z

Status: We've confirmed that our applied mitigation measures are working successfully and we're conducting an extended period of service monitoring to ensure that impact has been resolved. Next update by: Thursday, September 30, 2021, at 2:30 AM UTC; PublishedTime=2021-09-29T20:18:30.83Z

Status: Our service monitoring indicates that impact is intermittently recurring for some users. We're working to determine the cause of resumed impact and prepare potential mitigation actions. Next update by: Thursday, September 30, 2021, at 4:30 AM UTC; PublishedTime=2021-09-30T00:35:49.84Z

Status: After monitoring the affected environments, we've determined that no additional mitigation actions are needed and impact should be fully remediated at this time. We're continuing to monitor our telemetry to ensure the service remains stable and no additional scenarios are inadvertently affected. Next update by: Thursday, September 30, 2021, at 7:30 PM UTC; PublishedTime=2021-09-30T02:48:32.313Z

Status: Internal telemetry continues to indicate that impact has been resolved. We're persistently monitoring system telemetry to ensure the service remains stable and that no additional scenarios become inadvertently affected. Next update by: Thursday, September 30, 2021, at 11:00 PM UTC; PublishedTime=2021-09-30T17:58:20.507Z

Status: Our monitoring of the system is ongoing and indicates our mitigation efforts haven't produced any subsequent impact scenarios. We expect to be able to confirm full mitigation via our next scheduled update. Next update by: Friday, October 1, 2021, at 8:30 PM UTC; PublishedTime=2021-09-30T21:33:15.417Z

Status: Our monitoring processes continue, and we've not observed any residual impact thus far. We're continuing our extended monitoring to validate that all impact scenarios have been remediated and anticipate full mitigation by our next scheduled update. Next update by: Monday, October 4, 2021, at 8:30 PM UTC; PublishedTime=2021-10-01T19:02:45.31Z

Status: While we've not seen any new instances of the issue occurring, we're continuing our extended monitoring process to ensure the issue is fully resolved. Next update by: Wednesday, October 6, 2021, at 8:30 PM UTC; PublishedTime=2021-10-04T19:32:19.62Z

Status: We've confirmed we haven't seen any new occurrences of this issue, and will continue our extended monitoring process to ensure that the problem has fully resolved. Next update by: Friday, October 8, 2021, at 8:30 PM UTC; PublishedTime=2021-10-06T20:24:40.453Z

Final status: We've verified after extended monitoring of the service that rerouting affected traffic to alternate service infrastructure successfully resolved the issue. Next update by: Friday, October 8, 2021, at 8:30 PM UTC; PublishedTime=2021-10-06T20:24:40.453Z

--------------------------------------------------------------------------------------------------
Title Some users are unable to send email or receive using their Exchange ActiveSync (EAS) synced iOS or Android device
ID EX291497
Service Exchange Online
Feature E-Mail timely delivery
Status Service restored
Impact Users were unable to send email or receive using iOS or Android devices that leverage EAS.
Severity Sev2
Start time 2021-09-27T00:00:00Z
End time 2021-10-15T06:39:00Z
Last update 2021-10-15T17:02:19.433Z
More info Users were able to send and receive email as expected using the Outlook desktop client or Outlook on the web. Final status: We've confirmed that deployment has completed and verified through both telemetry and affected user reports that our fix to revert the Transport Layer Security (TLS) version 1.3 has mitigated impact. Scope of impact: This problem impacted some iOS and Android users who attempted to send or receive email through EAS synced devices. Start time: Monday, September 27, 2021, at 12:00 AM'UTC End time: Friday, October 15, 2021, at 6:39 AM'UTC Root cause: A Transport Layer Security (TLS) configuration issue was preventing users from sending or receiving email on mobile devices that leveraged EAS to connect to Exchange Online. Next steps: - We're reviewing our Transport Layer Security (TLS) code for improved performance and potential automated recovery options to reduce or avoid similar impact in the future. - We're reviewing reproductions of the issue to determine why this occurred within the TLS 1.3 version. This is the final update for the event.; PublishedTime=2021-10-15T17:02:19.433Z
Root cause A Transport Layer Security (TLS) configuration issue was preventing users from sending or receiving email on mobile devices that leveraged EAS to connect to Exchange Online. Next steps: - We're reviewing our Transport Layer Security (TLS) code for improved performance and potential automated recovery options to reduce or avoid similar impact in the future. - We're reviewing reproductions of the issue to determine why this occurred within the TLS 1.3 version. This is the final update for the event.; PublishedTime=2021-10-15T17:02:19.433Z

EX291497 Details

Status: We've identified that an authentication issue may be causing impact. We're reviewing ActiveSync logs to identify the source of the issue and establish a fix. Next update by: Thursday, October 14, 2021, at 1:30 PM UTC; PublishedTime=2021-10-14T11:27:47.043Z

Status: We've identified that unexpected throttling due to high levels of user traffic may be causing the issue. We're reviewing additional information provided by your organization to determine what is causing the throttling. Next update by: Thursday, October 14, 2021, at 3:30 PM UTC; PublishedTime=2021-10-14T13:06:04.517Z

Status: We're continuing to review user-provided information and network trace logs to identify what is causing the unexpected throttling. Next update by: Thursday, October 14, 2021, at 5:30 PMUTC; PublishedTime=2021-10-14T14:58:40.07Z

Status: Following our network trace review, we've identified a potential network authorization component configuration issue that may be preventing iOS and Android users from sending or receiving email within the Exchange Online service. We're analyzing the authorization component configuration to determine our next troubleshooting steps for remediating impact. Next update by: Thursday, October 14, 2021, at 7:30 PMUTC; PublishedTime=2021-10-14T17:10:17.983Z

Status: As a result of our additional investigation, we're now focused on a potential network authorization component configuration issue. We're disabling the network authorization component in an effort to remediate impact. Next update by: Thursday, October 14, 2021, at 9:30 PMUTC; PublishedTime=2021-10-14T19:05:35.143Z

Status: Our investigation into the configuration of a network authorization component has identified a Transport Layer Security (TLS) configuration issue is resulting in impact. We're updating the TLS configuration to remediate impact. Next update by: Thursday, October 14, 2021, at 11:30 PMUTC; PublishedTime=2021-10-14T21:15:37.22Z

Status: We're continuing to work on developing an update to TLS configuration. Next update by: Friday, October 15, 2021, at 1:30 AMUTC; PublishedTime=2021-10-14T23:20:25.073Z

Status: We've completed our development and started deployment of the TLS configuration change. We anticipate that this process will complete and resolve impact by our next scheduled update. Next update by: Friday, October 15, 2021, at 6:30 PMUTC; PublishedTime=2021-10-15T00:34:03.24Z

Final status: We've confirmed that deployment has completed and verified through both telemetry and affected user reports that our fix to revert the Transport Layer Security (TLS) version 1.3 has mitigated impact. Next update by: Friday, October 15, 2021, at 6:30 PMUTC; PublishedTime=2021-10-15T00:34:03.24Z

--------------------------------------------------------------------------------------------------
Title Users were seeing a blank page when using the classic experience to access SharePoint Online sites
ID SP287647
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users were seeing a blank page when using the classic experience to access SharePoint Online sites.
Severity Sev2
Start time 2021-09-26T23:47:00Z
End time 2021-09-28T21:30:00Z
Last update 2021-09-29T15:22:36.18Z
More info To bypass the impact, users could have enabled the modern experience in SharePoint Online to access sites. Final status: Our mitigation script has finished running, and we've confirmed with a sample previously affected users that impact is no longer occurring. Scope of impact: Impact was specific to some users who were served through the affected infrastructure. Start time: Sunday, September 26, 2021, at 11:47 PM UTC End time: Tuesday, September 28, 2021, at 9:30 PM UTC Root cause: A recent update contained a configuration change that inadvertently resulted in impact. Next steps: - We're reviewing our standard service update procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-09-29T14:55:26.667Z
Root cause A recent update contained a configuration change that inadvertently resulted in impact. Next steps: - We're reviewing our standard service update procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-09-29T14:55:26.667Z

SP287647 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're analyzing network trace logs provided by your organization to determine the source of the issue. Next update by: Monday, September 27, 2021, at 4:30 AMUTC; PublishedTime=2021-09-27T02:30:33.793Z

Status: Our analysis of network trace logs has been inconclusive. We're reviewing recent updates in your affected environment to see if any changes could be contributing to the impact. Next update by: Monday, September 27, 2021, at 6:30 AMUTC; PublishedTime=2021-09-27T04:25:59.007Z

Status: We've identified a potential deployment that could be resulting in impact. We're in the process of testing a possible mitigation to validate it's efficacy and confirm the root cause. Next update by: Monday, September 27, 2021, at 8:30 AMUTC; PublishedTime=2021-09-27T06:25:16.957Z

Status: We've identified that a recent update contained a configuration change, that inadvertently resulted in impact to the classic experience view. We've developed a fix and have begun deploying it to a portion of the affected infrastructure, to ensure it will mitigate the issue as expected. Next update by: Monday, September 27, 2021, at 10:30 AM UTC; PublishedTime=2021-09-27T08:15:54.837Z

Status: We've validated that the fix has successfully mitigated the issue for the portion of users that received it. We're deploying the fix to the remaining affected infrastructure to fully resolve the issue. Next update by: Monday, September 27, 2021, at 7:00 PM UTC; PublishedTime=2021-09-27T10:28:07.29Z

Status: We're continuing to apply the mitigation script on the affected infrastructure and expect to have a resolution timeline by the next scheduled update. Next update by: Tuesday, September 28, 2021, at 12:00 AMUTC; PublishedTime=2021-09-27T17:55:11.983Z

Status: Our application of the mitigation script to the affected infrastructure continues to progress, and we're expecting for the process to have completed and for impact to be resolved by our next scheduled update. Next update by: Tuesday, September 28, 2021, at 7:00 PMUTC; PublishedTime=2021-09-27T21:55:33.293Z

Status: Our mitigation script is taking longer than expected to deploy, but continues to make progress. We estimate that this fix will complete deployment and remediate impact by Tuesday, September 28, 2021, at 9:30 PM UTC. Next update by: Tuesday, September 28, 2021, at 11:30 PMUTC; PublishedTime=2021-09-28T17:32:23.107Z

Status: Our mitigation script has finished running, and we're working with affected users to ensure the issue is no longer occurring. Next update by: Wednesday, September 29, 2021, at 4:00 PMUTC; PublishedTime=2021-09-28T21:53:22.383Z

Final status: Our mitigation script has finished running, and we've confirmed with a sample previously affected users that impact is no longer occurring. Next update by: Wednesday, September 29, 2021, at 4:00 PMUTC; PublishedTime=2021-09-28T21:53:22.383Z

--------------------------------------------------------------------------------------------------
Title Some users may intermittently be unable to send or receive PTSN calls via Microsoft Teams
ID TM287505
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have intermittently been unable to send or receive PTSN calls via Microsoft Teams.
Severity Sev2
Start time 2021-09-25T19:45:00Z
End time 2021-09-25T20:33:00Z
Last update 2021-09-25T22:58:01.323Z
More info
Root cause A network issue between the Microsoft Teams service and a PSTN partner carrier resulted in PSTN call failures. Next steps: - We're collaborating with the partner carrier to more quickly detect and correct similar network issues that impact PSTN calling to prevent this problem from happening again in the future. This is the final update for the event.; PublishedTime=2021-09-25T22:45:46.61Z

TM287505 Details

Status: We've investigated a problem with the Microsoft Teams service in which a subset of users may have been unable to send or receive Public Switched Telephone Network (PTSN) calls intermittently and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.

Final status: We've identified that a network issue between the Microsoft Teams service and a PSTN partner carrier resulted in PSTN call failures. We've verified that our carrier partner applied a network configuration fix to remediate impact, and after monitoring the service, we've confirmed this action has successfully resolved impact.

--------------------------------------------------------------------------------------------------
Title Some users may intermittently be unable to send or receive PTSN calls via Microsoft Teams
ID TM287505
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have intermittently been unable to send or receive PTSN calls via Microsoft Teams.
Severity Sev2
Start time 2021-09-25T19:45:00Z
End time 2021-09-25T20:33:00Z
Last update 2021-09-25T22:58:01.323Z
More info
Root cause A network issue between the Microsoft Teams service and a PSTN partner carrier resulted in PSTN call failures. Next steps: - We're collaborating with the partner carrier to more quickly detect and correct similar network issues that impact PSTN calling to prevent this problem from happening again in the future. This is the final update for the event.; PublishedTime=2021-09-25T22:45:46.61Z

TM287505 Details

Status: We've investigated a problem with the Microsoft Teams service in which a subset of users may have been unable to send or receive Public Switched Telephone Network (PTSN) calls intermittently and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.

Final status: We've identified that a network issue between the Microsoft Teams service and a PSTN partner carrier resulted in PSTN call failures. We've verified that our carrier partner applied a network configuration fix to remediate impact, and after monitoring the service, we've confirmed this action has successfully resolved impact.

--------------------------------------------------------------------------------------------------
Title Users can't load Class Notebooks within the Microsoft Teams web and desktop clients
ID TM287186
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were unable to load Class Notebooks within the Microsoft Teams web and desktop clients.
Severity Sev2
Start time 2021-09-24T11:00:00Z
End time 2021-09-24T20:01:00Z
Last update 2021-09-24T20:15:02.327Z
More info Users may have also been unable to see the Class Notebook tab within the Microsoft Teams client. Users may have encountered an error indicating "Something Went Wrong" or "Still setting up your class" when attempting to load Class Notebooks. Final status: We've completed restarting the affected sections of service infrastructure, and after monitoring the environment we've confirmed these mitigation actions successfully resolved the service authentication issue and restored functionality. Scope of impact: Your organization was affected by this event, and any user attempting to load Class Notebooks may have been impacted. Start time: Friday, September 24, 2021, at 11:00 AM UTC End time: Friday, September 24, 2021, at 8:01 PM UTC Root cause: A service authentication issue was revealed by a recent standard service update, resulting in impact. Next steps: - Our investigation into the context surrounding the offending service update is ongoing to help us better understand why it resulted in impact. In parallel, we're closely monitoring the associated service authentication pathway to ensure this issue doesn't reoccur. This is the final update for the event.; PublishedTime=2021-09-24T20:15:02.327Z
Root cause A service authentication issue was revealed by a recent standard service update, resulting in impact. Next steps: - Our investigation into the context surrounding the offending service update is ongoing to help us better understand why it resulted in impact. In parallel, we're closely monitoring the associated service authentication pathway to ensure this issue doesn't reoccur. This is the final update for the event.; PublishedTime=2021-09-24T20:15:02.327Z

TM287186 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We've isolated a service authentication issue which was revealed by a recent standard service update. We're reviewing the change alongside the corresponding authentication errors to determine the most effective remediation strategy. Next update by: Friday, September 24, 2021, at 5:00 PMUTC; PublishedTime=2021-09-24T15:41:57.503Z

Status: We're continuing to investigate the details of a recent service update in tandem with the service telemetry which correlates with the corresponding authentication path to help us develop a targeted mitigation strategy. Next update by: Friday, September 24, 2021, at 7:00 PMUTC; PublishedTime=2021-09-24T16:41:05.21Z

Status: We've applied a targeted authentication configuration change to repair the issue, and we're restarting the affected sections of service infrastructure while monitoring the environment to ensure these mitigation actions are successful. Next update by: Friday, September 24, 2021, at 9:30 PMUTC; PublishedTime=2021-09-24T18:29:00.327Z

Final status: We've completed restarting the affected sections of service infrastructure, and after monitoring the environment we've confirmed these mitigation actions successfully resolved the service authentication issue and restored functionality. Next update by: Friday, September 24, 2021, at 9:30 PMUTC; PublishedTime=2021-09-24T18:29:00.327Z

--------------------------------------------------------------------------------------------------
Title Users can't create Microsoft Office files using the Power Automate tool within SharePoint Online
ID SP288333
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users were unable to create Microsoft Office files using the Power Automate tool within SharePoint Online.
Severity Sev2
Start time 2021-09-24T09:26:00Z
End time 2021-09-30T20:33:00Z
Last update 2021-09-30T22:08:17.64Z
More info This issue occurred when the 'Allow Chunking' option was enabled. As a workaround, users could have circumvented the issue by disabling the 'Allow Chunking' option and then splitting the "create file" and "update file" into two steps. Final status: We've reverted the change for all impacted infrastructure after successfully testing it in our internal environments and confirmed with affected users that impact has been mitigated. Scope of impact: Impact was specific to users who created Microsoft Office files with "Allow Chunking' enabled. Start time: Friday, September 24, 2021, at 9:26 AM'UTC End time: Thursday, September 30, 2021, at 8:33 PM'UTC Root cause: A recent API update which changed the way file storage is accessed resulted in impact. Next steps: - We're reviewing our API update procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-30T21:45:00.343Z
Root cause A recent API update which changed the way file storage is accessed resulted in impact. Next steps: - We're reviewing our API update procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-30T21:45:00.343Z

SP288333 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-09-30T11:48:41.023

Status: We're attempting to reproduce the issue in our internal testing environment, to help identify the root cause and develop a mitigation plan. Next update by: Thursday, September 30, 2021, at 7:00 PM UTC; PublishedTime=2021-09-30T12:31:49.023Z

Status: We've successfully reproduced the issue internally and our investigation indicates impact was caused by a recent API update which changed the way file storage is accessed. We're working to revert the change in a test environment to confirm this is the root cause prior to expanding to all impacted infrastructure. Next update by: Thursday, September 30, 2021, at 10:30 PMUTC; PublishedTime=2021-09-30T17:48:04.357Z

Final status: We've reverted the change for all impacted infrastructure after successfully testing it in our internal environments and confirmed with affected users that impact has been mitigated. Next update by: Thursday, September 30, 2021, at 10:30 PMUTC; PublishedTime=2021-09-30T17:48:04.357Z

--------------------------------------------------------------------------------------------------
Title Users may experience delays and timeouts when attempting to perform any actions within Power Apps
ID MM286942
Service Power Apps in Microsoft 365
Feature Service and web access issues
Status Service restored
Impact Users may have experienced delays and timeouts when attempting to perform any actions within Power Apps.
Severity Sev2
Start time 2021-09-23T15:25:00Z
End time 2021-09-23T16:10:00Z
Last update 2021-09-23T18:36:22.917Z
More info
Root cause A portion of storage infrastructure that facilitates Power Apps services unexpectedly restarted, resulting in impact. Next steps: - We're reviewing storage service logs to isolate the cause of the restart and develop automated recovery processes and procedures to prevent future occurrences. This is the final update for the event.; PublishedTime=2021-09-23T18:36:22.917Z

MM286942 Details

Status: We've investigated a problem with the Power Apps service in which users may have received errors or timeouts when performing any actions within Power Automate, and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-09-23T17:54:36.68

Final status: Our service monitoring notified us that a portion of storage infrastructure that facilitates Power Apps services was unexpectedly restarted, resulting in impact. We've monitored the environment and confirmed that service has been restored.

--------------------------------------------------------------------------------------------------
Title Users can't open PDF attachments when using Outlook on the web
ID EX287756
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users couldn't open PDF attachments when using Outlook on the web.
Severity Sev2
Start time 2021-09-23T02:00:00Z
End time 2021-09-29T18:57:00Z
Last update 2021-09-29T20:07:17.063Z
More info While we were working to resolve this issue, affected users could have opened PDF attachments via the Outlook desktop and mobile apps. Final status: We've confirmed via telemetry that the deployment has completed, and we've received reports from users that this has mitigated impact. Scope of impact: Your organization was affected and any user who attempted to open PDF attachments in Outlook on the web was impacted. Start time: Thursday, September 23, 2021, at 2:00 AM'UTC End time: Wednesday, September 29, 2021, at 6:57 PM'UTC Root cause: A recent Outlook Web App update inadvertently introduced a code issue affecting the PDF reader, resulting in the impact. Next steps: - We're reviewing our Outlook Web App update procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-29T20:05:55.86Z
Root cause A recent Outlook Web App update inadvertently introduced a code issue affecting the PDF reader, resulting in the impact. Next steps: - We're reviewing our Outlook Web App update procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-29T20:05:55.86Z

EX287756 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-09-27T21:31:50.573

Status: We're reviewing network traces provided by affected users to isolate the underlying cause and develop mitigation actions. Next update by: Monday, September 27, 2021, at 11:30 PMUTC; PublishedTime=2021-09-27T21:52:14.707Z

Status: Our network trace review has indicated 404 errors occurring, however the nature of the 404 errors isn't explicitly clear. We're continuing to review the trace logs for additional information and, in parallel, we're reaching out to some affected users to confirm reproduction steps to reproduce the issue in our environment for additional debugging. Next update by: Tuesday, September 28, 2021, at 1:00 AMUTC; PublishedTime=2021-09-27T23:09:33.163Z

Status: We're continuing to work in parallel with affected users to reproduce impact and capture reproduction steps to assist in further debugging the issue. Next update by: Tuesday, September 28, 2021, at 3:00 AMUTC; PublishedTime=2021-09-28T00:47:53.453Z

Status: We've contacted some affected users to request additional information for identifying the root cause and a potential mitigation plan. Please provide impact reproduction steps, as well as if all attachments or only specific types of attachments, such as PDF files, are affected. Next update by: Tuesday, September 28, 2021, at 10:30 PMUTC; PublishedTime=2021-09-28T02:41:32.943Z

Status: After collecting additional information from affected users and reviewing update history, we've determined that a recent Outlook Web App update inadvertently introduced a code issue affecting the PDF reader, resulting in the impact. We've developed and initiated the deployment of a fix for the PDF reader issue, and expect for the deployment to complete and resolve this issue by our next update. Next update by: Wednesday, September 29, 2021, at 10:30 PMUTC; PublishedTime=2021-09-28T21:06:02.217Z

Final status: We've confirmed via telemetry that the deployment has completed, and we've received reports from users that this has mitigated impact. Next update by: Wednesday, September 29, 2021, at 10:30 PMUTC; PublishedTime=2021-09-28T21:06:02.217Z

--------------------------------------------------------------------------------------------------
Title Users will see a frozen white screen when attempting to sign into the OneDrive for Business sync app
ID OD286672
Service OneDrive for Business
Feature OneDrive for Business
Status Service restored
Impact Users would have seen a frozen white screen when attempting to sign into the OneDrive for Business sync app.
Severity Sev2
Start time 2021-09-22T16:25:38Z
End time 2021-10-12T17:00:00Z
Last update 2021-10-22T18:30:01.28Z
More info This issue might have affected users who have the Microsoft Exploit Protection EAF (Export Address Filtering) feature enabled and have installed the September security updates. We've confirmed that users with Mobile Device Management (MDM) enabled devices could have added the following reg key as a workaround: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Policies\Microsoft\FeatureManagement\Override s /v 2767781516 /t REG_DWORD /d 0 /f We've also confirmed with affected users that disabling Export Address Filtering (EAF) was a potential workaround to alleviate the impact. Admins who wished to do so could have navigated to Windows Security > App & Browser Control > Exploit Protection Settings > Program Settings > OneDrive.exe > Turn off Export Address Filtering (EAF) and mitigated the impact. Alternatively, this issue could have been mitigated using Known Issue Rollback (KIR): https://techcommunity.microsoft.com/t5/windows-it-pro-blog/known-issue-rollback-helping-y ou-keep-windows-devices-protected/ba-p/2176831. For enterprise-managed devices that have installed an affected update and encountered this issue, it could have been mitigated by installing and configuring a special Group Policy installed via an MSI file available for: - Windows 10, version 1809 and Windows Server 2019: https://download.microsoft.com/downlo ad/7/f/1/7f194890-eea9-4cad-b19f-25ab67e41bbe/Windows%2010%20(1903%20&%201909)%20Known%20 Issue%20Rollback%20091721%2001.msi - Windows 10, version 1909: https://download.microsoft.com/download/7/f/1/7f194890-eea9-4 cad-b19f-25ab67e41bbe/Windows%2010%20(1903%20&%201909)%20Known%20Issue%20Rollback%2009172 1%2001.msi - Windows 10, version 2004, Windows 10, version 20H2 and Windows 10, version 21H1: https: //download.microsoft.com/download/7/f/1/7f194890-eea9-4cad-b19f-25ab67e41bbe/Windows%2010 %20(2004%20,%2020H2%20and%2021H1)%20Known%20Issue%20Rollback%20091721%2001.msi - Windows Server 2022: https://download.microsoft.com/download/7/f/1/7f194890-eea9-4cad-b 19f-25ab67e41bbe/Windows%20Server%202022%20Known%20Issue%20Rollback%20091821%2001.msi Note: Devices needed to be restarted after configuring the special Group Policy. For help, users could have reviewed the following: https://docs.microsoft.com/en-us/troubleshoot/windows-client/group-policy/use-group-polic y-to-deploy-known-issue-rollback For general information on using Group Policies, see: https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2- and-2012/hh831791(v=ws.11) Please note that many Mobile Device Management (MDM) customers couldn't deploy group policies. Customers that were unable to deploy group policy settings for the KIR may have opened support cases for further assistance. Final status: After an extended period of monitoring, we have confirmed that the fix within the October Windows update successfully resolves the issue and affected users have reported relief. Any user still experiencing impact will need to initiate the update on their device to remediate the issue. Scope of impact: Any user may have been prevented from signing into the OneDrive for Business sync app. Start time: Monday, September 13, 2021, at 8:26 PM UTC End time: Tuesday, October 12, 2021, at 5:00 PM UTC Preliminary root cause: A recent Windows patch deployment inadvertently introduced an Export Address Filtering (EAF) issue that's contributed to the impact. Next steps: - We're reviewing our update procedures to better identify similar issues during our development and testing cycles. - We're reviewing our standard service update procedures to avoid similar impact in the future. We'll publish a post-incident report within five business days.; PublishedTime=2021-10-22T18:30:01.28Z
Root cause A recent Windows patch deployment inadvertently introduced an Export Address Filtering (EAF) issue that's contributing to the impact. Next update by: Friday, September 24, 2021, at 11:00 PM UTC; PublishedTime=2021-09-22T20:12:41.437Z

OD286672 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-09-22T16:26:21.83

Status: We're reviewing system logs to isolate the origin of this issue. Next update by: Wednesday, September 22, 2021, at 6:00 PMUTC; PublishedTime=2021-09-22T16:42:39.78Z

Status: We're continuing our review of the available system logs to determine the underlaying cause of this issue. Next update by: Wednesday, September 22, 2021, at 7:00 PMUTC; PublishedTime=2021-09-22T17:08:58.853Z

Status: We've confirmed that a recent Windows patch deployment inadvertently introduced an EAF issue that's contributing to the impact. We've successfully reproduced the issue locally to test potential long-term solutions to resolve this problem. As we focus on a permanent solution, we've confirmed with affected users that following the steps in the KIR provided or disabling EAF are both functioning as a successful workaround to mitigate the impact. Next update by: Wednesday, September 22, 2021, at 9:00 PM UTC; PublishedTime=2021-09-22T19:26:17.753Z

Status: While focused on expediting a permanent fix for this issue, we've confirmed another workaround for users with Mobile Device Management (MDM) enabled devices by adding a specific reg key provided in the "more info" section. Next update by: Friday, September 24, 2021, at 11:00 PM UTC; PublishedTime=2021-09-22T20:12:41.437Z

Status: We've prepared a fix for this issue and are preparing it for deployment through our change management protocols. Due to the complexity and nature of our solution, we expect this process to take several weeks to complete. In the interim, users are encouraged to leverage the workarounds provided above. Next update by: Monday, October 11, 2021, at 11:30 PM UTC; PublishedTime=2021-09-24T21:48:40.703Z

Status: We're working through final preparations to deploy the fix for the affected customers, and we'll provide an updated ETA as soon as it becomes available. In the interim users are encouraged to leverage the workarounds provided above. Next update by: Wednesday, October 20, 2021, at 11:30 PM UTC; PublishedTime=2021-10-11T22:17:29.707Z

Status: The fix has been deployed and is contained within the October Windows update. Users will need to initiate the update on their device to remediate the issue. Next update by: Friday, October 22, 2021, at 11:30 PM UTC; PublishedTime=2021-10-19T18:22:19.51Z

Final status: After an extended period of monitoring, we have confirmed that the fix within the October Windows update successfully resolves the issue and affected users have reported relief. Any user still experiencing impact will need to initiate the update on their device to remediate the issue. Next update by: Friday, October 22, 2021, at 11:30 PM UTC; PublishedTime=2021-10-19T18:22:19.51Z

--------------------------------------------------------------------------------------------------
Title Admins' call record query searches through Graph API intermittently time-out or process slowly
ID TM286936
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Admins' call record query searches through Graph API intermittently timed-out or processed slowly.
Severity Sev2
Start time 2021-09-22T15:00:00Z
End time 2021-09-24T03:00:00Z
Last update 2021-09-24T04:48:58.513Z
More info This issue previously impacted Call Analytics query searches in the Microsoft Teams admin center. Final status: We've finished developing and deploying the secondary fix. While the fix was deploying, we found higher than normal usage on a portion of Graph API infrastructure that was contributing to impact. We've applied a change to alleviate the higher than normal usage and confirmed via telemetry that service is restored. Scope of impact: All admins utilizing CallRecords query searches through Graph API may have been intermittently impacted. Start time: Wednesday, September 22, 2021, at 3:00 PM'UTC End time: Friday, September 24, 2021, at 3:00 AM'UTC Root cause: Traffic processing inefficiencies related to the Graph API, combined with a higher than normal usage on a portion of Graph API infrastructure, resulted in impact. Next steps: - We're reviewing methods for improving the processing of traffic and preventing higher than normal usage in the future. This is the final update for the event.; PublishedTime=2021-09-24T04:47:53.813Z
Root cause Traffic processing inefficiencies related to the Graph API, combined with a higher than normal usage on a portion of Graph API infrastructure, resulted in impact. Next steps: - We're reviewing methods for improving the processing of traffic and preventing higher than normal usage in the future. This is the final update for the event.; PublishedTime=2021-09-24T04:47:53.813Z

TM286936 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-09-23T17:32:56.483

Status: We're continuing to investigate correlated service logs, infrastructure telemetry, and API response codes to isolate the underlying cause of this issue and help us develop a sustainable mitigation strategy. In parallel, we're rerouting the affected traffic to an alternate section of infrastructure in an attempt to quickly restore Call Analytics query search functionality. Next update by: Thursday, September 23, 2021, at 7:30 PMUTC; PublishedTime=2021-09-23T18:09:36.713Z

Status: We've rerouted the affected traffic to an alternate section of infrastructure to potentially mitigate some of the impact causing the Call Analytics query to misfunction. We're contacting some affected users to validate if this was effective. Our investigation into the root cause has been inconclusive; however, we're continuing to review available data for the cause of this event in order to find a path to a more permanent mitigation. Next update by: Thursday, September 23, 2021, at 9:30 PMUTC; PublishedTime=2021-09-23T19:08:27.067Z

Status: Through our service telemetry we've validated that our previous mitigation efforts were effective for restoring functionality for the Call Analytics queries through the Microsoft Teams admin center, although we're still investigating the root cause of that impact. Meanwhile, we've found that there's also impact within the call records query searches through Graph API, and we're rerouting traffic to alternate infrastructure and increasing concurrent connections to Graph API as a potential mitigation. We're also reviewing infrastructure telemetry and API response codes to diagnose the root cause. Next update by: Thursday, September 23, 2021, at 11:30 PMUTC; PublishedTime=2021-09-23T21:33:55.33Z

Status: After investigating the issue, we've found that traffic processing inefficiencies related to the Graph API are causing the issue. We've developed and validated a code fix that reduces memory consumption under certain circumstances, thus improving traffic processing speeds to resolve the issue. The fix has begun deployment and we expect to have an estimated time for completion by the next update time. Next update by: Friday, September 24, 2021, at 1:30 AMUTC; PublishedTime=2021-09-23T23:01:03.76Z

Status: The deployment has completed and has improved memory usage patterns related to the Graph API. It has not fully mitigated impact for some admins, and so we're developing a secondary deployment to resolve the residual impact. Next update by: Friday, September 24, 2021, at 3:30 AMUTC; PublishedTime=2021-09-24T00:31:05.197Z

Status: We're continuing to develop the secondary fix, which we'll then validate and deploy to resolve the residual impact. Next update by: Friday, September 24, 2021, at 9:00 PMUTC; PublishedTime=2021-09-24T01:53:45.643Z

Final status: We've finished developing and deploying the secondary fix. While the fix was deploying, we found higher than normal usage on a portion of Graph API infrastructure that was contributing to impact. We've applied a change to alleviate the higher than normal usage and confirmed via telemetry that service is restored. Next update by: Friday, September 24, 2021, at 9:00 PMUTC; PublishedTime=2021-09-24T01:53:45.643Z

--------------------------------------------------------------------------------------------------
Title Users can't open meeting options from the meeting window or calendar within any connection method of Microsoft Teams
ID TM286515
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users couldn't open meeting options from the meeting window or calendar within any connection method of Microsoft Teams.
Severity
Start time 2021-09-21T19:40:00Z
End time 2021-09-21T21:14:00Z
Last update 2021-09-21T22:22:39.823Z
More info
Root cause A recently deployed update meant to add features to the service contained a code issue that prevented users from accessing meeting options. Next steps: - We're reviewing our update procedures to better identify similar meeting option access issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-09-21T22:22:39.823Z

TM286515 Details

Status: We've investigated a problem in which users can't open meeting options from the meeting window or calendar within any connection method of Microsoft Teams, and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-09-21T21:44:55.817

Final status: We've determined that a recently deployed update meant to add features to the service contained a code issue that prevented users from accessing meeting options. We've rolled back this update and confirmed via internal monitoring that impact has been remediated.

--------------------------------------------------------------------------------------------------
Title Users may be unable to access Exchange Online using Representational State Transfer (REST) enabled connection methods
ID EX286189
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users may have intermittently been unable to connect to the Exchange Online service when using a REST connection method.
Severity
Start time 2021-09-20T02:21:00Z
End time 2021-09-20T02:49:00Z
Last update 2021-09-20T04:56:31.533Z
More info
Root cause

EX286189 Details

Status: We're analyzing service logs to determine the source of the issue. Next update by: Monday, September 20, 2021, at 4:30 AM UTC; PublishedTime=2021-09-20T03:14:08.133Z

Status: We're continuing to analyze service logs and available diagnostic data to isolate the problem. Next update by: Monday, September 20, 2021, at 6:00 AMUTC; PublishedTime=2021-09-20T04:22:00.243Z

Final status: While we were in the process of analyzing system logs to isolate the source of the problem and formulate a remediation plan, system monitoring indicated that the service had returned to normal health. We've confirmed that the problem is no longer occurring, and we'll continue to monitor the service to ensure that the problem does not happen again. Next update by: Monday, September 20, 2021, at 6:00 AMUTC; PublishedTime=2021-09-20T04:22:00.243Z

--------------------------------------------------------------------------------------------------
Title Some users may have been unable to access Exchange Online using any connection method
ID EX286116
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users may have been unable to access Exchange Online using any connection method.
Severity
Start time 2021-09-19T09:28:00Z
End time 2021-09-19T11:08:00Z
Last update 2021-09-19T12:29:10.27Z
More info
Root cause A portion of our DNS infrastructure was performing below expected thresholds. Next steps: - We're reviewing recent drops in service availability associated with DNS infrastructure to prevent further impact. This is the final update for the event.; PublishedTime=2021-09-19T12:09:26.423Z

EX286116 Details

Status: We're reviewing service monitoring telemetry to isolate the source of the issue and establish a fix. Next update by: Sunday, September 19, 2021, at 12:30 PM UTC; PublishedTime=2021-09-19T10:16:13.563Z

Final status: We've determined that a portion of our Domain Name System (DNS) infrastructure was performing below expected thresholds. Our automated recovery system repaired this problem, and we confirmed that service was restored after monitoring the environment. Next update by: Sunday, September 19, 2021, at 12:30 PM UTC; PublishedTime=2021-09-19T10:16:13.563Z

--------------------------------------------------------------------------------------------------
Title Some users were unable to connect to the Exchange Online service when using Representational State Transfer (REST)
ID EX286105
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users may have been unable to connect to the Exchange Online service when using a REST connection method.
Severity
Start time 2021-09-19T06:24:00Z
End time 2021-09-19T06:41:00Z
Last update 2021-09-19T07:27:37.79Z
More info
Root cause A portion of our DNS infrastructure was performing below expected thresholds, resulting in impact to the REST connection method. Next steps: - We're investigating the affected DNS infrastructure to determine the cause of the performance issue and find way to prevent this from reoccurring. This is the final update for the event.; PublishedTime=2021-09-19T07:27:37.79Z

EX286105 Details

Status: We're reviewing service monitoring telemetry to isolate the source of the issue and establish a fix. Next update by: Sunday, September 19, 2021, at 9:00 AM UTC; PublishedTime=2021-09-19T06:51:13.767Z

Final status: We've determined that a portion of our Domain Name System (DNS) infrastructure was performing below expected thresholds. Our automated recovery system repaired this problem, and we've confirmed that service was restored after monitoring the environment. Next update by: Sunday, September 19, 2021, at 9:00 AM UTC; PublishedTime=2021-09-19T06:51:13.767Z

--------------------------------------------------------------------------------------------------
Title Users of the iOS Outlook mobile app aren't getting push notifications after receiving email messages
ID EX286304
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users of the iOS Outlook mobile app aren't receiving push notifications after getting email messages.
Severity Sev2
Start time 2021-09-18T23:00:00Z
End time 2021-09-23T08:41:00Z
Last update 2021-09-25T00:31:33.817Z
More info
Root cause A code issue in a recent iOS Outlook mobile app update was causing push email notifications to fail. Next steps: - We're continuing the review of the impacted version to gather additional diagnostic data to isolate and prevent similar notification based issues in our validation and testing cycles. This is the final update for the event.; PublishedTime=2021-09-23T16:22:12.887Z

EX286304 Details

Status: We've received reports that users of the iOS Outlook mobile app are not getting push notifications after receiving email messages. We're reviewing update history and service telemetry to determine our next troubleshooting steps. Next update by: Monday, September 20, 2021, at 11:00 PM UTC; PublishedTime=2021-09-20T21:20:16.103Z

Status: We're continuing our initial review into update history and service telemetry to confirm the source for the event. Next update by: Tuesday, September 21, 2021, at 1:00 AMUTC; PublishedTime=2021-09-20T22:34:01.22Z

Status: We're continuing to investigate our update history and service telemetry while reproducing the issue internally in order to develop a plan to mitigate impact. Next update by: Tuesday, September 21, 2021, at 3:00 AMUTC; PublishedTime=2021-09-20T23:49:17.69Z

Status: After further review, we identified this issue to be with a recent iOS Outlook mobile app update. We've begun building a fix for the issue that once validated will be submitted to the App Store for iOS for expedited approval. Next update by: Tuesday, September 21, 2021, at 5:00 AMUTC; PublishedTime=2021-09-21T02:52:28.53Z

Status: We validated the fix and have submitted it to the App Store for iOS for expedited approval. Once approved, the update will be automatically released. Next update by: Tuesday, September 21, 2021, at 7:00 PMUTC; PublishedTime=2021-09-21T04:34:41.883Z

Status: The Outlook mobile iOS update continues through the expedited review process after being submitted for the App Store approval. Once the fix is approved, the update will be automatically released to users who have automatic updates configured on their devices and will be available for manual update or download to all users. Next update by: Wednesday, September 22, 2021, at 7:30 PMUTC; PublishedTime=2021-09-21T16:58:20.02Z

Status: The Outlook mobile iOS update is undergoing the final phases of the App Store expedited review process and upon approval will be automatically available for all users with devices configured to receive automatic updates. Next update by: Thursday, September 23, 2021, at 7:30 PMUTC; PublishedTime=2021-09-22T16:50:12.473Z

Final status: The iOS Outlook mobile app has been approved by Apple, and released to all users. Users should be able to update to version 4.2137.2 and no longer experience impact once complete. In addition, our telemetry has confirmed that this update has resolved the issue. Next update by: Thursday, September 23, 2021, at 7:30 PMUTC; PublishedTime=2021-09-22T16:50:12.473Z

--------------------------------------------------------------------------------------------------
Title Users may be intermittently unable to access the Exchange Online service through any connection method
ID EX286010
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users may have been intermittently unable to access the Exchange Online service through any connection method.
Severity
Start time 2021-09-18T13:48:00Z
End time 2021-09-18T14:18:00Z
Last update 2021-09-18T15:02:24.323Z
More info
Root cause

EX286010 Details

Final status: While we were in the process of analyzing system logs to isolate the source of the problem, system monitoring indicated that the service had returned to normal health. We've confirmed that the problem is no longer occurring, and we'll continue to monitor the service to ensure that the problem does not happen again.

--------------------------------------------------------------------------------------------------
Title Users with recently updated UPNs can't access OneDrive for Business content from Office.com
ID OD289670
Service OneDrive for Business
Feature OneDrive for Business
Status Service restored
Impact Users with recently updated UPNs are unable to access their OneDrive for Business content from Office.com.
Severity Sev2
Start time 2021-09-18T00:00:00Z
End time 2021-10-04T21:30:00Z
Last update 2021-10-09T00:12:03.02Z
More info Impact is specific to users whose User Principal Names (UPNs) changed between September 18, 2021, through October 2, 2021. Users will see a '404 not found' error. Users can self-mitigate this issue by navigating to their direct OneDrive for Business URL. Final status: As previously communicated, we've applied a code fix to resolve the underlying issue; however, additional action may be required to mitigate impact for affected users. To self-mitigate this issue, users need to access the OneDrive for Business service through their direct tenant URL. Additional information regarding impact and mitigation steps can be found in MC289953. Scope of impact: Users whose UPNs were updated between September 18, 2021, and October 2, 2021, are unable to access their OneDrive for Business content through Office.com. Start time: Saturday, September 18, 2021, at 12:00 AM UTC End time: Monday, October 4, 2021, at 9:30 PM UTC Root cause: A recent change made to correct the OneDrive for Business URL caching infrastructure unexpectedly resulted in impact. Next steps: - We're reviewing our change processes to prevent similar incidents from happening in the future. This is the final update for the event.; PublishedTime=2021-10-08T18:59:27.193Z
Root cause A recent change made to correct the OneDrive for Business URL caching infrastructure unexpectedly resulted in impact. Next steps: - We're reviewing our change processes to prevent similar incidents from happening in the future. This is the final update for the event.; PublishedTime=2021-10-08T18:59:27.193Z

OD289670 Details

Status: We're reviewing available diagnostic data to isolate the source of impact and identify troubleshooting actions. Next update by: Wednesday, October 6, 2021, at 8:30 PMUTC; PublishedTime=2021-10-06T19:35:13.567Z

Status: We've identified a recent change made to correct the OneDrive for Business URL caching infrastructure unexpectedly resulted in impact. We've applied a code fix to resolve the underlying problem and prevent additional users from impact, however, users whose UPNs were updated between September 18, 2021, and October 2, 2021, are unable to access their OneDrive for Business content through Office.com. This issue can be self-mitigated by navigating to users' OneDrive for Business account through a direct OneDrive for Business link. Next update by: Friday, October 8, 2021, at 8:30 PM UTC; PublishedTime=2021-10-06T20:34:05.777Z

Final status: As previously communicated, we've applied a code fix to resolve the underlying issue; however, additional action may be required to mitigate impact for affected users. To self-mitigate this issue, users need to access the OneDrive for Business service through their direct tenant URL. Additional information regarding impact and mitigation steps can be found in MC289953. Next update by: Friday, October 8, 2021, at 8:30 PM UTC; PublishedTime=2021-10-06T20:34:05.777Z

--------------------------------------------------------------------------------------------------
Title Users may have been unable to access Tasks, Shifts, or theWalkie Talkie app via any connection method of Microsoft Teams
ID TM285885
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have been unable to access Tasks, Shifts, or thWalkie Talkie app via any connection method of Microsoft Teams.
Severity
Start time 2021-09-17T19:30:00Z
End time 2021-09-18T05:42:00Z
Last update 2021-09-18T07:35:23.37Z
More info
Root cause A recent network infrastructure change, unexpectedly caused a portion of networking infrastructure to perform below acceptable performance thresholds, resulting in access issues for the Tasks, Shifts, and Walkie Talkie app. Next steps: - We're reviewing our network infrastructure change procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-18T07:34:39.527Z

TM285885 Details

Status: Our internal monitoring solutions have identified a potential issue impacting the Tasks, Shifts, or Walkie Talkie app within Microsoft Teams. We're reviewing service telemetry and update history to determine our next troubleshooting steps. Next update by: Saturday, September 18, 2021, at 4:00 AMUTC; PublishedTime=2021-09-18T00:08:46.663Z

Status: We've improved the performance on a portion of networking infrastructure to assist with traffic management and our analysis indicates that service is recovering to expected levels again. We're continuing to monitor telemetry to ensure impact is fully remediated. Next update by: Saturday, September 18, 2021, at 9:00 AMUTC; PublishedTime=2021-09-18T03:27:41.97Z

Final status: We've confirmed after a period of monitoring system telemetry, that impact is fully remediated. Next update by: Saturday, September 18, 2021, at 9:00 AMUTC; PublishedTime=2021-09-18T03:27:41.97Z

--------------------------------------------------------------------------------------------------
Title Some users may have been unable to access the Exchange Online service via any connection method
ID EX285738
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users may have been unable to access the Exchange Online service via any connection method.
Severity
Start time 2021-09-17T07:40:00Z
End time 2021-09-17T08:50:00Z
Last update 2021-09-17T09:54:54.51Z
More info
Root cause A section of infrastructure, responsible for directing user requests, was performing below acceptable performance thresholds. Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-17T09:53:39.833Z

EX285738 Details

Status: We're reviewing service monitoring telemetry to isolate the source of the issue and establish a fix. Next update by: Friday, September 17, 2021, at 11:00 AM UTC; PublishedTime=2021-09-17T09:00:24.19Z

Final status: We've identified a section of infrastructure, responsible for directing user requests, was performing below acceptable performance thresholds. We've restarted the affected infrastructure, to remediate impact. We've confirmed after monitoring the environment that the issue has been resolved. Next update by: Friday, September 17, 2021, at 11:00 AM UTC; PublishedTime=2021-09-17T09:00:24.19Z

--------------------------------------------------------------------------------------------------
Title Users can't log out of the Power Apps Maker Portal
ID MM285243
Service Power Apps in Microsoft 365
Feature Service and web access issues
Status Service restored
Impact Users were unable to log out of the Power Apps Maker Portal.
Severity
Start time 2021-09-15T18:10:00Z
End time 2021-09-15T23:18:29.217Z
Last update 2021-09-15T23:18:29.217Z
More info
Root cause A recent update to the Office header functionality contained a regression, causing impact. Next steps: - We're reviewing our deployment and provisioning procedures to understand why this recent update contained a regression to help prevent similar problems from reoccurring in the future. This is the final update for the event.; PublishedTime=2021-09-15T23:18:29.217Z

MM285243 Details

Final status: We've determined that a recent update to the Office header functionality contained a regression, causing impact. We've reverted this update and after monitoring telemetry, we've confirmed that impact has been fully remediated.

--------------------------------------------------------------------------------------------------
Title Users can't log out of the Power Apps Maker Portal
ID MM285243
Service Power Apps in Microsoft 365
Feature Service and web access issues
Status Service restored
Impact Users were unable to log out of the Power Apps Maker Portal.
Severity
Start time 2021-09-15T18:10:00Z
End time 2021-09-15T23:18:29.217Z
Last update 2021-09-15T23:18:29.217Z
More info
Root cause A recent update to the Office header functionality contained a regression, causing impact. Next steps: - We're reviewing our deployment and provisioning procedures to understand why this recent update contained a regression to help prevent similar problems from reoccurring in the future. This is the final update for the event.; PublishedTime=2021-09-15T23:18:29.217Z

MM285243 Details

Final status: We've determined that a recent update to the Office header functionality contained a regression, causing impact. We've reverted this update and after monitoring telemetry, we've confirmed that impact has been fully remediated.

--------------------------------------------------------------------------------------------------
Title Some users may receive errors when interacting with bots in the Microsoft Teams desktop client
ID TM286587
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have received errors when interacting with bots in the Microsoft Teams desktop client.
Severity Sev2
Start time 2021-09-15T01:45:00Z
End time 2021-10-08T18:14:00Z
Last update 2021-10-11T19:46:31.713Z
More info Users may have seen the message "There was a problem reaching this app" While we were focused on remediation, users may have been able to access the bots through personal chats in the Microsoft Teams desktop client as well as through the Microsoft Teams web client. Final status: We've confirmed that the fix has completed rolling and that the underlying issue has been addressed. Impacted users may need to use the check for updates feature, or restart the desktop client to receive the fix. Scope of impact: Some users interacting with bots using Microsoft Teams desktop version 1.4.00.22926 may have been impacted. Start time: Wednesday, September 15, 2021, at 1:45 AM'UTC End time: Friday, October 8, 2021, at 6:14 PM'UTC Root cause: An unexpected issue within a recent Microsoft Teams version was impacting interacting with bots in the Microsoft Teams desktop client. This is the final update for the event.; PublishedTime=2021-10-11T19:46:31.713Z
Root cause An unexpected issue within a recent Microsoft Teams version was impacting interacting with bots in the Microsoft Teams desktop client. This is the final update for the event.; PublishedTime=2021-10-11T19:46:31.713Z

TM286587 Details

Status: We're investigating a potential issue and checking for impact. We'll provide an update within 30 minutes.; PublishedTime=2021-09-22T05:06:13.067

Status: We identified an issue with a recent Microsoft Teams version that is unexpectedly causing errors when interacting with bots. We've developed a fix and working to validate it before deploying it to the affected environment. Next update by: Thursday, September 23, 2021, at 2:00 AMUTC; PublishedTime=2021-09-22T05:38:26.203Z

Status: We've begun deployment of the fix and are monitoring it as it progresses. Users may see remediation as the fix saturates the affected environment. Next update by: Friday, September 24, 2021, at 6:30 PMUTC; PublishedTime=2021-09-23T00:51:00.677Z

Status: The fix is continuing to deploy as anticipated and we're monitoring its progress to ensure it completes as expected. We'll provide a more precise estimated time to arrival via our next scheduled update. Next update by: Monday, September 27, 2021, at 6:30 PMUTC; PublishedTime=2021-09-24T18:27:28.167Z

Status: During the deployment, we ran into an issue that requires us to roll it back while we work to resolve the problem. After that issue is fixed, we'll restart the deployment and hope to be able to provide a timeline for mitigation then. Next update by: Tuesday, September 28, 2021, at 7:30 PMUTC; PublishedTime=2021-09-27T17:41:36.083Z

Status: We've corrected the issue affecting our deployment infrastructure and are re-initiating deployment of the fix. We're going to deploy the fix via staged rollout to ensure no secondary impacts occur and aim to provide a timeline to deployment completion with our next scheduled update. Next update by: Thursday, September 30, 2021, at 8:00 PMUTC; PublishedTime=2021-09-28T18:17:43.837Z

Status: We've confirmed via service telemetry that the rollout of the fix has resumed and is progressing as expected. We're continuing to monitor its staged deployment progress and currently project it will reach full saturation in all affected environments by Friday, October 8, 2021. Next update by: Friday, October 8, 2021, at 8:00 PMUTC; PublishedTime=2021-09-30T19:31:41.69Z

Status: We're continuing to monitor the staged deployment of the fix via service telemetry. While this continues to progress, it's taking longer than previously estimated. We now expect the fix to reach full saturation in all affected environments by Monday, October 11, 2021. Next update by: Monday, October 11, 2021, at 8:30 PMUTC; PublishedTime=2021-10-08T18:24:06.61Z

Final status: We've confirmed that the fix has completed rolling and that the underlying issue has been addressed. Impacted users may need to use the check for updates feature, or restart the desktop client to receive the fix. Next update by: Monday, October 11, 2021, at 8:30 PMUTC; PublishedTime=2021-10-08T18:24:06.61Z

--------------------------------------------------------------------------------------------------
Title Some users may encounter authentication errors when accessing Microsoft 365 iOS mobile apps
ID MO286520
Service Microsoft 365 suite
Feature Portal
Status Post-incident report published
Impact Users may have encountered authentication errors when accessing Microsoft 365 iOS mobile apps.
Severity Sev2
Start time 2021-09-15T00:27:00Z
End time 2021-09-26T19:49:00Z
Last update 2021-10-14T22:46:49.743Z
More info Users will continue to receive errors with subsequent authentication attempts. As a workaround, users experiencing impact may need to remove their Work Account from the Authenticator application. To do so, users may navigate to the Authenticator app and select their Work Account. On the following screen, users can select the Gear icon; and proceed to choose the "Remove account" option on the next screen. Users will receive a prompt to remove for one application or all applications, users may select "choose all Applications." After completing these steps, users can go to the application and sign in again which will add the user back to the Authenticator application and fully mitigate the issue. Current status: After deploying our fix, we've seen a reduction of errors in our service health telemetry, indicating users are seeing relief and impact is reducing. However, existing users who previously experienced impact may still need to utilize the workaround above to resolve the issue. We're preparing an additional fix for deployment to mitigate the remaining errors and resolve the issue. Scope of impact: Your organization is affected by this event and any user attempting to authenticate when accessing Microsoft 365 iOS mobile apps may be impacted. Start time: Wednesday, September 15, 2021, at 9:15 AM UTC Root cause: A recent configuration change performed in our service inadvertently introduced a code defect which is resulting in impact. Next update by: Tuesday, September 28, 2021, at 1:30 AM UTC; PublishedTime=2021-09-23T23:36:41.943Z
Root cause A recent deployment introduced a code defect resulting in authentication responses being returned, which the receiving applications were unable to handle. The defect itself was rooted in a new form of authentication responses that was supposed to be scoped to a specific scenario but led to refresh tokens getting corrupted as a result of a design flaw. Even though this change was rolled out gradually, we failed to detect the issue either in testing or through monitoring until it was reported by customers on September 21, 2021, at 9:16 PM UTC. Even after this change was reverted, the corrupted refresh tokes needed to be manually cleared to mitigate the issue. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. A PIR will be published for this incident by Thursday, October 14, 2021; PublishedTime=2021-09-28T00:07:34.73Z

MO286520 Details

Status: We're investigating an issue with our service in which it appears that some users are experiencing authentication errors when accessing any Microsoft 365 iOS mobile apps. We've determined that a recent configuration change performed in our service inadvertently introduced a code defect which is resulting in impact. To mitigate, we're reverting this change and we're monitoring this process to ensure it completes as expected. We'll provide a more precise estimated time to mitigation via our next scheduled update. Next update by: Wednesday, September 22, 2021, at 12:30 AMUTC; PublishedTime=2021-09-21T22:18:03.517Z

Status: We've completed our revert of the impacting change, although users may continue to see impact. In order to fully mitigate, users who experienced impact may need to remove their Work Account from the Authenticator application. To do so, users may navigate to the Authenticator app and select their Work Account. On the following screen, users can select the Gear icon; and proceed to choose the "Remove account" option on the next screen. Users will receive a prompt to remove for one application or all applications, users may select "choose all Applications." After completing these steps, users can go to the application and sign in again which will add the user back to the Authenticator application and fully mitigate the issue. Next update by: Friday, September 24, 2021, at 1:00 AM UTC; PublishedTime=2021-09-21T23:58:21.157Z

Status: After deploying our fix, we've seen a reduction of errors in our service health telemetry, indicating users are seeing relief and impact is reducing. However, existing users who previously experienced impact may still need to utilize the workaround above to resolve the issue. We're preparing an additional fix for deployment to mitigate the remaining errors and resolve the issue. Next update by: Tuesday, September 28, 2021, at 1:30 AM UTC; PublishedTime=2021-09-23T23:36:41.943Z

Final status: Our fix for the existing users who were previously experiencing impact has completed deployment. We've confirmed through telemetry that the issue has been mitigated and users will no longer experience authentication errors. Next update by: Tuesday, September 28, 2021, at 1:30 AM UTC; PublishedTime=2021-09-23T23:36:41.943Z

--------------------------------------------------------------------------------------------------
Title Users can't see digital signatures within .pdf files when using the SharePoint Online previewer
ID SP289898
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users couldn't see digital signatures within .pdf files when using the SharePoint Online previewer.
Severity Sev2
Start time 2021-09-14T13:00:00Z
End time 2021-10-11T22:00:00Z
Last update 2021-10-12T18:51:05.277Z
More info Impact was specific to the digital signatures within the previewer; the rest of the content was visible. Users could have seen the signatures after downloading the file or by using the "Open in Browser" feature. Final status: We've confirmed that the fix has completed deployment and verified via telemetry and customer report that this has mitigated impact. Scope of impact: Any user who attempted to view digital signatures within .pdf files using the SharePoint Online previewer experienced impact. Start time: Tuesday, September 14, 2021, at 1:00 PM'UTC End time: Monday, October 11, 2021, at 10:00 PM'UTC Root cause: A recent JavaScript update contained a configuration issue which resulted in impact. Next steps: - We're reviewing our JavaScript update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-12T18:51:05.277Z
Root cause A recent JavaScript update contained a configuration issue which resulted in impact. Next steps: - We're reviewing our JavaScript update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-12T18:51:05.277Z

SP289898 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-10-07T15:35:26.223

Status: We're reviewing the SharePoint Online .pdf file previewing code to assist in the root cause investigation. Next update by: Thursday, October 7, 2021, at 5:00 PMUTC; PublishedTime=2021-10-07T15:49:06.933Z

Status: We're continuing our review of the previewer code to isolate the underlying cause and develop mitigation actions. Next update by: Thursday, October 7, 2021, at 7:00 PMUTC; PublishedTime=2021-10-07T16:54:04.893Z

Status: Our investigation has confirmed that a recent Javascript update contained a configuration error, resulting in impact. We're in the process of developing a fix and expect to have a mitigation timeline by the next scheduled update. Next update by: Friday, October 8, 2021, at 2:30 AMUTC; PublishedTime=2021-10-07T18:08:56.33Z

Status: We've developed and deployed the fix. We're monitoring its progress and expect service will be restored by Tuesday, October 12, 2021. Next update by: Friday, October 8, 2021, at 10:30 PM UTC; PublishedTime=2021-10-08T01:34:49.213Z

Status: We've confirmed that deployment of the fix has reached 50 percent of the affected environment, and continue to expect deployment to complete to resolve this issue 'by Tuesday, October 12, 2021 Next update by: Tuesday, October 12, 2021, at 8:00 PMUTC; PublishedTime=2021-10-08T21:06:20.51Z

Final status: We've confirmed that the fix has completed deployment and verified via telemetry and customer report that this has mitigated impact. Next update by: Tuesday, October 12, 2021, at 8:00 PMUTC; PublishedTime=2021-10-08T21:06:20.51Z

--------------------------------------------------------------------------------------------------
Title Users intermittently can't load users' Organization chart in the Microsoft Teams desktop client
ID TM284736
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users intermittently couldn't load users' Organization chart in the Microsoft Teams desktop client.
Severity Sev2
Start time 2021-09-13T09:09:00Z
End time 2021-09-20T00:00:00Z
Last update 2021-09-24T02:24:00.93Z
More info
Root cause A large influx of data to the call log data stream was creating an unusually high workload on the components responsible for data transmission. Next steps: - We're reviewing our automated rebalancing processes and related performance logs to determine how we can better handle large influxes of data and better re-distribute workload. This is the final update for the event.; PublishedTime=2021-09-24T02:23:52.14Z

TM284736 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 60 minutes.; PublishedTime=2021-09-13T16:38:49.053

Status: We're reviewing system logs and any recent changes that may have contributed to the problem to determine our next troubleshooting steps. Next update by: Monday, September 13, 2021, at 7:30 PMUTC; PublishedTime=2021-09-13T17:26:18.487Z

Status: We're attempting to reproduce the issue locally to investigate what we suspect is an issue occurring when specific calls are being made from the dial-pad to generate dial-pad suggestions. Next update by: Monday, September 13, 2021, at 9:30 PMUTC; PublishedTime=2021-09-13T19:05:55.03Z

Status: Our effort to reproduce the problem locally is ongoing. We're continuing to review the influx in API calls from the dial-pad we suspect is causing the problem to allow us to formulate potential mitigations actions. Next update by: Monday, September 13, 2021, at 11:30 PMUTC; PublishedTime=2021-09-13T21:28:10.473Z

Status: We've been unable to reproduce the issue. We're continuing to investigate the source of the influx of the API calls to help identify any recent changes that could be contributing to the impact. Next update by: Tuesday, September 14, 2021, at 1:30 AMUTC; PublishedTime=2021-09-13T23:21:11.03Z

Status: At this point in our investigation, we can see a large influx of call events in our call log updates. We're reviewing recent updates and features that could be responsible for the increase in events we're seeing in the logs. Next update by: Tuesday, September 14, 2021, at 8:30 PMUTC; PublishedTime=2021-09-13T23:58:27.903Z

Status: Our investigation has determined that a large influx of data to the call log data stream is creating an unusually high workload on the components responsible for data transmission, resulting in impact. We're reviewing call logs and other telemetry to determine the source of this data influx and formulate a mitigation strategy. Next update by: Tuesday, September 14, 2021, at 10:30 PMUTC; PublishedTime=2021-09-14T20:16:19.02Z

Status: Our investigation has determined that a large influx of data to the call log data stream is creating an unusually high workload on the components responsible for data transmission, resulting in impact. We're developing a targeted update to mitigate this data influx and restore full functionality. Next update by: Wednesday, September 15, 2021, at 12:30 AMUTC; PublishedTime=2021-09-14T22:20:30.617Z

Status: Our investigation has determined that a large influx of data to the call log data stream is creating an unusually high workload on the components responsible for data transmission, resulting in impact. We're developing a targeted fix to mitigate this data influx and restore full functionality. This change will take some time to deploy, and we'll provide more information by our next scheduled communications update. Next update by: Wednesday, September 15, 2021, at 11:00 PMUTC; PublishedTime=2021-09-15T00:09:34.23Z

Status: Our fix has completed the initial testing and validation phase and is now beginning deployment to a small portion of affected infrastructure. We're continuing to monitor the deployment process to ensure it completes as expected. Some users will experience mitigation by our next update, and we expect that the fix will be deployed and enabled in all environments by late next week. Next update by: Tuesday, September 21, 2021, at 11:00 PMUTC; PublishedTime=2021-09-15T22:29:09.237Z

Status: We've confirmed that deployment of the fix has successfully completed, and the mitigating change is being enabled in affected environments. As this process goes forward, users will start experiencing relief from the problem and we anticipate that full mitigation will be achieved by Saturday, September 25, 2021. Next update by: Friday, September 24, 2021, at 11:00 PMUTC; PublishedTime=2021-09-21T22:33:57.147Z

Final status: We've validated that the fix has fully saturated and our internal monitoring shows that service is restored. Next update by: Friday, September 24, 2021, at 11:00 PMUTC; PublishedTime=2021-09-21T22:33:57.147Z

--------------------------------------------------------------------------------------------------
Title Admins can't create Data Subject Requests or User Data Search cases within the Microsoft 365 compliance center
ID EX284902
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Admins were unable to create Data Subject Requests or User Data Search cases within the Microsoft 365 compliance center.
Severity
Start time 2021-09-13T08:41:00Z
End time 2021-09-20T21:06:00Z
Last update 2021-09-20T22:26:44.303Z
More info While we were working on mitigation, affected admins could utilize Core eDiscovery to create and access the same data. Final status: The deployment of the fix is complete and we've determined that this issue is resolved. Scope of impact: Any admin running Data Subject Requests or User Data Searches may have been impacted. Start time: Thursday, September 9, 2021, at 12:55 PM'UTC End time: Monday, September 20, 2021, at 9:06 PM'UTC Root cause: A recent service update contained a code issue, resulting in impact. Next steps: - We're reviewing our validation and deployment procedures to prevent issues like this prior to impact in the future. This is the final update for the event.; PublishedTime=2021-09-20T21:30:53.17Z
Root cause A recent service update contained a code issue, resulting in impact. Next steps: - We're reviewing our validation and deployment procedures to prevent issues like this prior to impact in the future. This is the final update for the event.; PublishedTime=2021-09-20T21:30:53.17Z

EX284902 Details

Status: We're reviewing support case data to determine the next troubleshooting steps. Next update by: Tuesday, September 14, 2021, at 12:00 PM UTC; PublishedTime=2021-09-14T09:40:50.807Z

Status: We've identified an issue within the service's User Interface (UI), which is causing impact. We're developing a fix to resolve this issue. Next update by: Tuesday, September 14, 2021, at 8:00 PM UTC; PublishedTime=2021-09-14T11:22:03.073Z

Status: Our continued review of system telemetry has confirmed that impact is caused by a code issue within a recent service update. We're continuing our development of the fix and expect to have a mitigation timeline by the next scheduled update. Next update by: Wednesday, September 15, 2021, at 8:00 PMUTC; PublishedTime=2021-09-14T18:58:32.777Z

Status: The process of internally validating the fix is taking longer than initially anticipated; however, the testing phase is nearly complete and we expect deployment to begin within the next 24 hours. We'll provide an update on the fix deployment status and a more precise expected timeline to mitigation via our next scheduled update. Next update by: Thursday, September 16, 2021, at 7:30 PM UTC; PublishedTime=2021-09-15T19:28:24.357Z

Status: The deployment of the fix has once again been delayed as it undergoes final, rigorous testing to verify its efficacy in restoring our service's optimal functionality. Once verification completes, we'll provide additional details regarding the fix's status and anticipated time to mitigation. Next update by: Monday, September 20, 2021, at 10:30 PM UTC; PublishedTime=2021-09-16T19:31:13.317Z

Final status: The deployment of the fix is complete and we've determined that this issue is resolved. Next update by: Monday, September 20, 2021, at 10:30 PM UTC; PublishedTime=2021-09-16T19:31:13.317Z

--------------------------------------------------------------------------------------------------
Title Mac users may be unable to sign into or run multiple OneDrive accounts on a single device
ID OD284958
Service OneDrive for Business
Feature OneDrive for Business
Status Service restored
Impact Mac users may have been unable to sign into or run multiple OneDrive accounts on a single device.
Severity
Start time 2021-09-13T07:00:00Z
End time 2021-09-17T00:00:00Z
Last update 2021-09-17T01:57:24.833Z
More info This problem mostly affected users attempting to sign into both a personal account and a business account on the same device, but also impacted users signing into two business accounts. The issue only impacted users who have updated their devices to macOS 11.6. The issue also impacted both the OneDrive for Mac standalone sync client and the OneDrive on the Mac app acquired through the App Store. OneDrive for Mac standalone sync client users could have manually downloaded the updated build with a fix if they hadn't automatically received it from the following link: https://oneclient.sfx.ms/Mac/Prod/21.160.0808.0004/OneDrive.zip Final status: We've developed and deployed an additional fix for the OneDrive app through the App Store that is now available and impacted users can manually update their app to immediately resolve this issue. As previously mentioned, the OneDrive for Mac standalone sync client fix has been completed and users can manually download the updated build that includes the fix at: https://oneclient.sfx.ms/Mac/Prod/21.160.0808.0004/OneDrive.zip if their clients aren't already updated automatically. Scope of impact: This issue could have impacted any of your Mac users who recently updated to macOS 11.6 and were attempting to sign into multiple OneDrive accounts on the same device. Start time: Monday, September 13, 2021, at 7:00 AM UTC End time: Friday, September 17, 2021, at 12:00 AM UTC Root cause: A compatibility issue between macOS 11.6 and multi-account OneDrive for Business facilitation resulted in access issues for Mac users attempting to access multiple accounts from a single device. This is the final update for the event.; PublishedTime=2021-09-17T01:57:24.833Z
Root cause A compatibility issue between macOS 11.6 and multi-account OneDrive for Business facilitation resulted in access issues for Mac users attempting to access multiple accounts from a single device. This is the final update for the event.; PublishedTime=2021-09-17T01:57:24.833Z

OD284958 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 60 minutes.

Status: We're reviewing system logs to determine if the macOS update is a contributing factor for this issue. Next update by: Tuesday, September 14, 2021, at 9:00 PMUTC; PublishedTime=2021-09-14T17:55:29.857Z

Status: We're working with Apple to better understand multi-account OneDrive for Business compatibility issues for Mac devices that started with the release and adoption of macOS 11.6 to better understand the underlying cause and to develop a solution. Next update by: Wednesday, September 15, 2021, at 10:00 PMUTC; PublishedTime=2021-09-14T20:58:44.113Z

Status: We've developed and are validating a temporary fix for the OneDrive for Mac standalone sync client that will allow affected users to again utilize multiple OneDrive accounts, both personal and business, on the same macOS 11.6 updated device. Once testing and validation is complete, we'll provide an estimate on the deployment timeline. In parallel, we're looking into workarounds for users that leverage the OneDrive on the Mac app acquired through the App Store. Next update by: Thursday, September 16, 2021, at 1:00 AMUTC; PublishedTime=2021-09-15T20:22:22.447Z

Status: We've developed and are validating a temporary fix for the OneDrive for Mac standalone sync client that will allow affected users to again utilize multiple OneDrive accounts, both personal and business, on the same macOS 11.6 updated device. Once testing and validation is complete, we'll provide an estimate on the deployment timeline. In parallel, we're looking into workarounds for users that leverage the OneDrive app on macOS acquired through the App Store. Next update by: Thursday, September 16, 2021, at 9:00 PM UTC; PublishedTime=2021-09-15T23:32:58.77Z

Status: The OneDrive for Mac standalone sync client fix has completed and most users should have automatically received this update, although it may take a few more hours to reach some users that haven't yet received it. Users can also manually download the updated build that includes the fix at: https://oneclient.sfx.ms/Mac/Prod/21.160.0808.0004/OneDrive.zip. We're continuing to review how the OneDrive app on macOS acquired through the App Store is specifically affected by this problem. Next update by: Friday, September 17, 2021, at 9:30 PM UTC; PublishedTime=2021-09-16T18:48:40.423Z

Final status: We've developed and deployed an additional fix for the OneDrive app through the App Store that is now available and impacted users can manually update their app to immediately resolve this issue. As previously mentioned, the OneDrive for Mac standalone sync client fix has been completed and users can manually download the updated build that includes the fix at: https://oneclient.sfx.ms/Mac/Prod/21.160.0808.0004/OneDrive.zip if their clients aren't already updated automatically. Next update by: Friday, September 17, 2021, at 9:30 PM UTC; PublishedTime=2021-09-16T18:48:40.423Z

--------------------------------------------------------------------------------------------------
Title User are unable to use captions or transcripts in Microsoft Teams meetings or live events
ID TM284372
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact User weren't able to use captions or transcripts in Microsoft Teams meetings or live events.
Severity
Start time 2021-09-10T21:30:00Z
End time 2021-09-10T22:50:00Z
Last update 2021-09-10T23:56:32.673Z
More info Users with already existing caption or transcript sessions would see the captions or transcripts abruptly stop. Final status: We determined that an inadvertent DNS configuration change resulted in impact. We corrected the DNS configuration and our monitoring confirmed that the issue has been mitigated. Scope of impact: This issue may have potentially affected any of your users attempting to use captions and transcript features in Microsoft Teams meetings and live events. Start time: Friday, September 10, 2021, at 9:30 PM'UTC End time: Friday, September 10, 2021, at 10:50 PM'UTC Root cause: An inadvertent DNS configuration change resulted in users not seeing captions and transcripts. Next steps: - We've investigating the cause of the DNS configuration change to prevent this issue from reoccurring. This is the final update for the event.; PublishedTime=2021-09-10T23:56:32.673Z
Root cause An inadvertent DNS configuration change resulted in users not seeing captions and transcripts. Next steps: - We've investigating the cause of the DNS configuration change to prevent this issue from reoccurring. This is the final update for the event.; PublishedTime=2021-09-10T23:56:32.673Z

TM284372 Details

Status: We've investigated a problem with the Microsoft Teams service in which users may have been unable to use captions or transcripts in Microsoft Teams meetings and live events, and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-09-10T23:29:30.133

Final status: We determined that an inadvertent DNS configuration change resulted in impact. We corrected the DNS configuration and our monitoring confirmed that the issue has been mitigated.

--------------------------------------------------------------------------------------------------
Title Admins may be unable to review existing policies in the configured Settings Catalog (preview)
ID IT284228
Service Microsoft Intune
Feature Microsoft Intune
Status Service restored
Impact Admins may have been unable to review existing policies in the configured Settings Catalog (preview).
Severity
Start time 2021-09-09T22:22:00Z
End time 2021-09-10T02:47:00Z
Last update 2021-09-10T03:07:48.507Z
More info
Root cause A recent build deployment contained a code regression that prevents admins from reviewing existing policies in the configured Settings Catalog (preview). Next steps: - We're reviewing our build update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-09-10T03:06:38.47Z

IT284228 Details

Status: Our monitoring has detected an issue regarding admins being unable to review existing policies in the configured Settings Catalog (preview). We've determined that a recent build deployment contains a code regression that has resulted in impact. We're reverting the build to mitigate impact. Next update by: Friday, September 10, 2021, at 4:30 AMUTC; PublishedTime=2021-09-10T02:39:15.533Z

Final status: We've completed the reversion of the build and our telemetry shows that the issue is mitigated. Next update by: Friday, September 10, 2021, at 4:30 AMUTC; PublishedTime=2021-09-10T02:39:15.533Z

--------------------------------------------------------------------------------------------------
Title Users may be unable to place emergency calls through the Skype for Business service
ID LY284755
Service Skype for Business
Feature Audio and Video
Status Service restored
Impact Users may have been unable to place emergency calls through the Skype for Business service.
Severity
Start time 2021-09-09T21:00:00Z
End time 2021-09-13T20:30:00Z
Last update 2021-09-13T22:14:57.64Z
More info
Root cause Infrastructure used to route emergency calls through our partner carrier stopped processing traffic. As a result, communication efforts between Microsoft and our partner carrier failed, and prevented emergency calls from succeeding. This is the final update for the event.; PublishedTime=2021-09-13T22:14:57.64Z

LY284755 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-09-13T18:15:17.837

Status: Our investigation has determined that the infrastructure used to route emergency calls has entered an unhealthy state, resulting in impact. We're investigating system logs and telemetry from the affected infrastructure to formulate a mitigation strategy. Next update by: Monday, September 13, 2021, at 9:00 PMUTC; PublishedTime=2021-09-13T18:59:00.297Z

Status: Our investigation has determined that infrastructure used to route emergency calls through our partner carrier has stopped processing traffic, which is causing communication efforts between Microsoft and our partner carrier to fail. We've isolated this unhealthy infrastructure and are implementing call routing changes to mitigate the problem. We expect full functionality to be restored by our next scheduled update. Next update by: Monday, September 13, 2021, at 11:00 PMUTC; PublishedTime=2021-09-13T21:00:49.743Z

Final status: We've implemented the call routing changes and confirmed impact was resolved after monitoring the environment. Next update by: Monday, September 13, 2021, at 11:00 PMUTC; PublishedTime=2021-09-13T21:00:49.743Z

--------------------------------------------------------------------------------------------------
Title Admins are seeing some activities with incorrect locations within the Microsoft Cloud App Security portal
ID CS284737
Service Cloud App Security
Feature Cloud App Security
Status Service restored
Impact Admins were seeing some activities with incorrect locations within the Microsoft Cloud App Security portal.
Severity
Start time 2021-09-09T17:29:00Z
End time 2021-09-14T16:21:00Z
Last update 2021-09-14T17:40:23.677Z
More info This issue affected both Microsoft Cloud App Security and Office Cloud App Security services. Final status: We've developed and successfully deployed a fix, and confirmed that new activities should be tagged with the correct locations as expected; however, any existing affected activities will remain impacted until they they expire as per retention policy. Scope of impact: Your organization was affected by this event, and this issue may have potentially affected any of your admins attempting to view activity locations within the Microsoft Cloud App Security portal. Start time: Thursday, September 9, 2021, at 5:29 PM'UTC End time: Tuesday, September 14, 2021, at 4:21 PM'UTC Root cause: A recent update to the geo-location provider service was causing admins to see incorrect locations within the Microsoft Cloud App Security portal. Next steps: - We're reviewing our validation and update procedures to identify issues like this prior to deployment and prevent problems like this from occurring in the future. This is the final update for the event.; PublishedTime=2021-09-14T17:26:34.467Z
Root cause A recent update to the geo-location provider service was causing admins to see incorrect locations within the Microsoft Cloud App Security portal. Next steps: - We're reviewing our validation and update procedures to identify issues like this prior to deployment and prevent problems like this from occurring in the future. This is the final update for the event.; PublishedTime=2021-09-14T17:26:34.467Z

CS284737 Details

Status: We've identified a recent change in the geo-location provider service that is causing admins to see locations for some activities to be incorrectly reported as Tanzania within the Microsoft Cloud App Security portal. The team is working on a fix to revert the change and remediate impact. Next update by: Wednesday, September 15, 2021, at 2:00 PMUTC; PublishedTime=2021-09-13T16:39:36.123Z

Final status: We've developed and successfully deployed a fix, and confirmed that new activities should be tagged with the correct locations as expected; however, any existing affected activities will remain impacted until they they expire as per retention policy. Next update by: Wednesday, September 15, 2021, at 2:00 PMUTC; PublishedTime=2021-09-13T16:39:36.123Z

--------------------------------------------------------------------------------------------------
Title Admins may notice that some usage report data is delayed in the Microsoft 365 admin center
ID MO284660
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Admins may have noticed that some usage report data was delayed in the Microsoft 365 admin center.
Severity
Start time 2021-09-09T07:00:00Z
End time 2021-09-15T15:00:00Z
Last update 2021-09-16T03:10:30.387Z
More info Any usage report data starting from September 10, 2021, may have been delayed including: -Exchange Mailbox usage -Office 365 Groups activity -Office 365 License activity Final status: We've confirmed that the upstream data source delay is resolved and the usage report data is up to date. Scope of impact: This issue may have potentially affected any admin reviewing usage report data in the Microsoft 365 admin center. Start time: Thursday, September 9, 2021, at 7:00 AM'UTC End time: Wednesday, September 15, 2021, at 3:00 PM'UTC Root cause: An unexpected issue with the upstream data source responsible for facilitating the timely processing and delivery of usage report data was causing delays. Next steps: - We're reviewing the issue with the upstream data source to better our understanding of why it occurred and to prevent similar issues in the future. This is the final update for the event.; PublishedTime=2021-09-16T03:10:30.387Z
Root cause An unexpected issue with the upstream data source responsible for facilitating the timely processing and delivery of usage report data was causing delays. Next steps: - We're reviewing the issue with the upstream data source to better our understanding of why it occurred and to prevent similar issues in the future. This is the final update for the event.; PublishedTime=2021-09-16T03:10:30.387Z

MO284660 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-09-13T03:20:31.173

Status: We've identified an issue with the upstream data source responsible for facilitating the timely processing and delivery of usage report data. We're reviewing upstream data source logs to determine the best way to remediate the impact. Next update by: Tuesday, September 14, 2021, at 6:00 AMUTC; PublishedTime=2021-09-13T03:45:00.297Z

Status: We pinpointed the issue in the upstream data source logs that is preventing the report data from updating as expected. We're continuing our remediation efforts to fix the upstream data source delay and refresh the usage report data. Next update by: Thursday, September 16, 2021, at 5:00 AMUTC; PublishedTime=2021-09-14T04:47:24.467Z

Final status: We've confirmed that the upstream data source delay is resolved and the usage report data is up to date. Next update by: Thursday, September 16, 2021, at 5:00 AMUTC; PublishedTime=2021-09-14T04:47:24.467Z

--------------------------------------------------------------------------------------------------
Title Some users are unable to access their Exchange Online mailboxes via all connection methods
ID EX283808
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users were unable to access their Exchange Online mailboxes via all connection methods.
Severity
Start time 2021-09-08T18:25:00Z
End time 2021-09-08T19:35:00Z
Last update 2021-09-08T20:55:33.427Z
More info
Root cause A subset of mailbox database infrastructure was performing below acceptable standards and took longer than anticipated to restart, which subsequently inhibited users from accessing their Exchange Online mailboxes during the window of impact. Next steps: - We're continuing to investigate why the portion of infrastructure initially began operating below performance thresholds. - We're reviewing options to optimize our automated recovery system so we can more quickly remediate potential future impact instances. This is the final update for the event.; PublishedTime=2021-09-08T20:17:10.173Z

EX283808 Details

Status: We've investigated a problem with the Exchange Online service in which some users may have been unable to access their Exchange Online mailbox via any connection method, and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-09-08T19:25:29.603

Final status: Our investigation is complete, and we've determined that impact occurred due to a subset of mailbox database infrastructure that was performing below acceptable standards, which took longer than anticipated to restart and subsequently inhibited users from accessing their Exchange Online mailboxes. We've confirmed via monitoring service health telemetry that the affected portion of infrastructure has effectively recovered after being restarted and is performing at optimal functionality.

--------------------------------------------------------------------------------------------------
Title Emergency Public Switch Telephone Network (PSTN) calls may have not connected to local dispatch centers
ID TM283466
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Emergency PSTN calls may have connected to the global emergency center instead of the local dispatch center.
Severity
Start time 2021-09-07T06:23:00Z
End time 2021-09-07T08:10:00Z
Last update 2021-09-07T08:41:39.31Z
More info
Root cause Some recent changes made to the service as part of a standard service update, had inadvertently caused impact. Next steps: - We're reviewing our standard service update procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-09-07T08:41:39.31Z

TM283466 Details

Status: We're gathering diagnostic data for analysis. Next update by: Tuesday, September 7, 2021, at 10:00 AM UTC; PublishedTime=2021-09-07T08:02:23.01Z

Final status: We identified that some recent changes made to the service as part of a standard service update, had inadvertently caused impact. We've rolled back the update to a previous version, to resolve the issue. Next update by: Tuesday, September 7, 2021, at 10:00 AM UTC; PublishedTime=2021-09-07T08:02:23.01Z

--------------------------------------------------------------------------------------------------
Title Users may intermittently been unable to sign-in, view organization charts, send messages, and view files.
ID TM283251
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have intermittently been unable to sign-in, view organization charts, send messages, and view files
Severity
Start time 2021-09-06T01:56:00Z
End time 2021-09-06T04:40:00Z
Last update 2021-09-06T05:45:04.33Z
More info Any cached data may have intermittently been affected, such as user presence and organization charts. Users may have also intermittently been unable to add/remove users to teams and channels and sync OneNote with Microsoft Teams. Final status: We've finished reverting the configuration change and confirmed via telemetry that service is restored. Scope of impact: This issue intermittently affected users attempting to sign-in, view organization charts, send messages, and view files. Start time: Monday, September 6, 2021, at 1:56 AM'UTC End time: Monday, September 6, 2021, at 4:40 AM'UTC Root cause: A configuration change for regulating requests inadvertently encountered issues, resulting in impact. This is the final update for the event.; PublishedTime=2021-09-06T05:24:10.653Z
Root cause A configuration change for regulating requests inadvertently encountered issues, resulting in impact. This is the final update for the event.; PublishedTime=2021-09-06T05:24:10.653Z

TM283251 Details

Status: We've detected an issue in which users intermittently see stale presence, can't send chat messages, and new users can't sign-in. We're analyzing service logs to determine the source of the issue. Next update by: Monday, September 6, 2021, at 4:30 AMUTC; PublishedTime=2021-09-06T03:16:42.46Z

Status: We've determined that a configuration change for regulating requests inadvertently encountered issues, resulting in impact. We're reverting the configuration change to mitigate impact. Next update by: Monday, September 6, 2021, at 5:30 AMUTC; PublishedTime=2021-09-06T03:37:34.26Z

Final status: We've finished reverting the configuration change and confirmed via telemetry that service is restored. Next update by: Monday, September 6, 2021, at 5:30 AMUTC; PublishedTime=2021-09-06T03:37:34.26Z

--------------------------------------------------------------------------------------------------
Title Admins may have noticed delays from various Microsoft 365 reports starting from September 1, 2021
ID MO283134
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Admins may have noticed delays from some Microsoft 365 reports in the Microsoft 365 admin center.
Severity
Start time 2021-09-05T00:00:00Z
End time 2021-09-06T11:30:00Z
Last update 2021-09-06T12:42:31.5Z
More info The following reports were affected from September 1, 2021: - Microsoft 365 Group Activity reports - Exchange Online mailbox usage reports Final status: We've finished processing the backlog of data and confirmed that the issue is resolved. Scope of impact: This issue could have potentially affected any admin accessing the affected reports listed in the More info section through the Microsoft 365 admin center. Start time: Sunday, September 5, 2021, at 12:00 AM UTC End time: Monday, September 6, 2021, at 11:30 AM UTC Root cause: A data source stream delay was causing impact to the pipeline responsible for processing the report data. Next steps: - We're reviewing our data stream pipeline to refine its procedures and find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-06T12:42:31.5Z
Root cause A data source stream delay was causing impact to the pipeline responsible for processing the report data. Next steps: - We're reviewing our data stream pipeline to refine its procedures and find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-06T12:42:31.5Z

MO283134 Details

Status: Our internal monitoring notified us of an ongoing delay to Microsoft 365 Group Activity reports, Exchange Online mailbox usage reports, and multi-geo reports starting from September 1, 2021. Our investigation indicates that the data sources for the affected reports may be causing impact. We're analyzing the data source and pertinent requests to determine the reasoning behind the delays and how to best address it. Next update by: Monday, September 6, 2021, at 9:00 AMUTC; PublishedTime=2021-09-05T01:15:59.16Z

Status: We've resolved the delays within the data source, preventing further impact. We're processing the backlog of reports to fully resolve the issue, which we anticipate will be complete within the next 8 hours. We've confirmed that impact to Multi-geo reports has been resolved. Next update by: Monday, September 6, 2021, at 4:00 PM UTC; PublishedTime=2021-09-06T08:01:45.963Z

Final status: We've finished processing the backlog of data and confirmed that the issue is resolved. Next update by: Monday, September 6, 2021, at 4:00 PM UTC; PublishedTime=2021-09-06T08:01:45.963Z

--------------------------------------------------------------------------------------------------
Title Users on two specific apps may be unable to update data source credentials for those apps and see an error
ID PB284853
Service Power BI
Feature PowerBI.com
Status Service restored
Impact Users on two specific apps may have been unable to update data source credentials for those apps and see an error.
Severity
Start time 2021-09-04T12:43:00Z
End time 2021-09-20T02:15:00Z
Last update 2021-09-20T05:12:06.58Z
More info The Azure Cost Management and Sustainability Calculator apps were impacted. The error message read, 'We cannot convert a value of type Binary to type Function'. Final status: The deployment of the fix completed and we confirmed via monitoring that the impact was remediated. Scope of impact: Impact was specific to users who were served through the affected infrastructure. Start time: Saturday, September 4, 2021, at 12:43 PM'UTC End time: Monday, September 20, 2021, at 2:15 AM'UTC Root cause: A recent update to how the service handles retry requests contained a code regression, resulting in impact. Next steps: -We're analyzing our service update procedures to help identify similar code regressions prior to deployment. This is the final update for the event.; PublishedTime=2021-09-20T05:12:06.58Z
Root cause A recent update to how the service handles retry requests contained a code regression, resulting in impact. Next steps: -We're analyzing our service update procedures to help identify similar code regressions prior to deployment. This is the final update for the event.; PublishedTime=2021-09-20T05:12:06.58Z

PB284853 Details

Status: We've responded to reports of an issue where users on the Azure Cost Management and Sustainability Calculator apps may be unable to update data source credentials for those apps and see an error that reads, 'We cannot convert a value of type Binary to type Function'. We've determined that a recent update to how the service handles retry requests contained a code regression, resulting in impact. We're working to develop a fix, which we'll begin deploying as soon it as it completes testing and validation. Next update by: Tuesday, September 14, 2021, at 2:30 AMUTC; PublishedTime=2021-09-14T01:34:11.697Z

Status: We're continuing to develop a fix for the issue. We'll provide a timeline for deployment when available. Next update by: Tuesday, September 14, 2021, at 4:30 AMUTC; PublishedTime=2021-09-14T02:17:43.19Z

Status: We've completed our fix, which we're testing and validating in our internal environments. Once this is completed, we'll begin deployment and provide a deployment timeline. Next update by: Tuesday, September 14, 2021, at 9:30 AMUTC; PublishedTime=2021-09-14T04:24:39.693Z

Status: We're in the final stages of our internal validation process. Once this is concluded, we'll initiate deployment to the affected environment. Next update by: Friday, September 17, 2021, at 7:00 AM UTC; PublishedTime=2021-09-14T09:05:06.843Z

Status: We've completed our validation process and have begun deployment of the fix to the affected environment. We'll monitor the deployment progress to ensure the impact is remediated and we expect that the fix will finish deployment by Monday, September 20, 2021. Next update by: Monday, September 20, 2021, at 7:00 AMUTC; PublishedTime=2021-09-17T05:31:33.783Z

Final status: The deployment of the fix completed and we confirmed via monitoring that the impact was remediated. Next update by: Monday, September 20, 2021, at 7:00 AMUTC; PublishedTime=2021-09-17T05:31:33.783Z

--------------------------------------------------------------------------------------------------
Title Admins may have noticed Microsoft 365 app usage and Active Users reports data were delayed after September 4, 2021
ID MO283026
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Admins may have noticed Microsoft 365 app usage and Active Users reports data were delayed after September 4, 2021.
Severity
Start time 2021-09-04T00:00:00Z
End time 2021-09-04T13:30:00Z
Last update 2021-09-04T14:18:51.52Z
More info Admins may have also noticed that Microsoft Teams activity reports were delayed after September 4, 2021, and may have been unable to retrieve active users' report data via Microsoft Graph APIs. Final status: We've confirmed that all reports are now up to date and the issue has been resolved. Scope of impact: This issue could have potentially affected any admin if they were hosted through the affected infrastructure. Start time: Saturday, September 4, 2021, at 12:00 AM UTC End time: Saturday, September 4, 2021, at 1:30 PM UTC Root cause: A data source delay prevented the affected usage reports from appearing within the Microsoft 365 admin center within the expected timeframe. Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-04T14:18:51.52Z
Root cause A data source delay prevented the affected usage reports from appearing within the Microsoft 365 admin center within the expected timeframe. Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-04T14:18:51.52Z

MO283026 Details

Status: Our monitoring has detected an issue where Microsoft 365 app usage and Active Users reports data are delayed after September 4, 2021 in the Microsoft 365 admin center. We've identified a data source delay as the source of impact, and we're working to resolve that delay. Additionally, we're replaying the delayed data to ensure the reports are available to admins. Next update by: Saturday, September 4, 2021, at 10:00 AMUTC; PublishedTime=2021-09-04T03:08:41.19Z

Status: We've deployed a fix on the affected infrastructure and can confirm from monitoring that all reports, with the exception of the O365 Active User Report, have recovered. We estimate that the O365 Active User Report should recover by Saturday, September 4, 2021, at 1:00 PM UTC. Next update by: Saturday, September 4, 2021, at 1:00 PM UTC; PublishedTime=2021-09-04T08:19:46.917Z

Status: The O365 Active User Report is taking longer than expected to recover. We're continuing to monitor its progress, and estimate that it will be resolved by Sunday, September 5, 2021. Next update by: Sunday, September 5, 2021, at 11:00 AM UTC; PublishedTime=2021-09-04T11:17:29.407Z

Final status: We've confirmed that all reports are now up to date and the issue has been resolved. Next update by: Sunday, September 5, 2021, at 11:00 AM UTC; PublishedTime=2021-09-04T11:17:29.407Z

--------------------------------------------------------------------------------------------------
Title Some users can't view chat messages in the Microsoft Teams desktop client
ID TM285554
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users couldn't see chat messages in the Microsoft Teams desktop client.
Severity Sev2
Start time 2021-09-03T19:00:00Z
End time 2021-09-21T19:00:00Z
Last update 2021-09-28T23:28:50.393Z
More info To bypass the impact, affected users could have used the Microsoft Teams web app, mobile app, or signed out and back in to the desktop client. Final status: We developed and deployed a fix to remediate the impact with chat messages and slowness when switching chat windows. We've advised that any user sign out and restart their clients to ensure they have the latest version of Microsoft Teams. Please contact support and file a support case if you are continuing to experiences issues, and we'll resume our analysis of the problem. Scope of impact: This issue may have affected some users attempting to view chat messages in the Microsoft Teams desktop client. Start time: Friday, September 3, 2021, at 7:00 PM UTC End time: Tuesday, September 21, 2021, at 7:00 PM UTC Root cause: A recent change intended to provide general optimizations to the Microsoft Teams service contains a code issue that resulted in impact. Next steps: -We're continuing to analyze our optimization changes to ensure there is no similar impact in the future -We're actively investigating potential issues with multi-tenant multi-accounts related to the optimization changes. This is the final update for the event.; PublishedTime=2021-09-21T23:55:27.46Z
Root cause A recent change intended to provide general optimizations to the Microsoft Teams service contains a code issue that resulted in impact. Next steps: -We're continuing to analyze our optimization changes to ensure there is no similar impact in the future -We're actively investigating potential issues with multi-tenant multi-accounts related to the optimization changes. This is the final update for the event.; PublishedTime=2021-09-21T23:55:27.46Z

TM285554 Details

Status: After reverting the change for a subset of affected users, we've received reports that impact persists for some users, as well as some minor secondary impacts occurring. We're reviewing fresh logs captured from affected users still experiencing the issue to continue formulating potential mitigation options. Next update by: Friday, September 17, 2021, at 10:00 PMUTC; PublishedTime=2021-09-16T22:26:13.093Z

Status: Our telemetry as well as user reports indicate that reverting the change appears to have resolved the issue for a majority of affected users; however, we're seeing that a small subset of users are still impacted. We're continuing to capture and review telemetry data from users who are still experiencing the issue to determine what additional mitigation actions may be required to achieve remediation. Next update by: Tuesday, September 21, 2021, at 1:00 AMUTC; PublishedTime=2021-09-17T20:46:41.42Z

Status: While a majority of affected users are confirmed mitigated through our monitoring after they restarted their clients, there is still a small subset of users who are experiencing impact. We've advised that users sign out and restart their clients and that if they are still experiencing issues to capture telemetry data for us to review so we can assist with the proper remediation steps. Next update by: Wednesday, September 22, 2021, at 1:00 AMUTC; PublishedTime=2021-09-21T00:09:29.54Z

Final status: We developed and deployed a fix to remediate the impact with chat messages and slowness when switching chat windows. We've advised that any user sign out and restart their clients to ensure they have the latest version of Microsoft Teams. Please contact support and file a support case if you are continuing to experiences issues, and we'll resume our analysis of the problem. Next update by: Wednesday, September 22, 2021, at 1:00 AMUTC; PublishedTime=2021-09-21T00:09:29.54Z

--------------------------------------------------------------------------------------------------
Title Some users were unable to search for other users via any connection method of Microsoft Teams
ID TM282949
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were unable to search for other users via any connection method of Microsoft Teams.
Severity
Start time 2021-09-03T14:07:00Z
End time 2021-09-03T15:59:00Z
Last update 2021-09-03T17:23:31.17Z
More info Users' presence may have also been stale. Final status: We've identified that a section of infrastructure that manages user presence and discovery in Microsoft Teams wasn't processing a high volume of traffic as efficiently as expected, resulting in impact. While investigating internal telemetry to identify the cause of the traffic processing issue and create a strategy that remediates impact, system monitoring indicated that the service had returned to normal health. We've confirmed that the problem is no longer occurring, and we'll continue to monitor the service to ensure that the problem does not happen again. Scope of impact: Your organization was affected by this event, and some users attempting to search for other users or see presence information may have experienced impact. Start time: Friday, September 3, 2021, at 2:07 PM UTC End time: Friday, September 3, 2021, at 3:59 PM UTC Root cause: A section of infrastructure that manages user presence and discovery in Microsoft Teams wasn't processing a high volume of traffic as efficiently as expected, resulting in stale presence and degrading the ability to search for users in Microsoft Teams. Next steps: - We're analyzing performance data and trends on the infrastructure that processes user presence and discovery data to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-03T16:52:45.087Z
Root cause A section of infrastructure that manages user presence and discovery in Microsoft Teams wasn't processing a high volume of traffic as efficiently as expected, resulting in stale presence and degrading the ability to search for users in Microsoft Teams. Next steps: - We're analyzing performance data and trends on the infrastructure that processes user presence and discovery data to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-03T16:52:45.087Z

TM282949 Details

Status: We've investigated a problem with the Microsoft Teams service in which users were unable to search for other users via any connection method and their presence may have been stale. We've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-09-03T16:25:13.553

Final status: We've identified that a section of infrastructure that manages user presence and discovery in Microsoft Teams wasn't processing a high volume of traffic as efficiently as expected, resulting in impact. While investigating internal telemetry to identify the cause of the traffic processing issue and create a strategy that remediates impact, system monitoring indicated that the service had returned to normal health. We've confirmed that the problem is no longer occurring, and we'll continue to monitor the service to ensure that the problem does not happen again.

--------------------------------------------------------------------------------------------------
Title Problems connecting to Microsoft 365 services
ID MO282828
Service Microsoft 365 suite
Feature Portal
Status Investigation suspended
Impact Users may be experiencing issues when attempting to access any Microsoft 365 service.
Severity
Start time 2021-09-03T01:07:00Z
End time 2021-09-03T02:54:40Z
Last update 2021-09-03T02:54:42.92Z
More info
Root cause An external networking issue resulted in impact. Next steps: -We'll continue working with network providers to identify the underlying cause of the problem.; PublishedTime=2021-09-03T02:54:42.92Z

MO282828 Details

Status: We've detected an issue in which New Zealand based users may be experiencing issues connecting to any Microsoft 365 service. We're working with the network providers within the impacted region to determine our next steps.

Final status: Our investigation determined that an external networking issue occurred for Internet Service Providers (ISPs) in New Zealand. Using our available monitoring services, we've determined that service access has recovered for the region.

--------------------------------------------------------------------------------------------------
Title Admins may notice that some reports haven't updated since September 3, 2021 in the Microsoft 365 Admin center
ID MO283306
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Admins may have noticed that some reports hadn't updated since September 3, 2021 in the Microsoft 365 Admin center.
Severity
Start time 2021-09-03T00:00:00Z
End time 2021-09-10T05:00:00Z
Last update 2021-09-10T06:14:25.077Z
More info The following reports were impacted: - Teams User activity - Teams device usage - License Activity Final status: We monitored the processing of report data and confirmed that it's complete and up to date. Scope of impact: Impact was specific to of users who were served through the affected infrastructure. Start time: Friday, September 3, 2021, at 12:00 AM'UTC End time: Friday, September 10, 2021, at 5:00 AM'UTC Root cause: An issue with the upstream data source schema unexpectedly caused report data delays. Next steps: - We're reviewing our upstream data source pipeline configuration to help find ways of avoiding similar situations in the future. This is the final update for the event.; PublishedTime=2021-09-10T06:14:25.077Z
Root cause An issue with the upstream data source schema unexpectedly caused report data delays. Next steps: - We're reviewing our upstream data source pipeline configuration to help find ways of avoiding similar situations in the future. This is the final update for the event.; PublishedTime=2021-09-10T06:14:25.077Z

MO283306 Details

Status: We've identified a section of infrastructure, used as the data source for the reports, is experiencing an unexpected delay in processing the data. We're reviewing the data source to isolate the root cause of impact and determine our next troubleshooting steps. Next update by: Tuesday, September 7, 2021, at 5:30 AM UTC; PublishedTime=2021-09-06T13:40:38.053Z

Status: We've discovered a delay in the upstream data source and are working to isolate the cause of the problem. Next update by: Friday, September 10, 2021, at 5:30 AMUTC; PublishedTime=2021-09-07T03:57:08.493Z

Status: We've updated the upstream data source schema and can see that report data is processing. We'll continue to monitor the processing of report data to ensure that it completes and is up to date. Next update by: Saturday, September 11, 2021, at 5:30 AMUTC; PublishedTime=2021-09-10T04:00:26.99Z

Final status: We monitored the processing of report data and confirmed that it's complete and up to date. Next update by: Saturday, September 11, 2021, at 5:30 AMUTC; PublishedTime=2021-09-10T04:00:26.99Z

--------------------------------------------------------------------------------------------------
Title Users can't create new e-sign approvals in the Microsoft Teams Approvals app
ID TM282720
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users couldn't create new e-sign approvals in the Microsoft Teams Approvals app.
Severity
Start time 2021-09-02T17:25:00Z
End time 2021-09-02T19:40:00Z
Last update 2021-09-02T21:41:05.627Z
More info Additionally, users weren't receiving notifications for changes to existing e-sign approvals. Users would have experienced impact through all Microsoft Teams connection methods. Final status: Our investigation is complete, and we've confirmed that the affected authentication component intended to optimize the e-sign approval creation process in the Microsoft Teams Approvals app was exported in a way that didn't preserve key elements. When applied within the app, this led to inhibited functionality and subsequent impact. To mitigate, we've re-applied the authentication component via an alternative export methodology, and have confirmed through monitoring service telemetry that alleviation has been achieved. Scope of impact: Any user may have experienced impact. Start time: Thursday, September 2, 2021, at 5:25 PM UTC End time: Thursday, September 2, 2021, at 7:40 PM UTC Root cause: An authentication component intended to optimize the e-sign approval creation process in the Microsoft Teams Approvals app was exported in a way that didn't preserve it's full functionality. When applied within the app, this led to impact. Next steps: - We're reviewing the methodology by which authentication components are exported and applied in the Microsoft Teams Approvals app so we can prevent future instances of impact by ensuring all key elements are present. This is the final update for the event.; PublishedTime=2021-09-02T21:41:05.627Z
Root cause An authentication component intended to optimize the e-sign approval creation process in the Microsoft Teams Approvals app was exported in a way that didn't preserve it's full functionality. When applied within the app, this led to impact. Next steps: - We're reviewing the methodology by which authentication components are exported and applied in the Microsoft Teams Approvals app so we can prevent future instances of impact by ensuring all key elements are present. This is the final update for the event.; PublishedTime=2021-09-02T21:41:05.627Z

TM282720 Details

Status: We're investigating a potential issue with the Microsoft Teams service in which users are unable to create e-sign approvals. We're analyzing diagnostic data from recent changes performed on the service so we can better determine the underlying cause of the issue and develop a subsequent mitigation plan. Next update by: Thursday, September 2, 2021, at 8:00 PMUTC; PublishedTime=2021-09-02T19:00:36.63Z

Status: Our investigation so far suggests impact is occurring due to a recent change intended to optimize the authentication component utilized in the creation of e-sign approvals, which is causing impact as the authentication process isn't completing as expected. We're continuing to investigate precisely why this component isn't performing as expected, and we're simultaneously exploring options to expedite a mitigation strategy moving forward. Next update by: Thursday, September 2, 2021, at 10:00 PMUTC; PublishedTime=2021-09-02T20:01:49.717Z

Final status: Our investigation is complete, and we've confirmed that the affected authentication component intended to optimize the e-sign approval creation process in the Microsoft Teams Approvals app was exported in a way that didn't preserve key elements. When applied within the app, this led to inhibited functionality and subsequent impact. To mitigate, we've re-applied the authentication component via an alternative export methodology, and have confirmed through monitoring service telemetry that alleviation has been achieved. Next update by: Thursday, September 2, 2021, at 10:00 PMUTC; PublishedTime=2021-09-02T20:01:49.717Z

--------------------------------------------------------------------------------------------------
Title Users' locations weren't available to external users when making emergency calls via Microsoft Teams
ID TM282680
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users' locations weren't available to external users when making emergency calls via Microsoft Teams.
Severity
Start time 2021-09-02T13:07:00Z
End time 2021-09-02T15:51:00Z
Last update 2021-09-02T16:21:06.417Z
More info This impacted all connection methods of Microsoft Teams and would have required emergency call center agents to manually identify the users location, possibly resulting in a delayed emergency response time. Final status: We've identified that the infrastructure that routes location information to emergency call center users was performing below acceptable service thresholds, causing these location requests to timeout. We've routed service traffic to an alternate section of infrastructure and our internal monitoring indicates that location data is now being sent, remediating impact. Scope of impact: Your organization was affected by this event, and any user making an emergency call may have experienced impact. Start time: Thursday, September 2, 2021, at 1:07 PM UTC End time: Thursday, September 2, 2021, at 3:51 PM UTC Root cause: The infrastructure that routes location information to emergency call center users was performing below acceptable service thresholds, causing these location requests to timeout and resulting in impact. Next steps: - We're analyzing performance data and trends on the affected infrastructure to identify the cause of these timeouts and help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-02T16:21:06.417Z
Root cause The infrastructure that routes location information to emergency call center users was performing below acceptable service thresholds, causing these location requests to timeout and resulting in impact. Next steps: - We're analyzing performance data and trends on the affected infrastructure to identify the cause of these timeouts and help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-02T16:21:06.417Z

TM282680 Details

Status: We're analyzing internal telemetry to determine the source of the issue. Next update by: Thursday, September 2, 2021, at 4:30 PMUTC; PublishedTime=2021-09-02T15:02:15.583Z

Final status: We've identified that the infrastructure that routes location information to emergency call center users was performing below acceptable service thresholds, causing these location requests to timeout. We've routed service traffic to an alternate section of infrastructure and our internal monitoring indicates that location data is now being sent, remediating impact. Next update by: Thursday, September 2, 2021, at 4:30 PMUTC; PublishedTime=2021-09-02T15:02:15.583Z

--------------------------------------------------------------------------------------------------
Title Users may be unable to view transcripts for meetings in Microsoft Teams
ID TM282946
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have been unable to view transcripts for meetings in Microsoft Teams.
Severity
Start time 2021-09-02T12:30:00Z
End time 2021-09-03T23:30:00Z
Last update 2021-09-04T00:59:36.953Z
More info
Root cause

TM282946 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're reviewing service logs to gain a better understanding of the issue and inform our next troubleshooting steps. Next update by: Friday, September 3, 2021, at 6:00 PMUTC; PublishedTime=2021-09-03T16:27:05.027Z

Status: Our investigation so far has been inconclusive. We're continuing to analyze service logs in conjunction with gathering Object IDs which will allow us to more thoroughly understand the root cause of the issue and identify potential mitigating actions. Next update by: Friday, September 3, 2021, at 9:00 PMUTC; PublishedTime=2021-09-03T17:47:55.373Z

Status: We've identified reoccurring timeout errors related to the service on which the transcript function relies. We're analyzing a recent spike in these errors via telemetry and are reviewing request IDs to identify what is causing these timeouts and if this is a contributing factor of impact. Next update by: Saturday, September 4, 2021, at 12:00 AMUTC; PublishedTime=2021-09-03T20:51:14.967Z

Status: We're investigating the reoccurring timeout errors related to the service on which the transcript function relies on, as well as the recent spike in these errors so we can work on developing a fix for the impacted users. In parallel, we restarted affected portions of infrastructure to determine if this helps immediately remediate impact but early results indicate that impact still persists. Next update by: Saturday, September 4, 2021, at 3:00 AMUTC; PublishedTime=2021-09-03T23:47:58.173Z

Final status: While we were in the process of investigating the timeout errors to isolate the source of the problem, system monitoring indicated that the service had returned to normal health. We've confirmed that the problem is no longer occurring, and we'll continue to monitor the service to ensure that the problem does not happen again. Next update by: Saturday, September 4, 2021, at 3:00 AMUTC; PublishedTime=2021-09-03T23:47:58.173Z

--------------------------------------------------------------------------------------------------
Title Admins not seeing up to date or accurate information for mail flow status reports within the Exchange admin center (EAC)
ID EX283033
Service Exchange Online
Feature Management and Provisioning
Status Service restored
Impact Admins weren't seeing up to date or accurate information for mail flow status reports within the EAC.
Severity
Start time 2021-09-02T08:00:00Z
End time 2021-09-04T15:46:00Z
Last update 2021-09-04T16:22:50.377Z
More info
Root cause A portion of the infrastructure responsible for facilitating mail status report data was unexpectedly performing below the expected service threshold, resulting in impact. Next steps: - We're reviewing our code for improved EAC and reporting infrastructure performance to help us find ways to reduce or avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-09-04T16:22:50.377Z

EX283033 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-09-04T03:50:54.243

Status: We're reviewing provided trace logs to determine what's preventing the reports from showing as expected. Next update by: Saturday, September 4, 2021, at 5:30 AMUTC; PublishedTime=2021-09-04T04:15:23.513Z

Status: We identified that a portion of infrastructure responsible for facilitating mail status report data that is performing below the expected service threshold. We're working on a fix to restore the generation of the mail flow status reports to remediate the impact. Next update by: Saturday, September 4, 2021, at 7:30 AMUTC; PublishedTime=2021-09-04T05:28:43.097Z

Status: We've developed a fix to restore the generation of the mail flow status reports and are applying it to the affected environment. Next update by: Saturday, September 4, 2021, at 6:00 PMUTC; PublishedTime=2021-09-04T06:23:55.017Z

Final status: We've successfully applied the targeted fix to repair the infrastructure degradation, and we've internally validated that the issue is no longer occurring. Next update by: Saturday, September 4, 2021, at 6:00 PMUTC; PublishedTime=2021-09-04T06:23:55.017Z

--------------------------------------------------------------------------------------------------
Title Users can't automate Power Automate items in their document libraries within SharePoint Online
ID SP282669
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users couldn't automate Power Automate items in their document libraries within SharePoint Online.
Severity
Start time 2021-09-02T03:02:00Z
End time 2021-09-02T17:30:00Z
Last update 2021-09-02T18:12:54.69Z
More info Impact was specific to manually created flows. Flows created via template were not affected. Final status: We determined that impact was caused by a recent service update which contained a change that altered the way category names are defined. We've reverted the change and confirmed mitigation with affected users. Scope of impact: This issue may have impacted any user. Start time: Thursday, September 2, 2021, at 3:02 AM'UTC End time: Thursday, September 2, 2021, at 5:30 PM'UTC Root cause: A recent service update intended to change the method category names are defined contained a code issue that inadvertently caused impact. Next steps: - We're reviewing the category name definition code changes to isolate the impacting change to better identify similar issues within the validation process in the future. This is the final update for the event.; PublishedTime=2021-09-02T18:12:54.69Z
Root cause A recent service update intended to change the method category names are defined contained a code issue that inadvertently caused impact. Next steps: - We're reviewing the category name definition code changes to isolate the impacting change to better identify similar issues within the validation process in the future. This is the final update for the event.; PublishedTime=2021-09-02T18:12:54.69Z

SP282669 Details

Status: We're reviewing the details provided in the support case and our internal reproduction of the issue to determine the root cause of impact. Next update by: Thursday, September 2, 2021, at 7:30 PM UTC; PublishedTime=2021-09-02T13:42:53.01Z

Final status: We determined that impact was caused by a recent service update which contained a change that altered the way category names are defined. We've reverted the change and confirmed mitigation with affected users. Next update by: Thursday, September 2, 2021, at 7:30 PM UTC; PublishedTime=2021-09-02T13:42:53.01Z

--------------------------------------------------------------------------------------------------
Title Users were unable to open notebooks in Microsoft OneNote
ID OP283538
Service Microsoft 365 Apps
Feature Office Client issues
Status Service restored
Impact Users were unable to open notebooks in Microsoft OneNote.
Severity
Start time 2021-09-01T17:30:00Z
End time 2021-09-07T19:00:00Z
Last update 2021-09-08T04:50:47.553Z
More info Impact was specific to all versions of Microsoft OneNote excluding OneNote 2016 and web client. Windows 10 desktop client, iOS and Android mobile app versions were all impacted. Final status: We've confirmed that a recent deployment intended to provide optimizations for service stability inadvertently introduced a regression in service logic, resulting in impact. We've reverted the affecting change and confirmed that impact is mitigated. Scope of impact: Your organization was affected by this event, and all users were impacted. Start time: Wednesday, September 1, 2021, at 5:30 PM'UTC End time: Tuesday, September 7, 2021, at 7:00 PM'UTC Root cause: A recent deployment intended to provide optimizations for service stability inadvertently introduced a regression in service logic, resulting in impact. Next steps: - We're reviewing our update procedures to identify how impact was missed during testing and validation to better prevent similar future impact. This is the final update for the event.; PublishedTime=2021-09-07T19:25:58.897Z
Root cause A recent deployment intended to provide optimizations for service stability inadvertently introduced a regression in service logic, resulting in impact. Next steps: - We're reviewing our update procedures to identify how impact was missed during testing and validation to better prevent similar future impact. This is the final update for the event.; PublishedTime=2021-09-07T19:25:58.897Z

OP283538 Details

Status: We've investigated a problem with the Microsoft OneNote service in which users were unable to open notebooks, and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.

Final status: We've confirmed that a recent deployment intended to provide optimizations for service stability inadvertently introduced a regression in service logic, resulting in impact. We've reverted the affecting change and confirmed that impact is mitigated.

--------------------------------------------------------------------------------------------------
Title Some users aren't receiving channel notifications or can't update their channel settings in Microsoft Teams.
ID TM282380
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users weren't receiving channel notifications or couldn't update their channel settings in Microsoft Teams.
Severity
Start time 2021-09-01T11:47:00Z
End time 2021-09-01T14:47:00Z
Last update 2021-09-01T15:50:28.587Z
More info
Root cause A portion of infrastructure that facilitates Microsoft Teams notifications and channel settings experienced an increase in latency, resulting in impact. Next steps: - We're reviewing system telemetry to isolate the underlying cause of the latency and develop prevention strategies to prevent similar impact in the future. This is the final update for the event.; PublishedTime=2021-09-01T15:50:28.587Z

TM282380 Details

Status: We've identified a section of infrastructure is experiencing higher than expected latency. We're reviewing service logs to isolate where the latency is occurring and determine our next troubleshooting steps. Next update by: Wednesday, September 1, 2021, at 4:00 PM UTC; PublishedTime=2021-09-01T13:50:01.817Z

Final status: We've rebalanced load from the affected infrastructure and confirmed mitigation via telemetry. Next update by: Wednesday, September 1, 2021, at 4:00 PM UTC; PublishedTime=2021-09-01T13:50:01.817Z

--------------------------------------------------------------------------------------------------
Title Users may experience delays of up to two hours when sending email from on-premises to Exchange Online
ID EX282101
Service Exchange Online
Feature E-Mail timely delivery
Status Post-incident report published
Impact Users may have experienced delays of up to two hours when sending email from on-premises to Exchange Online.
Severity
Start time 2021-08-31T14:40:00Z
End time 2021-08-31T18:45:00Z
Last update 2021-09-07T18:27:36.193Z
More info
Root cause To protect the overall mail flow service, Exchange Online implements outbound traffic rules that include throttling limits on emails. Some of these rules are designed to target trial memberships sending large volumes of email. The rules are specifically designed to exclude organizations that send a high volume of emails daily. Anti-Spam features use datasets from other upstream Microsoft services to gather organizational licensing information, which filter into some outbound traffic rules. An upstream service made a change to a specific dataset, which caused data streams used by the Anti-Spam features to produce invalid results. This resulted in mail flow components not recognizing the exception organizations, and imposed the throttling limits. As a result, email messages sent from on-premises users to Exchange Online mailboxes were delayed and queued for delivery once the throttle limit had been reached. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. A post-incident report will be produced within five business days.; PublishedTime=2021-08-31T20:17:06.263Z

EX282101 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're reviewing support case information including examples of error messages to isolate potential sources of impact. Next update by: Tuesday, August 31, 2021, at 7:00 PMUTC; PublishedTime=2021-08-31T17:52:10.617Z

Status: Our review of the error messages identified some network connectivity errors occurring in parallel with access denied errors. We're continuing to investigate these errors to determine what is generating the errors and identify potential mitigation options. Next update by: Tuesday, August 31, 2021, at 9:00 PMUTC; PublishedTime=2021-08-31T18:26:02.94Z

Final status: We've determined that an outbound traffic rule intended to exclude high volume senders from traffic management activities was inadvertently overwritten. As a result, email messages sent from on-premises users to Exchange Online users were throttled and queued for delivery. We've reverted the rule and confirmed with affected users that email is no longer queuing and queued email messages have been delivered. Next update by: Tuesday, August 31, 2021, at 9:00 PMUTC; PublishedTime=2021-08-31T18:26:02.94Z

--------------------------------------------------------------------------------------------------
Title Some users may have been unable to access the Yammer service
ID YA281987
Service Yammer Enterprise
Feature Yammer Components
Status Service restored
Impact Users may have been unable to access the Yammer service.
Severity
Start time 2021-08-31T06:04:00Z
End time 2021-08-31T06:42:00Z
Last update 2021-08-31T17:17:54.453Z
More info
Root cause A subset of database infrastructure was performing below acceptable performance thresholds. Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-08-31T07:33:02.503Z

YA281987 Details

Status: We're reviewing service monitoring telemetry to isolate the source of the issue and establish a fix. Next update by: Tuesday, August 31, 2021, at 9:00 AM UTC; PublishedTime=2021-08-31T06:50:15.25Z

Final status: We've identified a subset of database infrastructure was performing below acceptable performance thresholds. We've run commands which optimized traffic and performance on the affected infrastructure. We've confirmed the problem has been resolved after monitoring the impacted environment. Next update by: Tuesday, August 31, 2021, at 9:00 AM UTC; PublishedTime=2021-08-31T06:50:15.25Z

--------------------------------------------------------------------------------------------------
Title Admins may have seen no data for specific reports after August 31, 2021
ID MO282045
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Admins may have seen no data for specific reports after August 31, 2021.
Severity
Start time 2021-08-30T19:00:00Z
End time 2021-09-02T10:00:00Z
Last update 2021-09-02T11:40:59.17Z
More info Impacted reports included: -OneDrive Activity Report. -OneDrive Site Usage Report. -SharePoint Activity Report. -SharePoint Site Usage Report. -ProplusUsage Report. -Forms Report. -FormPro Report. -LicenseActivity Report. -MultiGeo Report. -Power BI Report. Final status: We've confirmed by monitoring that all reports are now displaying accurate data and this issue has been resolved. Scope of impact: This issue may have impacted admins running any of the listed reports from August 31, 2021. Start time: Monday, August 30, 2021, at 7:00 PM UTC End time: Thursday, September 2, 2021, at 10:00 AM UTC Root cause: An upstream data issue associated with loading tenant profile and usage report data, provided unexpected results for specific reports. Next steps: - We're analyzing performance data and trends on the affected infrastructure to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-02T10:35:00.257Z
Root cause An upstream data issue associated with loading tenant profile and usage report data, provided unexpected results for specific reports. Next steps: - We're analyzing performance data and trends on the affected infrastructure to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-02T10:35:00.257Z

MO282045 Details

Status: We've identified an upstream data issue associated with loading tenant profile and usage report data. We're developing a fix, to remediate impact. Next update by: Thursday, September 2, 2021, at 9:00 AM UTC; PublishedTime=2021-08-31T12:17:15.727Z

Status: We've deployed the fix to the affected infrastructure and can confirm from monitoring that all reports, with the exception of the ProPlusUsage report, are now showing results correctly. We anticipate that the ProPlusUsage report will recover by Friday, September 3, 2021. Next update by: Friday, September 3, 2021, at 9:00 AM UTC; PublishedTime=2021-09-02T07:24:23.297Z

Final status: We've confirmed by monitoring that all reports are now displaying accurate data and this issue has been resolved. Next update by: Friday, September 3, 2021, at 9:00 AM UTC; PublishedTime=2021-09-02T07:24:23.297Z

--------------------------------------------------------------------------------------------------
Title Some users may have been unable to access the device management tab within the Microsoft Teams admin center
ID TM281722
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have been unable to access the device management tab within the Microsoft Teams admin center.
Severity
Start time 2021-08-30T08:56:00Z
End time 2021-08-30T10:59:00Z
Last update 2021-08-30T11:43:39.073Z
More info
Root cause A recent configuration change, which was part of a scheduled maintenance update, caused a section of database infrastructure to generate exceptions when processing admin center device requests. Next steps: - We're reviewing our standard service update procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-08-30T11:43:39.073Z

TM281722 Details

Status: We're reviewing service monitoring telemetry to isolate the source of the issue and establish a fix. Next update by: Monday, August 30, 2021, at 12:00 PM UTC; PublishedTime=2021-08-30T10:06:33.917Z

Final status: We've identified that a recent configuration change, which was part of a scheduled maintenance update, caused a section of database infrastructure to generate exceptions when processing admin center device requests. We've reverted the offending change and confirmed that impact is mitigated after monitoring the affected environment. Next update by: Monday, August 30, 2021, at 12:00 PM UTC; PublishedTime=2021-08-30T10:06:33.917Z

--------------------------------------------------------------------------------------------------
Title Some users are missing from various daily reports generated by admins from August 29, 2021 to October 6, 2021
ID MO291140
Service Microsoft 365 suite
Feature Administration
Status Restoring service
Impact Some users are missing from various daily reports generated by admins from August 29, 2021 to October 6, 2021.
Severity Sev2
Start time 2021-08-29T00:00:00Z
End time
Last update 2021-10-22T06:11:27.977Z
More info The affected reports are generated via the Microsoft 365 admin center, the Microsoft 365 Usage Analytics, the PowerBI Template App and other portals. The affected reports include, but are not limited to: - User Teams Activity - User Teams Activity Current status: We've verified that the data backfill for Exchange Online, SharePoint Online, and OneDrive for Business has finished. We're monitoring the backfill progress for the remaining services which includes Teams and PowerBI. Scope of impact: This issue could potentially affect any of your admins who are generating the affected reports. Start time: Sunday, August 29, 2021, at 12:00 AM'UTC Estimated time to resolve: We expect this issue to be completely resolved for all affected admins by Thursday, October 28, 2021, as the data is thoroughly backfilled. Root cause: An upstream data resource was experiencing issues extracting data. Next update by: Thursday, October 28, 2021, at 8:00 AMUTC; PublishedTime=2021-10-22T06:11:27.977Z
Root cause An upstream data resource was experiencing issues extracting data. Next update by: Thursday, October 28, 2021, at 8:00 AMUTC; PublishedTime=2021-10-22T06:11:27.977Z

MO291140 Details

Status: We've determined that an upstream data resource was experiencing issues extracting data. We've repaired the upstream data resources and current reports reflect the accurate data. Additionally, we're backfilling the data for the impacted time period. We expect this process to be complete by Thursday, October 28, 2021. Next update by: Friday, October 15, 2021, at 5:00 AM UTC; PublishedTime=2021-10-13T04:16:54.723Z

Status: We're continuing to monitor the backfill of data and are on track to have it complete by Thursday, October 28, 2021. Next update by: Wednesday, October 20, 2021, at 5:00 AMUTC; PublishedTime=2021-10-15T03:35:25.6Z

Status: We've confirmed that the data backfill for Exchange Online, SharePoint Online, and OneDrive for Business have completed and we're verifying the results to ensure the data is complete. The data backfill for the remaining services is still underway and we're monitoring the progress. We're still on track to complete all backfills by Thursday, October 28, 2021. Next update by: Friday, October 22, 2021, at 8:00 AM UTC; PublishedTime=2021-10-20T03:35:59.737Z

Status: We've verified that the data backfill for Exchange Online, SharePoint Online, and OneDrive for Business has finished. We're monitoring the backfill progress for the remaining services which includes Teams and PowerBI. Next update by: Thursday, October 28, 2021, at 8:00 AMUTC; PublishedTime=2021-10-22T06:11:27.977Z

--------------------------------------------------------------------------------------------------
Title Some admins are unable to access audit events for delegate access via the Security & Compliance Center (SCC).
ID EX290151
Service Exchange Online
Feature Management and Provisioning
Status Service restored
Impact Admins were unable to access audit events for delegate access via the SCC.
Severity Sev2
Start time 2021-08-28T00:00:00Z
End time 2021-10-11T23:03:00Z
Last update 2021-10-11T23:21:09.52Z
More info Impact was specific to some admins with configured delegated access permissions. Subscribers of the Management API may have been similarly impacted. Final status: We've developed and deployed a fix for this issue, and we've verified via internal testing that the fix has mitigated impact. Scope of impact: A subset of admins attempting to access audit events for delegate access via the SCC may have been impacted. Start time: Saturday, August 28, 2021, at 12:00 AM'UTC End time: Monday, October 11, 2021, at 11:03 PM'UTC Root cause: A recent service update designed to improve auditing diagnostics was preventing admins from accessing audit events for delegate access via the SCC. Next steps: - We're exploring options to catch similar potentially impacting updates before they deploy. This is the final update for the event.; PublishedTime=2021-10-11T23:21:09.52Z
Root cause A recent service update designed to improve auditing diagnostics was preventing admins from accessing audit events for delegate access via the SCC. Next steps: - We're exploring options to catch similar potentially impacting updates before they deploy. This is the final update for the event.; PublishedTime=2021-10-11T23:21:09.52Z

EX290151 Details

Status: We're collecting diagnostic data gathered from reproductions of the issue performed in our testing environment so we can better determine the underlying origin of impact as we develop our next troubleshooting steps. Next update by: Saturday, October 9, 2021, at 2:00 AMUTC; PublishedTime=2021-10-08T19:05:57.33Z

Status: We're reviewing the diagnostic data gathered from our internal reproductions of the issue to help us determine a root cause and formulate a mitigation plan. Next update by: Sunday, October 10, 2021, at 2:00 AMUTC; PublishedTime=2021-10-09T00:27:16.393Z

Status: We're continuing to review the internal reproduction of the issue to help us better understand the cause and determine our next steps. Next update by: Sunday, October 10, 2021, at 11:00 PMUTC; PublishedTime=2021-10-10T00:36:42.43Z

Status: Following our internal reproduction of the issue, we've identified a recent service update which we believe is preventing admins from accessing audit events for delegate access via the SCC. We're developing and internally validating a fix designed to revert the offending update and remediate impact. Next update by: Monday, October 11, 2021, at 6:00 PMUTC; PublishedTime=2021-10-10T21:27:39.967Z

Status: We've successfully reverted the offending update and are validating with test users to ensure this issue is fully resolved. Next update by: Tuesday, October 12, 2021, at 1:00 AMUTC; PublishedTime=2021-10-11T17:07:30.463Z

Final status: We've developed and deployed a fix for this issue, and we've verified via internal testing that the fix has mitigated impact. Next update by: Tuesday, October 12, 2021, at 1:00 AMUTC; PublishedTime=2021-10-11T17:07:30.463Z

--------------------------------------------------------------------------------------------------
Title Some admins may have been unable to access the Microsoft Cloud App Security portal
ID CS281212
Service Cloud App Security
Feature Microsoft Cloud App Security
Status Service restored
Impact Admins may have been unable to access the Microsoft Cloud App Security portal (MCAS).
Severity
Start time 2021-08-27T09:49:00Z
End time 2021-08-27T12:00:00Z
Last update 2021-08-27T12:57:56.29Z
More info
Root cause A Secure Sockets Layer (SSL) configuration issue may have prevented admins from accessing the MCAS portal. Next steps: - We're reviewing our Secure Sockets Layer (SSL) configuration procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-08-27T12:57:56.29Z

CS281212 Details

Status: We're identified that admins may be unable to access the MCAS portal. We're analyzing system telemetry, to determine the next troubleshooting steps. Next update by: Friday, August 27, 2021, at 1:00 PM UTC; PublishedTime=2021-08-27T10:56:45.247Z

Final status: We've determined that a Secure Sockets Layer (SSL) configuration issue may have prevented admins from accessing the MCAS portal. We've corrected the configuration issue and confirmed that impact has been remediated. Next update by: Friday, August 27, 2021, at 1:00 PM UTC; PublishedTime=2021-08-27T10:56:45.247Z

--------------------------------------------------------------------------------------------------
Title A limited amount of users may experience search failures when using Outlook Win32 version
ID EX283383
Service Exchange Online
Feature Backend Data Processing
Status Service restored
Impact A limited amount of users may experience search failures when using Outlook Win32 version.
Severity
Start time 2021-08-27T09:00:00Z
End time 2021-09-07T16:57:00Z
Last update 2021-09-08T00:44:21.19Z
More info
Root cause We recently enabled a service efficiency improvement that reduced CPU and memory usage, which caused increased rates of query handling failures. Next steps: - To help prevent similar impact in the future, we're further reviewing why the change caused increased rates of query handling failures and how we can improve our validation procedures to catch such issues before deployment. This is the final update for the event.; PublishedTime=2021-09-08T00:44:21.19Z

EX283383 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We've discovered that after enabling a service efficiency improvement that reduces CPU and memory usage, we're seeing increased rates of query handling failures. We're now in the process of reverting this change, which should mitigate impact. Next update by: Wednesday, September 8, 2021, at 2:00 AMUTC; PublishedTime=2021-09-07T01:47:28.57Z

Final status: We've finished reverting the service efficiency change and confirmed via telemetry that service is restored. Next update by: Wednesday, September 8, 2021, at 2:00 AMUTC; PublishedTime=2021-09-07T01:47:28.57Z

--------------------------------------------------------------------------------------------------
Title Users' all day events may show as spanning two days instead of only one in Outlook on the web calendar
ID EX281692
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users' all day events showed as spanning two days instead of one in Outlook on the web calendar's month and week views.
Severity
Start time 2021-08-27T05:04:00Z
End time 2021-08-30T22:00:00Z
Last update 2021-08-31T03:28:47.803Z
More info Users with access to the Outlook desktop client could've used that connection method to bypass impact. Final status: We've finished deploying our code fix and confirmed with some affected users that service is restored. Scope of impact: Any user viewing all day events in Outlook on the web calendar may have been impacted. Start time: Friday, August 27, 2021, at 5:04 AM'UTC End time: Monday, August 30, 2021, at 10:00 PM'UTC Root cause: A recent update to optimize how specific event dates are displayed within the Outlook calendar inadvertently introduced a code problem that caused impact. Next steps: - To help prevent similar impact in the future, we're further reviewing why the code problem occurred and how we can improve our validation procedures to catch such issues before updates are deployed. This is the final update for the event.; PublishedTime=2021-08-31T03:28:47.803Z
Root cause A recent update to optimize how specific event dates are displayed within the Outlook calendar inadvertently introduced a code problem that caused impact. Next steps: - To help prevent similar impact in the future, we're further reviewing why the code problem occurred and how we can improve our validation procedures to catch such issues before updates are deployed. This is the final update for the event.; PublishedTime=2021-08-31T03:28:47.803Z

EX281692 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're analyzing network trace logs provided by some affected users to determine the source of the issue. Next update by: Monday, August 30, 2021, at 5:00 AMUTC; PublishedTime=2021-08-30T03:58:47.07Z

Status: We're reproducing the impact in a test account and analyzing Hypertext Transfer Protocol (HTTP) traces provided by some affected users to isolate the problem. Next update by: Monday, August 30, 2021, at 6:30 AMUTC; PublishedTime=2021-08-30T04:58:44.937Z

Status: We've found that this impact doesn't occur in Outlook on the web build 20210823010 and later. We're analyzing our reproduction of the issue and reaching out to some affected users to determine if clearing the browser cache and refreshing Outlook on the web provides affected users with the most recent build and mitigates impact. Next update by: Monday, August 30, 2021, at 8:30 AMUTC; PublishedTime=2021-08-30T06:36:40.147Z

Status: We've identified an issue within older build versions of Outlook on the web. We're deploying a fix to remediate the issue. Next update by: Monday, August 30, 2021, at 7:00 PM UTC; PublishedTime=2021-08-30T08:29:17.627Z

Status: We've confirmed that a recent update to optimize how specific event dates are displayed within the Outlook calendar inadvertently introduced code problem that's causing this issue. We're developing a fix to resolve this problem, which we expect to begin deploying to the affected environments in the next few hours. We expect we'll have a better estimation on the completion of the fix deployment by our next update. Next update by: Monday, August 30, 2021, at 10:30 PMUTC; PublishedTime=2021-08-30T18:00:05.543Z

Status: Our code fix has been successfully validated and is now being deployed. We'll provide an estimated time for completion by our next update time. Next update by: Tuesday, August 31, 2021, at 5:00 AMUTC; PublishedTime=2021-08-30T21:25:23.853Z

Final status: We've finished deploying our code fix and confirmed with some affected users that service is restored. Next update by: Tuesday, August 31, 2021, at 5:00 AMUTC; PublishedTime=2021-08-30T21:25:23.853Z

--------------------------------------------------------------------------------------------------
Title Users in New Zealand unable to access Microsoft 365 services
ID MO281120
Service Microsoft 365 suite
Feature Portal
Status Post-incident report published
Impact Affected users were unable to access multiple Microsoft 365 services.
Severity
Start time 2021-08-26T20:25:00Z
End time 2021-08-26T22:45:00Z
Last update 2021-09-03T05:53:41.593Z
More info Initial reports indicated that users were unable to access Microsoft Teams, Outlook, and SharePoint Online, though additional services may have also been impacted. Final status: We've determined that a network routing issue between a third-party ISP and our service inadvertently caused users to be unable to access various Microsoft 365 services. The issue was resolved by correcting the connection routing configuration between the ISP and Microsoft services. Scope of impact: This issue affected users located in New Zealand. Start time: Thursday, August 26, 2021, at 8:25 PM UTC End time: Thursday, August 26, 2021, at 10:45 PM UTC Preliminary root cause: A network routing issue between a third-party ISP and our services caused users to be unable to access multiple Microsoft 365 services. Further details can be found in the Post Incident Review document. Next steps: - We're partnering with the third-party ISP to understand the underlying root cause of the issue between the affected parties. - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. A post-incident report will be published within five business days.; PublishedTime=2021-08-26T23:23:45.047Z
Root cause

MO281120 Details

Status: We've received reports of issues affecting access to multiple Microsoft services from users in New Zealand. We're investigating the issue and will provide an update as soon as possible.

Status: We're continuing to investigate reports of impact originating from New Zealand-based users. We're focused on verifying the health of network components in our managed environment and other network pathways used to serve our users.

Final status: We've determined that a network routing issue between a third-party ISP and our service inadvertently caused users to be unable to access various Microsoft 365 services. The issue was resolved by correcting the connection routing configuration between the ISP and Microsoft services.

--------------------------------------------------------------------------------------------------
Title Unable to access the admin center.
ID MO281098
Service Microsoft 365 suite
Feature Portal
Status Post-incident report published
Impact Affected admins and users may have been unable to access the admin center.
Severity
Start time 2021-08-26T18:30:00Z
End time 2021-08-26T20:36:00Z
Last update 2021-09-03T11:15:02.417Z
More info As a reminder, customers could go to status.office.com for updates on major issues if there were problems accessing the portal or the SHD. Final status: We've remediated impact by routing traffic through healthy channels and performing targeted restarts on affected infrastructure. We confirmed that admin and user access to the admin center had been restored. Scope of impact: Any user routed through the affected infrastructure in North America, attempting to access the admin center or view information within the admin center, could have been affected by this event. Start time: Thursday, August 26, 2021, at 6:30 PM UTC End time: Thursday, August 26, 2021, at 8:36 PM UTC Preliminary root cause: A recent deployment caused an unexpected increase in traffic load and CPU usage for portions of infrastructure responsible for access to the admin center, though the underlying cause remains under investigation. Next Steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document.; PublishedTime=2021-08-26T21:12:59.847Z
Root cause

MO281098 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-08-26T18:45:48.727

Status: We've received reports of an issue affecting access to the admin center. We're investigating to determine the cause of the issue and are working to identify remediation actions. We'll provide an estimated time to restoration as soon as one becomes available. Next update by: Thursday, August 26, 2021, at 8:30 PM UTC; PublishedTime=2021-08-26T19:11:13.25Z

Status: We've identified an ongoing deployment that may be contributing to high CPU usage for a portion of infrastructure responsible for admin center access. We're working to stop the deployment in an effort to remediate impact while we continue to investigate the underlying cause of the issue. Additionally, we've routed all affected traffic through healthy channels and we're working to restart the affected infrastructure as needed. We'll provide an estimated time to resolution as soon as one becomes available. Next update by: Thursday, August 26, 2021, at 9:30 PM UTC; PublishedTime=2021-08-26T20:00:56.183Z

Final status: We've remediated impact by routing traffic through healthy channels and performing targeted restarts on affected infrastructure. We confirmed that admin and user access to the admin center had been restored. Next update by: Thursday, August 26, 2021, at 9:30 PM UTC; PublishedTime=2021-08-26T20:00:56.183Z

--------------------------------------------------------------------------------------------------
Title Users' OneDrive for Business storage limits are lower than expected
ID OD280960
Service OneDrive for Business
Feature OneDrive for Business
Status Post-incident report published
Impact Users' OneDrive for Business storage limits were lower than expected.
Severity
Start time 2021-08-25T02:31:00Z
End time 2021-09-10T04:30:00Z
Last update 2021-09-17T04:21:27.227Z
More info For users that were impacted, any quota which exceeded the 1TB storage, users may have only been able to see their content in read-only mode. Whilst in read-only mode, users were able to access their content in OneDrive for Business, but were unable to make changes. Instructions to manually set quotas could have been found here: https://docs.microsoft.com/en-us/onedrive/change-user-storage Final status: We've completed our mitigation efforts and confirmed that the issue has been successfully resolved. We appreciate your patience during the duration of this incident. Scope of impact: This issue affected a large subset of users worldwide, who have a storage quota that exceeds 1TB. Start time: Wednesday, August 25, 2021, at 2:31 AM'UTC End time: Friday, September 10, 2021, at 4:30 AM'UTC Root Cause: As part of feature improvements intended to address inefficient logic managing OneDrive for Business quotas, a code change was introduced. On August 25, 2021 a previously unidentified exception occurred which correlated with the recent change. This exception gave the impression that a user had no licenses assigned. The quota management component would then automatically assign a 1TB quota limit, which is the default value if a user had no explicit license enabled. This unexpected change then caused impact as many organizations set larger quote limits by default, or give specific users larger limits than the standard 1TB. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. A post-incident report has been published; PublishedTime=2021-09-10T04:44:20.7Z
Root cause An exception is not recognizing user licenses and reverting the storage quota limit to the default settings of 1TB. Next update by: Thursday, August 26, 2021, at 5:00 PM UTC; PublishedTime=2021-08-26T14:54:53.117Z

OD280960 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're reviewing available system data to determine possible underlying causes of this issue. Next update by: Thursday, August 26, 2021, at 7:30 AMUTC; PublishedTime=2021-08-26T06:16:16.693Z

Status: We're continuing to review our available system data, whilst additionally working with your representatives to gather diagnostic logs to isolate the root cause of impact. Next update by: Thursday, August 26, 2021, at 9:30 AM UTC; PublishedTime=2021-08-26T07:07:37.187Z

Status: We've determined from the gathered logs that an exception is not recognizing user licenses and reverting the storage quota limit to the default settings of 1TB. We're changing the way the quota is calculated, which should mitigate the issue. Next update by: Thursday, August 26, 2021, at 11:30 AM UTC; PublishedTime=2021-08-26T09:05:57.047Z

Status: We're preparing to revert recent changes made to the quota settings to resolve the issue. Next update by: Thursday, August 26, 2021, at 1:00 PM UTC; PublishedTime=2021-08-26T11:03:35.96Z

Status: We've identified a fix that involves reverting the recent changes in addition to re-aligning user quotas to what they are licensed for and reverting user's quota to previously set state. Next update by: Thursday, August 26, 2021, at 3:00 PM UTC; PublishedTime=2021-08-26T12:55:21.523Z

Status: We're gathering the list of affected users in order to apply the mitigation steps. Meanwhile, we're validating our mitigation steps with a sample of users before applying them to all of the affected infrastructure. Next update by: Thursday, August 26, 2021, at 5:00 PM UTC; PublishedTime=2021-08-26T14:54:53.117Z

Status: We've validated that the mitigation steps successfully resolve this issue and we'll begin deployment once the full scope of impacted users has been determined. Our effort to determine the full scope of impacted users is ongoing, and due to the complex nature of this activity, make take an extended amount of time to complete. Next update by: Thursday, August 26, 2021, at 7:00 PM UTC; PublishedTime=2021-08-26T16:00:33.783Z

Status: We've initiated the deployment of the fix in a limited environment to ensure it resolves impact as expected. Once confirmed, we'll proceed with expanding the deployment to all affected users. By design, this fix is expected to restore all affected users to the default storage quota of five terabytes. Next update by: Thursday, August 26, 2021, at 9:00 PM UTC; PublishedTime=2021-08-26T18:57:45.287Z

Status: The targeted deployment remediated impact as expected, and we're in the process of expanding the deployment to all affected users. We'll continue to monitor the environment while the fix saturates to ensure it completes successfully. Next update by: Thursday, August 26, 2021, at 11:00 PM UTC; PublishedTime=2021-08-26T21:00:32.513Z

Status: The deployment has completed successfully. Additionally, we've identified that the fix will take approximately 24 hours to take full effect. We'll continue to closely monitor the environment during this timeframe to ensure the service remains in a healthy state, and that the fix progresses as intended. Next update by: Friday, August 27, 2021, at 11:00 PM UTC; PublishedTime=2021-08-26T22:11:39.617Z

Status: Our telemetry indicates that the fix has taken full effect for users that have default and or non-custom quotas, though some users may see a delay in mitigation until they navigate to their OneDrive for Business account via the web, which will initiate a backend refresh. Additionally, we have received some reports that this issue is not resolved for users with custom quotas. Further investigation is required to confirm this behavior. Next update by: Saturday, August 28, 2021, at 5:00 AM UTC; PublishedTime=2021-08-27T23:01:17.93Z

Status: Our data continues to indicate that the fix is progressing and users are experiencing incremental resolution if they have a default or non-custom quota set. We're expecting some users may see a delay and if necessary can expedite this process by navigating to their OneDrive for Business account via the web. Additionally, we've determined that some users who have a custom quota set were impacted by this issue. Our initial investigation did not properly account for a specific variable and we have adjusted our telemetry to ensure we have identified all impacting scenarios. We're working on a fix to address users in this state and will provide a recovery timeline as soon as possible. We have provided an interim solution that may be viable for some admins that can be found in the 'More Info' section above. We understand how impacting this issue has on your organization and we want to assure you that we are treating the issue with the utmost priority. We appreciate your patience and will continue to update the Service Health Dashboard daily. Next update by: Saturday, August 28, 2021, at 11:00 PM UTC; PublishedTime=2021-08-28T04:51:48.167Z

Status: Our telemetry indicates the fix continues to progress for users with a default or non-custom quota set. As we've determined that some users who have a custom quota set were impacted by this issue, we're in the process of developing and testing a separate mitigation activity that will temporarily increase the quota to a value greater than 1TB, and then subsequently apply the correct value. Similar to the fix for non-custom quotas, this mitigation activity will require users to initiate a refresh activity such as navigating to the OneDrive for Business account via the web. We expect development of this mitigation to be completed over the weekend, and for deployment to begin early next week. In tandem, and to accelerate this process for both users with default quotas and custom quotas, we're developing a mitigation activity to initiate an automated refresh, which will automatically revert the quota to the correct value. We understand how impacting this issue has been on your organization and we want to assure you that we are treating the issue with the utmost priority. We appreciate your patience and will continue to update the Service Health Dashboard daily. Next update by: Monday, August 30, 2021, at 12:00 AM UTC; PublishedTime=2021-08-28T22:29:01.79Z

Status: We've completed development of the mitigation for users with custom quotas and are completing final validations prior to deployment. Additionally, our effort to develop a mitigation activity to initiate an automated refresh is ongoing. Our telemetry indicates the initial mitigation activity continues to provide relief for users with a default or non-custom quota set. We understand how impacting this issue has been on your organization and we want to assure you that we are treating the issue with the utmost priority. We appreciate your patience and will continue to update the Service Health Dashboard daily. Next update by: Monday, August 30, 2021, at 11:00 PM UTC; PublishedTime=2021-08-29T22:40:38.703Z

Status: In order to ensure we're following safe deployment practices, we're in the process of completing our final validations within our internal environments prior to initiating a targeted release. Our development of a mitigation activity to initiate an automated refresh is ongoing. Our monitoring indicates users with a default or non-custom quota set continue to see relief. We understand how impacting this issue has on your organization and we want to assure you that we are treating the issue with the utmost priority. We appreciate your patience and will continue to update the Service Health Dashboard daily. Next update by: Tuesday, August 31, 2021, at 11:00 PM UTC; PublishedTime=2021-08-30T22:36:39.227Z

Status: We've completed the validation process and are deploying our solution for users with applied custom quotas. We'll continue to monitor the deployment as it progresses safely through our change management systems. For users with a default or non-custom quota set, our telemetry indicates that users are continuing to see relief. We understand how impacting this issue has on your organization and we want to assure you that we are treating the issue with the utmost priority. We appreciate your patience and will continue to update the Service Health Dashboard daily. Next update by: Wednesday, September 1, 2021, at 11:00 PM UTC; PublishedTime=2021-08-31T21:58:25.197Z

Status: The deployment continues to progress and users with custom quotas will experience relief as the fix saturates. As part of our overall mitigation strategy, we intend to trigger an automatic refresh of all affected OneDrive for Business clients; however, users with non-custom quotas can manually trigger a refresh through various actions within OneDrive. For example, accessing OneDrive for Business account from the web or selecting a file. We understand how impacting this issue has on your organization and we want to assure you that we are treating the issue with the utmost priority. We appreciate your patience and will continue to update the Service Health Dashboard daily. Next update by: Thursday, September 2, 2021, at 11:00 PM UTC; PublishedTime=2021-09-01T23:01:45.957Z

Status: The deployment is nearing completion, and we expect to begin post deployment validations by the end of day, September 2, 2021. Once the fix has taken full effect, both users with custom and non-custom quotas can manually trigger a refresh through various actions within OneDrive to resolve this issue. For example, by accessing their OneDrive for Business account from the web or selecting a file. Our automatic refresh activity is on schedule to be completed and initiated by end of day, September 3, 2021. We understand how impacting this issue has on your organization and we want to assure you that we are treating the issue with the utmost priority. We appreciate your patience and will continue to update the Service Health Dashboard daily. Next update by: Friday, September 3, 2021, at 11:00 PM UTC; PublishedTime=2021-09-02T22:20:21.937Z

Status: Our telemetry indicates that the fix is functioning as expected post deployment; however, we identified a subset of infrastructure that is taking longer than expected to update. We'll continue to monitor the status of the deployment to ensure it completes successfully. Our automatic refresh activity has been initiated and will take effect over the following days. We understand how impacting this issue has been on your organization and we want to assure you that we are treating the issue with the utmost priority. Next update by: Sunday, September 5, 2021, at 11:00 PM UTC; PublishedTime=2021-09-03T22:03:29.607Z

Status: We successfully initiated the manual refresh activity and have validated that approximately 20 percent of the remaining non-custom accounts have been addressed. In parallel, we're continuing to deploy the fix that will address the custom quota impact. Once complete, we'll initiate a manual refresh for the accounts with the custom quotas. We understand how impacting this issue has been on your organization and we want to assure you that we are treating the issue with the utmost priority. Next update by: Monday, September 6, 2021, at 11:00 PM UTC; PublishedTime=2021-09-05T17:38:10.983Z

Status: We continue to perform the manual refresh activity, and approximately 40 percent of the remaining non-custom accounts have been completed. Additionally, the fix that will address the custom quota impact continues deployment and has already been deployed to the majority of the service. We've activated the fix on 50 percent of the infrastructure that the fix has been deployed to, and expect the fix to be activated on 100 percent of the components within the next six hours. We understand how impacting this issue has been on your organization and we want to assure you that we are treating the issue with the utmost priority. Next update by: Tuesday, September 7, 2021, at 11:00 PM UTC; PublishedTime=2021-09-06T17:49:45.337Z

Status: The fix has been activated on 100 percent of the affected components. At this time, users with non-custom and custom quotas can manually trigger a refresh through various actions within OneDrive to remediate any lingering impact. For example, accessing a OneDrive for Business account from the web or selecting a file within OneDrive will trigger such a refresh. Additionally, we are entering an extended period of monitoring to ensure that users continue to experience resolution. We're continuing to complete refresh activities designed to automatically trigger a refresh without manual intervention. We understand how impacting this issue has been on your organization and we want to assure you that we are treating the issue with the utmost priority. Next update by: Wednesday, September 8, 2021, at 11:00 PM UTC; PublishedTime=2021-09-07T18:47:34.65Z

Status: Our telemetry data indicates that this issue has been resolved for the majority of users, and the subset of users still impacted are continuing to experience incremental relief. We'll continue to monitor as we complete our refresh activities designed to resolve this issue without manual intervention on the part of individual users. We understand how impacting this issue has been on your organization and we want to assure you that we are treating the issue with the utmost priority. Next update by: Thursday, September 9, 2021, at 11:00 PM UTC; PublishedTime=2021-09-08T22:05:40.98Z

Status: We expect the remaining impacted users to become fully resolved within the next 48-hours. We'll continue to monitor progress towards full remediation for the isolated subset of users who are continuing to experience incremental relief. We understand how impacting this issue has been on your organization and we want to assure you that we are treating the issue with the utmost priority. Next update by: Friday, September 10, 2021, at 11:00 PM UTC; PublishedTime=2021-09-09T22:04:39.89Z

Status: We expect the remaining impacted users to become fully resolved within the next 48-hours. We'll continue to monitor progress towards full remediation for the isolated subset of users who are continuing to experience incremental relief. We understand how impacting this issue has been on your organization and we want to assure you that we are treating the issue with the utmost priority. Next update by: Friday, September 10, 2021, at 11:00 PM UTC; PublishedTime=2021-09-09T22:10:44.52Z

Final status: We've completed our mitigation efforts and confirmed that the issue has been successfully resolved. We appreciate your patience during the duration of this incident. Next update by: Friday, September 10, 2021, at 11:00 PM UTC; PublishedTime=2021-09-09T22:10:44.52Z

--------------------------------------------------------------------------------------------------
Title Users that utilize the Microsoft Graph service to query Azure Directory Objects may have had intermittent service unavai
ID MO280680
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Users that utilize the Microsoft Graph service to query Azure Directory Objects may have had intermittent service unavai
Severity
Start time 2021-08-25T01:15:00Z
End time 2021-08-25T07:15:00Z
Last update 2021-08-25T09:02:08.927Z
More info The impacted queries are listed below - https://docs.microsoft.com/en-us/graph/aad-advanced-queries Final status: We've successfully restarted the affected infrastructure along with performing a configuration change to optimize traffic. We've confirmed that impact as been mitigated after monitoring the affected environment. Scope of impact: Your organization was affected by this event, and the issue intermittently impacted users utilizing Microsoft Graph to query the listed Azure Active Directory (Azure AD) objects. Start time: Wednesday, August 25, 2021, at 1:15 AM UTC End time: Wednesday, August 25, 2021, at 7:15 AM UTC Root cause: An unexpected spike in network traffic resulted in issues with Microsoft Graph queries. Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-08-25T08:31:18.153Z
Root cause An unexpected spike in network traffic resulted in issues with Microsoft Graph queries. Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-08-25T08:31:18.153Z

MO280680 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're checking system diagnostic logs to identify the cause of impact. Next update by: Wednesday, August 25, 2021, at 7:00 AMUTC; PublishedTime=2021-08-25T04:54:34.443Z

Status: We've discovered that an unexpected spike in network traffic, is resulting in Microsoft Graph query issues. We're restarting a portion of the affected infrastructure, this will provide mitigation. We will have more information on a mitigation timeline at our next scheduled update. Next update by: Wednesday, August 25, 2021, at 8:30 AMUTC; PublishedTime=2021-08-25T06:26:39.547Z

Final status: We've successfully restarted the affected infrastructure along with performing a configuration change to optimize traffic. We've confirmed that impact as been mitigated after monitoring the affected environment. Next update by: Wednesday, August 25, 2021, at 8:30 AMUTC; PublishedTime=2021-08-25T06:26:39.547Z

--------------------------------------------------------------------------------------------------
Title Users are unable to submit elevated access requests via the Exchange Management Shell
ID EX281865
Service Exchange Online
Feature Management and Provisioning
Status Service restored
Impact Users were unable to submit elevated access requests via the Exchange Management Shell.
Severity
Start time 2021-08-24T16:18:00Z
End time 2021-08-30T20:00:00Z
Last update 2021-08-31T17:17:57.273Z
More info Impact was limited to the New-ElevatedAccessRequest, and users attempting to submit these requests were receiving timeout errors. Final status: While working to merge and deploy the updated build to gather additional logs for the investigation, we've confirmed that impact was caused by a recent authentication component that wasn't properly linked, resulting in impact. We've linked the component and confirmed mitigation via telemetry. Scope of impact: This issue may have affected any user attempting to submit a new elevated access request. Start time: Tuesday, August 24, 2021, at 4:18 PM'UTC End time: Monday, August 30, 2021, at 8:00 PM'UTC Root cause: A recent authentication update contained a component that wasn't properly linked, resulting in impact. Next steps: - We're reviewing our authentication component update procedures to better identify similar linking issues to prevent future occurrences. This is the final update for the event.; PublishedTime=2021-08-31T16:55:32.57Z
Root cause A recent authentication update contained a component that wasn't properly linked, resulting in impact. Next steps: - We're reviewing our authentication component update procedures to better identify similar linking issues to prevent future occurrences. This is the final update for the event.; PublishedTime=2021-08-31T16:55:32.57Z

EX281865 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes. Next update by: Tuesday, August 31, 2021, at 1:30 AMUTC; PublishedTime=2021-08-30T20:58:33.857Z

Status: We've confirmed that timeouts are occurring when users submit new elevated access requests via the Exchange Management Shell, and that calls from the Privileged Access Management (PAM) service to the Active Directory Authentication Library (ADAL) are failing to retrieve the expected access tokens as a result. We're developing a code change to apply to the affected environment to provide additional logging information from the affected infrastructure for additional insight into the errors occurring and to isolate the source of the timeouts to assist in formulating potential mitigation options. Next update by: Tuesday, August 31, 2021, at 1:30 AMUTC; PublishedTime=2021-08-30T21:16:49.387Z

Status: While developing our code change to enable additional logging on the affected infrastructure, we've determined that an official build will be needed in order to implement the logging as expected. We're developing an official build that will facilitate the code change deployment to continue our investigation into the errors occurring. Next update by: Tuesday, August 31, 2021, at 6:00 PMUTC; PublishedTime=2021-08-31T00:38:21.35Z

Final status: While working to merge and deploy the updated build to gather additional logs for the investigation, we've confirmed that impact was caused by a recent authentication component that wasn't properly linked, resulting in impact. We've linked the component and confirmed mitigation via telemetry. Next update by: Tuesday, August 31, 2021, at 6:00 PMUTC; PublishedTime=2021-08-31T00:38:21.35Z

--------------------------------------------------------------------------------------------------
Title Users were unable to add shared mailboxes via the Outlook mobile app
ID EX284109
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users were unable to add shared mailboxes via the Outlook mobile app.
Severity Sev2
Start time 2021-08-24T07:00:00Z
End time 2021-10-08T13:30:00Z
Last update 2021-10-08T13:57:14.2Z
More info Outlook desktop and Outlook on the web were unaffected. Users would've encountered the following error when attempting to add shared mailboxes via the Outlook mobile app: "Authentication Failed: That didn't work. You may have not have the right permissions to add this mailbox or the mailbox doesn't exist." Final status: The fix has now successfully completed and the issue has been mitigated. Scope of impact: Any user attempting to add shared mailboxes via the Outlook mobile app may have been impacted. Start time: Tuesday, August 24, 2021, at 7:00 AM UTC End time: Friday, October 8, 2021, at 1:30 PM UTC Root cause: A recent update intended to optimize the error code for disabled accounts incidentally introduced incorrect code logic and caused impact. Next steps: - We're reviewing our update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-08T13:57:14.2Z
Root cause A recent update intended to optimize the error code for disabled accounts incidentally introduced incorrect code logic and caused impact. Next steps: - We're reviewing our update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-08T13:57:14.2Z

EX284109 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're reviewing recent service updates to isolate the source of the problem and determine the most expedient mitigation approach. Next update by: Thursday, September 9, 2021, at 6:30 PMUTC; PublishedTime=2021-09-09T17:01:16.55Z

Status: We've identified a recent update intended to optimize the error code for disabled accounts which incidentally introduced an incorrect logic and caused impact. We're reverting this change across all affected environments which we anticipate will resolve the issue by the next scheduled update. Next update by: Thursday, September 9, 2021, at 8:30 PMUTC; PublishedTime=2021-09-09T17:46:03.09Z

Status: We've determined that the reversion of the offending change across the affected environments requires additional validation and processing time, and we now anticipate resolution by Friday, September 10, 2021. Next update by: Friday, September 10, 2021, at 8:30 PMUTC; PublishedTime=2021-09-09T19:26:55.623Z

Status: The reversion of the offending change has completed the validation process, however this change will take some time to saturate across all affected environments. Users will incrementally experience mitigation as the saturation continues, and we expect completion for all users by Monday, September 13, 2021. Next update by: Monday, September 13, 2021, at 10:00 PMUTC; PublishedTime=2021-09-10T18:14:14.013Z

Status: We've confirmed with affected users that the fix is incrementally remediating impact. Our fix deployment timeline has been slightly extended, and we're now expecting for the fix deployment to be complete and for impact to be resolved by Tuesday, September 14, 2021, at 10:30 PM UTC. Next update by: Tuesday, September 14, 2021, at 10:30 PMUTC; PublishedTime=2021-09-13T20:29:11.52Z

Status: We're monitoring our fix deployment which continues to progress slower than earlier anticipated. The fix has now saturated approximately 83 percent of the affected environment, and based on current deployment projections we're expecting for the deployment to have completed by Wednesday, September 15, 2021. Next update by: Wednesday, September 15, 2021, at 11:30 PMUTC; PublishedTime=2021-09-14T20:15:22.157Z

Status: We've determined that the progression of our deployment is taking more time than estimated. We're expecting to be able to provide an updated estimate for the fix deployment timeline and the resolution of impact by our next scheduled update. Next update by: Friday, September 17, 2021, at 11:30 PM UTC; PublishedTime=2021-09-15T21:51:20.43Z

Status: While telemetry indicates that this issue is likely resolved for most affected users, we'll continue monitoring the final stages of the deployment over the weekend to ensure it completes as expected to resolve the issue for all affected users. Next update by: Monday, September 20, 2021, at 11:30 PMUTC; PublishedTime=2021-09-17T22:28:23.663Z

Status: Our internal telemetry validates that the issue has been resolved for a majority of affected users. The fix deployment is 98 percent saturated, and we're continuing to monitor the final aspects of the deployment to ensure it completes and ensure that impact has been completely resolved. Next update by: Thursday, September 23, 2021, at 11:30 PMUTC; PublishedTime=2021-09-20T16:35:42.503Z

Status: While impact should be resolved for most affected users, our telemetry indicates that the fix deployment is still in its final stages. We expect it to complete deployment and resolve the issue for all affected users by our next scheduled update. Next update by: Monday, September 27, 2021, at 11:30 PMUTC; PublishedTime=2021-09-23T15:17:31.293Z

Status: We're continuing to monitor the deployment of our fix to mitigate the issue for the remaining affected users, and while it has progressed, a small percentage of users haven't yet received the fix. We're continuing to monitor its progress as we work to establish an updated estimate for the remainder of deployment. Next update by: Tuesday, September 28, 2021, at 10:00 AMUTC; PublishedTime=2021-09-27T22:30:54.59Z

Status: The fix is approaching the final stages of deployment and we're continuing to monitor it as it progresses towards completion. Next update by: Wednesday, September 29, 2021, at 3:00 PM UTC; PublishedTime=2021-09-28T09:13:26.267Z

Status: The fix has reached the final stages of completion. We estimate that the fix will be fully deployed by Monday, October 4, 2021. Next update by: Monday, October 4, 2021, at 3:00 PM UTC; PublishedTime=2021-09-29T14:15:30.51Z

Status: The fix is taking longer than expected to complete. Our new estimate for resolution is Wednesday, October 6, 2021. Next update by: Wednesday, October 6, 2021, at 3:00 PM UTC; PublishedTime=2021-10-04T13:34:36.913Z

Status: The fix is close to completion and the error count has decreased to a minimal amount. We estimate that fix should complete by Friday, October 8, 2021. Next update by: Friday, October 8, 2021, at 3:00 PM UTC; PublishedTime=2021-10-06T13:43:56.177Z

Final status: The fix has now successfully completed and the issue has been mitigated. Next update by: Friday, October 8, 2021, at 3:00 PM UTC; PublishedTime=2021-10-06T13:43:56.177Z

--------------------------------------------------------------------------------------------------
Title Users were unable to add shared mailboxes via the Outlook mobile app
ID EX284109
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users were unable to add shared mailboxes via the Outlook mobile app.
Severity Sev2
Start time 2021-08-24T07:00:00Z
End time 2021-10-08T13:30:00Z
Last update 2021-10-08T13:57:14.2Z
More info Outlook desktop and Outlook on the web were unaffected. Users would've encountered the following error when attempting to add shared mailboxes via the Outlook mobile app: "Authentication Failed: That didn't work. You may have not have the right permissions to add this mailbox or the mailbox doesn't exist." Final status: The fix has now successfully completed and the issue has been mitigated. Scope of impact: Any user attempting to add shared mailboxes via the Outlook mobile app may have been impacted. Start time: Tuesday, August 24, 2021, at 7:00 AM UTC End time: Friday, October 8, 2021, at 1:30 PM UTC Root cause: A recent update intended to optimize the error code for disabled accounts incidentally introduced incorrect code logic and caused impact. Next steps: - We're reviewing our update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-08T13:57:14.2Z
Root cause A recent update intended to optimize the error code for disabled accounts incidentally introduced incorrect code logic and caused impact. Next steps: - We're reviewing our update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-10-08T13:57:14.2Z

EX284109 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're reviewing recent service updates to isolate the source of the problem and determine the most expedient mitigation approach. Next update by: Thursday, September 9, 2021, at 6:30 PMUTC; PublishedTime=2021-09-09T17:01:16.55Z

Status: We've identified a recent update intended to optimize the error code for disabled accounts which incidentally introduced an incorrect logic and caused impact. We're reverting this change across all affected environments which we anticipate will resolve the issue by the next scheduled update. Next update by: Thursday, September 9, 2021, at 8:30 PMUTC; PublishedTime=2021-09-09T17:46:03.09Z

Status: We've determined that the reversion of the offending change across the affected environments requires additional validation and processing time, and we now anticipate resolution by Friday, September 10, 2021. Next update by: Friday, September 10, 2021, at 8:30 PMUTC; PublishedTime=2021-09-09T19:26:55.623Z

Status: The reversion of the offending change has completed the validation process, however this change will take some time to saturate across all affected environments. Users will incrementally experience mitigation as the saturation continues, and we expect completion for all users by Monday, September 13, 2021. Next update by: Monday, September 13, 2021, at 10:00 PMUTC; PublishedTime=2021-09-10T18:14:14.013Z

Status: We've confirmed with affected users that the fix is incrementally remediating impact. Our fix deployment timeline has been slightly extended, and we're now expecting for the fix deployment to be complete and for impact to be resolved by Tuesday, September 14, 2021, at 10:30 PM UTC. Next update by: Tuesday, September 14, 2021, at 10:30 PMUTC; PublishedTime=2021-09-13T20:29:11.52Z

Status: We're monitoring our fix deployment which continues to progress slower than earlier anticipated. The fix has now saturated approximately 83 percent of the affected environment, and based on current deployment projections we're expecting for the deployment to have completed by Wednesday, September 15, 2021. Next update by: Wednesday, September 15, 2021, at 11:30 PMUTC; PublishedTime=2021-09-14T20:15:22.157Z

Status: We've determined that the progression of our deployment is taking more time than estimated. We're expecting to be able to provide an updated estimate for the fix deployment timeline and the resolution of impact by our next scheduled update. Next update by: Friday, September 17, 2021, at 11:30 PM UTC; PublishedTime=2021-09-15T21:51:20.43Z

Status: While telemetry indicates that this issue is likely resolved for most affected users, we'll continue monitoring the final stages of the deployment over the weekend to ensure it completes as expected to resolve the issue for all affected users. Next update by: Monday, September 20, 2021, at 11:30 PMUTC; PublishedTime=2021-09-17T22:28:23.663Z

Status: Our internal telemetry validates that the issue has been resolved for a majority of affected users. The fix deployment is 98 percent saturated, and we're continuing to monitor the final aspects of the deployment to ensure it completes and ensure that impact has been completely resolved. Next update by: Thursday, September 23, 2021, at 11:30 PMUTC; PublishedTime=2021-09-20T16:35:42.503Z

Status: While impact should be resolved for most affected users, our telemetry indicates that the fix deployment is still in its final stages. We expect it to complete deployment and resolve the issue for all affected users by our next scheduled update. Next update by: Monday, September 27, 2021, at 11:30 PMUTC; PublishedTime=2021-09-23T15:17:31.293Z

Status: We're continuing to monitor the deployment of our fix to mitigate the issue for the remaining affected users, and while it has progressed, a small percentage of users haven't yet received the fix. We're continuing to monitor its progress as we work to establish an updated estimate for the remainder of deployment. Next update by: Tuesday, September 28, 2021, at 10:00 AMUTC; PublishedTime=2021-09-27T22:30:54.59Z

Status: The fix is approaching the final stages of deployment and we're continuing to monitor it as it progresses towards completion. Next update by: Wednesday, September 29, 2021, at 3:00 PM UTC; PublishedTime=2021-09-28T09:13:26.267Z

Status: The fix has reached the final stages of completion. We estimate that the fix will be fully deployed by Monday, October 4, 2021. Next update by: Monday, October 4, 2021, at 3:00 PM UTC; PublishedTime=2021-09-29T14:15:30.51Z

Status: The fix is taking longer than expected to complete. Our new estimate for resolution is Wednesday, October 6, 2021. Next update by: Wednesday, October 6, 2021, at 3:00 PM UTC; PublishedTime=2021-10-04T13:34:36.913Z

Status: The fix is close to completion and the error count has decreased to a minimal amount. We estimate that fix should complete by Friday, October 8, 2021. Next update by: Friday, October 8, 2021, at 3:00 PM UTC; PublishedTime=2021-10-06T13:43:56.177Z

Final status: The fix has now successfully completed and the issue has been mitigated. Next update by: Friday, October 8, 2021, at 3:00 PM UTC; PublishedTime=2021-10-06T13:43:56.177Z

--------------------------------------------------------------------------------------------------
Title Some users may be unable to view or download transcripts via the Microsoft Teams desktop client
ID TM280181
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have been unable to view or download transcripts via the Microsoft Teams desktop client.
Severity
Start time 2021-08-23T13:00:00Z
End time 2021-08-24T05:30:00Z
Last update 2021-08-24T06:19:42.637Z
More info
Root cause A portion of Microsoft Teams infrastructure that facilitates viewing and downloading transcripts was experiencing issues. Next steps: - To help prevent similar impact in the future, we're further reviewing the underlying cause of the performance issues. This is the final update for the event.; PublishedTime=2021-08-24T06:08:10.393Z

TM280181 Details

Status: We're investigating an issue with Microsoft Teams desktop client in which some users in your organization are unable to view or download transcripts from meetings. To determine the underlying cause of the issue, we're analyzing diagnostic data from the networking path so we can develop a subsequent mitigation strategy moving forward. Next update by: Monday, August 23, 2021, at 7:30 PMUTC; PublishedTime=2021-08-23T17:40:34.337Z

Status: Our preliminary analysis of diagnostic data from the networking path suggests that impact may be occurring due to high latency in the calls made to a portion of infrastructure responsible for routing these requests. This in turn is resulting in timeout errors and subsequent retries, which are further obscuring the precise cause of impact while simultaneously increasing latency. We're developing a change which will limit the number of retries attempted for each failed request so we can make the system more resilient to transient latency conditions and synchronously formulate the most expedient option to remediate impact. Next update by: Tuesday, August 24, 2021, at 12:00 AMUTC; PublishedTime=2021-08-23T19:29:42.15Z

Status: We're continuing our development and testing of a change which will limit the number of retries attempted for each failed request to make the system more resilient to transient latency conditions. In parallel, we're making a change to collect additional diagnostics to further the root cause analysis for the latency issues encountered. Next update by: Tuesday, August 24, 2021, at 1:30 AMUTC; PublishedTime=2021-08-23T22:54:23.087Z

Status: We're deploying our change intended to limit the number of retries attempted for each failed request; and to collect additional diagnostics for root cause analysis to a portion of the affected infrastructure. Once completed, we'll monitor to confirm the fix is working as intended and continue deployment to additional environments. Next update by: Tuesday, August 24, 2021, at 5:00 AMUTC; PublishedTime=2021-08-24T00:04:32.443Z

Status: We're currently validating the fix in our testing environment. Once we confirm that this would mitigate the issue, we will deploy to the affected environments. Next update by: Tuesday, August 24, 2021, at 7:00 AMUTC; PublishedTime=2021-08-24T03:49:41.67Z

Final status: We've determined that a portion of Microsoft Teams infrastructure that facilitates viewing and downloading transcripts was experiencing issues. We've confirmed that the fix worked in our testing environment. We've deployed to all affected environments, and telemetry shows that service is restored. Next update by: Tuesday, August 24, 2021, at 7:00 AMUTC; PublishedTime=2021-08-24T03:49:41.67Z

--------------------------------------------------------------------------------------------------
Title Some users are missing some settings within Outlook on the web, and some admins can't access Exchange Online PowerShell
ID EX280152
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users were missing some settings within Outlook on the web, and admins were unable to access Exchange Online PowerShell.
Severity
Start time 2021-08-23T12:09:00Z
End time 2021-09-09T18:52:00Z
Last update 2021-09-09T19:23:51.53Z
More info Users reported that the missing settings included, but may have not been limited to, the following: Attachments Change Language Customize Actions Rules Signatures Sweep Impact was specific to users and admins configured with non-English languages. We had confirmed reports of impact from users and admins set to Lithuanian, Norwegian, and Swedish. Our internal testing indicated that affected users and admins could set their language to English to bypass impact; however, users and admins may have been unable to change their language back to their desired language choice until the issue was resolved. Final status: Our deployment of the fix has saturated throughout the majority of the affected environment, and we've determined via telemetry that this issue is resolved. Scope of impact: The impact was limited to users and admins configured with non-English languages, such as Lithuanian, Norwegian, and Swedish. Start time: Monday, August 23, 2021, at 12:09 PM UTC End time: Thursday, September 9, 2021, at 6:52 PM UTC Root cause: A recent change made to the platform to better align with localization per industry standards inadvertently introduced a regression that resulted in impact. Next steps: - We're reviewing our validation and testing of localization updates to prevent issues like this from occurring in the future. This is the final update for the event.; PublishedTime=2021-09-09T19:23:51.53Z
Root cause A recent change made to the platform to better align with localization per industry standards inadvertently introduced a regression that resulted in impact. Next steps: - We're reviewing our validation and testing of localization updates to prevent issues like this from occurring in the future. This is the final update for the event.; PublishedTime=2021-09-09T19:23:51.53Z

EX280152 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-08-23T15:48:46.677

Status: We're working with affected users to gather network traces to assist in the root cause investigation. Next update by: Monday, August 23, 2021, at 5:30 PMUTC; PublishedTime=2021-08-23T16:12:28.33Z

Status: We're continuing our efforts to reproduce the issue and gather network traces of the event while also looking into recent code changes which may have inadvertently caused impact. Next update by: Monday, August 23, 2021, at 7:30 PMUTC; PublishedTime=2021-08-23T17:14:11.087Z

Status: Our review of system logs has shown that some Role Based Access Control (RBAC) calls are failing which manifests as the page not rendering the affected settings. We're continuing our review of these RBAC logs to confirm the root cause and develop mitigation actions. Next update by: Monday, August 23, 2021, at 9:30 PMUTC; PublishedTime=2021-08-23T19:12:08.997Z

Status: We're expanding our investigation to compare RBAC calls against the code that generates them to get additional context and confirm the underlying cause of impact. Next update by: Tuesday, August 24, 2021, at 3:00 AMUTC; PublishedTime=2021-08-23T20:13:22.013Z

Status: Our investigation so far has been inconclusive. We're reproducing the issue in our testing environment to gather additional logging to assist with determining the underlying root cause. Next update by: Tuesday, August 24, 2021, at 9:00 PMUTC; PublishedTime=2021-08-24T01:38:23.653Z

Status: We're we're continuing our efforts to reproduce the issue in our testing environment to gather additional logging to assist with determining the underlying cause. Next update by: Tuesday, August 24, 2021, at 9:00 PMUTC; PublishedTime=2021-08-24T15:55:15.853Z

Status: We've successfully reproduced the issue internally and were able to gather additional diagnostic data, including network traces and browser logs. In parallel we're working to develop workaround procedures to alleviate impact for those that are unable to manually change their language. Next update by: Thursday, August 26, 2021, at 9:00 PMUTC; PublishedTime=2021-08-24T20:00:14.367Z

Status: We've confirmed that a recent change made to the platform to better align with localization per industry standards inadvertently introduced a regression that's resulting in the impact. We've developed and are validating a fix to be deployed as safely and efficiently as possible, and we expect we'll have a better estimation on the deployment timeline by our next update. Next update by: Monday, August 30, 2021, at 6:00 PM UTC; PublishedTime=2021-08-26T20:55:26.6Z

Status: We've successfully validated the fix in our testing environment and are now ensuring through an additional internal environment that this fix fully resolves the impact without any unintended effects. We expect that this validation will complete and that the deployment to all environments will begin by our next update, at which time we'll have a better estimation on full impact resolution. Next update by: Wednesday, September 1, 2021, at 6:00 PM UTC; PublishedTime=2021-08-30T16:33:46.873Z

Status: Final validation of the fix within our internal environment is taking longer than initially expected; however, we expect this process to complete and for deployment to begin by the end of day. Deployment of the fix to the affected environment is expected to take some time, and we'll have a better estimation for the deployment timeline by our next update. Next update by: Tuesday, September 7, 2021, at 8:00 PM UTC; PublishedTime=2021-09-01T16:55:53.55Z

Status: Final validation of the fix within our internal environment is taking longer than initially expected; however, we expect this process to complete and for deployment to begin by the end of day. Deployment of the fix to the affected environment is expected to take some time, and we'll have a better estimation for the deployment timeline by our next update. Next update by: Tuesday, September 7, 2021, at 8:00 PM UTC; PublishedTime=2021-09-03T16:12:07.61Z

Status: Our deployment of the fix has saturated throughout approximately 75 percent of the affected environment, and we're continuing to monitor its progress. We'll have a better estimation for completion of the deployment by our next scheduled update. Next update by: Thursday, September 9, 2021, at 8:30 PM UTC; PublishedTime=2021-09-07T18:55:33.527Z

Final status: Our deployment of the fix has saturated throughout the majority of the affected environment, and we've determined via telemetry that this issue is resolved. Next update by: Thursday, September 9, 2021, at 8:30 PM UTC; PublishedTime=2021-09-07T18:55:33.527Z

--------------------------------------------------------------------------------------------------
Title Some users may have experienced an approximate 10 second delay when receiving Microsoft Kaizala messages
ID KZ280051
Service Microsoft Kaizala
Feature Kaizala Messaging
Status Service restored
Impact Users may have experienced an approximate 10 second delay when receiving Microsoft Kaizala messages.
Severity
Start time 2021-08-23T06:15:00Z
End time 2021-08-23T10:15:00Z
Last update 2021-08-23T11:32:18.43Z
More info
Root cause An unexpected spike in traffic caused a subset of message processing infrastructure to perform below acceptable thresholds, resulting in impact. Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-08-23T11:25:41.843Z

KZ280051 Details

Status: We're reviewing service monitoring telemetry to isolate the source of the issue and establish a fix. Next update by: Monday, August 23, 2021, at 11:30 AM UTC; PublishedTime=2021-08-23T09:25:14.213Z

Final status: We've identified that an unexpected spike in traffic caused a subset of message processing infrastructure to perform below acceptable thresholds, resulting in impact. We've run commands which optimized traffic and performance on the affected infrastructure. After monitoring the affected environment, we've determined that impact has been remediated. Next update by: Monday, August 23, 2021, at 11:30 AM UTC; PublishedTime=2021-08-23T09:25:14.213Z

--------------------------------------------------------------------------------------------------
Title Users can't move items within shared folders when using Outlook on the web
ID EX283525
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users were unable to move items within shared folders when using Outlook on the web.
Severity
Start time 2021-08-20T07:00:00Z
End time 2021-09-09T17:00:00Z
Last update 2021-09-10T17:46:51.143Z
More info Users saw an error message indicating "something went wrong" when trying to move items within shared folders. While we were focused on the investigation, affected users may have been able to move items by using the "Search for a folder" section and typing the name of the destination folder. Final status: We've confirmed with affected users that impact has been mitigated and service is restored. Scope of impact: Any user who tried to move items within shared folders while using Outlook on the web may have experienced impact. Start time: Friday, August 20, 2021, at 7:00 AM'UTC End time: Thursday, September 9, 2021, at 5:00 PM'UTC Root cause: A change regarding folder suggestions inadvertently prevented users from moving items within shared folders when using Outlook on the web. Next steps: - We're reviewing our Outlook on the web folder update procedures to find ways to prevent similar problems from happening again. This is the final update for the event.; PublishedTime=2021-09-10T17:46:51.143Z
Root cause A change regarding folder suggestions inadvertently prevented users from moving items within shared folders when using Outlook on the web. Next steps: - We're reviewing our Outlook on the web folder update procedures to find ways to prevent similar problems from happening again. This is the final update for the event.; PublishedTime=2021-09-10T17:46:51.143Z

EX283525 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-09-07T17:01:46.32

Status: We're reviewing network traces provided by affected users to assist in the root cause investigation and develop mitigation actions. Next update by: Tuesday, September 7, 2021, at 6:30 PMUTC; PublishedTime=2021-09-07T17:18:47.077Z

Status: We're continuing our review of the network traces and expanding our investigation to include recent service updates to the folder feature set. Next update by: Wednesday, September 8, 2021, at 8:00 PMUTC; PublishedTime=2021-09-07T18:23:35.547Z

Status: Our review of recent service updates to the folder feature set revealed a latent code regression which we suspect is leading to impact. We're now analyzing the associated section code to validate this and determine the appropriate mitigation actions. Next update by: Wednesday, September 8, 2021, at 11:30 PMUTC; PublishedTime=2021-09-08T20:03:50.66Z

Status: Our analysis confirmed that a change regarding folder suggestions is inadvertently preventing users from moving items within shared folders when using Outlook on the web. We're reverting the change to mitigate impact. Next update by: Thursday, September 9, 2021, at 1:00 AMUTC; PublishedTime=2021-09-08T23:33:17.133Z

Status: We're continuing our mitigation efforts to revert the change. The fix is developed and is undergoing validation before being deployed to the affected environment. Next update by: Friday, September 10, 2021, at 12:00 AMUTC; PublishedTime=2021-09-09T00:59:16.663Z

Status: We've completed our revert and fix deployment processes to restore service and we're reaching out to affected users to confirm impact is mitigated. Next update by: Friday, September 10, 2021, at 10:00 PMUTC; PublishedTime=2021-09-10T00:18:04.197Z

Final status: We've confirmed with affected users that impact has been mitigated and service is restored. Next update by: Friday, September 10, 2021, at 10:00 PMUTC; PublishedTime=2021-09-10T00:18:04.197Z

--------------------------------------------------------------------------------------------------
Title Users can't run 2013 workflows in SharePoint Online
ID SP279230
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users were unable to run 2013 workflows in SharePoint Online.
Severity
Start time 2021-08-19T06:50:00Z
End time 2021-08-19T19:00:00Z
Last update 2021-08-19T20:41:56.477Z
More info
Root cause A portion of infrastructure that supports workflow calls unexpectedly stopped performing within acceptable standards, causing impact. Next steps: - We're reviewing this portion of infrastructure to understand why it unexpectedly stopped performing within acceptable standards to help prevent similar impact from reoccurring in the future. - We're reviewing our monitoring services to look for ways to reduce detection time and more quickly restore service. This is the final update for the event.; PublishedTime=2021-08-19T20:41:56.477Z

SP279230 Details

Status: We're reviewing details provided by affected users to determine the next steps needed to resolve this issue. Next update by: Thursday, August 19, 2021, at 4:00 PM UTC; PublishedTime=2021-08-19T14:05:54.1Z

Status: We're reviewing back-end service logs to better understand the issue and determine our next troubleshooting steps. Next update by: Thursday, August 19, 2021, at 6:00 PMUTC; PublishedTime=2021-08-19T15:40:11.887Z

Status: We've noticed some errors during our review, leading us to suspect that a networking issue may be the cause. We're reviewing the API version of the service, as well as performing connectivity tests to isolate the cause of impact and determine our next steps. Next update by: Thursday, August 19, 2021, at 8:00 PMUTC; PublishedTime=2021-08-19T17:08:46.287Z

Status: We've confirmed that a portion of infrastructure that supports workflow calls unexpectedly stopped performing within acceptable standards, causing impact. We've re-routed traffic to an alternate infrastructure and are restarting it, after which we'll review telemetry and reach out to affected users to confirm impact remediation. Next update by: Thursday, August 19, 2021, at 9:30 PMUTC; PublishedTime=2021-08-19T19:08:30.907Z

Final status: We've completed restarting the infrastructure, and after reviewing workflow telemetry and reaching out to affected users, we've confirmed that impact has been fully remediated. Next update by: Thursday, August 19, 2021, at 9:30 PMUTC; PublishedTime=2021-08-19T19:08:30.907Z

--------------------------------------------------------------------------------------------------
Title Users may have intermittently been unable to load their Yammer feed
ID YA279165
Service Yammer Enterprise
Feature Yammer Components
Status Service restored
Impact Users may have intermittently been unable to load their Yammer feed.
Severity
Start time 2021-08-19T05:20:00Z
End time 2021-08-19T06:35:00Z
Last update 2021-08-19T07:25:51.12Z
More info
Root cause A collection of content storage infrastructure was performing below acceptable performance thresholds, causing latency to user Yammer feeds. Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-08-19T07:21:53.01Z

YA279165 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're investigating a possible service database availability issue that may be contributing to impact. Next update by: Thursday, August 19, 2021, at 7:30 AMUTC; PublishedTime=2021-08-19T06:33:57.89Z

Final status: We've determined that a collection of content storage infrastructure was performing below acceptable performance thresholds, causing latency to user Yammer feeds. We've run commands that optimized traffic and performance on the affected infrastructure, which remediated impact. Next update by: Thursday, August 19, 2021, at 7:30 AMUTC; PublishedTime=2021-08-19T06:33:57.89Z

--------------------------------------------------------------------------------------------------
Title Admins may be unable to renew subscriptions or make subscription service changes to Volume Licensing subscriptions
ID MO279591
Service Microsoft 365 suite
Feature Portal
Status Service restored
Impact Admins were unable to renew subscriptions or make subscription service changes to Volume Licensing subscriptions.
Severity
Start time 2021-08-19T03:38:00Z
End time 2021-08-21T07:50:00Z
Last update 2021-08-21T17:48:30.61Z
More info
Root cause A recent change intended to provide general optimizations, coupled with an authentication misconfiguration, resulted in impact. Next steps: - Our investigation into the details of the offending change is continuing to help us find ways avoid similar issues during our ongoing service optimization efforts. This is the final update for the event.; PublishedTime=2021-08-21T17:48:30.61Z

MO279591 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-08-20T18:25:00.173

Status: We're reviewing Internet Information Services (IIS) logs for servers exhibiting impact for error analysis to determine potential sources of impact. Next update by: Friday, August 20, 2021, at 8:00 PMUTC; PublishedTime=2021-08-20T18:42:21.963Z

Status: We've determined that the inclusion of two incorrect host files caused an authentication configuration issue, impacting our Volume Licensing Integration service. We've removed these host files in an attempt to mitigate impact; however, an additional problem with the Commerce Transaction Platform service is contributing to the impact. We've identified a recent update that likely related to this issue, and we're reviewing options to revert this change to mitigate impact. Next update by: Friday, August 20, 2021, at 10:00 PMUTC; PublishedTime=2021-08-20T19:48:23.153Z

Status: We've completed deployment of our revert to a portion of the affected infrastructure. While the deployment was successful, we have confirmed some additional errors occurring which seem to be related to a previous authentication configuration change made. We're analyzing the errors to confirm potential mitigation pathways to correct the issue. Next update by: Saturday, August 21, 2021, at 12:00 AMUTC; PublishedTime=2021-08-20T21:41:31.813Z

Status: We've mitigated the additional authentication configuration issue which was causing errors after our reversion of the affecting change. We've confirmed this portion of infrastructure is now processing requests as expected, although there is a backlog of requests which is continuing to drain. We're proceeding with reverting this change on the remaining portion of infrastructure, after which we'll route traffic through both portions to complete our mitigation. We're also reviewing options to expedite the backlogged requests once completed. Next update by: Saturday, August 21, 2021, at 2:00 AMUTC; PublishedTime=2021-08-20T23:42:10.63Z

Status: We're continuing to monitor the process of reverting the offending change on the remaining portion of infrastructure so we can route the traffic through both portions of infrastructure to mitigate the issue. Additionally, we're continuing to review and test options on potentially expediting the backlogged requests. Next update by: Saturday, August 21, 2021, at 6:00 AMUTC; PublishedTime=2021-08-21T01:47:26.083Z

Status: We've completed reverting the offending change on the portion of infrastructure and are routing the traffic through both portions of infrastructure. Additionally, we've validated and are running a manual code change to expedite the processing of backlogged requests. Next update by: Saturday, August 21, 2021, at 7:00 PMUTC; PublishedTime=2021-08-21T05:51:42.323Z

Final status: We've completed processing the backlogged requests to fully mitigate the impact and restore the expected functionality. Next update by: Saturday, August 21, 2021, at 7:00 PMUTC; PublishedTime=2021-08-21T05:51:42.323Z

--------------------------------------------------------------------------------------------------
Title Users are unable to upload files to Project Online through the Project app in Microsoft Teams
ID TM281114
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were unable to upload files to Project Online through the Project app in Microsoft Teams.
Severity
Start time 2021-08-18T23:31:00Z
End time 2021-09-14T07:30:00Z
Last update 2021-09-14T18:34:20.987Z
More info Files could have been successfully uploaded via Project for the web or the Project Online desktop client. Final status: We've confirmed that the fix has fully saturated the affected environment, and we've validated resolution with previously affected users. Scope of impact: Your organization was affected by this event, and any user who attempted to upload files to Project Online using the Project app in Microsoft Teams may have experienced impact. Start time: Wednesday, August 18, 2021, at 11:31 PM'UTC End time: Tuesday, September 14, 2021, at 7:30 AM'UTC Root cause: An authentication token which facilitates the exchange of credentials between users and the Microsoft-managed environment when uploading files to Project Online through the Project app in Microsoft Teams wasn't configured correctly, resulting in impact. Next steps: - We're further reviewing why the affected authentication token wasn't properly configured to prevent similar problems from happening in the future. This is the final update for the event.; PublishedTime=2021-09-14T18:26:51.443Z
Root cause An authentication token which facilitates the exchange of credentials between users and the Microsoft-managed environment when uploading files to Project Online through the Project app in Microsoft Teams wasn't configured correctly, resulting in impact. Next steps: - We're further reviewing why the affected authentication token wasn't properly configured to prevent similar problems from happening in the future. This is the final update for the event.; PublishedTime=2021-09-14T18:26:51.443Z

TM281114 Details

Status: We've determined that an authentication component necessary for uploading files to Project Online through the Project app in Microsoft Teams isn't configured correctly, resulting in impact. We've developed a fix to ensure the token is set correctly and we're internally validating this fix. We expect that testing will be complete and the fix will be ready for deployment by Thursday, September 9, 2021. We'll provide additional information regarding the fix deployment status and estimated timeline at our next scheduled update. Next update by: Thursday, September 9, 2021, at 7:00 PM UTC; PublishedTime=2021-08-26T21:24:09.543Z

Status: We've initiated deployment of the fix to the affected environment. We're monitoring the ongoing deployment to ensure it completes as expected and fully resolves this issue. We expect to have an estimated time for impact resolution by our next update. Next update by: Tuesday, September 14, 2021, at 7:00 PMUTC; PublishedTime=2021-09-09T17:39:34.44Z

Final status: We've confirmed that the fix has fully saturated the affected environment, and we've validated resolution with previously affected users. Next update by: Tuesday, September 14, 2021, at 7:00 PMUTC; PublishedTime=2021-09-09T17:39:34.44Z

--------------------------------------------------------------------------------------------------
Title Users can't modify the layout of SharePoint Online modern sites that have web parts
ID SP279577
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users were unable to modify the layout of SharePoint Online modern sites that have web parts.
Severity
Start time 2021-08-18T23:00:00Z
End time 2021-08-20T17:56:00Z
Last update 2021-08-20T18:53:47.72Z
More info When users attempted to edit the desired section, they received an error stating "No settings to change for this web part." Final status: We've identified that a recent change intended to update the Canvas Zone SharePoint Online library unexpectedly resulted in impact. We've disabled this offending change and after monitoring the service and verifying with affected users, we've ensured that this action has successfully resolved the problem. Scope of impact: Impact may have occurred for any user attempting to modify the layout of SharePoint Online modern sites that have web parts. Start time: Wednesday, August 18, 2021, at 11:00 PM UTC End time: Friday, August 20, 2021, at 5:56 PM UTC Root cause: A recent change intended to update the Canvas Zone SharePoint Online library unexpectedly resulted in impact. Next steps: - We're investigating why the recent change intended to update the Canvas Zone SharePoint Online library resulted in impact to prevent this problem from happening again in the future. - We're also assessing why the problem wasn't caught prior to deploying the update so that we can improve our update testing procedures. This is the final update for the event.; PublishedTime=2021-08-20T18:51:38.57Z
Root cause A recent change intended to update the Canvas Zone SharePoint Online library unexpectedly resulted in impact. Next steps: - We're investigating why the recent change intended to update the Canvas Zone SharePoint Online library resulted in impact to prevent this problem from happening again in the future. - We're also assessing why the problem wasn't caught prior to deploying the update so that we can improve our update testing procedures. This is the final update for the event.; PublishedTime=2021-08-20T18:51:38.57Z

SP279577 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're investigating available diagnostic data, such as any recent changes that may have contributed to this impact, in order to isolate the source of this issue and identify troubleshooting actions. Next update by: Friday, August 20, 2021, at 5:30 PMUTC; PublishedTime=2021-08-20T16:30:27.99Z

Status: We've identified a recent change that may have resulted in the impact, and we're disabling this flight in our test environment to confirm our findings and determine next steps. Next update by: Friday, August 20, 2021, at 7:30 PMUTC; PublishedTime=2021-08-20T17:13:49.4Z

Final status: We've identified that a recent change intended to update the Canvas Zone SharePoint Online library unexpectedly resulted in impact. We've disabled this offending change and after monitoring the service and verifying with affected users, we've ensured that this action has successfully resolved the problem. Next update by: Friday, August 20, 2021, at 7:30 PMUTC; PublishedTime=2021-08-20T17:13:49.4Z

--------------------------------------------------------------------------------------------------
Title Admins are unable to access Service Health Dashboard and Message Center blades in the Microsoft Endpoint Manager
ID IT279120
Service Microsoft Intune
Feature Microsoft Intune
Status Service restored
Impact Admins were unable to access Service Health and Message Center blades in the Microsoft Endpoint Manager (MEM).
Severity
Start time 2021-08-18T21:03:00Z
End time 2021-08-19T00:50:00Z
Last update 2021-08-19T01:02:37.733Z
More info Affected admins would have seen the following pop-up error in the upper-right portion of the MEM: "The portal is having issues getting an authentication token. The experience rendered may be degraded..." This issue only impacted the Service Health and Message Center blades. All other MEM functions were unaffected. Intune admins were able to view Service Health and Message Center messages at https://portal.office.com/AdminPortal/Home#/servicehealth Final status: We've determined that a recent update to the URL that the MEM uses to retrieve Service Health and Message Center information resulted in impact. We've reverted the update and confirmed through our internal testing that the issue is mitigated. Scope of impact: This issue could've impacted any Intune admin attempting to view Service Health and Message Center messages through the MEM. Start time: Wednesday, August 18, 2021, at 9:03 PM UTC End time: Thursday, August 19, 2021, at 12:50 AM UTC Root cause: A recent update to the URL that the MEM uses to retrieve Service Health and Message Center information resulted in impact. Next steps: - We're reviewing the dependent URL update deployment procedures to help prevent similar problems in the future. This is the final update for the event.; PublishedTime=2021-08-19T01:02:37.733Z
Root cause A recent update to the URL that the MEM uses to retrieve Service Health and Message Center information resulted in impact. Next steps: - We're reviewing the dependent URL update deployment procedures to help prevent similar problems in the future. This is the final update for the event.; PublishedTime=2021-08-19T01:02:37.733Z

IT279120 Details

Status: We're reviewing the error message that admins are receiving and investigating a potential request connection issue that may be resulting in impact. Next update by: Thursday, August 19, 2021, at 1:30 AMUTC; PublishedTime=2021-08-19T00:29:28.953Z

Final status: We've determined that a recent update to the URL that the MEM uses to retrieve Service Health and Message Center information resulted in impact. We've reverted the update and confirmed through our internal testing that the issue is mitigated. Next update by: Thursday, August 19, 2021, at 1:30 AMUTC; PublishedTime=2021-08-19T00:29:28.953Z

--------------------------------------------------------------------------------------------------
Title Admins are unable to see call records between August 18, 2021 and August 23, 2021 in the Security and Compliance Center
ID EX280618
Service Exchange Online
Feature Management and Provisioning
Status Service restored
Impact Admins were unable to see call records between August 18, 2021 and August 23, 2021 in the SCC.
Severity
Start time 2021-08-18T20:30:00Z
End time 2021-09-13T16:15:00Z
Last update 2021-09-14T20:11:14.287Z
More info Impact was specific to admins with the specific role entitled eDiscovery Managers. Additionally, impact was specific to Microsoft Teams records. Final status: We've confirmed that the backfill of the missing data has completed and impact has been mitigated. Scope of impact: Any admin who attempted to see Microsoft Teams call records within the Security and Compliance Center within the dates of August 18, 2021 to August 23, 2021 was affected. Start time: Wednesday, August 18, 2021, at 8:30 PM'UTC End time: Monday, September 13, 2021, at 4:15 PM'UTC Root cause: An outdated authentication component prevented Microsoft Teams call records from being posted to eDiscovery, which resulted in impact. Next steps: - We're reviewing our authentication component update procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-14T20:11:14.287Z
Root cause An outdated authentication component prevented Microsoft Teams call records from being posted to eDiscovery, which resulted in impact. Next steps: - We're reviewing our authentication component update procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-14T20:11:14.287Z

EX280618 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-08-24T23:29:17.23

Status: We've identified there was an outdated authentication configuration which prevented Microsoft Teams call records from being posted to eDiscovery. We've corrected the issue for Tuesday August 24, 2021 and onwards. We are now in the process of backfilling the data from Wednesday August 18, 2021 to Monday August 23, 2021. Next update by: Wednesday, September 1, 2021, at 1:00 AMUTC; PublishedTime=2021-08-24T23:48:33.363Z

Status: We're in the process of preparing the missing data as well as preparing the pipeline before we initiate the backfill process. Everything is progressing as expected and we have an anticipated mitigation date of Friday, September 10, 2021. Next update by: Wednesday, September 8, 2021, at 1:30 AMUTC; PublishedTime=2021-08-31T23:48:38.987Z

Status: We've confirmed that the data and pipeline have been fully staged and the backfill of the missing data is now in progress. We anticipate the backfill to be completed by Saturday, September 11, 2021. Next update by: Saturday, September 11, 2021, at 12:00 AMUTC; PublishedTime=2021-09-07T23:51:45.12Z

Status: The backfill of the missing data is progressing more slowly than previously anticipated. We now expect the backfill to be completed by Thursday, September 16, 2021. Next update by: Thursday, September 16, 2021, at 11:30 PMUTC; PublishedTime=2021-09-10T22:11:41.75Z

Final status: We've confirmed that the backfill of the missing data has completed and impact has been mitigated. Next update by: Thursday, September 16, 2021, at 11:30 PMUTC; PublishedTime=2021-09-10T22:11:41.75Z

--------------------------------------------------------------------------------------------------
Title Some users may not see members that are added or removed via Microsoft Teams in the Shifts app membership
ID TM279051
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have not seen members that were added or removed via Microsoft Teams in the Shifts app membership.
Severity
Start time 2021-08-18T14:58:00Z
End time 2021-08-18T22:00:00Z
Last update 2021-08-18T22:24:28.273Z
More info
Root cause A portion of Microsoft Teams infrastructure responsible for membership change events wasn't updating the Shifts app membership roster as intended, resulting in impact. Next steps: - We're investigating why the portion of Microsoft Teams infrastructure responsible for membership change events wasn't updating the Shifts app membership roster as intended to prevent this problem from happening again in the future. This is the final update for the event.; PublishedTime=2021-08-18T22:23:33.387Z

TM279051 Details

Status: We're reviewing available diagnostic data to isolate the source of this impact and identify troubleshooting actions. Next update by: Wednesday, August 18, 2021, at 6:30 PMUTC; PublishedTime=2021-08-18T17:37:06.567Z

Status: We've identified that a portion of Microsoft Teams infrastructure responsible for membership change events isn't updating the Shifts app membership roster as intended, resulting in impact. We've redirected the affected traffic to alternate infrastructure and we're monitoring the service to ensure that this action successfully alleviates the issue. Next update by: Wednesday, August 18, 2021, at 8:30 PMUTC; PublishedTime=2021-08-18T18:15:36.743Z

Status: We're continuing to monitor the service to ensure that redirecting the affected traffic to alternate infrastructure has mitigated the impact in full. Next update by: Wednesday, August 18, 2021, at 10:30 PMUTC; PublishedTime=2021-08-18T20:17:23.733Z

Final status: We've redirected the affected traffic to alternate infrastructure, and after monitoring the service, we've ensured that this action has successfully alleviated impact. Next update by: Wednesday, August 18, 2021, at 10:30 PMUTC; PublishedTime=2021-08-18T20:17:23.733Z

--------------------------------------------------------------------------------------------------
Title Users experiencing intermittent navigation errors and connectivity issues when accessing the Microsoft Teams service
ID TM279042
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have experienced intermittent navigation errors and connectivity issues within any Microsoft Teams client.
Severity
Start time 2021-08-18T14:46:00Z
End time 2021-08-18T15:09:00Z
Last update 2021-08-18T15:46:14.01Z
More info Affected users may have been able to navigate or connect upon automatic client retries, reducing the duration and degree to which users would've experienced this impact. At most, users may have seen impact persist for approximately ten minutes. Final status: We identified a section of traffic management infrastructure which wasn't performing within expected thresholds, resulting in intermittent impact to users served through the affected section of infrastructure in the central United States. We've rebalanced the traffic management infrastructure, and we've monitored the service telemetry to confirm that the impact is mitigated. Scope of impact: Any user served through the affected section of infrastructure in the central United States may have been intermittently impacted. Start time: Wednesday, August 18, 2021, at 2:46 PM UTC End time: Wednesday, August 18, 2021, at 3:09 PM UTC Root cause: A section of traffic management infrastructure wasn't performing within expected thresholds, resulting in intermittent impact. Next steps: - We'll continue to investigate the context in which the traffic management infrastructure became degraded, and we'll work to identify possible improvements for our automated service recovery processes to help us prevent the issue from reoccurring. This is the final update for the event.; PublishedTime=2021-08-18T15:46:14.01Z
Root cause A section of traffic management infrastructure wasn't performing within expected thresholds, resulting in intermittent impact. Next steps: - We'll continue to investigate the context in which the traffic management infrastructure became degraded, and we'll work to identify possible improvements for our automated service recovery processes to help us prevent the issue from reoccurring. This is the final update for the event.; PublishedTime=2021-08-18T15:46:14.01Z

TM279042 Details

Status: We've investigated a problem with the Microsoft Teams service in which users may have experienced intermittent navigation errors and connectivity issues within any Microsoft Teams client, and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-08-18T15:20:25.373

Final status: We identified a section of traffic management infrastructure which wasn't performing within expected thresholds, resulting in intermittent impact to users served through the affected section of infrastructure in the central United States. We've rebalanced the traffic management infrastructure, and we've monitored the service telemetry to confirm that the impact is mitigated.

--------------------------------------------------------------------------------------------------
Title Users may be unable to make calls or join meetings through any Skype for Business connection method
ID LY278898
Service Skype for Business
Feature Audio and Video
Status Service restored
Impact Users may have been unable to make calls or join meetings through any Skype for Business connection method.
Severity
Start time 2021-08-17T22:18:00Z
End time 2021-08-18T00:08:00Z
Last update 2021-08-18T00:39:30.617Z
More info
Root cause A recent deployment introduced a misconfiguration on a portion of infrastructure that facilitates Skype for Business calling and meetings, which caused impact. Next steps: - We're reviewing our deployment and provisioning procedures to help prevent similar configuration change problems in the future. This is the final update for the event.; PublishedTime=2021-08-18T00:39:30.617Z

LY278898 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We've identified a coding error in a recent change made to our environment which is causing impact. We've corrected the configuration change and are deploying the updated configuration to the affected environment. Next update by: Wednesday, August 18, 2021, at 1:00 AMUTC; PublishedTime=2021-08-17T23:55:19.603Z

Final status: Our deployment of the fix has completed and we've confirmed via service health monitoring that impact is mitigated. Next update by: Wednesday, August 18, 2021, at 1:00 AMUTC; PublishedTime=2021-08-17T23:55:19.603Z

--------------------------------------------------------------------------------------------------
Title Some users can't see some chat messages in the Microsoft Teams desktop app
ID TM278944
Service Microsoft Teams
Feature Teams Components
Status Post-incident report published
Impact Users couldn't see some chat messages in the Microsoft Teams desktop app.
Severity Sev2
Start time 2021-08-16T00:00:00Z
End time 2021-09-20T17:30:00Z
Last update 2021-09-27T22:05:55.11Z
More info Some users may have seen an Oops page when signing in, couldn't receive messages when switching between tenants, or couldn't see channels and chats on the sidebar. Additionally, previously posted chat messages may not have been visible. Users with access to the Microsoft Teams web app could use that connection method to avoid impact. Final status: After further review, we've established that the underlying cause of this event has been addressed. If users are still impacted, it's because of the cascading impact associated with TM285554. Please see TM285554 for additional insights in the event your organization remains affected. Scope of impact: A limited number of users leveraging the Microsoft Teams desktop app may have been affected. Start time: Monday, August 16, 2021, at 12:00 AM UTC End time: Monday, September 20, 2021, at 5:30 PM UTC Preliminary root cause: A service update intended to improve the Microsoft Teams desktop client performance encountered various transient issues during the implementation process that resulted in impact for a limited number of users. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document.; PublishedTime=2021-09-20T18:28:23.373Z
Root cause An ongoing service update intended to improve the desktop client performance is encountering various transient issues during the implementation process for a limited number of users. Next update by: Friday, August 27, 2021, at 1:00 AMUTC; PublishedTime=2021-08-26T22:27:43.897Z

TM278944 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're analyzing service logs to determine the source of the issue. Next update by: Thursday, August 19, 2021, at 12:30 AMUTC; PublishedTime=2021-08-18T05:41:52.783Z

Status: Our investigation has found a potential code issue that may be contributing to the impact. We're continuing to investigate the potential code issue to determine our next steps in the mitigation process. Next update by: Thursday, August 19, 2021, at 11:30 PMUTC; PublishedTime=2021-08-18T23:03:42.943Z

Status: Our efforts to determine a root cause are ongoing, however we suspect a new "common data later" (CDL) code path could resolve the issue. We've enabled this CDL path for a single affected user to test as a potential mitigation, and are awaiting confirmation of the results of the testing. Next update by: Friday, August 20, 2021, at 11:00 PMUTC; PublishedTime=2021-08-19T22:07:25.557Z

Status: We suspect that a communication issue between chat components is resulting in the impact. We've developed a fix which we're deploying incrementally to a subset of affected users to ensure the fix is successful. Next update by: Saturday, August 21, 2021, at 1:30 AMUTC; PublishedTime=2021-08-20T17:56:31.703Z

Status: Our deployment of the fix to a subset of affected users has completed and we ensured the fix was successful. We're continuing our deployment of the fix incrementally to affected tenants slowly to provide as little affect to service as possible. Next update by: Monday, August 23, 2021, at 11:30 PMUTC; PublishedTime=2021-08-21T00:43:46.177Z

Status: During our incremental deployment, we've determined that our developed fix will not be a viable solution for remediating impact. We're developing and validating a new fix, and expect we can provide an updated fix deployment timeline once validation has completed. Next update by: Tuesday, August 24, 2021, at 11:30 PMUTC; PublishedTime=2021-08-23T21:31:50.383Z

Status: We're continuing to develop our new fix, and ensure it doesn't experience the problems that prevented our first fix deployment from completing as expected. We're expecting to provide an estimate for the remediation of impact once the new fix has been validated. Next update by: Wednesday, August 25, 2021, at 11:00 PMUTC; PublishedTime=2021-08-24T21:34:38.473Z

Status: Our development of the new fix is ongoing to ensure it doesn't experience the issues that prevented our first fix deployment from completing as expected. We expect we'll have an estimated timeline for impact remediation as soon as the new fix is successfully validated. Next update by: Thursday, August 26, 2021, at 11:00 PMUTC; PublishedTime=2021-08-25T22:24:26.763Z

Status: We've completed deployment of the fix to a subset of users and are now reaching out to them to confirm if the impact was successfully remediated. We're also developing corresponding targeted fixes for the remaining affected users. Next update by: Friday, August 27, 2021, at 1:00 AMUTC; PublishedTime=2021-08-26T22:27:43.897Z

Status: We're continuing to reach out to the subset of users who have received the fix to validate mitigation. In parallel, we're continuing to develop the corresponding targeted fixes for the remaining affected users. Next update by: Friday, August 27, 2021, at 6:00 AMUTC; PublishedTime=2021-08-27T00:32:15.147Z

Status: We've reached out to some affected users; however, we've not yet received any responses. This may be due to the intermittency of the issue. While we wait for users responses, we're continuing to develop a targeted fix for the remaining impacted users. We'll have a more accurate mitigation timeline by our next scheduled update. Next update by: Wednesday, September 1, 2021, at 6:00 AMUTC; PublishedTime=2021-08-27T04:50:27.653Z

Status: We've received reports from some affected users that impact is mitigated, and we're continuing to await further responses. Additionally, we're continuing to develop the targeted fix, which we expect will be deployed to remaining affected users by early next week. Next update by: Wednesday, September 8, 2021, at 6:00 AMUTC; PublishedTime=2021-09-01T04:43:41.087Z

Status: We're beginning deployment of the targeted fix to the remaining users who reported impact, and we expect this to be completed by the end of the day on Wednesday, September 8, 2021. Next update by: Tuesday, September 14, 2021, at 6:00 AM UTC; PublishedTime=2021-09-08T04:45:40.627Z

Status: We're continuing to wait to hear back from reported users that the issue is resolved after implementing the fix. Next update by: Tuesday, September 21, 2021, at 6:00 AMUTC; PublishedTime=2021-09-14T04:52:52Z

Status: We've received reports that various users are unable to see or receive some chat messages, may see an Oops page when signing in, or can't see the channels and chat in the sidebar when using the desktop client. We've determined that an ongoing service update intended to improve the desktop client performance is encountering various transient issues during the implementation process for a limited number of users. We expect that the vast majority of users can resolve this issue by either fully closing the desktop client (Right-click on Microsoft Teams on the taskbar and select 'Quit') and starting Microsoft Teams again, or signing out and signing in again. If users are still affected after performing the previous steps, we suggest leveraging the web app, if possible, and reaching out to support for additional follow up. In parallel, we're working on developing a fix to remediate impact for continuously impacted users and expect to have an update by Tuesday, September 21, 2021 on its progress. Next update by: Tuesday, September 21, 2021, at 6:00 AM UTC; PublishedTime=2021-09-17T08:00:16.787Z

Final status: After further review, we've established that the underlying cause of this event has been addressed. If users are still impacted, it's because of the cascading impact associated with TM285554. Please see TM285554 for additional insights in the event your organization remains affected. Next update by: Tuesday, September 21, 2021, at 6:00 AM UTC; PublishedTime=2021-09-17T08:00:16.787Z

--------------------------------------------------------------------------------------------------
Title Admins are unable to change OneDrive for Business user storage limits
ID OD280715
Service OneDrive for Business
Feature OneDrive for Business
Status Service restored
Impact Admins were unable to change OneDrive for Business user storage limits.
Severity
Start time 2021-08-13T01:22:00Z
End time 2021-08-25T20:07:00Z
Last update 2021-08-25T20:29:40.64Z
More info
Root cause A recent service update designed to redistribute load within the SharePoint Online Directory resulted in impact. Next steps: - We're reviewing our update procedures to better understand why the change produced impact and to identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-08-25T20:29:40.64Z

OD280715 Details

Status: We're investigating a potential issue where admins are unable to change OneDrive for Business user storage limits. We'll provide an update within 30 minutes.; PublishedTime=2021-08-25T10:03:27.807

Status: We're reviewing details provided in the support case to determine the next steps needed to resolve this issue. Next update by: Wednesday, August 25, 2021, at 12:30 PM UTC; PublishedTime=2021-08-25T10:14:57.273Z

Status: We're analyzing Unified Logging Service (ULS) logs to isolate the source of the issue and develop a mitigation plan. Next update by: Wednesday, August 25, 2021, at 2:30 PM UTC; PublishedTime=2021-08-25T12:27:55.803Z

Status: We're continuing to analyze Unified Logging Service (ULS) logs from our reproduction of the issue, to determine our next troubleshooting steps. Next update by: Wednesday, August 25, 2021, at 6:00 PM UTC; PublishedTime=2021-08-25T13:25:54.853Z

Status: Our analysis of the ULS data from our reproduction of the issue is ongoing. In parallel, we're reviewing recent service changes to further our efforts to isolate the source of the problem. Next update by: Wednesday, August 25, 2021, at 9:30 PMUTC; PublishedTime=2021-08-25T17:01:50.1Z

Final status: Following our ULS data analysis, we've identified a recent service update that was preventing admins from changing OneDrive for Business user storage limits. We've reverted the recent update, and confirmed via internal telemetry that it has successfully remediated impact. Next update by: Wednesday, August 25, 2021, at 9:30 PMUTC; PublishedTime=2021-08-25T17:01:50.1Z

--------------------------------------------------------------------------------------------------
Title Admins are unable to set menu options for auto-attendant calls in the Microsoft Teams admin center
ID TM277594
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Admins were unable to set menu options for auto-attendant calls in the Microsoft Teams admin center.
Severity
Start time 2021-08-12T14:13:00Z
End time 2021-08-12T20:10:00Z
Last update 2021-08-12T21:31:09.713Z
More info In the auto-attendant section of the admin center, no options for menu options were displayed after admins selected "Play menu options" in the Advanced Settings -> Call flow section. Final status: We've completed reverting the problematic update and confirmed with affected users and through monitoring telemetry that the issue has been resolved. Scope of impact: This issue potentially affected any of your admins attempting to set menu options for auto-attendant calls in the Microsoft Teams admin center. Start time: Thursday, August 12, 2021, at 2:13 PM'UTC End time: Thursday, August 12, 2021, at 8:10 PM'UTC Root cause: A recent update to the Microsoft Teams admin center, intended to enhance its functionality, affected a subsection of system code, resulted in impact. Next steps: - We're reviewing our Microsoft Teams admin center update validation and testing procedures to ensure potential code issues are isolated and remediated prior to future impact scenarios. This is the final update for the event.; PublishedTime=2021-08-12T21:30:11.24Z
Root cause A recent update to the Microsoft Teams admin center, intended to enhance its functionality, affected a subsection of system code, resulted in impact. Next steps: - We're reviewing our Microsoft Teams admin center update validation and testing procedures to ensure potential code issues are isolated and remediated prior to future impact scenarios. This is the final update for the event.; PublishedTime=2021-08-12T21:30:11.24Z

TM277594 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-08-12T15:05:28.69

Status: We're reviewing a subsection of system code and a group of recent service updates as we determine the source for this event. Next update by: Thursday, August 12, 2021, at 8:00 PMUTC; PublishedTime=2021-08-12T15:18:57.907Z

Status: Through our analysis of a subsection of system code, we've determined that impact was caused by a recent update to the Microsoft Teams admin center intended to enhance its functionality. We're reverting this change to mitigate impact and expect that this process will complete by our next scheduled update. Next update by: Thursday, August 12, 2021, at 11:30 PMUTC; PublishedTime=2021-08-12T19:58:18.027Z

Final status: We've completed reverting the problematic update and confirmed with affected users and through monitoring telemetry that the issue has been resolved. Next update by: Thursday, August 12, 2021, at 11:30 PMUTC; PublishedTime=2021-08-12T19:58:18.027Z

--------------------------------------------------------------------------------------------------
Title Users are unable to view PowerPoint Live presentations when using the Microsoft Teams web platform.
ID TM277395
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were unable to view PowerPoint Live presentations when using the Microsoft Teams web platform.
Severity
Start time 2021-08-11T19:45:00Z
End time 2021-08-11T21:00:00Z
Last update 2021-08-11T21:19:51.313Z
More info Users could view presentations when using the Microsoft Teams desktop client or mobile app. Final status: We've determined that a recent setting change to the PowerPoint Live feature was preventing users from viewing PowerPoint Live presentations when using the Microsoft Teams web platform. We manually updated the setting, which successfully remediated the impact. Scope of impact: This issue may have potentially affected any of your users attempting to view PowerPoint Live presentations when using the Microsoft Teams web platform. Start time: Wednesday, August 11, 2021, at 7:45 PM'UTC End time: Wednesday, August 11, 2021, at 9:00 PM'UTC Root cause: A recent settings change to the PowerPoint Live feature was preventing users from viewing PowerPoint Live presentations when using the Microsoft Teams web platform. Next steps: - We're reviewing our settings change procedures to better identify similar issues before impact is experienced. This is the final update for the event.; PublishedTime=2021-08-11T21:15:48.55Z
Root cause A recent settings change to the PowerPoint Live feature was preventing users from viewing PowerPoint Live presentations when using the Microsoft Teams web platform. Next steps: - We're reviewing our settings change procedures to better identify similar issues before impact is experienced. This is the final update for the event.; PublishedTime=2021-08-11T21:15:48.55Z

TM277395 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-08-11T20:56:54.117

Final status: We've determined that a recent setting change to the PowerPoint Live feature was preventing users from viewing PowerPoint Live presentations when using the Microsoft Teams web platform. We manually updated the setting, which successfully remediated the impact.

--------------------------------------------------------------------------------------------------
Title Users aren't receiving Briefing email
ID EX277384
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users weren't receiving Briefing email.
Severity
Start time 2021-08-11T15:20:00Z
End time 2021-08-12T01:00:00Z
Last update 2021-08-12T03:23:24.053Z
More info
Root cause A portion of infrastructure which facilitates receiving Briefing email in Exchange Online wasn't performing as efficiently as expected, which resulted in impact. Next steps: - We're investigating why the portion of infrastructure wasn't performing as efficiently as expected, so that we can help prevent similar problems in the future. This is the final update for the event.; PublishedTime=2021-08-12T03:23:24.053Z

EX277384 Details

Status: We've identified a potential issue with Briefing messages and have disabled the feature as a precautionary measure. We're reviewing the logs and code of the feature to isolate the potential issue and develop mitigation actions. Next update by: Wednesday, August 11, 2021, at 10:00 PMUTC; PublishedTime=2021-08-11T19:06:20.933Z

Status: We're continuing our review of Briefing code and logs to determine the underlying cause of impact and develop mitigation actions as needed. Next update by: Thursday, August 12, 2021, at 4:00 AMUTC; PublishedTime=2021-08-11T21:46:27.227Z

Final status: Our investigation determined that a portion of infrastructure which facilitates receiving Briefing email in Exchange Online wasn't performing as efficiently as expected, which resulted in impact. We've restarted the affected portion of infrastructure and confirmed with internal telemetry that the issue is resolved. Next update by: Thursday, August 12, 2021, at 4:00 AMUTC; PublishedTime=2021-08-11T21:46:27.227Z

--------------------------------------------------------------------------------------------------
Title Some users' My Apps page may not fully load, preventing them from accessing some apps
ID MO277119
Service Microsoft 365 suite
Feature Portal
Status Service restored
Impact Users' My Apps page may not have fully loaded, preventing them from accessing some apps.
Severity
Start time 2021-08-10T18:12:00Z
End time 2021-08-10T20:45:00Z
Last update 2021-08-11T04:17:34.72Z
More info
Root cause A service misconfiguration was introduced by a recent standard service update, resulting in app access issues for some users attempting to load the My Apps page. Next steps: - We're continuing to review the recent standard service update to help us better understand the context for the misconfiguration which resulted in impact so we can prevent this issue from happening again in future updates. This is the final update for the event.; PublishedTime=2021-08-10T22:07:23.6Z

MO277119 Details

Status: We've investigated a problem in which users may have been unable to access the My Apps page, and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-08-10T21:52:24.7

Final status: We've identified a service misconfiguration which was introduced by a recent standard service update, resulting in app access issues for some users attempting to load the My Apps page. We've reverted the update to mitigate the impact, and we've confirmed the issue is resolved.

--------------------------------------------------------------------------------------------------
Title Users may experience intermittent Power Platform dataflow refresh failures
ID MM277143
Service Power Apps in Microsoft 365
Feature Service and web access issues
Status Service restored
Impact Users may have experienced intermittent Power Platform dataflow refresh failures.
Severity
Start time 2021-08-10T08:30:00Z
End time 2021-08-11T12:00:00Z
Last update 2021-08-11T19:58:48.05Z
More info While we were focused on remediation, users may have retried Power Platform dataflow refreshes as a workaround. Final status: We've developed and completed deploying the fix to the affected environments, reverting the impacting update. We then reviewed our service telemetry and confirmed that the issue is now fully resolved. Scope of impact: This issue may have intermittently affected any user performing Power Platform dataflow refreshes. Start time: Tuesday, August 10, 2021, at 8:30 AM'UTC End time: Wednesday, August 11, 2021, at 12:00 PM'UTC Root cause: A recent standard service update to the Microsoft Authentication Library unexpectedly caused heavy traffic, which caused some Power Platform dataflow refresh failures. Next steps: - We're reviewing this recent service update to understand why it caused heavy traffic and better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-08-11T19:58:48.05Z
Root cause A recent standard service update to the Microsoft Authentication Library unexpectedly caused heavy traffic, which caused some Power Platform dataflow refresh failures. Next steps: - We're reviewing this recent service update to understand why it caused heavy traffic and better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-08-11T19:58:48.05Z

MM277143 Details

Status: A recent update to the Microsoft Authentication Library is unexpectedly causing dataflow refresh failures. We're developing a fix to revert the update and remediate the impact. Next update by: Wednesday, August 11, 2021, at 8:00 PM UTC; PublishedTime=2021-08-10T23:59:45.74Z

Final status: We've developed and completed deploying the fix to the affected environments, reverting the impacting update. We then reviewed our service telemetry and confirmed that the issue is now fully resolved. Next update by: Wednesday, August 11, 2021, at 8:00 PM UTC; PublishedTime=2021-08-10T23:59:45.74Z

--------------------------------------------------------------------------------------------------
Title Users were experiencing delays in file content scanning in Cloud App Security
ID CS277018
Service Cloud App Security
Feature Cloud App Security
Status Service restored
Impact Users were experiencing delays in file content scanning in Cloud App Security.
Severity
Start time 2021-08-10T05:53:00Z
End time 2021-08-10T15:00:00Z
Last update 2021-08-11T07:28:32.16Z
More info
Root cause A network authorization component misconfiguration, caused delays in file content scanning. Next steps: - We're reviewing our network authorization component configuration procedures, to help prevent similar problems in the future. This is the final update for the event.; PublishedTime=2021-08-11T07:10:35.44Z

CS277018 Details

Status: We've identified a network authorization component misconfiguration, is causing delays of up to 24 hours in file content scanning. We've deployed a fix, and are processing the backlog of file content scanning. Next update by: Wednesday, August 11, 2021, at 5:00 PM UTC; PublishedTime=2021-08-10T14:36:17.05Z

Final status: The backlog of data has been successfully processed and we can confirm the issue has been mitigated. Next update by: Wednesday, August 11, 2021, at 5:00 PM UTC; PublishedTime=2021-08-10T14:36:17.05Z

--------------------------------------------------------------------------------------------------
Title Users may be unable to add a specific third-party connector to their flows or run flows with that connector
ID MM276941
Service Power Apps in Microsoft 365
Feature Service and web access issues
Status Service restored
Impact Users may have been unable to add a specific third-party connector to their flows or run flows with that connector.
Severity
Start time 2021-08-09T23:20:00Z
End time 2021-08-10T00:05:00Z
Last update 2021-08-10T02:27:32.247Z
More info
Root cause

MM276941 Details

Status: We've investigated a problem with the Microsoft Power Apps service in which users may have been unable to add a specific third-party connector to their flows and were unable to run a flow that contained the third party connector. We've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-08-10T01:50:49.447

Final status: We've identified that a third party web and mobile application was experiencing issues. We've determined that our automated recovery system repaired this problem, and we confirmed that service was restored after monitoring the environment.

--------------------------------------------------------------------------------------------------
Title Users may be unable to add a specific third-party connector to their flows or run flows with that connector
ID MM276941
Service Power Apps in Microsoft 365
Feature Service and web access issues
Status Service restored
Impact Users may have been unable to add a specific third-party connector to their flows or run flows with that connector.
Severity
Start time 2021-08-09T23:20:00Z
End time 2021-08-10T00:05:00Z
Last update 2021-08-10T02:27:32.247Z
More info
Root cause

MM276941 Details

Status: We've investigated a problem with the Microsoft Power Apps service in which users may have been unable to add a specific third-party connector to their flows and were unable to run a flow that contained the third party connector. We've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-08-10T01:50:49.447

Final status: We've identified that a third party web and mobile application was experiencing issues. We've determined that our automated recovery system repaired this problem, and we confirmed that service was restored after monitoring the environment.

--------------------------------------------------------------------------------------------------
Title Users may experience policy update delays, device action failures, and app installs not working on iOS devices
ID IT276862
Service Microsoft Intune
Feature Microsoft Intune
Status Service restored
Impact Users may have experienced policy update delays, device action failures, and app installs not working on iOS devices.
Severity
Start time 2021-08-09T20:45:00Z
End time 2021-08-09T21:55:00Z
Last update 2021-08-09T23:37:25.683Z
More info
Root cause

IT276862 Details

Status: We've investigated a problem with the Microsoft Intune service in which users may have experienced delays in receiving policy updates, device actions may have failed, and available application installs may not have started as expected. We've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-08-09T22:50:39.39

Final status: We've determined that a third party software provider was experiencing a service degradation. While we were in the process of analyzing service logs to identify a mitigation, system monitoring indicated that the service had returned to normal health. We've confirmed that the problem is no longer occurring, and we'll continue to monitor the service to ensure that the problem does not happen again.

--------------------------------------------------------------------------------------------------
Title Admins may have seen delays with license reports in the admin center
ID MO276700
Service Microsoft 365 suite
Feature Portal
Status Service restored
Impact Admins may have seen delays with license reports in the admin center.
Severity
Start time 2021-08-07T07:00:00Z
End time 2021-08-13T00:00:00Z
Last update 2021-08-13T09:36:44.853Z
More info Impacted reports may have shown decreased activity starting from Saturday, August 7, 2021. Final status: We've replayed the outstanding delayed reports and confirmed the issue has been resolved. Scope of impact: This issue may have impacted any report that was calling license data. Start time: Saturday, August 7, 2021, at 7:00 AM UTC End time: Friday, August 13, 2021, at 12:00 AM UTC Root cause: A configuration change in our upstream data sources caused delays with some report data. Next steps: - We're reviewing our configuration change procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-08-13T09:36:44.853Z
Root cause A configuration change in our upstream data sources caused delays with some report data. Next steps: - We're reviewing our configuration change procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-08-13T09:36:44.853Z

MO276700 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-08-09T03:03:05.817

Status: We determined that a recent configuration change is unexpectedly causing upstream data delays with license report generation. We've corrected the configuration and are replaying the data to generate the delayed reports and remediated impact. Next update by: Friday, August 13, 2021, at 11:00 AMUTC; PublishedTime=2021-08-09T03:37:02.47Z

Final status: We've replayed the outstanding delayed reports and confirmed the issue has been resolved. Next update by: Friday, August 13, 2021, at 11:00 AMUTC; PublishedTime=2021-08-09T03:37:02.47Z

--------------------------------------------------------------------------------------------------
Title Users may receive an error when attempting to submit a response in Microsoft Forms
ID FM277484
Service Microsoft Forms
Feature Service
Status Service restored
Impact Users may have received an error when attempting to submit a response in Microsoft Forms.
Severity
Start time 2021-08-06T01:15:00Z
End time 2021-08-12T03:50:00Z
Last update 2021-08-12T06:08:24.1Z
More info Users may have noticed the error: "You don't have permission to view this Form" Users could've submited their response again without issue. Final status: We've discovered we enabled Web-Auth Cookies for the environment. By enabling this feature, we created a coding issue in our service. We've disabled the offending flight, which has mitigated impact. To ensure service is restored, users should refresh their webpage. Scope of impact: Any user attempting to submit a non-anonymous Microsoft Forms response was affected. Start time: Friday, August 6, 2021, at 1:15 AM'UTC End time: Thursday, August 12, 2021, at 3:50 AM'UTC Root cause: Enabling Web-Auth Cookies has inadvertently caused impact to our service. Next steps: - We're reviewing our deployment and provisioning procedures to help prevent similar problems in the future. This is the final update for the event.; PublishedTime=2021-08-12T06:05:13.063Z
Root cause Enabling Web-Auth Cookies has inadvertently caused impact to our service. Next steps: - We're reviewing our deployment and provisioning procedures to help prevent similar problems in the future. This is the final update for the event.; PublishedTime=2021-08-12T06:05:13.063Z

FM277484 Details

Status: We've investigated a problem with the Microsoft Forms service in which users may have received an error when attempting to submit a response in Microsoft Forms, and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-08-12T05:29:39.033

Final status: We've discovered we enabled Web-Auth Cookies for the environment. By enabling this feature, we created a coding issue in our service. We've disabled the offending flight, which has mitigated impact. To ensure service is restored, users should refresh their webpage.

--------------------------------------------------------------------------------------------------
Title Users intermittently are unable to navigate to folders via the SharePoint Online library
ID SP277379
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users intermittently were unable to navigate to folders via the SharePoint Online library.
Severity
Start time 2021-08-03T08:28:00Z
End time 2021-08-11T22:33:00Z
Last update 2021-08-12T18:05:57.653Z
More info Users may have noticed intermittent redirection to a level up when they attempted to access a specific folder in SharePoint Online library. Due to the intermittent nature of the issue, subsequent attempts to access the folder may have been successful. Final status: We've confirmed with some previously affected users that by reverting the recent update, impact has been remediated and the issue resolved. Scope of impact: Impact may have occurred for all users intermittently when attempting to navigate to folders via the SharePoint Online library. Start time: Tuesday, August 3, 2021, at 8:28 AM UTC End time: Wednesday, August 11, 2021, at 10:33 PM'UTC Root cause: A recent service update that introduced a regression in a portion of code related to the item selection pane within the SharePoint Online library resulted in impact. Next steps: - We're reviewing our standard service update development and testing procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-08-12T17:46:56.67Z
Root cause A recent service update that introduced a regression in a portion of code related to the item selection pane within the SharePoint Online library resulted in impact. Next steps: - We're reviewing our standard service update development and testing procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-08-12T17:46:56.67Z

SP277379 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-08-11T18:02:42.343

Status: We're investigating available diagnostic data, such as network traces, to isolate the source of impact and our troubleshooting actions. Additionally, we're attempting to internally reproduce this issue to gather additional telemetry for assessment. Next update by: Wednesday, August 11, 2021, at 8:00 PMUTC; PublishedTime=2021-08-11T18:22:06.533Z

Status: We're continuing our efforts to internally reproduce this issue in order to gather additional telemetry for assessment. Our review of currently available diagnostic data is ongoing in parallel. Next update by: Wednesday, August 11, 2021, at 10:00 PMUTC; PublishedTime=2021-08-11T19:40:40.54Z

Status: Our investigation is taking longer than anticipated. We're continuing our efforts to review current available diagnostic data, and in parallel, we're also continuing our attempt to internally reproduce this issue to isolate the source of impact. Next update by: Thursday, August 12, 2021, at 1:30 AMUTC; PublishedTime=2021-08-11T20:39:18.58Z

Status: We've identified a recent service update that introduced a regression in a portion of code related to the item selection pane within the SharePoint Online library, resulting in impact. We've reverted the update, and we're reaching out to affected users to confirm that this has remediated impact. Next update by: Thursday, August 12, 2021, at 7:00 PMUTC; PublishedTime=2021-08-11T23:58:06.927Z

Final status: We've confirmed with some previously affected users that by reverting the recent update, impact has been remediated and the issue resolved. Next update by: Thursday, August 12, 2021, at 7:00 PMUTC; PublishedTime=2021-08-11T23:58:06.927Z

--------------------------------------------------------------------------------------------------
Title Custom connector added to a DLP policy via PowerShell may be removed if policy is edited in Power Platform admin center.
ID CR275975
Service Dynamics 365 Apps
Feature Other
Status Service restored
Impact Custom connector added to a DLP policy via PowerShell may be removed if policy is edited in Power Platform admin center.
Severity
Start time 2021-08-02T14:36:00Z
End time 2021-08-06T20:25:00Z
Last update 2021-08-06T20:28:36.537Z
More info
Root cause

CR275975 Details

--------------------------------------------------------------------------------------------------
Title Admins' custom connectors on Data Loss Prevention (DLP) policies are removed when edited in Power Platform admin center
ID MO275990
Service Microsoft 365 suite
Feature Administration
Status Extended recovery
Impact Admins' custom connectors on DLP policies are removed when edited in the Power Platform admin center.
Severity Sev2
Start time 2021-08-02T13:36:00Z
End time
Last update 2021-10-14T19:30:28.56Z
More info This doesn't affect DLP policies that were created in the Power Platform admin center. Only policies that were created through the legacy PowerShell cmdlet are affected. Impact occurs in the following scenario: 1. An admin opens the DLP policies page in the Power Platform admin center in a web browser. 2. A custom connector is added to the policy using the "Add-CustomConnectorToPolicy" PowerShell cmdlet. 3. Without refreshing the policy list, the admin then edits and saves the same policy in the Power Platform admin center. 4. The previously added custom connector gets removed from the policy. We recommend avoiding the above process by instead performing DLP policy updates using the Power Platform admin center interface. Current status: Deployment of the fix is progressing as expected, and admins that are on infrastructure that has already received the fix are experiencing relief from impact. We anticipate impact to be mitigated by our next scheduled update. Scope of impact: Your organization is affected by this event, and any admin attempting to add custom connectors to PowerShell-created DLP policies in the Power Platform admin center may be affected. Start time: Monday, August 2, 2021, at 1:36 PM UTC Root cause: A latent code issue revealed when a custom connector is added to a DLP policy via the legacy PowerShell cmdlet and then edited using a cached version of the DLP policies page in the Power Platform admin center is resulting in impact. Next update by: Friday, November 5, 2021, at 9:00 PM UTC; PublishedTime=2021-10-14T19:30:28.56Z
Root cause A latent code issue revealed when a custom connector is added to a DLP policy via the legacy PowerShell cmdlet and then edited using a cached version of the DLP policies page in the Power Platform admin center is resulting in impact. Next update by: Friday, November 5, 2021, at 9:00 PM UTC; PublishedTime=2021-10-14T19:30:28.56Z

MO275990 Details

Status: We're examining service telemetry and recent service updates to determine the root cause of this issue. Next update by: Friday, August 6, 2021, at 9:00 PMUTC; PublishedTime=2021-08-06T19:24:24.82Z

Status: We've determined that a latent code issue is resulting in impact, and we're in the initial stages of developing a fix. We expect to have a deployment timeline by our next scheduled update. Next update by: Friday, August 13, 2021, at 9:00 PM UTC; PublishedTime=2021-08-06T20:48:32.397Z

Status: The process of developing a fix is ongoing and is expected to take up to two weeks to complete. Once developed, we anticipate that the deployment process will take an additional two weeks. We'll provide more accurate development and deployment timelines once available. Next update by: Friday, August 20, 2021, at 9:00 PM UTC; PublishedTime=2021-08-13T19:37:27.68Z

Status: The development of the fix for this issue is ongoing; however, the user experience fix is taking more time than expected, and could take an additional week to develop. We will have a better estimation on the deployment timeline once the development is complete in about two weeks. Next update by: Friday, September 3, 2021, at 9:00 PM UTC; PublishedTime=2021-08-20T19:28:45.4Z

Status: We've developed a fix for this issue that we are validating in an internal test environment. Once validation has completed, we expect this fix will deploy by Saturday, September 18, 2021. The user experience fix is still being developed, but we expect it will complete deployment by Saturday, September 25, 2021. We'll provide better estimations on the timeline for remediation once the validation and development of these fixes completes. Next update by: Friday, September 17, 2021, at 9:00 PM UTC; PublishedTime=2021-09-03T19:42:55.5Z

Status: Validation of the main fix is in its final stages and we expect to begin the deployment process on Monday, September 20, 2021. The fix for the user experience has completed the development and validation process, and is on track to be deployed earlier than originally anticipated, alongside the main fix. We expect both fixes will complete their deployment by Monday, October 4, 2021. We'll monitor these deployments and provide better estimations on the timeline for remediation as they progress. Next update by: Friday, October 1, 2021, at 9:00 PM UTC; PublishedTime=2021-09-17T19:24:25.543Z

Status: We've encountered an issue with services that facilitate fix deployment which are preventing the release of the fix. We're reviewing the processes and services responsible for deployment release to determine the cause of the delay. Next update by: Friday, October 8, 2021, at 9:00 PM UTC; PublishedTime=2021-10-01T19:58:18.847Z

Status: We've addressed the hinderances of deployment and are now deploying a fix for the issue. Functionality will be restored for affected users as soon as the fix reaches their environment. We're continuing to monitor the progress of the fix to ensure it completes as expected and aim to provide an estimated time to impact remediation by our next scheduled update. Next update by: Thursday, October 14, 2021, at 9:00 PM UTC; PublishedTime=2021-10-08T17:16:45.68Z

Status: Deployment of the fix is progressing as expected, and admins that are on infrastructure that has already received the fix are experiencing relief from impact. We anticipate impact to be mitigated by our next scheduled update. Next update by: Friday, November 5, 2021, at 9:00 PM UTC; PublishedTime=2021-10-14T19:30:28.56Z

--------------------------------------------------------------------------------------------------
Title Access and slowness issues with multiple Microsoft 365 services
ID MO275087
Service Microsoft 365 suite
Feature Portal
Status Service restored
Impact Users may have experienced access issues or latency when utilizing multiple Microsoft 365 services and features.
Severity
Start time 2021-08-02T10:19:00Z
End time 2021-08-02T17:29:00Z
Last update 2021-08-02T18:05:45.847Z
More info Impacted services included but were not limited to: - Microsoft Teams - Admin center Final status: We've confirmed that a recent networking policy change resulted in networking traffic being routed incorrectly, causing impact. We've confirmed that automated recovery systems corrected the traffic routing issue to mitigate impact. In parallel, we've updated the networking policy to ensure impact is fully remediated. Scope of impact: Impact was specific to a subset of users who were served through the affected infrastructure. Start time: Monday, August 2, 2021, at 10:19 AM'UTC End time: Monday, August 2, 2021, at 5:29 PM'UTC Root cause: A recent networking policy change resulted in networking traffic being routed incorrectly, causing impact. Next steps: - We're reviewing our policy update procedures to identify opportunities for improvement to better prevent similar future impact. This is the final update for the event.; PublishedTime=2021-08-02T18:05:45.847Z
Root cause A recent networking policy change resulted in networking traffic being routed incorrectly, causing impact. Next steps: - We're reviewing our policy update procedures to identify opportunities for improvement to better prevent similar future impact. This is the final update for the event.; PublishedTime=2021-08-02T18:05:45.847Z

MO275087 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-08-02T16:58:56.033

Status: We're reviewing system diagnostics to identify the source of the issue. Initial indications are that the service has recovered; however, we're continuing to investigate. Next update by: Monday, August 2, 2021, at 6:30 PM UTC; PublishedTime=2021-08-02T17:20:59.55Z

Final status: We've confirmed that a recent networking policy change resulted in networking traffic being routed incorrectly, causing impact. We've confirmed that automated recovery systems corrected the traffic routing issue to mitigate impact. In parallel, we've updated the networking policy to ensure impact is fully remediated. Next update by: Monday, August 2, 2021, at 6:30 PM UTC; PublishedTime=2021-08-02T17:20:59.55Z

--------------------------------------------------------------------------------------------------
Title Some users were unable to use the "Approvals" application within Microsoft Teams
ID TM275074
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were unable to use the "Approvals" application within Microsoft Teams.
Severity
Start time 2021-08-02T08:00:00Z
End time 2021-08-02T12:26:00Z
Last update 2021-08-02T13:51:26.983Z
More info Impacted users may have seen a 404 error code. Final status: We've identified a recent build, intended to update the "Approvals" application, did not complete successfully within the affected infrastructure. We've redeployed the build and confirmed the issue is resolved among previously affected users. Scope of impact: Impact was specific to some users served through the affected infrastructure. Start time: Monday, August 2, 2021, at 8:00 AM UTC End time: Monday, August 2, 2021, at 12:26 PM UTC Root cause: A recent build intended to update the "Approvals" application did not complete within the affected infrastructure. Next steps: - We're reviewing our deployment and provisioning procedures to help prevent similar problems in the future. This is the final update for the event.; PublishedTime=2021-08-02T13:49:02.6Z
Root cause A recent build intended to update the "Approvals" application did not complete within the affected infrastructure. Next steps: - We're reviewing our deployment and provisioning procedures to help prevent similar problems in the future. This is the final update for the event.; PublishedTime=2021-08-02T13:49:02.6Z

TM275074 Details

Status: We're analyzing network trace logs to determine the source of the issue. Next update by: Monday, August 2, 2021, at 2:30 PM UTC; PublishedTime=2021-08-02T12:38:42.333Z

Final status: We've identified a recent build, intended to update the "Approvals" application, did not complete successfully within the affected infrastructure. We've redeployed the build and confirmed the issue is resolved among previously affected users. Next update by: Monday, August 2, 2021, at 2:30 PM UTC; PublishedTime=2021-08-02T12:38:42.333Z

--------------------------------------------------------------------------------------------------
Title Errors when connecting to Power Apps
ID MO274372
Service Microsoft 365 suite
Feature Portal
Status Post-incident report published
Impact Users may have experienced connectivity issues with Power Apps connections.
Severity
Start time 2021-07-30T21:27:00Z
End time 2021-07-30T23:59:00Z
Last update 2021-08-06T15:22:01.353Z
More info Users may have received different error messages depending on the scenario and connection type being leveraged. Most error messages would've indicated that the specified record or connector was not found. In some scenarios, users may have received 404 or 500 errors when attempting to connect Power Apps with SharePoint Online. Final status: Our investigation determined that a recent deployment update to a portion of infrastructure responsible for routing traffic contained a misconfiguration which was causing connector errors. We've successfully rerouted traffic through alternate infrastructure which has restored connections to all previously impacted customers. Scope of impact: This issue could have affected any user with PowerApps connections deployed if environment was hosted in Europe or North America region. Start time: Friday, July 30, 2021, at 9:27 PM UTC End time: Friday, July 30, 2021, at 11:59 PM UTC Root cause: A change to the Power Platform connector infrastructure in Europe and United States regions inadvertently corrupted the Application Programming Interface (API) Management policies for all Power Platform connector requests. This resulted in user request failures for Power Apps and Power Automate. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. We'll publish a post-incident report within five business days.; PublishedTime=2021-07-31T00:23:22.577Z
Root cause A change to the Power Platform connector infrastructure in Europe and United States regions inadvertently corrupted the Application Programming Interface (API) Management policies for all Power Platform connector requests. This resulted in user request failures for Power Apps and Power Automate. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. We'll publish a post-incident report within five business days.; PublishedTime=2021-07-31T00:23:22.577Z

MO274372 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-07-30T22:38:57.547

Status: We're reviewing diagnostic data to better understand the issue and determine our next troubleshooting steps. Next update by: Saturday, July 31, 2021, at 12:00 AM UTC; PublishedTime=2021-07-30T22:55:31.277Next update by: Saturday, July 31, 2021, at 12:00 AM UTC; PublishedTime=2021-07-30T22:55:31.277Z

Status: We're continuing to narrow down the root cause of the impact by analyzing recent deployments. Concurrently, we are evaluating troubleshooting options to help mitigate impact more quickly. Next update by: Saturday, July 31, 2021, at 2:00 AMUTC; PublishedTime=2021-07-30T23:34:12.2Z

Final status: Our investigation determined that a recent deployment update to a portion of infrastructure responsible for routing traffic contained a misconfiguration which was causing connector errors. We've successfully rerouted traffic through alternate infrastructure which has restored connections to all previously impacted customers. Next update by: Saturday, July 31, 2021, at 2:00 AMUTC; PublishedTime=2021-07-30T23:34:12.2Z

--------------------------------------------------------------------------------------------------
Title Users can't sync profiles created before July 30, 2021, in School Data Sync if they're hosted in specific regions
ID OE276115
Service School Data Sync
Feature School Data Sync
Status Service restored
Impact Users couldn't sync profiles created before July 30, 2021, in School Data Sync if they were hosted in specific regions.
Severity
Start time 2021-07-30T19:00:00Z
End time 2021-08-10T05:00:00Z
Last update 2021-08-10T18:16:16.473Z
More info This issue only affected users hosted within the United States or Europe. Final status: We've confirmed that our fix deployment has completed, and that impact has been remediated. Scope of impact: This issue could have potentially affected any of your users trying to sync profiles if created before Friday, July 30, 2021, and if they were hosted within the United States or Europe. Start time: Friday, July 30, 2021, at 7:00 PM'UTC End time: Tuesday, August 10, 2021, at 5:00 AM'UTC Root cause: An authentication issue in the sync service for customer data operations before Friday, July 30, 2021, was resulting in impact. Next steps: - We're reviewing our customer data service sync procedures to find ways to prevent this problem from happening again. - We're reviewing our monitoring services to look for ways to reduce detection time and more quickly restore service. This is the final update for the event.; PublishedTime=2021-08-10T18:16:16.473Z
Root cause An authentication issue in the sync service for customer data operations before Friday, July 30, 2021, was resulting in impact. Next steps: - We're reviewing our customer data service sync procedures to find ways to prevent this problem from happening again. - We're reviewing our monitoring services to look for ways to reduce detection time and more quickly restore service. This is the final update for the event.; PublishedTime=2021-08-10T18:16:16.473Z

OE276115 Details

Status: We've received reports that users are unable to sync profiles created before July 30, 2021 if they're hosted in the United States or Europe. We've identified an authentication issue in the sync service for customer data operations that's causing impact. We're determining our potential mitigation options and we'll provide details by our next scheduled update. Next update by: Saturday, August 7, 2021, at 10:00 PMUTC; PublishedTime=2021-08-07T00:23:06.427Z

Status: We've deployed a fix to repair the affected accounts and we're monitoring its progress to ensure it successfully alleviates the problem. We suspect that we'll be able to provide an estimated timeline for completion by our next scheduled update. Next update by: Sunday, August 8, 2021, at 6:30 PM UTC; PublishedTime=2021-08-07T20:04:32.537Z

Status: We've encountered some issues which have delayed the deployment process. We're now working on resolving those issues prior to continuing deployment of the fix to repair the affected accounts, which we anticipate will mitigate impact. Once those issues are resolved, we'll be able to provide an estimated timeline for completion. Next update by: Monday, August 9, 2021, at 6:00 AMUTC; PublishedTime=2021-08-08T17:15:07.927Z

Status: We resolved the issues with the fix deployment. We can see via monitoring that the fix deployment has finished deploying to some regions. We estimate that the deployment will fully complete by end of day on Monday, August 9, 2021. Next update by: Monday, August 9, 2021, at 8:30 PMUTC; PublishedTime=2021-08-09T04:54:29.61Z

Status: We're monitoring the final aspects of our fix deployment, which continues to progress as expected. We're expecting for the deployment to have completed and for impact to be resolved by Monday, August 9, 2021, at 11:30 PM UTC. Next update by: Monday, August 9, 2021, at 11:30 PMUTC; PublishedTime=2021-08-09T17:48:14.857Z

Status: The fix has completed deployment to the United States region but is taking longer than expected to complete in other regions. We're continuing to monitor the fix as it progresses and expect complete mitigation by the next scheduled update. Next update by: Tuesday, August 10, 2021, at 5:00 AMUTC; PublishedTime=2021-08-09T21:58:04.657Z

Status: The impact is resolved for users in the United States and most of Europe. The deployment is still running, we expect to have full mitigation by our next scheduled communication update. Next update by: Tuesday, August 10, 2021, at 8:00 PM UTC; PublishedTime=2021-08-10T03:31:23.073Z

Final status: We've confirmed that our fix deployment has completed, and that impact has been remediated. Next update by: Tuesday, August 10, 2021, at 8:00 PM UTC; PublishedTime=2021-08-10T03:31:23.073Z

--------------------------------------------------------------------------------------------------
Title Users' Windows devices are unable to download their Autopilot profiles during Out Of Box Experience (OOBE)
ID IT274426
Service Microsoft Intune
Feature Microsoft Intune
Status Service restored
Impact Users' Windows devices were unable to download their Autopilot profiles during OOBE.
Severity
Start time 2021-07-30T18:00:00Z
End time 2021-07-31T00:45:00Z
Last update 2021-07-31T02:09:22.187Z
More info
Root cause A portion of the infrastructure that facilitates Autopilot downloading was performing below manageable thresholds, resulting in impact. Next steps: - We're investigating why the affected infrastructure's performance was degraded to better understand how to prevent this issue from reoccurring. This is the final update for the event.; PublishedTime=2021-07-31T02:09:22.187Z

IT274426 Details

Status: We've received reports were users' Windows devices are unable to download their Autopilot profiles during OOBE. We're investigating diagnostic logs to determine the underlying cause of the issue. Next update by: Saturday, July 31, 2021, at 2:00 AMUTC; PublishedTime=2021-07-31T00:58:21.483Z

Status: We determined that a portion of the infrastructure that facilitates Autopilot downloading is performing below manageable thresholds, resulting in impact. We've restarted the affected infrastructure and our telemetry as well as our internal testing indicates that the impact is mitigated. We're reaching out to affected users to confirm mitigation. Next update by: Saturday, July 31, 2021, at 3:00 AMUTC; PublishedTime=2021-07-31T01:55:05.26Z

Final status: We're confirmed with several affected users that the mitigation steps we performed successfully remediated impact. Next update by: Saturday, July 31, 2021, at 3:00 AMUTC; PublishedTime=2021-07-31T01:55:05.26Z

--------------------------------------------------------------------------------------------------
Title Some users may be experiencing issues joining or initiating Live Events within Microsoft Teams
ID TM274264
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may experience issues joining or initiating Live Events within Microsoft Teams.
Severity
Start time 2021-07-30T13:52:08Z
End time 2021-07-30T14:30:00Z
Last update 2021-07-30T16:26:17.327Z
More info
Root cause A code issue unexpectedly caused some applications that are required for Microsoft Teams live events to be disabled, resulting in impact. Next steps: - We're further investigating the source of the code issue to determine whether a long-term solution is required. This is the final update for the event.; PublishedTime=2021-07-30T16:09:51.887Z

TM274264 Details

Status: We're analyzing system telemetry to determine the source of the issue. Next update by: Friday, July 30, 2021, at 4:00 PM UTC; PublishedTime=2021-07-30T13:54:25.98Z

Final status: Our investigation determined that a code issue unexpectedly caused some applications that are required for Microsoft Teams live events to be disabled. We re-enabled these applications and confirmed via telemetry that impact was mitigated. Next update by: Friday, July 30, 2021, at 4:00 PM UTC; PublishedTime=2021-07-30T13:54:25.98Z

--------------------------------------------------------------------------------------------------
Title Users' flows fail to run when they have a "when email arrives" trigger enabled
ID MM275110
Service Power Apps in Microsoft 365
Feature Service and web access issues
Status Service restored
Impact Users' flows failed to run when they had a "when email arrives" trigger enabled.
Severity
Start time 2021-07-30T12:02:00Z
End time 2021-08-05T00:34:00Z
Last update 2021-08-05T01:00:01.987Z
More info Affected flows will be labeled as "Skipped' in the Status column of the My flows page, these flows my also result in an error message in the Checks section of the flow or Power Automate maker portal. As a workaround, users can disable and re-enable impacted flows in the flow or Power Automate maker portal to restore functionality. Current status: We've completed internally testing and validating the fix and have begun its deployment. We anticipate that the fix will be fully saturated and impact fully remediated by the next scheduled update. Scope of impact: Your organization is affected by this event, and this affects any user with a flow that includes an enabled "when email arrives" trigger. Start time: Friday, July 30, 2021, at 12:02 PM'UTC Root cause: A recent update to a portion of infrastructure responsible for routing traffic contains a misconfiguration that's preventing flows from running when they leverage a 'when email arrives' trigger. Next update by: Thursday, August 5, 2021, at 2:30 AMUTC; PublishedTime=2021-08-04T16:04:18.1Z
Root cause A recent update to a portion of infrastructure responsible for routing traffic contained a misconfiguration that prevented flows from running when they leverage a 'when email arrives' trigger. Next steps: - We're reviewing our automated alerting services to determine how we can identify and remediate similar issues in a more timely fashion. - We're reviewing the recent service update to determine how this was missed during our validation procedures to prevent issues from deploying and affecting users. This is the final update for the event.; PublishedTime=2021-08-05T01:00:01.987Z

MM275110 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We've confirmed this issue was caused due to a recent update to a portion of infrastructure responsible for routing traffic contained a misconfiguration which resulted in errors. We're reviewing mitigation options including a potential code fix to determine the swiftest path to remediation. Next update by: Tuesday, August 3, 2021, at 5:30 PMUTC; PublishedTime=2021-08-02T20:37:01.8Z

Status: We've determined that a code fix deployment will be the most expedient method for remediating impact. We're completing the internal testing and validation of our code fix, which is expected to begin deployment shortly. We're expecting to be able to provide an update on the fix deployment and the remediation of impact by Wednesday, August 4, 2021. Next update by: Wednesday, August 4, 2021, at 5:00 PMUTC; PublishedTime=2021-08-03T16:00:38.813Z

Status: We've completed internally testing and validating the fix and have begun its deployment. We anticipate that the fix will be fully saturated and impact fully remediated by the next scheduled update. Next update by: Thursday, August 5, 2021, at 2:30 AMUTC; PublishedTime=2021-08-04T16:04:18.1Z

Final status: We've confirmed that the deployment of the fix has completed and impact has been remediated. Next update by: Thursday, August 5, 2021, at 2:30 AMUTC; PublishedTime=2021-08-04T16:04:18.1Z

--------------------------------------------------------------------------------------------------
Title Microsoft Defender for Office 365 admins can't run Attack simulation training in Microsoft Defender Security Center
ID MO274281
Service Microsoft 365 suite
Feature Portal
Status Service restored
Impact Admins experienced failures and were unable to run Attack simulation training in the Microsoft Defender Security Center.
Severity
Start time 2021-07-30T12:00:00Z
End time 2021-07-30T14:00:00Z
Last update 2021-07-30T16:26:35.85Z
More info
Root cause A recent Azure AD configuration update changed the logic in the way inventory is read from databases, resulting in impact. Next steps: - We're reviewing our Azure AD configuration validation procedures to isolate and correct similar issues in the future. This is the final update for the event.; PublishedTime=2021-07-30T16:26:35.85Z

MO274281 Details

Status: We've investigated a problem with the Microsoft Defender for Office 365 service in which admins may have experienced failures when running Attack simulation training, and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-07-30T16:02:27.823

Final status: Our monitoring identified that a recent Azure AD configuration update changed the logic in the way inventory is read from databases, resulting in impact. We've corrected the configuration and verified impact mitigation via telemetry.

--------------------------------------------------------------------------------------------------
Title Users can't see a specific third-party social media web part in SharePoint Online modern pages
ID SP274278
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users were unable to see a specific third-party social media web part in SharePoint Online modern pages.
Severity
Start time 2021-07-29T23:33:00Z
End time 2021-08-02T11:50:00Z
Last update 2021-08-02T17:26:39.55Z
More info
Root cause A code regression occurred within the Microsoft-managed environment, specific to the embedded code path of the web part, when the third-party social media web part provider deployed an update. Next steps: - We're reviewing our third-party social media web part procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-08-02T17:26:39.55Z

SP274278 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-07-30T15:57:22.247

Status: We're analyzing networking traces and Unified Logging Service (ULS) data provided by affected users to help us isolate the source of the issue. Next update by: Friday, July 30, 2021, at 5:30 PMUTC; PublishedTime=2021-07-30T16:08:19.53Z

Status: We're investigating the embedded code path which supports the affected functionality to help us further isolate the source of the problem and determine a mitigation strategy. Next update by: Friday, July 30, 2021, at 7:30 PMUTC; PublishedTime=2021-07-30T17:12:32.807Z

Status: We've identified that a code regression occurred within the Microsoft-managed environment when the third-party social media web part provider deployed an update. We're developing and internally validating a targeted code fix for this. We're also reaching out to the third-party social media provider to help us better understand the context of the recent changes to aid our mitigation efforts. Next update by: Saturday, July 31, 2021, at 1:30 AMUTC; PublishedTime=2021-07-30T18:31:28.807Z

Status: We've completed development of our fix, and are our validation efforts are proceeding to ensure it mitigates the issue as expected. In parallel, we're continuing to work with the third-party social media provider regarding their recent changes for additional support in our mitigation efforts. Next update by: Tuesday, August 3, 2021, at 1:00 AMUTC; PublishedTime=2021-07-30T23:30:01.947Z

Final status: Following our internal testing and validation, we've confirmed with affected users that the deployment of our code fix has successfully remediated impact. Next update by: Tuesday, August 3, 2021, at 1:00 AMUTC; PublishedTime=2021-07-30T23:30:01.947Z

--------------------------------------------------------------------------------------------------
Title Can't perform billing purchase and manage capabilities within the Microsoft 365 admin center
ID MO274017
Service Microsoft 365 suite
Feature Purchase and Billing
Status Service restored
Impact Admins may have been unable to perform billing purchase and manage capabilities within the Microsoft 365 admin center.
Severity
Start time 2021-07-29T08:45:00Z
End time 2021-07-29T15:40:00Z
Last update 2021-07-29T18:58:45.81Z
More info
Root cause The database infrastructure upon which the invoice service was located wasn't working as expected, causing impact. Next steps: - We're reviewing this portion of affected infrastructure to understand why it stopped working as expected to prevent similar impact from reoccurring in the future. This is the final update for the event.; PublishedTime=2021-07-29T17:59:13.557Z

MO274017 Details

Status: We're analyzing system logs to determine the source of the issue. Next update by: Thursday, July 29, 2021, at 4:00 PM UTC; PublishedTime=2021-07-29T13:55:56.06Z

Status: We've determined that the database infrastructure upon which the invoice service is located isn't working as expected, causing impact. We've rerouted traffic and restarted the service. We're now monitoring service telemetry to ensure that impact is fully mitigated. Next update by: Thursday, July 29, 2021, at 6:00 PMUTC; PublishedTime=2021-07-29T15:26:01.18Z

Final status: After monitoring and reviewing service telemetry, we've confirmed that rerouting traffic and restarting the impacted infrastructure has mitigated impact. Next update by: Thursday, July 29, 2021, at 6:00 PMUTC; PublishedTime=2021-07-29T15:26:01.18Z

--------------------------------------------------------------------------------------------------
Title Unable to access Microsoft 365 services
ID MO273940
Service Microsoft 365 suite
Feature Portal
Status Service restored
Impact Users may have been unable to access one or more Microsoft 365 services or experienced degraded functionality.
Severity
Start time 2021-07-29T06:15:00Z
End time 2021-07-29T07:00:00Z
Last update 2021-07-29T15:24:52.293Z
More info Services impacted may have included, but were not limited to: - Microsoft Teams - Skype for Business - Exchange Online (Outlook Desktop and Outlook on the web) - Microsoft 365 admin center - SharePoint Online - OneDrive for Business Final status: We've confirmed that redirecting user traffic to alternative channels has been successful in resolving the issue. Scope of impact: Impact was specific to users served through the affected third-party Internet Service Provider (ISP) infrastructure in Germany. Start time: Thursday, July 29, 2021, at 6:15 AM UTC End time: Thursday, July 29, 2021, at 7:00 AM UTC Root cause: A third-party Internet Service Provider (ISP) link was not processing user traffic to Microsoft services as expected. Next steps: - We're following up with our third party ISP provider to determine the root cause for the issue and to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-07-29T09:20:56.13Z
Root cause A third-party Internet Service Provider (ISP) link was not processing user traffic to Microsoft services as expected. Next steps: - We're following up with our third party ISP provider to determine the root cause for the issue and to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-07-29T09:20:56.13Z

MO273940 Details

Status: We're investigating reports of a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're reviewing service telemetry to isolate the root cause of impact and determine our next troubleshooting steps. Next update by: Thursday, July 29, 2021, at 9:30 AM UTC; PublishedTime=2021-07-29T07:23:34.293Z

Final status: We've confirmed that redirecting user traffic to alternative channels has been successful in resolving the issue. Next update by: Thursday, July 29, 2021, at 9:30 AM UTC; PublishedTime=2021-07-29T07:23:34.293Z

--------------------------------------------------------------------------------------------------
Title Some admins may have been unable to see auditing logs for the Microsoft Forms service
ID FM274212
Service Microsoft Forms
Feature Service
Status Service restored
Impact Admins may have been unable to see auditing logs for the Microsoft Forms service.
Severity
Start time 2021-07-29T03:42:00Z
End time 2021-07-30T08:04:00Z
Last update 2021-07-30T10:42:51.273Z
More info Admins may see missing logs from Thursday, July 29, 2021, at 3:42 AM UTC to Friday, July 30, 2021, at 8:04 AM UTC. Final status: We've identified a service setting change, in the infrastructure responsible for recording auditing logs, may have caused admins to be unable to see auditing logs for the Microsoft Forms service. We've run commands to revert the change and have confirmed the logs auditing service has been restored, after monitoring the affected environment. Scope of impact: Your organization was affected by this event and impacted admins attempting to view auditing logs for the Microsoft Forms service. Start time: Thursday, July 29, 2021, at 3:42 AM UTC End time: Friday, July 30, 2021, at 8:04 AM UTC Root cause: A service setting change, in the infrastructure responsible for recording auditing logs, may have caused admins to be unable to see auditing logs for the Microsoft Forms service. Next steps: - We're reviewing our standard service update procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-07-30T10:42:51.273Z
Root cause A service setting change, in the infrastructure responsible for recording auditing logs, may have caused admins to be unable to see auditing logs for the Microsoft Forms service. Next steps: - We're reviewing our standard service update procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-07-30T10:42:51.273Z

FM274212 Details

Status: We've investigated a problem with the Microsoft Forms service in which users may have been unable to see auditing logs for the Microsoft Forms service, and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-07-30T09:53:02.577

Final status: We've identified a service setting change, in the infrastructure responsible for recording auditing logs, may have caused admins to be unable to see auditing logs for the Microsoft Forms service. We've run commands to revert the change and have confirmed the logs auditing service has been restored, after monitoring the affected environment.

--------------------------------------------------------------------------------------------------
Title Admins were unable to access "Managed app" configuration policies in the Microsoft Endpoint Manager (MEM) portal
ID IT273980
Service Microsoft Intune
Feature Microsoft Intune
Status Service restored
Impact Admins were unable to access "Managed app" configuration policies in the MEM portal.
Severity
Start time 2021-07-28T22:51:00Z
End time 2021-07-29T13:15:00Z
Last update 2021-07-29T13:45:10.457Z
More info
Root cause A recent flight contained a code regression, resulting in the infrastructure responsible for app configuration policies to perform unexpectedly. Next steps: - We're reviewing our deployment procedures to help prevent similar problems in the future. This is the final update for the event.; PublishedTime=2021-07-29T13:45:10.457Z

IT273980 Details

Status: We're analyzing diagnostic data to determine the source of the issue. Next update by: Thursday, July 29, 2021, at 12:00 PM UTC; PublishedTime=2021-07-29T09:53:42.183Z

Status: We've determined a recent flight contains a coding issue, resulting in the infrastructure responsible for app configuration policies to perform unexpectedly. We're developing a fix to resolve this issue. Next update by: Thursday, July 29, 2021, at 2:00 PM UTC; PublishedTime=2021-07-29T11:47:04.58Z

Final status: We've confirmed the fix has been successfully deployed to the affected environment. We've validated the issue has been resolved after testing with previously affected users. Next update by: Thursday, July 29, 2021, at 2:00 PM UTC; PublishedTime=2021-07-29T11:47:04.58Z

--------------------------------------------------------------------------------------------------
Title North America users' activity feed notifications intermittently delayed
ID TM273735
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users' activity feed notifications may have been intermittently delayed.
Severity
Start time 2021-07-28T14:00:00Z
End time 2021-07-28T23:20:00Z
Last update 2021-07-29T10:10:34.82Z
More info Some users may have not received activity feed notifications, including missed call notifications, for more than one hour. Final status: We confirmed that impact was caused by an unexpected spike in user traffic, which resulted in the service throttling the notification queues. We've confirmed that our rebalancing efforts have successfully drained the notification queues and impact has been remediated. Scope of impact: Any user serviced through the affected infrastructure may have been intermittently impacted. Start time: Wednesday, July 28, 2021, at 2:00 PM UTC End time: Wednesday, July 28, 2021, at 11:20 PM UTC Root cause: An unexpected spike in user traffic, which resulted in the service throttling the notification queues. Next steps: - We're running service optimization commands to improve service performance and to prevent this issue from happening again. This is the final update for the event.; PublishedTime=2021-07-29T09:59:21.463Z
Root cause An unexpected spike in user traffic, which resulted in the service throttling the notification queues. Next steps: - We're running service optimization commands to improve service performance and to prevent this issue from happening again. This is the final update for the event.; PublishedTime=2021-07-29T09:59:21.463Z

TM273735 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-07-28T15:21:31.313

Status: We're comparing this event to the prior incident to confirm the root cause and determine why those mitigation actions were ineffective. Next update by: Wednesday, July 28, 2021, at 5:00 PMUTC; PublishedTime=2021-07-28T15:43:54.307Z

Status: We've confirmed that the previous mitigation was effective, and the faulty networking infrastructure hasn't been inadvertently re-added into service. We're checking the remaining network infrastructure to isolate the underlying cause of impact and develop mitigation actions. Next update by: Wednesday, July 28, 2021, at 7:00 PMUTC; PublishedTime=2021-07-28T16:45:51.54Z

Status: As a cautionary measure, we've rebalanced load of the infrastructure that facilitates notification services and telemetry is showing an improvement in processing times. We're monitoring as service is restored while in parallel continuing our review of network devices to determine where the connection is being interrupted and causing timeouts and queuing of notifications. Next update by: Wednesday, July 28, 2021, at 9:00 PMUTC; PublishedTime=2021-07-28T18:34:21.6Z

Status: Telemetry has confirmed that the rebalancing efforts have relieved some pressure and we're monitoring as the queued notifications are processed. Additionally, our investigation indicates that this isn't a network issue as initially thought and we're turning our investigation to review downstream processes which are being queued which we expect are causing impact. As this issue has reoccurred once already we'll be monitoring telemetry trends to gather additional insight into the cause and develop mitigation actions. Next update by: Thursday, July 29, 2021, at 10:00 AMUTC; PublishedTime=2021-07-28T20:39:41.64Z

Final status: We confirmed that impact was caused by an unexpected spike in user traffic, which resulted in the service throttling the notification queues. We've confirmed that our rebalancing efforts have successfully drained the notification queues and impact has been remediated. Next update by: Thursday, July 29, 2021, at 10:00 AMUTC; PublishedTime=2021-07-28T20:39:41.64Z

--------------------------------------------------------------------------------------------------
Title Users in Brazil can't access or experience degraded functionality in some Microsoft 365 services
ID MO273734
Service Microsoft 365 suite
Feature Portal
Status Service restored
Impact Users may have been unable to access or experience degraded functionality in some Microsoft 365 services.
Severity
Start time 2021-07-28T13:48:00Z
End time 2021-07-28T15:20:00Z
Last update 2021-07-28T17:20:30.75Z
More info Some affected impact scenarios may have included, but weren't limited to, the following: Users may have been unable to access their Exchange Online mailboxes via any connection method. Users may have experienced message and notification delivery delays in Microsoft Teams. Users may have also been unable to update their presence or may have seen stale presence information for others. Users may have been intermittently unable to access documents via Office for the web. Users may have also have been unable to sign in, navigate sites, download, or upload files in SharePoint Online. Final status: We've identified a section of traffic management networking infrastructure which wasn't processing traffic as efficiently as expected due to a network misconfiguration which was revealed by a recent routine service update, resulting in impact across multiple Microsoft 365 services. We've rerouted the affected traffic across alternative sections of networking infrastructure, and we've confirmed after monitoring network telemetry that the impact is now mitigated. Scope of impact: Any user served through the affected section of infrastructure in Brazil may have been impacted. Start time: Wednesday, July 28, 2021, at 1:48 PM UTC End time: Wednesday, July 28, 2021, at 3:20 PM UTC Root cause: A section of traffic management networking infrastructure wasn't processing traffic as efficiently as expected due to a network misconfiguration which was revealed by a recent routine service update, resulting in impact across multiple Microsoft 365 services. Next steps: - We're continuing our investigation of the impacted section of traffic management networking infrastructure to help us better understand the context of the issue and prevent this impact from reoccurring. This is the final update for the event.; PublishedTime=2021-07-28T16:00:54.707Z
Root cause A section of traffic management networking infrastructure wasn't processing traffic as efficiently as expected due to a network misconfiguration which was revealed by a recent routine service update, resulting in impact across multiple Microsoft 365 services. Next steps: - We're continuing our investigation of the impacted section of traffic management networking infrastructure to help us better understand the context of the issue and prevent this impact from reoccurring. This is the final update for the event.; PublishedTime=2021-07-28T16:00:54.707Z

MO273734 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-07-28T15:01:50.747

Status: We've identified a possible networking issue which is causing impact to multiple Microsoft 365 services. We're investigating the corresponding portion of infrastructure to isolate the cause of the issue and determine the necessary mitigation actions. Next update by: Wednesday, July 28, 2021, at 4:30 PMUTC; PublishedTime=2021-07-28T15:32:02.51Z

Final status: We've identified a section of traffic management networking infrastructure which wasn't processing traffic as efficiently as expected due to a network misconfiguration which was revealed by a recent routine service update, resulting in impact across multiple Microsoft 365 services. We've rerouted the affected traffic across alternative sections of networking infrastructure, and we've confirmed after monitoring network telemetry that the impact is now mitigated. Next update by: Wednesday, July 28, 2021, at 4:30 PMUTC; PublishedTime=2021-07-28T15:32:02.51Z

--------------------------------------------------------------------------------------------------
Title Admins may notice an unexpected decrease in app discovery report and policy alert data
ID CS275779
Service Cloud App Security
Feature Cloud App Security Discovery
Status Service restored
Impact Admins may notice an unexpected decrease in app discovery report and policy alert data.
Severity
Start time 2021-07-28T13:00:00Z
End time 2021-08-06T06:34:41Z
Last update 2021-08-06T06:35:09.603Z
More info This impacted data streams from Microsoft Defender for Endpoint in the Cloud Discovery dashboard. Final status: We confirmed that the fix deployment completed and monitored the environment to verify that the service was restored. Scope of impact: Your organization was affected by this event, and all admins attempting to view app discovery report and policy alert data in the Cloud Discovery dashboard may have experienced impact. Start time: Wednesday, July 28, 2021, at 1:00 PM UTC End time: Friday, August 6, 2021, at 3:00 AM UTC Root cause: A recent update contained a configuration meant to distinguish different sets of Microsoft Defender for Endpoint event data, which allowed them to be routed to various services. This configuration contained an error that caused a conflict between two routing components, which resulted in the event data not reaching the Cloud Discovery dashboard. Next steps: -We're continuing our analysis of the routing configuration to help better understand how the events are routed to the different services which will assist us in preventing similar incidents in the future. This is the final update for the event.; PublishedTime=2021-08-06T06:35:09.603Z
Root cause A recent update contained a configuration meant to distinguish different sets of Microsoft Defender for Endpoint event data, which allowed them to be routed to various services. This configuration contained an error that caused a conflict between two routing components, which resulted in the event data not reaching the Cloud Discovery dashboard. Next steps: -We're continuing our analysis of the routing configuration to help better understand how the events are routed to the different services which will assist us in preventing similar incidents in the future. This is the final update for the event.; PublishedTime=2021-08-06T06:35:09.603Z

CS275779 Details

Status: We're analyzing the data stream from Microsoft Defender for Endpoint to determine the cause of this unexpected reduction in report and policy alert data. Next update by: Thursday, August 5, 2021, at 9:00 PMUTC; PublishedTime=2021-08-05T18:45:55.86Z

Status: We've identified that a recent update contained a configuration meant to distinguish different sets of Microsoft Defender for Endpoint event data, allowing them to be routed to various services. This configuration contained an error that caused a conflict between two routing components, resulting in the event data not reaching the Cloud Discovery dashboard. We're attempting to remove one of the routing components to prevent the conflict from occurring, allowing Microsoft Defender for Endpoint event data to reach the Cloud Discovery dashboard. Next update by: Thursday, August 5, 2021, at 9:30 PM UTC; PublishedTime=2021-08-05T19:35:13.377Z

Status: We're continuing to deploy a fix that will remove one of the offending routing components, allowing Microsoft Defender for Endpoint event data to reach the Cloud Discovery dashboard and remediating impact. Next update by: Thursday, August 5, 2021, at 10:30 PMUTC; PublishedTime=2021-08-05T21:15:39.207Z

Status: The fix has completed deployment and we're monitoring the environment to confirm that impact has been completely remediated. Some admins may gradually experience impact remediation as the fix is picked up by their environment. Next update by: Friday, August 6, 2021, at 8:00 AMUTC; PublishedTime=2021-08-05T22:28:08.083Z

Final status: We confirmed that the fix deployment completed and monitored the environment to verify that the service was restored. Next update by: Friday, August 6, 2021, at 8:00 AMUTC; PublishedTime=2021-08-05T22:28:08.083Z

--------------------------------------------------------------------------------------------------
Title Admins will see duplicate events when searching via the Microsoft 365 Suite service
ID MO278847
Service Microsoft 365 suite
Feature Portal
Status Service restored
Impact Admins may have seen duplicate events when searching via the Microsoft 365 Suite service.
Severity
Start time 2021-07-28T00:00:00Z
End time 2021-07-29T00:00:00Z
Last update 2021-08-18T04:08:04.123Z
More info Admins would have seen consistent duplicate events when searching in the Microsoft 365 Audit Log Search and Office 365 Management Activity API. This issue only duplicated events from July 28, 2021 12:00 AM UTC to July 29, 2021 12:00 AM UTC. Admins may have seen duplicate events for the following services: - Exchange - Microsoft Teams - SharePoint - OneDrive - OneDrive For Business Final status: A recent configuration issue in a service update unexpectedly caused events sent to audit to get duplicated. We reverted the update to remediate the impact. Scope of impact: This issue may have potentially affected any admins searching in the Microsoft 365 Audit Log Search and Office Activity API during the affected time period. Start time: Wednesday, July 28, 2021, at 12:00 AM UTC End time: Thursday, July 29, 2021, at 12:00 AM UTC Root cause: A configuration issue in a recent service update caused duplicate events to appear in the Microsoft 365 Audit Log Search and Office 365 Management Activity API. Next steps: -We're reviewing our service update procedures to better catch similar potential configuration issues prior to their deployment. This is the final update for the event.; PublishedTime=2021-08-17T22:35:23.237Z
Root cause A configuration issue in a recent service update caused duplicate events to appear in the Microsoft 365 Audit Log Search and Office 365 Management Activity API. Next steps: -We're reviewing our service update procedures to better catch similar potential configuration issues prior to their deployment. This is the final update for the event.; PublishedTime=2021-08-17T22:35:23.237Z

MO278847 Details

Final status: A recent configuration issue in a service update unexpectedly caused events sent to audit to get duplicated. We reverted the update to remediate the impact.

--------------------------------------------------------------------------------------------------
Title Approximately 85 percent of United States users' activity feed notifications intermittently delayed
ID TM273438
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users' activity feed notifications may be intermittently delayed.
Severity
Start time 2021-07-27T14:00:00Z
End time 2021-07-28T00:00:00Z
Last update 2021-07-28T03:52:30.97Z
More info Some users may have not received activity feed notifications, including missed call notifications, for more than one minute. Final status: We rebalanced the affected traffic across alternative traffic management infrastructure and monitored the service telemetry to verify that the backlog of traffic was processed without delay. We confirmed through monitoring that the impact was remediated. Scope of impact: Approximately 85 percent of users across the United States may have been impacted by this event. Start time: Tuesday, July 27, 2021, at 2:00 PM'UTC End time: Wednesday, July 28, 2021, at 12:00 AM'UTC Root cause: A traffic processing issue specific to a traffic management networking component resulted in impact. Next steps: -We're analyzing how requests are routed through the infrastructure to help identify ways to improve traffic management and avoid similar issues in the future. This is the final update for the event.; PublishedTime=2021-07-28T03:52:30.97Z
Root cause A traffic processing issue specific to a traffic management networking component resulted in impact. Next steps: -We're analyzing how requests are routed through the infrastructure to help identify ways to improve traffic management and avoid similar issues in the future. This is the final update for the event.; PublishedTime=2021-07-28T03:52:30.97Z

TM273438 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-07-27T15:48:48.97

Status: We've identified a possible networking configuration issue which is resulting in intermittent activity feed notification delays for approximately 85 percent of users across the United States. We're reviewing the corresponding service telemetry and network configurations to further isolate the source of the issue and determine the most expedient mitigation strategy. Next update by: Tuesday, July 27, 2021, at 5:30 PMUTC; PublishedTime=2021-07-27T16:10:30.8Z

Status: We're continuing our review of the associated network configurations to help us isolate the source of the issue and develop a corresponding mitigation approach. Next update by: Tuesday, July 27, 2021, at 7:30 PMUTC; PublishedTime=2021-07-27T17:05:38.443Z

Status: Our investigation is ongoing and is focused on the service-to-service networking configuration as we work to identify the source of the problem and determine the necessary steps to remediate the impact. Next update by: Tuesday, July 27, 2021, at 9:30 PMUTC; PublishedTime=2021-07-27T19:24:36.273Z

Status: Our investigation of the service-to-service networking configuration identified a traffic processing issue specific to a traffic management networking component, which resulted in impact. We've rebalanced the affected traffic across alternative traffic management infrastructure to restore the service, and we're monitoring the correlated service telemetry to ensure the backlog of traffic is processed without additional delay. We anticipate full resolution by the next scheduled update. Next update by: Wednesday, July 28, 2021, at 5:30 AMUTC; PublishedTime=2021-07-27T20:35:07.69Z

Final status: We rebalanced the affected traffic across alternative traffic management infrastructure and monitored the service telemetry to verify that the backlog of traffic was processed without delay. We confirmed through monitoring that the impact was remediated. Next update by: Wednesday, July 28, 2021, at 5:30 AMUTC; PublishedTime=2021-07-27T20:35:07.69Z

--------------------------------------------------------------------------------------------------
Title Some users may have been unable to access their mailbox via Outlook on the web
ID EX280059
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users may have been unable to access their mailbox via Outlook on the web.
Severity
Start time 2021-07-27T12:00:00Z
End time 2021-08-30T10:37:00Z
Last update 2021-08-30T11:04:06.407Z
More info Users affected by this event may receive the following error: Native Host Version:undefined err: Microsoft.Exchange.Clients.Owa2.Server.Core.OwaUserHasNoMailboxAnd NoLicenseAssignedException Current status: We've identified that two separate processes are attempting to update a user at the same time, causing contention when applying their license. We've developed an automated script that will resolve the contention. We're deploying the script throughout the affected environment and affected users will experience relief as this completes. Scope of impact: This issue impacts a small number of users that have recently had their Exchange Online license created or updated. Start time: Tuesday, July 27, 2021, at 12:00 PM UTC Root cause: Impact is caused by two separate processes attempting to update a user at the same time, causing contention when applying their license. Next update by: Monday, August 30, 2021, at 12:00 PM UTC; PublishedTime=2021-08-23T11:59:32.317Z
Root cause Impact was caused by two separate processes attempting to update a user at the same time, causing contention when applying their licence. Next steps: - We're reviewing the two processes for additional automated recovery options to reduce or avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-08-30T11:04:06.407Z

EX280059 Details

Status: We're analyzing network trace logs provided by affected users to determine the source of the issue. Next update by: Monday, August 23, 2021, at 12:00 PM UTC; PublishedTime=2021-08-23T10:04:51.2Z

Status: We've identified that two separate processes are attempting to update a user at the same time, causing contention when applying their license. We've developed an automated script that will resolve the contention. We're deploying the script throughout the affected environment and affected users will experience relief as this completes. Next update by: Monday, August 30, 2021, at 12:00 PM UTC; PublishedTime=2021-08-23T11:59:32.317Z

Final status: We've confirmed that the automated script has been successfully deployed across the affected environment. Our monitoring indicates that contention between the two processes is being resolved as expected. Next update by: Monday, August 30, 2021, at 12:00 PM UTC; PublishedTime=2021-08-23T11:59:32.317Z

--------------------------------------------------------------------------------------------------
Title Users are seeing titles rendering on top of images in news web parts on SharePoint Online sites
ID SP273435
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users were seeing titles rendering on top of images in news web parts on SharePoint Online sites.
Severity
Start time 2021-07-27T01:42:00Z
End time 2021-07-27T15:32:00Z
Last update 2021-07-27T15:56:24.173Z
More info
Root cause A code issue was causing users to see titles rendering on top of images in news web parts on SharePoint Online sites. Next steps: - We're further reviewing the news web parts code to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-07-27T15:56:24.173Z

SP273435 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-07-27T14:44:06.273

Status: We're reviewing Unified Logging Service (ULS) logs and the details provided by affected users as we work to determine the source for this event. Next update by: Tuesday, July 27, 2021, at 5:00 PMUTC; PublishedTime=2021-07-27T15:02:21Z

Final status: Following our review of ULS logs, we've identified a code issue that was causing users to see titles rendering on top of images in news web parts on SharePoint Online sites. We've manually updated the code, and confirmed with affected users that it has remediated impact. Next update by: Tuesday, July 27, 2021, at 5:00 PMUTC; PublishedTime=2021-07-27T15:02:21Z

--------------------------------------------------------------------------------------------------
Title Some users may have seen delays of up to 12 hours in the processing of alerts and activities
ID CS273671
Service Cloud App Security
Feature Cloud App Security
Status Service restored
Impact Users may see delays of up to 12 hours in the processing of alerts and activities for threat protection and discovery.
Severity Sev2
Start time 2021-07-27T01:22:00Z
End time 2021-09-27T10:00:00Z
Last update 2021-09-28T12:23:22.377Z
More info Approximately 50% of users served through our infrastructure in West US and users served through our East US infrastructure may see delays from Monday, August 2, 2021, at 4:00 PM UTC. We've fixed the issue for the remaining users that are served through our West US infrastructure, who may have seen delays from Tuesday, August 10, 2021, at 6:50 AM UTC. Current status: Our investigation in to the remaining impact within the East US infrastructure is taking longer than anticipated. We're continuing to troubleshoot the issue to develop a mitigation plan. Scope of impact: Impact is specific to some users who are served through the affected infrastructure. Start time: Tuesday, July 27, 2021, at 1:22 AM UTC Root cause: Some infrastructure responsible for processing threat protection, discovery alerts and activities, is performing below acceptable performance thresholds. Next update by: Thursday, September 23, 2021, at 1:00 PM UTC; PublishedTime=2021-09-16T11:21:44.013Z
Root cause Some infrastructure responsible for processing threat protection, discovery alerts and activities, was performing below acceptable performance thresholds. Next steps: - We're analyzing performance data and trends on the affected systems to help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-09-28T12:16:38.107Z

CS273671 Details

Status: We've determined a section of infrastructure, responsible for processing events notifications within the Microsoft Cloud App Security service, was performing unexpectedly. We've run commands which resolved the source issue, however, event notifications are still delayed due to a backlog. We've confirmed the backlog is draining and we will continue to monitor the environment to ensure this issue is resolved successfully. Next update by: Friday, July 30, 2021, at 12:30 PM UTC; PublishedTime=2021-07-28T10:50:07.873Z

Status: The backlog of event notifications continues to drain as expected and we're monitoring its progress until impact is fully mitigated. Next update by: Sunday, August 1, 2021, at 12:30 PM UTC; PublishedTime=2021-07-30T11:52:05.953Z

Status: We're continuing to monitor the backlog drain of event notifications. Completion is anticipated to be in a week. Next update by: Sunday, August 8, 2021, at 12:30 PM UTC; PublishedTime=2021-08-01T11:47:09.947Z

Status: We're monitoring the backlog of event notifications as it continues to drain. We anticipate the queue will be fully processed by Tuesday, August 10, 2021. Next update by: Tuesday, August 10, 2021, at 12:30 PM UTC; PublishedTime=2021-08-08T09:55:50.503Z

Status: The backlog of event notifications is taking longer than expected to complete. We're continuing to monitor the backlog as it drains until completion. Next update by: Monday, August 16, 2021, at 12:30 PM UTC; PublishedTime=2021-08-10T12:17:36.577Z

Status: Whilst we continue to monitor the processing of event notifications, we're optimizing the environment to deal with these request more efficiently. Next update by: Wednesday, August 18, 2021, at 12:30 PM UTC; PublishedTime=2021-08-16T11:49:54.423Z

Status: We've optimized the environment to deal with the requests more efficiently. We anticipate the backlog of events will be processed by Sunday, August 22, 2021. Next update by: Sunday, August 22, 2021, at 1:00 PM UTC; PublishedTime=2021-08-18T11:24:48.213Z

Status: We're continuing to review our processes to develop alternative means to resolve impact. High levels of traffic have slowed the progress of the previously applied optimization steps. We expect this traffic to subside and the issue to be mitigated by Monday, August 23, 2021. Next update by: Sunday, August 22, 2021, at 1:00 PM UTC; PublishedTime=2021-08-20T10:47:54.817Z

Status: We've reviewed our monitoring and determined that more time is required to process the backfill of event notifications. We're continuing to investigate alternative means to resolve impact. We expect the queue to complete by Friday, August 27, 2021, at 1:00 PM. Next update by: Friday, August 27, 2021, at 1:00 PM UTC; PublishedTime=2021-08-22T12:49:56.687Z

Status: We've confirmed that the process to backfill the event notifications is progressing as expected and that users will experience relief as this data is processed. We estimate that this should complete by Monday, August 30, 2021. Next update by: Monday, August 30, 2021, at 1:00 PM UTC; PublishedTime=2021-08-24T11:37:27.003Z

Status: We're in the final stages of the backfill completion. We're continuing to monitor the process as it completes. Next update by: Monday, September 6, 2021, at 1:00 PM UTC; PublishedTime=2021-08-30T12:03:57.057Z

Status: We've confirmed that the backfill has been completed for two of the affected regions, leaving approximately half of users within West US impacted. We've identified that a different mitigation strategy is required for the remaining infrastructure. We're analyzing the remaining backlog of alerts to determine the next troubleshooting steps. Next update by: Thursday, September 9, 2021, at 1:00 PM UTC; PublishedTime=2021-09-06T12:23:28.34Z

Status: We've identified that the issue in our East US infrastructure is not fully mitigated and users served through that infrastructure may still experience impact. We're continuing to investigate the underlying cause to help develop a mitigation plan that provides relief across the all affected infrastructure. Next update by: Thursday, September 16, 2021, at 1:00 PM UTC; PublishedTime=2021-09-09T08:59:19.323Z

Status: Our investigation in to the remaining impact within the East US infrastructure is taking longer than anticipated. We're continuing to troubleshoot the issue to develop a mitigation plan. Next update by: Thursday, September 23, 2021, at 1:00 PM UTC; PublishedTime=2021-09-16T11:21:44.013Z

Status: We've confirmed that impact within the East US infrastructure is fully mitigated. We've established that the backlog is progressing as expected, which we estimate to complete by Tuesday, September 28, 2021. Next update by: Tuesday, September 28, 2021, at 1:00 PM UTC; PublishedTime=2021-09-23T12:49:40.25Z

Final status: The backlog has completed and has successfully mitigated the issue. Next update by: Tuesday, September 28, 2021, at 1:00 PM UTC; PublishedTime=2021-09-23T12:49:40.25Z

--------------------------------------------------------------------------------------------------
Title Users are receiving an error that prevents delisting blocked IP addresses at https://sender.office.com
ID EX274026
Service Exchange Online
Feature Management and Provisioning
Status Service restored
Impact Users were receiving an error when attempting to delist blocked IP addresses at https://sender.office.com.
Severity
Start time 2021-07-25T21:12:00Z
End time 2021-07-29T17:20:00Z
Last update 2021-07-29T18:57:23.947Z
More info Users were seeing the following error message when attempting to delist blocked IP addresses: "5.1.0 - Unknown address error 550-'5.7.606 Access denied" Final status: Following our additional internal validation, we've confirmed that impact has been resolved. Scope of impact: This issue may have impacted any user attempting to delist blocked IP addresses at https://sender.office.com. Start time: Sunday, July 25, 2021, at 9:12 PM'UTC End time: Thursday, July 29, 2021, at 5:20 PM'UTC Root cause: A recent configuration change designed to migrate system-generated outbound mail was preventing users from delisting blocked IP addresses at https://sender.office.com. Next steps: - We're reviewing our update procedures to better understand why the configuration issue wasn't identified during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-07-29T18:57:23.947Z
Root cause A recent configuration change designed to migrate system-generated outbound mail was preventing users from delisting blocked IP addresses at https://sender.office.com. Next steps: - We're reviewing our update procedures to better understand why the configuration issue wasn't identified during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-07-29T18:57:23.947Z

EX274026 Details

Status: We're reviewing the error and correlating it to our service to see where is the point of failure is occurring. Next update by: Thursday, July 29, 2021, at 6:30 PMUTC; PublishedTime=2021-07-29T16:36:50.25Z

Status: We identified a recent configuration change prevented users from delisting blocked IP addresses. We've reverted the configuration change; however, we're performing additional validation to confirm that the reversion was successful. Next update by: Thursday, July 29, 2021, at 8:30 PMUTC; PublishedTime=2021-07-29T18:12:31.89Z

Final status: Following our additional internal validation, we've confirmed that impact has been resolved. Next update by: Thursday, July 29, 2021, at 8:30 PMUTC; PublishedTime=2021-07-29T18:12:31.89Z

--------------------------------------------------------------------------------------------------
Title Approved actions may have appeared as queued in the action center within Microsoft Suite
ID MO278781
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Admins may have experienced approved actions appearing as queued in the action center within Microsoft Suite.
Severity
Start time 2021-07-25T13:30:00Z
End time 2021-09-05T13:00:00Z
Last update 2021-09-06T13:25:12.553Z
More info
Root cause A recent change was sending cancellation actions to parent AIR investigations unintentionally. The cancelled status of these investigations wasn't getting passed on to the action center, causing conflicting statuses. Next steps: - We're reviewing our update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-09-06T13:25:12.553Z

MO278781 Details

Status: We've identified a code issue which is impacting actions within the action center. We're validating a fix we developed before rolling it out to the affected environment. Next update by: Tuesday, August 24, 2021, at 3:00 PM UTC; PublishedTime=2021-08-17T13:40:58.657Z

Status: We've completed validating the fix and have started deploying it to the affected environment. Next update by: Wednesday, September 1, 2021, at 3:00 PM UTC; PublishedTime=2021-08-24T13:36:04.37Z

Status: We're continuing to monitor the fix, which has passed 70% completion, as it saturates across the affected environment. We anticipate that the deployment will be complete by Monday 6th September 2021. Next update by: Monday, September 6, 2021, at 3:00 PM UTC; PublishedTime=2021-09-01T13:25:19.357Z

Final status: We've confirmed that the fix has completed and any new actions should appear with the correct status. Any old actions will take approximately 7 days to clear, as expected. Next update by: Monday, September 6, 2021, at 3:00 PM UTC; PublishedTime=2021-09-01T13:25:19.357Z

--------------------------------------------------------------------------------------------------
Title Some users may experience transcriptions or captions being unavailable in Microsoft Teams live events
ID TM272009
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may experience transcriptions or captions being unavailable in Microsoft Teams live events.
Severity
Start time 2021-07-21T07:32:00Z
End time 2021-07-21T09:39:00Z
Last update 2021-07-21T11:44:46.97Z
More info
Root cause Some speech infrastructure, responsible for facilitating the transcription services for Teams live events, had unexpectedly become unresponsive. Next steps: -We're investigating the affected speech infrastructure to identify the impact trigger to prevent similar issues in the future. This is the final update for the event.; PublishedTime=2021-07-21T11:44:46.97Z

TM272009 Details

Status: We're reviewing service monitoring telemetry to isolate the source of the issue and establish a fix. Next update by: Wednesday, July 21, 2021, at 11:00 AM UTC; PublishedTime=2021-07-21T08:49:32.03Z

Status: We've identified that some speech infrastructure, responsible for facilitating the transcription services for Teams live events, has unexpectedly become unresponsive. We're restarting the affected infrastructure to resolve the issue. Next update by: Wednesday, July 21, 2021, at 1:00 PM UTC; PublishedTime=2021-07-21T10:48:32.207Z

Final status: We've confirmed, after a period of monitoring the service, that restarting the affected infrastructure has mitigated the issue. Next update by: Wednesday, July 21, 2021, at 1:00 PM UTC; PublishedTime=2021-07-21T10:48:32.207Z

--------------------------------------------------------------------------------------------------
Title Auto Attendant welcome messages configured using text to speech were not playing within Microsoft Teams
ID TM272013
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were experiencing welcome messages configured using text to speech not playing within Microsoft Teams.
Severity
Start time 2021-07-21T06:35:00Z
End time 2021-07-21T09:30:00Z
Last update 2021-07-21T12:52:07.517Z
More info
Root cause A section of infrastructure, responsible for facilitating text to speech within Microsoft Teams, was performing unexpectedly. Next steps: - We're analyzing data on the affected text to speech infrastructure to prevent similar issues in the future. This is the final update for the event.; PublishedTime=2021-07-21T12:52:07.517Z

TM272013 Details

Status: We've determined that a section of infrastructure, responsible for facilitating text to speech within Microsoft Teams, is performing unexpectedly. We're restarting the affected infrastructure to mitigate the impact. Next update by: Wednesday, July 21, 2021, at 1:30 PM UTC; PublishedTime=2021-07-21T11:42:35.977Z

Final status: We've confirmed, after monitoring the environment, that restarting the affected infrastructure has mitigated the issue. Next update by: Wednesday, July 21, 2021, at 1:30 PM UTC; PublishedTime=2021-07-21T11:42:35.977Z

--------------------------------------------------------------------------------------------------
Title Users may have experienced delays or problems loading SharePoint Online sites
ID SP271897
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users may have experienced intermittent delays or navigation errors when accessing SharePoint sites.
Severity
Start time 2021-07-21T05:45:00Z
End time 2021-07-21T06:40:00Z
Last update 2021-07-21T07:57:58.503Z
More info Users may have seen the error message "The server is busy right now. Try again later." Final status: Whilst monitoring service recovery, we've identified that a recent update could have potentially been causing impact to the service infrastructure. We've stopped the deployment of this update whilst we continue to investigate the details of this upgrade. After a period of extended monitoring, we've confirmed that the actions taken by our automated system have restored the service. Scope of impact: This issue may have affected any user accessing SharePoint Online sites. Start time: Wednesday, July 21, 2021, at 5:45 AM UTC End time: Wednesday, July 21, 2021, at 6:40 AM UTC Root cause: A portion of infrastructure facilitating access to SharePoint Online content was performing below the expected service threshold. Next steps: - We're continuing to review system telemetry and diagnostic data from affected components to isolate the source of the issue. - We're analyzing the contents of the update to identify if it was the cause of the issue. - If the update is confirmed as the cause of the issue, we'll review our service update procedures to identify why impact was not correctly identify during testing cycles to prevent this from happening again. This is the final update for the event.; PublishedTime=2021-07-21T07:57:58.503Z
Root cause A portion of infrastructure facilitating access to SharePoint Online content was performing below the expected service threshold. Next steps: - We're continuing to review system telemetry and diagnostic data from affected components to isolate the source of the issue. - We're analyzing the contents of the update to identify if it was the cause of the issue. - If the update is confirmed as the cause of the issue, we'll review our service update procedures to identify why impact was not correctly identify during testing cycles to prevent this from happening again. This is the final update for the event.; PublishedTime=2021-07-21T07:57:58.503Z

SP271897 Details

Status: A portion of infrastructure facilitating access to SharePoint Online sites is performing below the expected service threshold. Our automated recovery system has rerouted traffic to alternate infrastructure to mitigate impact. We are monitoring to ensure that these actions were successful and that service is restored. Next update by: Wednesday, July 21, 2021, at 8:00 AMUTC; PublishedTime=2021-07-21T06:58:04.173Z

Final status: Whilst monitoring service recovery, we've identified that a recent update could have potentially been causing impact to the service infrastructure. We've stopped the deployment of this update whilst we continue to investigate the details of this upgrade. After a period of extended monitoring, we've confirmed that the actions taken by our automated system have restored the service. Next update by: Wednesday, July 21, 2021, at 8:00 AMUTC; PublishedTime=2021-07-21T06:58:04.173Z

--------------------------------------------------------------------------------------------------
Title Users may have experienced delays or problems loading SharePoint Online sites
ID SP271897
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users may have experienced intermittent delays or navigation errors when accessing SharePoint sites.
Severity
Start time 2021-07-21T05:45:00Z
End time 2021-07-21T06:40:00Z
Last update 2021-07-21T07:57:58.503Z
More info Users may have seen the error message "The server is busy right now. Try again later." Final status: Whilst monitoring service recovery, we've identified that a recent update could have potentially been causing impact to the service infrastructure. We've stopped the deployment of this update whilst we continue to investigate the details of this upgrade. After a period of extended monitoring, we've confirmed that the actions taken by our automated system have restored the service. Scope of impact: This issue may have affected any user accessing SharePoint Online sites. Start time: Wednesday, July 21, 2021, at 5:45 AM UTC End time: Wednesday, July 21, 2021, at 6:40 AM UTC Root cause: A portion of infrastructure facilitating access to SharePoint Online content was performing below the expected service threshold. Next steps: - We're continuing to review system telemetry and diagnostic data from affected components to isolate the source of the issue. - We're analyzing the contents of the update to identify if it was the cause of the issue. - If the update is confirmed as the cause of the issue, we'll review our service update procedures to identify why impact was not correctly identify during testing cycles to prevent this from happening again. This is the final update for the event.; PublishedTime=2021-07-21T07:57:58.503Z
Root cause A portion of infrastructure facilitating access to SharePoint Online content was performing below the expected service threshold. Next steps: - We're continuing to review system telemetry and diagnostic data from affected components to isolate the source of the issue. - We're analyzing the contents of the update to identify if it was the cause of the issue. - If the update is confirmed as the cause of the issue, we'll review our service update procedures to identify why impact was not correctly identify during testing cycles to prevent this from happening again. This is the final update for the event.; PublishedTime=2021-07-21T07:57:58.503Z

SP271897 Details

Status: A portion of infrastructure facilitating access to SharePoint Online sites is performing below the expected service threshold. Our automated recovery system has rerouted traffic to alternate infrastructure to mitigate impact. We are monitoring to ensure that these actions were successful and that service is restored. Next update by: Wednesday, July 21, 2021, at 8:00 AMUTC; PublishedTime=2021-07-21T06:58:04.173Z

Final status: Whilst monitoring service recovery, we've identified that a recent update could have potentially been causing impact to the service infrastructure. We've stopped the deployment of this update whilst we continue to investigate the details of this upgrade. After a period of extended monitoring, we've confirmed that the actions taken by our automated system have restored the service. Next update by: Wednesday, July 21, 2021, at 8:00 AMUTC; PublishedTime=2021-07-21T06:58:04.173Z

--------------------------------------------------------------------------------------------------
Title Admins are unable to make auto attendant changes in the Microsoft Teams admin center
ID TM272617
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Admins were unable to make auto attendant changes in the Microsoft Teams admin center.
Severity
Start time 2021-07-20T19:27:00Z
End time 2021-07-23T19:10:00Z
Last update 2021-07-23T22:41:47.843Z
More info Admins may have seen a spinning loading icon after hitting 'next' when attempting to apply changes. Final status: We've received confirmation from some affected users that shifting the UI to utilize an alternate API has effectively provided mitigation. Scope of impact: Any admin attempting to make auto attendant changes in the Microsoft Teams admin center was impacted. Start time: Tuesday, July 20, 2021, at 7:27 PM'UTC End time: Friday, July 23, 2021, at 7:10 PM'UTC Root cause: A recent back-end change implemented to improve UI functionality introduced a misconfiguration to the PowerShell service utilized by the UI when making auto attendant changes. Next steps: - We're reviewing our testing procedures to expose and remediate potential misconfiguration scenarios prior to the release of future changes. This is the final update for the event.; PublishedTime=2021-07-23T21:44:35.067Z
Root cause A recent back-end change implemented to improve UI functionality introduced a misconfiguration to the PowerShell service utilized by the UI when making auto attendant changes. Next steps: - We're reviewing our testing procedures to expose and remediate potential misconfiguration scenarios prior to the release of future changes. This is the final update for the event.; PublishedTime=2021-07-23T21:44:35.067Z

TM272617 Details

Status: We're analyzing case details and gathering diagnostic data from internal reproductions of the issue so we can better isolate the underlying cause of the issue and determine our next steps. Next update by: Friday, July 23, 2021, at 6:00 PM UTC; PublishedTime=2021-07-23T16:40:51.053Next update by: Friday, July 23, 2021, at 6:00 PM UTC; PublishedTime=2021-07-23T16:40:51.053Z

Status: Our analysis of case details and diagnostic data gathered from internal reproductions of the issue suggest impact may be occurring due to a recent change applied within our service to enhance UI functionality. To verify this, we're investigating details from this change so we can better develop a mitigation strategy moving forward. Next update by: Friday, July 23, 2021, at 8:00 PM UTC; PublishedTime=2021-07-23T17:54:12.277Z

Status: Our investigation has confirmed that a recent back-end change implemented to improve UI functionality introduced a misconfiguration to the PowerShell service utilized by the UI when making auto attendant changes. To remediate, we've enabled a change which allows the UI to utilize an alternate API rather than the problematic PowerShell service, and we're reaching out to affected users to confirm our mitigation strategy has provided alleviation. Next update by: Friday, July 23, 2021, at 10:00 PM UTC; PublishedTime=2021-07-23T20:01:15.917Z

Final status: We've received confirmation from some affected users that shifting the UI to utilize an alternate API has effectively provided mitigation. Next update by: Friday, July 23, 2021, at 10:00 PM UTC; PublishedTime=2021-07-23T20:01:15.917Z

--------------------------------------------------------------------------------------------------
Title Users were unable to create or utilize tags in all connection methods of Microsoft Teams
ID TM271753
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have been unable to create or utilize tags in all connection methods of Microsoft Teams.
Severity
Start time 2021-07-20T15:26:00Z
End time 2021-07-20T16:08:00Z
Last update 2021-07-20T17:27:44.617Z
More info
Root cause A recent configuration change meant to change the way we fetch settings on tags mistakenly disabled Microsoft Teams tag functionality. Next steps: - We're reviewing how impact was missed during our configuration change validation processes to prevent similar, future problems. This is the final update for the event.; PublishedTime=2021-07-20T17:27:44.617Z

TM271753 Details

Status: We've investigated a problem with the Microsoft Teams service in which users may have been unable to create or utilize tags and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-07-20T17:01:12.223

Final status: We've determined that a recent configuration change meant to change the way we fetch settings on tags mistakenly disabled Microsoft Teams tag functionality. We've rolled back this change and internal monitoring confirms that impact has been remediated.

--------------------------------------------------------------------------------------------------
Title Users' Windows Autopilot devices aren't receiving profiles from Microsoft Intune
ID IT289945
Service Microsoft Intune
Feature Microsoft Intune
Status Service degradation
Impact Users' Windows Autopilot devices aren't receiving profiles from Microsoft Intune.
Severity Sev2
Start time 2021-07-20T13:48:00Z
End time
Last update 2021-10-18T16:57:59.647Z
More info
Root cause A latent code issue surfaces when devices are incorrectly moved to 'obsolete' without removing specific property values pertaining to device records, leading to impact. Next update by: Monday, October 25, 2021, at 8:00 PMUTC; PublishedTime=2021-10-18T16:57:59.647Z

IT289945 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We've identified a previous issue where devices were incorrectly moved to obsolete without removing specific property values pertaining to device records. While that issue has been fixed, affected devices remain in this state, which is affecting profile deployment to newly enrolled devices. We're developing a fix to mitigate the issue by removing the property value requirement as a potential mitigation option. Next update by: Friday, October 8, 2021, at 12:00 AMUTC; PublishedTime=2021-10-07T22:33:11.33Z

Status: We've completed development of our fix and are preparing to deploy it to our internal testing environment to validate that our fix is effective without causing inadvertent impact to other services. Once successful, we'll establish a timeline for deployment to the affected environment. Next update by: Friday, October 8, 2021, at 6:30 PMUTC; PublishedTime=2021-10-07T23:46:54.237Z

Status: We're deploying the fix to our internal testing environment for further validation in order to ensure it successfully resolves the issue without causing inadvertent impact to other aspects of the service. Based on our current validation progress, we suspect that validation and a timeline for deployment to the affected environment will be provided by our next scheduled update. Next update by: Saturday, October 9, 2021, at 1:00 AMUTC; PublishedTime=2021-10-08T18:23:41.233Z

Status: We've completed internal validation of the fix and have started deployment to all impacted environments. We suspect to be able to provide an approximate deployment timeline by our next communications update. Next update by: Saturday, October 9, 2021, at 7:00 PMUTC; PublishedTime=2021-10-08T23:17:55.987Z

Status: We've verified that the fix is deploying successfully to the impacted environments and we suspect that the fix will complete on Sunday, October 10, 2021. Next update by: Sunday, October 10, 2021, at 8:30 PMUTC; PublishedTime=2021-10-09T17:14:18.777Z

Status: We've confirmed that our fix has completed deployment to the impacted environment. We're awaiting a system sync that will apply the fix to affected devices and resolve the impact by Monday, October 18, 2021, at approximately 8:00 PM UTC. Next update by: Monday, October 18, 2021, at 8:00 PMUTC; PublishedTime=2021-10-10T18:56:27.793Z

Status: The system sync has completed, and the fix mitigated impact on approximately 85 percent of affected devices. We've developed an additional fix to resolve the issue for the remaining devices which is expected to deploy on Monday, October 18, 2021, and we anticipate full impact remediation by Monday, October 25, 2021. Next update by: Monday, October 25, 2021, at 8:00 PMUTC; PublishedTime=2021-10-18T16:57:59.647Z

--------------------------------------------------------------------------------------------------
Title Users may have been unable to install apps from the "Build by your Colleague" section within Microsoft Teams
ID CR272113
Service Dynamics 365 Apps
Feature Other
Status Service restored
Impact Users may have been unable to install apps from the "Build by your Colleague" section within Microsoft Teams,
Severity
Start time 2021-07-20T05:00:00Z
End time 2021-07-21T21:16:00Z
Last update 2021-07-21T21:43:49.19Z
More info
Root cause

CR272113 Details

--------------------------------------------------------------------------------------------------
Title Some admins can't access the Exchange admin center (EAC)
ID EX271749
Service Exchange Online
Feature Management and Provisioning
Status Service restored
Impact Admins were unable to access the EAC.
Severity
Start time 2021-07-20T00:25:00Z
End time 2021-07-20T18:35:00Z
Last update 2021-07-20T19:22:46.837Z
More info Admins encountering this problem would receive an error with the description "AADSTS50011: The reply URL specified in the request does not match the reply URLS configured for the application." Final status: From our review of the available data and by reproducing the issue in our internal environment, we determined that impact was occurring due to a recent standard service update which introduced a code regression. We rolled back this change and confirmed that impact is no longer occurring. Scope of impact: Some admins attempting to access the EAC may have experienced impact. Start time: Tuesday, July 20, 2021, at 12:25 AM'UTC End time: Tuesday, July 20, 2021, at 6:35 PM'UTC Root cause: A recent standard service update contained a code regression which resulted in impact. Next steps: - We're reviewing our standard service update testing and validation procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-07-20T19:18:49.12Z
Root cause A recent standard service update contained a code regression which resulted in impact. Next steps: - We're reviewing our standard service update testing and validation procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-07-20T19:18:49.12Z

EX271749 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 60 minutes.; PublishedTime=2021-07-20T16:27:10.09

Status: We're reviewing available data to gain insight into the root cause of this issue. In parallel, we're requesting that impacted admins provide a Fiddler network trace log that details the reproduction of the issue to assist with our investigation. Next update by: Tuesday, July 20, 2021, at 7:30 PMUTC; PublishedTime=2021-07-20T17:28:26.387Z

Final status: From our review of the available data and by reproducing the issue in our internal environment, we determined that impact was occurring due to a recent standard service update which introduced a code regression. We rolled back this change and confirmed that impact is no longer occurring. Next update by: Tuesday, July 20, 2021, at 7:30 PMUTC; PublishedTime=2021-07-20T17:28:26.387Z

--------------------------------------------------------------------------------------------------
Title Users aren't receiving push notifications and can't register devices for push notifications
ID MM271596
Service Power Apps in Microsoft 365
Feature Service and web access issues
Status Service restored
Impact Users weren't receiving push notifications and couldn't register devices for push notifications.
Severity
Start time 2021-07-19T23:08:00Z
End time 2021-07-20T02:21:00Z
Last update 2021-07-20T02:50:54.06Z
More info
Root cause High traffic within the service caused push notifications to be unavailable. Next steps: - We're reviewing the service to better understand how impact began, and to explore options to prevent similar impact in the future. This is the final update for the event.; PublishedTime=2021-07-20T02:50:54.06Z

MM271596 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-07-20T01:54:26.713

Status: We've responded to monitoring raised reports of an issue where users aren't receiving push notifications and can't register devices for push notifications in Microsoft Power Apps. We're reviewing available system data to determine possible underlying causes of this issue. Next update by: Tuesday, July 20, 2021, at 3:30 AMUTC; PublishedTime=2021-07-20T02:10:55.197Z

Final status: We've determined that high traffic within the service caused push notifications to be unavailable. We've developed and deployed a fix for this issue and confirmed via service monitoring that this has remediated impact. Next update by: Tuesday, July 20, 2021, at 3:30 AMUTC; PublishedTime=2021-07-20T02:10:55.197Z

--------------------------------------------------------------------------------------------------
Title Users aren't receiving push notifications and can't register devices for push notifications
ID MM271596
Service Power Apps in Microsoft 365
Feature Service and web access issues
Status Service restored
Impact Users weren't receiving push notifications and couldn't register devices for push notifications.
Severity
Start time 2021-07-19T23:08:00Z
End time 2021-07-20T02:21:00Z
Last update 2021-07-20T02:50:54.06Z
More info
Root cause High traffic within the service caused push notifications to be unavailable. Next steps: - We're reviewing the service to better understand how impact began, and to explore options to prevent similar impact in the future. This is the final update for the event.; PublishedTime=2021-07-20T02:50:54.06Z

MM271596 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-07-20T01:54:26.713

Status: We've responded to monitoring raised reports of an issue where users aren't receiving push notifications and can't register devices for push notifications in Microsoft Power Apps. We're reviewing available system data to determine possible underlying causes of this issue. Next update by: Tuesday, July 20, 2021, at 3:30 AMUTC; PublishedTime=2021-07-20T02:10:55.197Z

Final status: We've determined that high traffic within the service caused push notifications to be unavailable. We've developed and deployed a fix for this issue and confirmed via service monitoring that this has remediated impact. Next update by: Tuesday, July 20, 2021, at 3:30 AMUTC; PublishedTime=2021-07-20T02:10:55.197Z

--------------------------------------------------------------------------------------------------
Title Users are unable to manage distribution lists via Outlook on the web
ID EX271802
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users were unable to manage distribution lists via Outlook on the web.
Severity
Start time 2021-07-19T21:15:00Z
End time 2021-07-20T23:40:00Z
Last update 2021-07-21T03:48:12.25Z
More info
Root cause A recent change intended to provide optimizations for Content Security Policy (CSP) inadvertently resulted in impact to Outlook on the web. Next steps: - We're reviewing the affecting change to identify how impact was missed during testing and validation to better prevent similar future impact. This is the final update for the event.; PublishedTime=2021-07-21T00:03:05.573Z

EX271802 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Final status: A recent change intended to provide optimizations for Content Security Policy (CSP) inadvertently resulted in impact to Outlook on the web. We've reverted the affecting change and confirmed after testing with some affected users that impact is mitigated. Additionally, users may have to refresh their browsers to ensure they have received the fix.

--------------------------------------------------------------------------------------------------
Title Users can't install 'Built by your colleagues' apps from Microsoft Teams app store
ID TM272111
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were unable to install apps from the 'Built by your colleagues' section of the Microsoft Teams app store.
Severity
Start time 2021-07-19T17:00:00Z
End time 2021-07-21T21:16:00Z
Last update 2021-07-21T21:53:56.727Z
More info
Root cause A recent standard service update included a code regression which resulted in impact. Next steps: - We're reviewing our standard update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-07-21T21:53:56.727Z

TM272111 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We've identified a code regression introduced by a recent service update that is resulting in impact. We're rolling back this change and monitoring its progress to ensure this remediates impact for the issue. Next update by: Wednesday, July 21, 2021, at 11:00 PMUTC; PublishedTime=2021-07-21T21:17:53.217Z

Final status: We've confirmed via telemetry that rolling back the recent change has mitigated impact. Next update by: Wednesday, July 21, 2021, at 11:00 PMUTC; PublishedTime=2021-07-21T21:17:53.217Z

--------------------------------------------------------------------------------------------------
Title Users can't add attachments from Approvals apps in the "Browse Teams and Channel" section on the Teams desktop client
ID TM277461
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users couldn't add attachments in Approvals apps in the "Browse Teams and Channel" section on the Teams desktop client.
Severity
Start time 2021-07-19T14:42:00Z
End time 2021-08-23T05:00:00Z
Last update 2021-08-23T23:24:24.603Z
More info As an alternative method of adding attachments via the "Browse Teams and Channel" section in Microsoft Teams, we recommended users access the web client to avoid this issue while we remediated impact to the desktop client. Adding attachments in other sections of the Approvals app on the desktop client were unaffected. Final status: We've completed fix deployment to all affected environments, and we've confirmed via internal testing that this has remediated impact. Scope of impact: This issue could have affected any user that's adding attachments from the Approvals app in the "Browse teams and Channel" section on the desktop client only. Start time: Monday, July 19, 2021, at 2:42 PM UTC End time: Monday, August 23, 2021, at 5:00 AM UTC Root cause: An authentication issue was preventing users from adding attachments from Approvals apps. Next steps: - We're reviewing the affected authentication components to better understand how impact began and to explore options to prevent similar impact in the future. This is the final update for the event.; PublishedTime=2021-08-23T23:24:24.603Z
Root cause An authentication issue was preventing users from adding attachments from Approvals apps. Next steps: - We're reviewing the affected authentication components to better understand how impact began and to explore options to prevent similar impact in the future. This is the final update for the event.; PublishedTime=2021-08-23T23:24:24.603Z

TM277461 Details

Status: We've received reports of an issue in which users can't add attachments from Approvals apps in the "Browse Teams and Channel" section on the Teams desktop client. We're investigating a potential authentication issue which may be contributing to impact, as well as validating potential authentication code changes that may resolve this issue. Additionally, as an immediate workaround, we recommend users access the Approvals app from the web client as an alternative method of file attachment from the "Browse Teams and Channel" while we investigate and remediate this issue. Next update by: Friday, August 13, 2021, at 1:00 AM UTC; PublishedTime=2021-08-12T00:25:48.753Z

Status: We've determined that there is an issue within a library file. We have developed and are currently validating a fix within our testing environment. While we're validating this fix, users may use the web client as an alternative method of file attachment. We anticipate having a mitigation time by our next scheduled update. Next update by: Wednesday, August 18, 2021, at 1:00 AMUTC; PublishedTime=2021-08-12T23:41:08.19Z

Status: We've validated the fix and started deploying it out to the affected environment. We anticipate that the fix will completely saturate by Wednesday, August 25, 2021. Some users may see mitigation as the fix deployment progresses. Next update by: Tuesday, August 24, 2021, at 1:00 AMUTC; PublishedTime=2021-08-17T23:33:08.513Z

Final status: We've completed fix deployment to all affected environments, and we've confirmed via internal testing that this has remediated impact. Next update by: Tuesday, August 24, 2021, at 1:00 AMUTC; PublishedTime=2021-08-17T23:33:08.513Z

--------------------------------------------------------------------------------------------------
Title Users intermittently unable to enroll devices and admins unable to see enrollment blades in Microsoft Endpoint Manager
ID IT271494
Service Microsoft Intune
Feature Microsoft Intune
Status Service restored
Impact Users may have been intermittently unable to enroll devices within the Microsoft Intune service.
Severity
Start time 2021-07-18T20:00:00Z
End time 2021-07-19T18:00:00Z
Last update 2021-07-19T21:06:39.783Z
More info Admins may have been intermittently unable to see the enrollment blades in the Microsoft Endpoint Manager portal. Final status: While we were in the process of reviewing recent changes to identify sources of impact, system monitoring indicated that the service had returned to normal health. We've confirmed that the problem is no longer occurring. We'll continue to monitor the service to ensure that the problem does not happen again, while simultaneously exploring what caused the underlying issue. Scope of impact: This issue may have intermittently affect any of your users and admins. Start time: Sunday, July 18, 2021, at 8:00 PM'UTC End time: Monday, July 19, 2021, at 6:00 PM'UTC This is the final update for the event.; PublishedTime=2021-07-19T20:14:00.297Z
Root cause

IT271494 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're reviewing available telemetry information to confirm the behavior of underlying enrollment service instances to better understand how impact manifests and formulate our next troubleshooting steps. Next update by: Monday, July 19, 2021, at 7:30 PMUTC; PublishedTime=2021-07-19T18:04:58.7Z

Status: We've confirmed via telemetry that the service has recovered, however we're continuing to monitor service health to ensure impact doesn't reoccur. In parallel, we've identified that an unusually large set of requests on the service occurred around the same time as a recent service update, which may be responsible for impact. We're continuing to analyze this recent change to confirm this theory and assess if additional remediation actions are needed. Next update by: Monday, July 19, 2021, at 9:30 PMUTC; PublishedTime=2021-07-19T19:24:49.143Z

Final status: While we were in the process of reviewing recent changes to identify sources of impact, system monitoring indicated that the service had returned to normal health. We've confirmed that the problem is no longer occurring. We'll continue to monitor the service to ensure that the problem does not happen again, while simultaneously exploring what caused the underlying issue. Next update by: Monday, July 19, 2021, at 9:30 PMUTC; PublishedTime=2021-07-19T19:24:49.143Z

--------------------------------------------------------------------------------------------------
Title Users may be seeing delays in search results in Exchange Online
ID EX271423
Service Exchange Online
Feature Networking Issues
Status Service restored
Impact Users may have gotten delays of up to 10 seconds when performing searches in Exchange Online via any connection method.
Severity
Start time 2021-07-15T02:30:00Z
End time 2021-07-19T09:00:00Z
Last update 2021-07-20T23:35:09.15Z
More info
Root cause A recent traffic migration resulted in a failure of data files to deploy correctly to the needed systems, thus causing the search delays. Next steps: - We're reviewing our traffic migration procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-07-20T23:35:09.15Z

EX271423 Details

Status: We're analyzing system logs and specific error messages to isolate the source of the issue. Next update by: Monday, July 19, 2021, at 12:30 PM UTC; PublishedTime=2021-07-19T11:23:16.997Z

Status: We've identified a recent deployment, made to the infrastructure responsible for mail searches, may be the source of the issue. We're verifying this before we develop a mitigation plan. Next update by: Monday, July 19, 2021, at 2:30 PM UTC; PublishedTime=2021-07-19T12:19:10.1Z

Status: We've determined the suspected recent deployment is not the source of the issue. We're reviewing Portable Ranker logs to determine the next troubleshooting steps. Next update by: Monday, July 19, 2021, at 4:30 PM UTC; PublishedTime=2021-07-19T14:17:45.373Z

Status: Our review of the ranker logs indicates that the service isn't responding as expected. We're continuing our investigation to isolate the source of the delays. Next update by: Monday, July 19, 2021, at 6:30 PMUTC; PublishedTime=2021-07-19T16:19:27.843Z

Status: Our review of the ranker logs indicates that the service isn't responding as expected. We're continuing our investigation to isolate the source of the delays. Next update by: Monday, July 19, 2021, at 6:30 PMUTC; PublishedTime=2021-07-19T16:21:50.247Z

Status: Our review of the ranker logs indicates that impact is potentially caused by an increase in latency across a subset of Client Access Front End (CAFE) infrastructure. We're reviewing CAFE logs to confirm this is the cause and develop mitigation actions. Next update by: Tuesday, July 20, 2021, at 12:00 AMUTC; PublishedTime=2021-07-19T17:11:17.56Z

Status: Our review of the CAFE logs has revealed that data files are not deploying correctly, thus causing the search delays. We're rolling back a recent migration of traffic to its original infrastructure which we expect will mitigate impact. In parallel, we're continuing to investigate further mitigation options. Next update by: Wednesday, July 21, 2021, at 12:00 AMUTC; PublishedTime=2021-07-19T23:50:04.987Z

Final status: We've confirmed via telemetry that rolling back the migration of traffic to its original infrastructure has mitigated impact. Next update by: Wednesday, July 21, 2021, at 12:00 AMUTC; PublishedTime=2021-07-19T23:50:04.987Z

--------------------------------------------------------------------------------------------------
Title Users are seeing blank pages when opening some PDF files within the SharePoint Online service
ID SP273670
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users were seeing blank pages when opening some PDF files within the SharePoint Online service.
Severity
Start time 2021-07-15T00:00:00Z
End time 2021-07-29T06:20:00Z
Last update 2021-07-29T17:49:48.337Z
More info This only impacted files opened in browsers or shown in a preview window. As a workaround, users could refresh the page to view the PDF files. Final status: We've deployed the fix to the affected environment and confirmed with impacted users that impact was remediated. Users may need to clear their browser cache to experience complete impact remediation. Scope of impact: This issue may have impact any of your users attempting to open some PDF files within the SharePoint Online service. Start time: Thursday, July 15, 2021, at 12:00 AM'UTC End time: Thursday, July 29, 2021, at 6:20 AM'UTC Root cause: A code issue in a recent deployment resulted in PDF files not opening as expected in a browser. Next steps: - We're reviewing our update procedures to better identify similar code issues in our service updates during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-07-29T17:49:48.337Z
Root cause A code issue in a recent deployment resulted in PDF files not opening as expected in a browser. Next steps: - We're reviewing our update procedures to better identify similar code issues in our service updates during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-07-29T17:49:48.337Z

SP273670 Details

Status: We're reviewing our internal reproduction of the issue and the details provided in the support case to determine our next troubleshooting steps. Next update by: Wednesday, July 28, 2021, at 12:30 PM UTC; PublishedTime=2021-07-28T10:41:29.717Z

Status: We've identified a recent code change is causing certain files not to load when opened. We're rolling back this change to mitigate the issue. Next update by: Wednesday, July 28, 2021, at 7:00 PM UTC; PublishedTime=2021-07-28T11:38:07.753Z

Status: We've developing a code fix as most expedient method for remediating impact. We'll provide more detail on the validation and deployment timelines once available. Next update by: Wednesday, July 28, 2021, at 11:00 PMUTC; PublishedTime=2021-07-28T18:19:39.68Z

Status: We've completed developing and validating the code fix; however, we're now reviewing whether applying this code fix or reverting the offending change will be the best solution for mitigating impact. Next update by: Thursday, July 29, 2021, at 1:00 AMUTC; PublishedTime=2021-07-28T21:51:48.22Z

Status: We've validated the code fix and are preparing to deploy it to the affected environment to remediate this issue. We estimate that the fix will be applied within the next 24 hours, and we'll provide an update on the deployment progress by our next scheduled update. Next update by: Thursday, July 29, 2021, at 7:00 PMUTC; PublishedTime=2021-07-28T23:52:19.753Z

Final status: We've deployed the fix to the affected environment and confirmed with impacted users that impact was remediated. Users may need to clear their browser cache to experience complete impact remediation. Next update by: Thursday, July 29, 2021, at 7:00 PMUTC; PublishedTime=2021-07-28T23:52:19.753Z

--------------------------------------------------------------------------------------------------
Title Some users can't load the Approvals app within the Microsoft Teams iOS app
ID TM270210
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have been unable to load the Approvals app within the Microsoft Teams iOS app.
Severity
Start time 2021-07-14T20:00:00Z
End time 2021-07-15T20:30:00Z
Last update 2021-07-16T02:53:07.92Z
More info This only affected users who were on the 3.11 build version of the Microsoft Teams iOS app. Final status: We've deployed the code fix and confirmed through telemetry that the issue is resolved. Users without automatic updates enabled will need to manually update their Teams iOS app for alleviation. Scope of impact: Any user on the 3.11 build version of the Microsoft Teams iOS app may have been impacted. Start time: Wednesday, July 14, 2021, at 8:00 PM'UTC End time: Thursday, July 15, 2021, at 8:30 PM'UTC Root cause: A recent Microsoft Teams iOS app build contained an unexpected code issue that was impacting users access to the Approvals app. Next steps: - We're reviewing the unexpected code issue to better understand what caused it and to assess how the problem wasn't identified during our update testing procedures. This is the final update for the event.; PublishedTime=2021-07-16T02:53:07.92Z
Root cause A recent Microsoft Teams iOS app build contained an unexpected code issue that was impacting users access to the Approvals app. Next steps: - We're reviewing the unexpected code issue to better understand what caused it and to assess how the problem wasn't identified during our update testing procedures. This is the final update for the event.; PublishedTime=2021-07-16T02:53:07.92Z

TM270210 Details

Status: We're reviewing available service health telemetry and diagnostics to isolate the source of impact. Next update by: Wednesday, July 14, 2021, at 11:30 PMUTC; PublishedTime=2021-07-14T22:33:21.167Z

Status: We identified a code issue in the 3.11 build version of the Microsoft Team iOS app. We've halted the deployment of the build and are developing a fix to remediate the impact for users with the 3.11 build installed. Next update by: Thursday, July 15, 2021, at 4:00 AMUTC; PublishedTime=2021-07-14T23:26:41.193Z

Status: We're developing a fix to change the responsible code and resolve impact. We'll then validate the code change to ensure it will successfully mitigate the issue without causing additional impact. Next update by: Thursday, July 15, 2021, at 6:00 PMUTC; PublishedTime=2021-07-15T03:00:47.043Z

Status: We're continuing our efforts to develop and validate the fix that is intended to change the responsible code and resolve impact. Once the fix is validated, we'll provide a timeline for deployment. Next update by: Friday, July 16, 2021, at 4:00 AMUTC; PublishedTime=2021-07-15T16:41:28.967Z

Final status: We've deployed the code fix and confirmed through telemetry that the issue is resolved. Users without automatic updates enabled will need to manually update their Teams iOS app for alleviation. Next update by: Friday, July 16, 2021, at 4:00 AMUTC; PublishedTime=2021-07-15T16:41:28.967Z

--------------------------------------------------------------------------------------------------
Title Users intermittently can't start new federated chats in Microsoft Teams via all connection methods
ID TM270106
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have been intermittently unable to start new federated chats in Microsoft Teams via all connection methods.
Severity
Start time 2021-07-14T15:21:00Z
End time 2021-07-14T16:23:00Z
Last update 2021-07-14T17:55:03.017Z
More info Existing federated chats were not affected. Additionally, users may have seen federated partners as "Unknown User" in Microsoft Teams chats and calls throughout the duration of this event. Final status: While we were in the process of gathering additional network trace data to isolate the source of the problem, system monitoring indicated that the service had returned to normal health. We've confirmed that the problem is no longer occurring. We'll continue to monitor the service to ensure that the problem does not happen again and are also working to identify what caused the underlying issue so that we can help prevent future occurrences. Scope of impact: Your organization was affected, and any user attempting to start new federated chats via any Microsoft Teams connection method may have experienced intermittent impact. Start time: Wednesday, July 14, 2021, at 3:21 PM UTC End time: Wednesday, July 14, 2021, at 4:23 PM UTC This is the final update for the event.; PublishedTime=2021-07-14T17:22:02.333Z
Root cause

TM270106 Details

Status: We've investigated a problem with our service in which users may have been unable to start new federated chats in Microsoft Teams, and we've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-07-14T16:53:10.537

Final status: While we were in the process of gathering additional network trace data to isolate the source of the problem, system monitoring indicated that the service had returned to normal health. We've confirmed that the problem is no longer occurring. We'll continue to monitor the service to ensure that the problem does not happen again and are also working to identify what caused the underlying issue so that we can help prevent future occurrences.

--------------------------------------------------------------------------------------------------
Title Some OneNote content created between May 14th, and May 18th, 2021, was not accessible
ID OP268688
Service Microsoft 365 Apps
Feature Office Client issues
Status Service restored
Impact Users may have noticed that some OneNote edits created between May 14th, 2021, and May 18th, 2021, were unavailable.
Severity
Start time 2021-07-12T21:34:58Z
End time 2021-08-03T23:30:00Z
Last update 2021-08-13T21:48:46.863Z
More info
Root cause A code issue caused OneNote content between May 14th, and May 18th, 2021, to not be saved properly. We've mitigated the issue going forward and included additional validations to prevent future occurrences of this issue.; PublishedTime=2021-08-05T23:58:56.167Z

OP268688 Details

Status: We've discovered an issue in which some of your users may notice that OneNote content that was edited by a user between May 14th, and May 18th, 2021, is unavailable. We're currently working to restore access to the impacted OneNote content, and we'll provide a timeline for restoration as soon as it becomes available. Next update by: Friday, June 16, 2021, at 11:30 PM UTC; PublishedTime=2021-07-12T21:39:36.543Z

Status: We're continuing to restore access to the impacted OneNote content. We expect this process to take an extended period to ensure access is properly restored. Next update by: Tuesday, July 20, 2021, at 11:30 PM UTC; PublishedTime=2021-07-16T22:57:11.097Z

Status: Our restoration efforts are progressing as expected. We continue to expect this work to take an extended period of time. At this point in the incident, we expect at least 95 percent of all users to be mitigated. Next update by: Friday, July 23, 2021, at 11:30 PM UTC; PublishedTime=2021-07-20T23:16:57.523Z

Status: Our recovery efforts are ongoing and approximately 98 percent of all users are no longer impacted by this event. We'll continue to work on restoring the remaining two percent. Next update by: Wednesday, July 28, 2021, at 11:30 PM UTC; PublishedTime=2021-07-23T23:12:01.117Z

Status: We've restored more than 99.5 percent of the impacted users; however, we're continuing our work to ensure all of our users have been successfully mitigated. We expect the remaining subset of users to be restored by our next update. Next update by: Wednesday, August 4, 2021, at 11:30 PM UTC; PublishedTime=2021-07-28T22:44:35.99Z

Status: We've completed our restoration efforts and we're continuing to monitor the service and for any customer reports that the issue is persisting. Next update by: Monday, August 9, 2021, at 11:30 PM UTC; PublishedTime=2021-08-03T22:59:10.013Z

Final status: We've received confirmation via internal monitoring and customer reports that our restoration efforts were successful. Next update by: Monday, August 9, 2021, at 11:30 PM UTC; PublishedTime=2021-08-03T22:59:10.013Z

--------------------------------------------------------------------------------------------------
Title Users may be intermittently unable to enroll devices and devices may fail to check-in, amongst other impact scenarios
ID IT268650
Service Microsoft Intune
Feature Microsoft Intune
Status Service restored
Impact Users may have been intermittently unable to enroll and check-in devices, amongst other impact scenarios.
Severity
Start time 2021-07-12T16:30:00Z
End time 2021-07-12T18:26:00Z
Last update 2021-07-12T20:10:01.097Z
More info Intermittent failures were occurring via multiple scenarios, though users may have predominantly noticed that device enrollments and check-ins failed. Final status: We've determined that an unexpected influx in naming services requests caused a subset of infrastructure that facilitates the affected Microsoft Intune scenarios to operate in a suboptimal state, which resulted in the impact. We've restarted the affected infrastructure to mitigate the impact and confirmed via telemetry that the service has fully recovered and this issue is resolved. Scope of impact: Any user hosted on the affected infrastructure may have experienced impact intermittently. Start time: Monday, July 12, 2021, at 4:30 PM UTC End time: Monday, July 12, 2021, at 6:26 PM UTC Root cause: An unexpected influx in naming services requests caused a subset of infrastructure that facilitates the affected Microsoft Intune scenarios to operate in a suboptimal state, resulting in impact. Next steps: - We're continuing our investigation to determine what caused the sudden influx in requests to occur, and why the traffic wasn't routed differently via automation prior to the impact to avoid issues like this from occurring in the future. This is the final update for the event.; PublishedTime=2021-07-12T19:38:22.207Z
Root cause An unexpected influx in naming services requests caused a subset of infrastructure that facilitates the affected Microsoft Intune scenarios to operate in a suboptimal state, resulting in impact. Next steps: - We're continuing our investigation to determine what caused the sudden influx in requests to occur, and why the traffic wasn't routed differently via automation prior to the impact to avoid issues like this from occurring in the future. This is the final update for the event.; PublishedTime=2021-07-12T19:38:22.207Z

IT268650 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-07-12T17:52:36.51

Status: We're reviewing system logs to isolate the origin of this issue. Next update by: Monday, July 12, 2021, at 8:00 PMUTC; PublishedTime=2021-07-12T18:16:54.913Z

Final status: We've determined that an unexpected influx in naming services requests caused a subset of infrastructure that facilitates the affected Microsoft Intune scenarios to operate in a suboptimal state, which resulted in the impact. We've restarted the affected infrastructure to mitigate the impact and confirmed via telemetry that the service has fully recovered and this issue is resolved. Next update by: Monday, July 12, 2021, at 8:00 PMUTC; PublishedTime=2021-07-12T18:16:54.913Z

--------------------------------------------------------------------------------------------------
Title Users only see suggested sites in Microsoft Viva topic cards even though there are pinned sites for the topic
ID MV277223
Service Microsoft Viva
Feature Topics
Status Service restored
Impact Users only saw suggested sites in Microsoft Viva topic cards even though there were pinned sites for the topic.
Severity
Start time 2021-07-12T08:35:00Z
End time 2021-08-17T18:50:00Z
Last update 2021-08-18T01:19:24.317Z
More info
Root cause A recent feature update contained a code regression that caused a communication issue between topic card and service databases, resulting in impact. Next steps: - To help prevent similar impact in the future, we're further reviewing why the feature update contained a code regression and how we can improve our validation procedures to catch such issues before deployment. This is the final update for the event.; PublishedTime=2021-08-18T01:19:24.317Z

MV277223 Details

Status: We've responded to reports of an issue where users are only able to see suggested sites in Microsoft Viva topic cards even though there are pinned sites for the topic. Our investigation indicates that a recent feature update contained a code regression that caused a communication issue between topic card and service databases, resulting in impact. We're working to develop a fix for the issue. Next update by: Friday, August 13, 2021, at 6:00 AMUTC; PublishedTime=2021-08-11T04:10:31.863Z

Status: We've successfully developed a fix and the deployment of the fix is ongoing. We anticipate the deployment to complete by end of day on Monday, August 16, 2021. Next update by: Monday, August 16, 2021, at 6:00 AMUTC; PublishedTime=2021-08-13T04:50:44.543Z

Status: Due to a complication with the deployment, our fix is taking longer than previously expected to finish saturating. We've remedied the delay and the fix is currently deployed to 50 percent of the affected environment. We expect the fix will be fully saturated and impact will be mitigated by Thursday, August 19, 2021. Next update by: Thursday, August 19, 2021, at 6:00 AM UTC; PublishedTime=2021-08-16T05:06:48.203Z

Final status: We've finished deploying the fix and confirmed via telemetry that service is restored. To ensure you receive the fix, please edit and republish your topic page. Next update by: Thursday, August 19, 2021, at 6:00 AM UTC; PublishedTime=2021-08-16T05:06:48.203Z

--------------------------------------------------------------------------------------------------
Title Some users may experience a blank page when accessing internal PDF links within SharePoint Online
ID SP270773
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users may have experienced a blank page when accessing internal PDF links within SharePoint Online.
Severity
Start time 2021-07-08T22:37:00Z
End time 2021-07-16T17:51:00Z
Last update 2021-07-17T04:00:24.833Z
More info Impact occurred only when using Internet Explorer 11 as a web browser. Other web browsers were not impacted by this event. SharePoint will no longer be supported on Internet Explorer 11 beginning August 17, 2021. For more information regarding this change, as well as supported browser options, see: https://docs.microsoft.com/en-us/sharepoint/prepare-ie11 Final status: Our fix has reached full saturation, and we've confirmed via monitoring telemetry and data gathered from validation in our testing environment that impact has been mitigated. Should the issue persist, we recommend affected users clear their Internet Explorer cache before again attempting to access internal PDF links in SharePoint Online. Scope of impact: Any user accessing internal PDF links via Internet Explorer 11 may have experienced impact. Start time: Thursday, July 8, 2021, at 10:37 PM UTC End time: Friday, July 16, 2021, at 5:51 PM UTC Root cause: A coding issue within a recent deployment designed to improve service performance inadvertently inhibited users' access to internal PDF links within SharePoint Online. Next steps: - We're reviewing our procedures for testing and validating updates, so we can expose and remediate code issues prior to future impact scenarios. This is the final update for the event.; PublishedTime=2021-07-16T20:13:29.537Z
Root cause A coding issue within a recent deployment designed to improve service performance inadvertently inhibited users' access to internal PDF links within SharePoint Online. Next steps: - We're reviewing our procedures for testing and validating updates, so we can expose and remediate code issues prior to future impact scenarios. This is the final update for the event.; PublishedTime=2021-07-16T20:13:29.537Z

SP270773 Details

Status: We're analyzing system logs to determine the source of the issue. Next update by: Friday, July 16, 2021, at 9:30 AM UTC; PublishedTime=2021-07-16T07:14:48.803Z

Status: We've identified a coding issue within a recent deployment designed to improve service performance which is inadvertently causing impact. We're developing a fix which we will test and validate before deploying to the affected infrastructure. Next update by: Friday, July 16, 2021, at 11:00 AM UTC; PublishedTime=2021-07-16T08:42:21.563Z

Status: We've successfully developed the fix which we're now making preparations to validate it within our internal testing environment, before deploying to the affected infrastructure. Next update by: Friday, July 16, 2021, at 2:30 PM UTC; PublishedTime=2021-07-16T10:01:38.347Z

Status: We've started validating the fix within our internal testing environment, which we will deploy to the affected infrastructure once complete. Next update by: Friday, July 16, 2021, at 7:00 PM UTC; PublishedTime=2021-07-16T13:17:20.49Z

Status: We've completed internal testing and validation of our fix, and have begun deployment to the affected infrastructure. We expect full saturation and subsequent remediation by our next scheduled update. Next update by: Friday, July 16, 2021, at 9:00 PMUTC; PublishedTime=2021-07-16T19:00:54.763Z

Final status: Our fix has reached full saturation, and we've confirmed via monitoring telemetry and data gathered from validation in our testing environment that impact has been mitigated. Should the issue persist, we recommend affected users clear their Internet Explorer cache before again attempting to access internal PDF links in SharePoint Online. Next update by: Friday, July 16, 2021, at 9:00 PMUTC; PublishedTime=2021-07-16T19:00:54.763Z

--------------------------------------------------------------------------------------------------
Title Users are missing calls to their Microsoft Teams for iOS app
ID TM267852
Service Microsoft Teams
Feature Teams Components
Status Post-incident report published
Impact Users aren't receiving calls to their Microsoft Teams for iOS app.
Severity
Start time 2021-07-08T10:10:00Z
End time 2021-07-08T16:55:00Z
Last update 2021-07-15T16:41:28.167Z
More info
Root cause The Notification Filtering Service (NFS), that facilitates Teams call notifications for iOS devices, recently deployed an update to the service rules configuration, which inadvertently caused most call notifications to be filtered out on iOS user devices. Android devices were not impacted by this event, as the platform does not currently utilize NFS for Teams call notifications. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. We'll publish a post-incident report within five business days.; PublishedTime=2021-07-08T22:13:46.807Z

TM267852 Details

Status: We're looking into your reported issue and checking for impact to your organization. We'll provide an update within one hour.; PublishedTime=2021-07-08T14:05:02.17

Status: We're reviewing service telemetry to determine the root cause of impact. Next update by: Thursday, July 8, 2021, at 4:30 PM UTC; PublishedTime=2021-07-08T14:43:35.787Z

Status: Our review of telemetry indicates an issue with a notification service that facilitates iOS calls. We're continuing our review of telemetry in conjunction with call samples provided by affected users to isolate the underlying cause and determine our mitigation actions. Next update by: Thursday, July 8, 2021, at 5:30 PMUTC; PublishedTime=2021-07-08T15:38:38.41Z

Status: Our review of telemetry and CallIDs has shown that a recent configuration update to a subset of notification systems that facilitate iOS call notifications contained an error, resulting in impact. We've reverted the change and are receiving reports of mitigation from affected users. We're continuing to monitor as the service recovers. Next update by: Thursday, July 8, 2021, at 7:30 PMUTC; PublishedTime=2021-07-08T17:18:45.55Z

Status: We're continuing to receive reports that this issue has been resolved for most users but are pending client-side telemetry to confirm mitigation. Next update by: Thursday, July 8, 2021, at 11:30 PMUTC; PublishedTime=2021-07-08T18:40:13.967Z

Final status: We've confirmed with impacted users and through internal telemetry that reverting the problematic change has been resolved the issue. Next update by: Thursday, July 8, 2021, at 11:30 PMUTC; PublishedTime=2021-07-08T18:40:13.967Z

--------------------------------------------------------------------------------------------------
Title Users may experience failures when performing flow management actions in Microsoft Dataverse for Teams
ID MF268674
Service Microsoft Power Automate in Microsoft 365
Feature Service and web access issues
Status Service restored
Impact Users may have experienced failures when performing flow management actions in Microsoft Dataverse for Teams.
Severity
Start time 2021-07-07T23:57:00Z
End time 2021-07-12T20:47:00Z
Last update 2021-07-12T21:18:56.313Z
More info Users were receiving the "0x80090477" error message when attempting to perform form management actions. Final status: Following the deployment of our code fix, we've internally validated that impact has been resolved. Scope of impact: Your organization was affected by this event, and this issue was impacting all North American users attempting to perform flow management actions in Microsoft Dataverse for Teams. Start time: Wednesday, July 7, 2021, at 11:57 PM'UTC End time: Monday, July 12, 2021, at 8:47 PM'UTC Root cause: A recent service update introduced a configuration issue that was preventing flow management actions in Microsoft Dataverse for Teams. Next steps: -We're reviewing our service update procedures to help identify similar configuration issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-07-12T21:18:13.433Z
Root cause A recent service update introduced a configuration issue that was preventing flow management actions in Microsoft Dataverse for Teams. Next steps: -We're reviewing our service update procedures to help identify similar configuration issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-07-12T21:18:13.433Z

MF268674 Details

Status: We've determined that a recent service update has introduced a code configuration issue that is producing errors for users attempting to perform flow management actions in the Microsoft Dataverse for Teams. We're internally validating a code fix that is expected to remediate impact. Next update by: Monday, July 12, 2021, at 9:30 PMUTC; PublishedTime=2021-07-12T19:48:42.317Z

Final status: Following the deployment of our code fix, we've internally validated that impact has been resolved. Next update by: Monday, July 12, 2021, at 9:30 PMUTC; PublishedTime=2021-07-12T19:48:42.317Z

--------------------------------------------------------------------------------------------------
Title Admins can't view email previews from the Email entity page in the Microsoft 365 Defender portal
ID EX268922
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Admins were seeing a blank pane when opening email previews from the Email entity page within Microsoft 365 Defender.
Severity
Start time 2021-07-07T02:52:00Z
End time 2021-07-19T01:47:00Z
Last update 2021-07-19T19:32:40.927Z
More info
Root cause A recent service update introduced a code regression for email previews in the Microsoft 365 Defender portal. Next steps: - We're reviewing our service update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-07-19T19:32:40.927Z

EX268922 Details

Status: We're reviewing system telemetry and logs to isolate the underlying cause of impact and develop mitigation actions. Next update by: Tuesday, July 13, 2021, at 8:30 PMUTC; PublishedTime=2021-07-13T19:04:02.63Z

Status: We're expanding our investigation to include recent service updates to the Email entity page and viewing pane to assist in the root cause analysis. Next update by: Wednesday, July 14, 2021, at 2:00 AMUTC; PublishedTime=2021-07-13T19:46:02.867Z

Status: We're continuing our investigation by reviewing recent service updates to the Email entity page and viewing pane to help with determining the underlying root cause of this issue. Next update by: Wednesday, July 14, 2021, at 2:00 PMUTC; PublishedTime=2021-07-14T00:27:41.883Z

Status: We've confirmed that recent service updates to the email Entity page and viewing pane have not caused impact to occur. We're reviewing backend components that facilitate email details for preview, to help identify the underlying root cause of this issue. Next update by: Thursday, July 15, 2021, at 7:30 AM UTC; PublishedTime=2021-07-14T12:30:25.293Z

Status: Our investigation found that a recent service update introduced a code regression for email previews in the Microsoft 365 Defender portal. We've developed a fix for the regression and are undergoing validation procedures to ensure it successfully resolves impact. We expect it to start deployment by Thursday, July 15, 2021, and to be fully saturated by Sunday, July 18, 2021. Next update by: Thursday, July 15, 2021, at 7:30 PMUTC; PublishedTime=2021-07-15T06:01:35.53Z

Status: Validation of the fix is nearly complete, and we expect for the deployment to be initiated to the affected environment in the next few hours. Once deployment begins, we anticipate that the fix will take two to three days to fully saturate the affected environment. Impact will be mitigated for users as they receive the fix, and we expect for the issue to be fully resolved for all affected users by our next update. Next update by: Monday, July 19, 2021, at 7:30 PMUTC; PublishedTime=2021-07-15T18:09:21.04Z

Final status: We've validated our fix, began the deployment, and confirmed that this fix has been applied to most of the affected customers. Next update by: Monday, July 19, 2021, at 7:30 PMUTC; PublishedTime=2021-07-15T18:09:21.04Z

--------------------------------------------------------------------------------------------------
Title Some users may have been unable to perform search queries within the Exchange Online service
ID EX267002
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users may have been unable to perform search queries within the Exchange Online service.
Severity
Start time 2021-07-06T09:05:00Z
End time 2021-07-06T09:28:00Z
Last update 2021-07-06T10:17:07.057Z
More info
Root cause A recent service update, which was part of regularly scheduled maintenance, unexpectedly resulted in a configuration issue with a network authorization component. Next steps: - We're reviewing our deployment procedures to help prevent similar problems in the future. This is the final update for the event.; PublishedTime=2021-07-06T10:15:20.82Z

EX267002 Details

Status: We've identified that a recent deployment is causing a configuration issue, resulting in impact. We're rolling back this build to mitigate the issue. Next update by: Tuesday, July 6, 2021, at 11:30 AM UTC; PublishedTime=2021-07-06T09:25:20.087Z

Final status: We've confirmed that the roll back of the offending flight has completed and impact has been mitigated. Next update by: Tuesday, July 6, 2021, at 11:30 AM UTC; PublishedTime=2021-07-06T09:25:20.087Z

--------------------------------------------------------------------------------------------------
Title Around 25 percent of users may intermittently have been unable to access the Microsoft Yammer service in North America
ID YA267096
Service Yammer Enterprise
Feature Yammer Components
Status Service restored
Impact Users may have intermittently been unable to access the Microsoft Yammer service in North America
Severity
Start time 2021-07-05T19:04:00Z
End time 2021-07-06T20:15:07Z
Last update 2021-07-06T20:42:13.76Z
More info
Root cause An unexpected influx in traffic caused a subset of infrastructure that facilitated access to the Microsoft Yammer service to operate in a sub-optimal state, resulting in intermittent access issues. Next steps: - We're reviewing the scenario that led to the unexpected traffic spike to prevent issues like this from occurring in the future. This is the final update for the event.; PublishedTime=2021-07-06T20:15:07.563Z

YA267096 Details

Final status: We have determined that an unexpected influx in traffic caused a subset of infrastructure that facilitated access to the Microsoft Yammer service to operate in a sub-optimal state. This resulted in intermittent access issues. We successfully routed traffic through a healthy infrastructure and confirmed that this issue is resolved.

--------------------------------------------------------------------------------------------------
Title Admins were seeing extraneous Shadow IT telemetry for Windows Server 2019 devices in Microsoft Cloud App Security (MCAS)
ID CS279055
Service Cloud App Security
Feature Microsoft Cloud App Security
Status Service restored
Impact Admins were seeing extraneous Shadow IT telemetry for Windows Server 2019 devices in MCAS.
Severity
Start time 2021-07-05T00:00:00Z
End time 2021-08-23T11:00:00Z
Last update 2021-08-25T12:58:47.127Z
More info For more information regarding Microsoft Defender for Endpoint integration with Microsoft Cloud App Security, please reference the following link: https://docs.microsoft.com/en-us/cloud-app-security/mde-integration Final status: We've confirmed that the fix has fully saturated and that impact has been mitigated. Scope of impact: Your organization may have been affected by this event, and any admin may have been impacted. Start time: Monday, July 5, 2021, at 12:00 AM UTC End time: Monday, August 23, 2021, at 11:00 AM UTC Root cause: A configuration issue specific to MCAS was resulting in extraneous Shadow IT telemetry for Windows Server 2019 devices. Next steps: - We're reviewing our MCAS configuration procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-08-25T12:58:47.127Z
Root cause A configuration issue specific to MCAS was resulting in extraneous Shadow IT telemetry for Windows Server 2019 devices. Next steps: - We're reviewing our MCAS configuration procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-08-25T12:58:47.127Z

CS279055 Details

Status: We've identified a configuration issue specific to the MCAS service in which admins are seeing extraneous Shadow IT telemetry for Windows Server 2019 devices. To mitigate this issue and align with the intended functionality, we'll turn off Shadow IT reporting for Windows Server 2019 devices in MCAS on Monday, August 23, 2021. At that time, users may notice a drop in traffic reports in MCAS due to the lack in traffic from these types of devices. We'll provide an update on Monday, August 23, 2021, by 1:30 PM UTC, with additional information regarding the completed mitigation action. Next update by: Monday, August 23, 2021, at 1:30 PM UTC; PublishedTime=2021-08-18T18:36:10.7Z

Status: The mitigation action is taking longer than expected to fix the issue. We're continuing to monitor it's progress until the fix has fully saturated. Next update by: Wednesday, August 25, 2021, at 1:30 PM UTC; PublishedTime=2021-08-23T13:26:06.957Z

Final status: We've confirmed that the fix has fully saturated and that impact has been mitigated. Next update by: Wednesday, August 25, 2021, at 1:30 PM UTC; PublishedTime=2021-08-23T13:26:06.957Z

--------------------------------------------------------------------------------------------------
Title Users in Chile can't access one or more Microsoft 365 services
ID MO266345
Service Microsoft 365 suite
Feature Portal
Status Investigation suspended
Impact Users may be unable to access one or more Microsoft 365 services.
Severity
Start time 2021-07-02T11:30:00Z
End time 2021-07-02T13:45:00Z
Last update 2021-07-02T14:32:36.583Z
More info Users who are able to access the service may have experienced latency. Final status: We've identified that an issue within a local ISP's infrastructure is resulting in impact. We've confirmed that the Microsoft-managed environment is healthy and performing as expected. Scope of impact: Impact is specific to a subset of users who were accessing the service from geographical locations in Chile and surrounding areas. This is the final update for the event.; PublishedTime=2021-07-02T14:08:03.683Z
Root cause

MO266345 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're reviewing telemetry and gathering further diagnostic data from the network infrastructure to isolate the source of the issue. Next update by: Friday, July 2, 2021, at 3:00 PM UTC; PublishedTime=2021-07-02T13:28:09.367Z

Final status: We've identified that an issue within a local ISP's infrastructure is resulting in impact. We've confirmed that the Microsoft-managed environment is healthy and performing as expected. Next update by: Friday, July 2, 2021, at 3:00 PM UTC; PublishedTime=2021-07-02T13:28:09.367Z

--------------------------------------------------------------------------------------------------
Title Users can't see the "Allow external presenter" option when setting up a Microsoft Teams live event.
ID TM266885
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were unable to see the "Allow external presenter" option when setting up a Microsoft Teams live event.
Severity
Start time 2021-07-01T23:52:00Z
End time 2021-07-05T14:26:00Z
Last update 2021-07-05T15:31:14.23Z
More info
Root cause A recent configuration change to the feature contained an issue, causing impact. Next steps: - We're reviewing this recent configuration change to determine why it was misconfigured and how this wasn't caught in our configuration testing and validation phases to help prevent similar problems from reoccurring in the future. This is the final update for the event.; PublishedTime=2021-07-05T15:31:14.23Z

TM266885 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-07-05T14:58:22.097

Final status: We've determined that a recent configuration change to the feature contained an issue, causing impact. We've corrected the feature configuration, and we've confirmed through system telemetry and reports by affected users that the issue is resolved.

--------------------------------------------------------------------------------------------------
Title Poly Cloud Video Interop users may experience intermittent video issues in large Microsoft Teams meetings
ID TM270277
Service Microsoft Teams
Feature Teams Components
Status Investigation suspended
Impact Poly Cloud Video Interop users may experience intermittent video issues in large Microsoft Teams meetings.
Severity
Start time 2021-07-01T07:00:00Z
End time 2021-07-15T04:12:51Z
Last update 2021-07-15T04:28:07.573Z
More info The issue is specific to connections through Cloud Video Interop users calling into Microsoft Teams conferences. If the large gallery view is enabled in the call, users may no longer see video feed. Final status: After further investigation, we identified that this issue was specific to Poly Cloud Video Interop (CVI) users. The impact is specific to the Microsoft Large Gallery view in Microsoft Teams meetings. We're actively working with Polycom to implement a fix for the issue and remediate the impact. Further communication regarding this issue will be provided by Polycom. Scope of impact: This issue could potentially intermittently affect any Poly Cloud Video Interop user calling into Microsoft Teams conferences. Start time: Thursday, July 1, 2021, at 7:00 AM'UTC This is the final update for the event.; PublishedTime=2021-07-15T04:12:59.343Z
Root cause

TM270277 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-07-15T02:43:06.787

Status: We're reviewing meeting logs to help identify the source of the issue and to determine our next troubleshooting steps. Next update by: Thursday, July 15, 2021, at 4:30 AMUTC; PublishedTime=2021-07-15T03:06:50.827Z

Final status: After further investigation, we identified that this issue was specific to Poly Cloud Video Interop (CVI) users. The impact is specific to the Microsoft Large Gallery view in Microsoft Teams meetings. We're actively working with Polycom to implement a fix for the issue and remediate the impact. Further communication regarding this issue will be provided by Polycom. Next update by: Thursday, July 15, 2021, at 4:30 AMUTC; PublishedTime=2021-07-15T03:06:50.827Z

--------------------------------------------------------------------------------------------------
Title Admins can't update conditional access policies for their SharePoint Online site collections
ID SP274273
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Admins were unable to update conditional access policies for their SharePoint Online site collections.
Severity
Start time 2021-07-01T05:06:00Z
End time 2021-08-16T13:38:00Z
Last update 2021-08-18T19:20:49.653Z
More info As a workaround, affected admins could have used Plug and Play PowerShell to update conditional access policies. Final status: We've confirmed that the fix has completed deployment to all affected infrastructure and confirmed mitigation via telemetry. Scope of impact: This issue may have potentially affected any admins attempting to update conditional access policies for SharePoint Online site collections. Start time: Thursday, July 1, 2021, at 5:06 AM'UTC End time: Monday, August 16, 2021, at 1:38 PM'UTC Root cause: A recent service change introduced a code issue which resulted in impact. Next steps: - We're continuing our review of the code that facilitates conditional access to identify and prevent similar occurrences in the future. This is the final update for the event.; PublishedTime=2021-08-18T19:03:44.73Z
Root cause A recent service change introduced a code issue which resulted in impact. Next steps: - We're continuing our review of the code that facilitates conditional access to identify and prevent similar occurrences in the future. This is the final update for the event.; PublishedTime=2021-08-18T19:03:44.73Z

SP274273 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-07-30T14:45:30.71

Status: We're reviewing the networking traces, usage logs, and Unified Logging Service (ULS) data provided by affected admins to validate what we suspect is a recent service change which is resulting in this impact. To help us validate this hypothesis, we're also gathering additional detail pertaining to this change so we can establish a mitigation approach which is targeted accordingly. Next update by: Friday, July 30, 2021, at 7:30 PMUTC; PublishedTime=2021-07-30T15:04:10.747Z

Status: We've confirmed a recent service change introduced a code issue which resulted in impact. We've developed a code fix to mitigate the issue which is undergoing testing and validation prior to initiating deployment. Next update by: Monday, August 2, 2021, at 8:00 PMUTC; PublishedTime=2021-07-30T19:21:32.257Z

Status: Following our internal testing and validation, we've deployed our code fix to the affected environments. We're monitoring the fix deployment as it is safely implemented through our change management process, and are expecting for it to have remediated impact by Monday, August 23, 2021. Next update by: Monday, August 23, 2021, at 8:00 PMUTC; PublishedTime=2021-08-02T18:26:28.69Z

Final status: We've confirmed that the fix has completed deployment to all affected infrastructure and confirmed mitigation via telemetry. Next update by: Monday, August 23, 2021, at 8:00 PMUTC; PublishedTime=2021-08-02T18:26:28.69Z

--------------------------------------------------------------------------------------------------
Title Users who are members of groups were not getting Microsoft Teams group policies assigned
ID TM268026
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users who were members of groups were not getting Microsoft Teams group policies assigned.
Severity
Start time 2021-06-30T13:05:00Z
End time 2021-07-09T23:00:00Z
Last update 2021-07-10T09:15:07.573Z
More info
Root cause A recent build update intended as a general product update contained a regression that resulted in the Group Policy Worker service to fail to update users' group policies. Next steps: - We're reviewing our update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-07-10T09:15:07.573Z

TM268026 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We've determined that a recent build update intended as a general product update contained a regression that resulted in the Group Policy Worker service to fail to update users' group policies. We're reverting the update to mitigate impact. Next update by: Friday, July 9, 2021, at 11:00 AMUTC; PublishedTime=2021-07-09T06:45:18.483Z

Status: We're monitoring the service while we continue to revert the update, to mitigate the issue. Next update by: Saturday, July 10, 2021, at 11:00 AM UTC; PublishedTime=2021-07-09T10:37:39.56Z

Final status: We've reverted the update and can confirm the issue has been mitigated. Next update by: Saturday, July 10, 2021, at 11:00 AM UTC; PublishedTime=2021-07-09T10:37:39.56Z

--------------------------------------------------------------------------------------------------
Title Admins may see Microsoft 365 app usage and productivity score reports data delayed after June 30, 2021
ID MO266326
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Admins may have seen Microsoft 365 app usage and productivity score reports data delayed after June 30, 2021.
Severity
Start time 2021-06-30T00:00:00Z
End time 2021-07-08T00:00:00Z
Last update 2021-07-08T05:00:20.95Z
More info
Root cause A recent service update introduced a code regression in a service component responsible for tracking user activity throughout multiple Microsoft 365 apps, resulting in impact. Next steps: - We're reviewing our deployment and provisioning procedures to help identify and prevent similar problems from being introduced in the future. This is the final update for the event.; PublishedTime=2021-07-08T05:00:20.95Z

MO266326 Details

Status: We received alerts from our automated monitoring regarding data delays for Microsoft 365 app usage reports and productivity reports after June 30, 2021. Our investigation determined that there's an upstream data source issue that's resulting in impact. We're working to determine the cause of this issue and explore potential mitigation actions. Next update by: Saturday, July 3, 2021, at 7:30 AMUTC; PublishedTime=2021-07-02T05:54:00.587Z

Status: Our investigation has found that a recent service update introduced a code regression in a service component responsible for tracking user activity throughout multiple Microsoft 365 apps, resulting in impact. We've developed and deployed a fix for the issue, and we're working to replay the delayed data. Next update by: Sunday, July 4, 2021, at 7:30 AMUTC; PublishedTime=2021-07-03T06:03:25.05Z

Status: We're still in the process of replaying the delayed data. We'll have a more accurate timeline for completion on our next scheduled update. Next update by: Monday, July 5, 2021, at 7:30 AMUTC; PublishedTime=2021-07-04T05:52:23.413Z

Status: We've determined that the reports are still delayed. We're continuing our efforts to replay the delayed reporting data which we anticipate will mitigate impact. Next update by: Wednesday, July 7, 2021, at 7:00 AMUTC; PublishedTime=2021-07-05T05:47:56.107Z

Status: Our monitoring indicates that Microsoft 365 app usage reports are up to date, and we expect productivity score reports to be completely replayed by our next scheduled update. Next update by: Thursday, July 8, 2021, at 7:00 AMUTC; PublishedTime=2021-07-07T05:11:18.197Z

Final status: We've confirmed that productivity score reports are now up to date, remediating impact. Next update by: Thursday, July 8, 2021, at 7:00 AMUTC; PublishedTime=2021-07-07T05:11:18.197Z

--------------------------------------------------------------------------------------------------
Title Admins were unable to add, remove or update permissions for public folders in the Exchange Admin Center
ID EX272010
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Admins were unable to add, remove or update permissions for public folders in the Exchange Admin Center.
Severity
Start time 2021-06-29T08:51:00Z
End time 2021-07-25T00:00:00Z
Last update 2021-07-28T13:29:36.127Z
More info
Root cause The issue was caused by a null exception, resulting in impact. Next steps: - We're reviewing our deployment and provisioning procedures to help prevent similar problems in the future. This is the final update for the event.; PublishedTime=2021-07-28T13:06:03.383Z

EX272010 Details

Status: We're reviewing support case details to determine the next troubleshooting steps. Next update by: Wednesday, July 21, 2021, at 11:00 AM UTC; PublishedTime=2021-07-21T08:59:41.02Z

Status: We're analyzing system telemetry to identify the underlying root cause. Next update by: Wednesday, July 21, 2021, at 1:00 PM UTC; PublishedTime=2021-07-21T10:59:42.673Z

Status: We've determined that the issue is being caused by a null exception. We're adding a null check, which once deployed will fix the issue. The change will take approximately 3 to 4 weeks to be deployed. Next update by: Wednesday, July 28, 2021, at 1:00 PM UTC; PublishedTime=2021-07-21T13:11:42.96Z

Final status: The null check was successfully deployed and we've confirmed, after a period of monitoring, that the issue has been mitigated. Next update by: Wednesday, July 28, 2021, at 1:00 PM UTC; PublishedTime=2021-07-21T13:11:42.96Z

--------------------------------------------------------------------------------------------------
Title Admins can't access the Microsoft Secure Score webpage via the Microsoft 365 security center
ID MO265916
Service Microsoft 365 suite
Feature Portal
Status Service restored
Impact Admins were unable to access the Microsoft Secure Score webpage via the Microsoft 365 security center.
Severity
Start time 2021-06-29T08:13:00Z
End time 2021-06-30T20:11:00Z
Last update 2021-06-30T20:43:45.757Z
More info Admins may have seen the following error message when attempting to access the Microsoft Secure Score webpage via the Microsoft 365 security center: "Something went wrong We have encountered an error loading this page, please try again later: TyperError: Cannot read property 'UserName' of undefined." Final status: We've identified that a recent JavaScript package change to the Secure Score webpage unexpectedly resulted in impact. We've reverted this offending change and after monitoring the service, we've verified that this action has resolved the issue. Scope of impact: This impact may have occurred for all admins attempting to load the Microsoft Secure Score webpage via the Microsoft 365 security center. Start time: Tuesday, June 29, 2021, at 8:13 AM UTC End time: Wednesday, June 30, 2021, at 8:11 PM UTC Root cause: A recent JavaScript package change to the Secure Score webpage unexpectedly resulted in impact. Next steps: - We're investigating how impact was missed during our testing and validation processes for the offending change to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-06-30T20:42:49.26Z
Root cause A recent JavaScript package change to the Secure Score webpage unexpectedly resulted in impact. Next steps: - We're investigating how impact was missed during our testing and validation processes for the offending change to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-06-30T20:42:49.26Z

MO265916 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're reviewing available diagnostic data to isolate the source of impact and identify troubleshooting actions. Next update by: Wednesday, June 30, 2021, at 6:00 PMUTC; PublishedTime=2021-06-30T16:42:09.897Z

Status: We're continuing to review available diagnostic data, as well as recent service changes, to isolate the source of impact and identify our troubleshooting actions. Next update by: Wednesday, June 30, 2021, at 8:00 PMUTC; PublishedTime=2021-06-30T17:39:39.31Z

Status: Due to the complexity of this issue, we're continuing to assess any recent offending service changes to isolate the source of this issue and identify an optimal mitigation action. Next update by: Wednesday, June 30, 2021, at 10:00 PMUTC; PublishedTime=2021-06-30T19:42:03.943Z

Final status: We've identified that a recent JavaScript package change to the Secure Score webpage unexpectedly resulted in impact. We've reverted this offending change and after monitoring the service, we've verified that this action has resolved the issue. Next update by: Wednesday, June 30, 2021, at 10:00 PMUTC; PublishedTime=2021-06-30T19:42:03.943Z

--------------------------------------------------------------------------------------------------
Title Users can't schedule Microsoft Teams meetings in Google calendar utilizing a Microsoft Teams add-on
ID TM265933
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were unable to schedule Microsoft Teams meetings in Google calendar utilizing a Microsoft Teams add-on.
Severity
Start time 2021-06-29T07:59:00Z
End time 2021-07-01T01:00:00Z
Last update 2021-07-01T19:50:27.823Z
More info Users were prompted to sign into Microsoft Teams when scheduling Microsoft Teams meetings in Google calendar utilizing the Teams meeting add-on for Google Workspace. The sign in failed, and the users received the following error: 'Add-on error: Something went wrong when executing the add-on.' Final status: Our parallel investigation into the corresponding code portion and the third-party deployment process was unable to identify a problem with the Microsoft-managed service. However, we suspect a possible authentication configuration issue was introduced during a third-party deployment process, which has since been resolved. After extended monitoring we've confirmed the errors correlated with this issue are no longer present, and the impact is mitigated. Scope of impact: Any user attempting to schedule Microsoft Teams meetings in Google calendar utilizing the Teams meeting add-on for Google Workspace was impacted. Start time: Tuesday, June 29, 2021, at 7:59 AM UTC End time: Thursday, July 1, 2021, at 1:00 AM UTC Root cause: A suspected authentication configuration issue was introduced during a third-party deployment process, resulting in impact. Next steps: - We'll continue to monitor the service health to ensure the issue doesn't reoccur. In parallel, our efforts to coordinate with Google to validate the underlying cause of the suspected configuration issue are ongoing so that we can prevent similar problems from occurring in the future. This is the final update for the event.; PublishedTime=2021-07-01T19:50:27.823Z
Root cause A suspected authentication configuration issue was introduced during a third-party deployment process, resulting in impact. Next steps: - We'll continue to monitor the service health to ensure the issue doesn't reoccur. In parallel, our efforts to coordinate with Google to validate the underlying cause of the suspected configuration issue are ongoing so that we can prevent similar problems from occurring in the future. This is the final update for the event.; PublishedTime=2021-07-01T19:50:27.823Z

TM265933 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're analyzing user provided information related to impact, including screenshots of the issue, to determine the source of the issue. Next update by: Wednesday, June 30, 2021, at 10:30 PMUTC; PublishedTime=2021-06-30T18:00:27.843Z

Status: We've identified a possible authentication configuration issue which is presented when users are attempting to schedule Microsoft Teams meetings in Google calendar utilizing the Teams meeting add-on for Google Workspace. We're investigating the corresponding code portion to isolate the source of the issue. In parallel, we're working with Google to identify any possible discrepancies which may have been introduced during the third-party deployment process. Next update by: Thursday, July 1, 2021, at 9:00 PMUTC; PublishedTime=2021-06-30T21:06:34.48Z

Final status: Our parallel investigation into the corresponding code portion and the third-party deployment process was unable to identify a problem with the Microsoft-managed service. However, we suspect a possible authentication configuration issue was introduced during a third-party deployment process, which has since been resolved. After extended monitoring we've confirmed the errors correlated with this issue are no longer present, and the impact is mitigated. Next update by: Thursday, July 1, 2021, at 9:00 PMUTC; PublishedTime=2021-06-30T21:06:34.48Z

--------------------------------------------------------------------------------------------------
Title Microsoft Cloud App Security portal inaccessible
ID CS265041
Service Cloud App Security
Feature Cloud App Security
Status Service restored
Impact Users may have been unable to access the Microsoft Cloud App Security portal.
Severity
Start time 2021-06-27T12:27:00Z
End time 2021-06-27T13:55:00Z
Last update 2021-06-27T14:12:07.93Z
More info
Root cause Recent changes to our service routing infrastructure inadvertently caused impact. Next steps: - We're reviewing our service routing maintenance procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-06-27T14:12:07.93Z

CS265041 Details

Status: We're gathering diagnostic data for analysis. Next update by: Sunday, June 27, 2021, at 2:00 PM UTC; PublishedTime=2021-06-27T14:01:50.527Z

Final status: We identified that recent changes to some of our service routing infrastructure inadvertently caused impact. We've corrected these changes and confirmed that the issue is resolved, after a period of monitoring. Next update by: Sunday, June 27, 2021, at 2:00 PM UTC; PublishedTime=2021-06-27T14:01:50.527Z

--------------------------------------------------------------------------------------------------
Title Admins can't access the Cloud App Security portal
ID CS264832
Service Cloud App Security
Feature Microsoft Cloud App Security
Status Service restored
Impact Admins were unable to access the Cloud App Security portal.
Severity
Start time 2021-06-26T14:35:00Z
End time 2021-06-26T16:00:00Z
Last update 2021-06-26T16:56:57.953Z
More info
Root cause An authorization component misconfiguration in the Cloud App Security portal Domain Name System (DNS) occurred, resulting in impact. Next steps: - We're investigating how the authorization component misconfiguration occurred to prevent this problem from happening again in the future. This is the final update for the event.; PublishedTime=2021-06-26T16:55:02.767Z

CS264832 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We're reviewing available diagnostic data, such as application logs, to isolate the source of impact and identify troubleshooting actions. Next update by: Saturday, June 26, 2021, at 5:00 PMUTC; PublishedTime=2021-06-26T15:42:08.513Z

Final status: We've identified that an authorization component misconfiguration in the Cloud App Security portal Domain Name System (DNS) occurred, resulting in impact. We've connected the Cloud App Security portal DNS service to an alternative infrastructure to alleviate impact and after monitoring the service, we've verified that this action has successfully resolved the problem. Next update by: Saturday, June 26, 2021, at 5:00 PMUTC; PublishedTime=2021-06-26T15:42:08.513Z

--------------------------------------------------------------------------------------------------
Title Some users may be unable to use the search function in SharePoint Online
ID SP264644
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users may have been unable to use the search function in SharePoint Online.
Severity
Start time 2021-06-25T18:00:00Z
End time 2021-06-25T22:30:00Z
Last update 2021-06-25T22:59:11.127Z
More info
Root cause A recent update inadvertently caused a code issue where certain item properties returned by the SharePoint Online search weren't processed correctly to occur more frequently, resulting in impact. Next steps: - We're reviewing the code issue to better understand how impact began, and to explore potential options to prevent similar impact from recurring. This is the final update for the event.; PublishedTime=2021-06-25T22:59:11.127Z

SP264644 Details

Status: We're analyzing internal telemetry to determine the source of the issue. Next update by: Friday, June 25, 2021, at 10:30 PMUTC; PublishedTime=2021-06-25T21:14:51.133Z

Status: We're continuing to analyze internal telemetry data to confirm whether a recent change may be the cause of this issue, allowing us to create a strategy that remediates impact. Next update by: Saturday, June 26, 2021, at 12:30 AMUTC; PublishedTime=2021-06-25T22:08:32.86Z

Final status: We've confirmed that a recent update inadvertently caused a code issue where certain item properties returned by the SharePoint Online search weren't processed correctly to occur more frequently, resulting in impact. We've reverted that update and confirmed via service monitoring that this has remediated impact. Next update by: Saturday, June 26, 2021, at 12:30 AMUTC; PublishedTime=2021-06-25T22:08:32.86Z

--------------------------------------------------------------------------------------------------
Title Some admins may see a delay in updated data for Skype for Business usage reports within the Microsoft 365 admin center
ID MO265224
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Admins may have seen a delay in updated data for Skype for Business usage reports within the Microsoft 365 admin center.
Severity
Start time 2021-06-24T00:00:00Z
End time 2021-06-29T10:00:00Z
Last update 2021-07-01T06:06:18.61Z
More info Admins may have seen a delay in data beginning from June 24, 2021. Final status: We've identified failures within the downstream source path that are responsible for impact. We've developed and deployed a fix, and confirmed via system telemetry that admins are now able to see previously delayed data, mitigating impact. Scope of impact: Impact was specific to any admin running the Skype for Business usage report. Start time: Thursday, June 24, 2021, at 12:00 AM'UTC End time: Tuesday, June 29, 2021, at 10:00 AM'UTC Root cause: Failures in a downstream source path were causing Skype for Business report data to not update after June 24, 2021. Next steps: - We're reviewing the downstream source path to better understand how impact began, and to explore options to prevent similar impact in the future. This is the final update for the event.; PublishedTime=2021-06-30T06:14:12.59Z
Root cause Failures in a downstream source path were causing Skype for Business report data to not update after June 24, 2021. Next steps: - We're reviewing the downstream source path to better understand how impact began, and to explore options to prevent similar impact in the future. This is the final update for the event.; PublishedTime=2021-06-30T06:14:12.59Z

MO265224 Details

Status: We've identified a downstream source path is causing Skype for Business report data to not update after June 24, 2021. We're reviewing data source logs to identify the next steps to mitigate the issue. Next update by: Wednesday, June 30, 2021, at 8:00 AM UTC; PublishedTime=2021-06-28T08:59:13.073Z

Final status: We've identified failures within the downstream source path that are responsible for impact. We've developed and deployed a fix, and confirmed via system telemetry that admins are now able to see previously delayed data, mitigating impact. Next update by: Wednesday, June 30, 2021, at 8:00 AM UTC; PublishedTime=2021-06-28T08:59:13.073Z

--------------------------------------------------------------------------------------------------
Title Admins will see processing delays since Wednesday, June 23, 2021, within the Microsoft Cloud App Security activity log
ID CS264551
Service Cloud App Security
Feature Cloud App Security
Status Service restored
Impact Admins were seeing processing delays of up to 16 hours within the Microsoft Cloud App Security activity log.
Severity
Start time 2021-06-23T10:14:00Z
End time 2021-06-26T20:00:00Z
Last update 2021-06-27T15:46:34.317Z
More info
Root cause An expired authentication component in the affected environment caused a communication problem that delayed the processing of activities. This caused a backlog of activities to build up since Wednesday, June 23, 2021. Next steps: - We're assessing our authentication component expiration notification and renewal procedures to prevent similar issues from occurring in the future. This is the final update for the event.; PublishedTime=2021-06-27T15:46:08.073Z

CS264551 Details

Status: We're analyzing support case data to determine why activities aren't being processed as expected. Next update by: Friday, June 25, 2021, at 4:30 PMUTC; PublishedTime=2021-06-25T15:13:38.48Z

Status: We've identified that an expired authentication component in the affected environment caused a communication problem that prevented activities from being processed. This caused a backlog of activities to build up since Thursday, June 24, 2021. We've increased the components expiration date, allowing it to function normally. The activity log backlog delayed up to 16 hours but it's processing and may take some time to complete. We're monitoring the backlog to ensure it processes completely. Next update by: Monday, June 28, 2021, at 6:00 PMUTC; PublishedTime=2021-06-25T16:30:07.513Z

Final status: We've determined that impact from the backlogged activities began impacting users earlier than previously calculated, and users began to experience processing delays within the Microsoft Cloud App Security activity log starting on Wednesday, June 23, 2021. We've confirmed that extending the authentication component expiration and processing the activity backlog has resolved the impact. Next update by: Monday, June 28, 2021, at 6:00 PMUTC; PublishedTime=2021-06-25T16:30:07.513Z

--------------------------------------------------------------------------------------------------
Title Admins may be unable to export enrollment failure reports in the Microsoft Endpoint Manager admin center
ID IT265608
Service Microsoft Intune
Feature Microsoft Intune
Status Service restored
Impact Admins may have been unable to export enrollment failure reports in the Microsoft Endpoint Manager admin center.
Severity
Start time 2021-06-23T05:00:00Z
End time 2021-06-29T22:50:00Z
Last update 2021-06-30T00:25:10.463Z
More info Admins may have seen the following error message when attempting to export enrollment failure reports in the Microsoft Endpoint Manager admin center via Microsoft Intune: "File download error." Final status: Our fix has successfully saturated across all affected service environments, and we've confirmed via monitoring that impact has been mitigated. Scope of impact: Any admin trying to export enrollment failure reports may have been affected. Start time: Wednesday, June 23, 2021, at 5:00 AM'UTC End time: Tuesday, June 29, 2021, at 10:50 PM'UTC Root cause: A recent change that was intended to update the Microsoft Endpoint Manager admin center user interface functionality unexpectedly resulted in impact. Next steps: - We're reviewing the impacting update to better understand how impact began, and to explore options to prevent similar impact in the future. This is the final update for the event.; PublishedTime=2021-06-30T00:16:24.603Z
Root cause A recent change that was intended to update the Microsoft Endpoint Manager admin center user interface functionality unexpectedly resulted in impact. Next steps: - We're reviewing the impacting update to better understand how impact began, and to explore options to prevent similar impact in the future. This is the final update for the event.; PublishedTime=2021-06-30T00:16:24.603Z

IT265608 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.

Status: We've identified a recent change that was intended to update the Microsoft Endpoint Manager admin center user interface functionality unexpectedly resulted in impact. We've identified a potential fix for this issue and are validating the fix now before applying it to the affected environments. Next update by: Tuesday, June 29, 2021, at 7:00 PMUTC; PublishedTime=2021-06-29T17:36:52.04Z

Status: We're continuing to validate the potential fix before applying it to the affected environments. We'll provide an update to the deployment timeline once available. Next update by: Tuesday, June 29, 2021, at 9:00 PMUTC; PublishedTime=2021-06-29T18:38:04.573Z

Status: We've validated the fix and have begun deployment to the affected environments. Additionally, we're monitoring the fix as it saturates to ensure it completes successfully and expect to provide a timeline for mitigation by our next update. Next update by: Wednesday, June 30, 2021, at 2:00 AMUTC; PublishedTime=2021-06-29T20:22:22.73Z

Final status: Our fix has successfully saturated across all affected service environments, and we've confirmed via monitoring that impact has been mitigated. Next update by: Wednesday, June 30, 2021, at 2:00 AMUTC; PublishedTime=2021-06-29T20:22:22.73Z

--------------------------------------------------------------------------------------------------
Title Some users intermittently see only profile pictures or avatars instead of video during Microsoft Teams meetings
ID TM286944
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were intermittently seeing only profile pictures or avatars instead of video during Microsoft Teams meetings.
Severity Sev2
Start time 2021-06-23T00:00:00Z
End time 2021-10-12T11:35:00Z
Last update 2021-10-12T16:55:26.82Z
More info Users may have been able to disable and then re-enable their video in the meeting for temporary relief from this problem. Final status: The fix has completed deploying to all formerly affected environments and we've confirmed via monitoring telemetry that mitigation has been achieved for all affected users. Scope of impact: This issue may have intermittently impacted a subset of your users and our telemetry indicates that only a small number of organizations were impacted by this issue. Start time: Wednesday, June 23, 2021, at 12:00 AM'UTC End time: Tuesday, October 12, 2021, at 11:35 AM'UTC Root cause: A recent update to resolve an unrelated issue resulted in a configuration issue on a component that facilitates some Microsoft Teams meeting video tasks, resulting in impact. Next steps: - We're reviewing our update testing methodology to isolate and remediate potential configuration issues prior to future update deployments. This is the final update for the event.; PublishedTime=2021-10-12T16:28:29.497Z
Root cause A recent update to resolve an unrelated issue resulted in a configuration issue on a component that facilitates some Microsoft Teams meeting video tasks, resulting in impact. Next steps: - We're reviewing our update testing methodology to isolate and remediate potential configuration issues prior to future update deployments. This is the final update for the event.; PublishedTime=2021-10-12T16:28:29.497Z

TM286944 Details

Status: We've determined that a recent update to resolve an unrelated issue has resulted in a configuration issue on a component that facilitates some Microsoft Teams meeting video tasks, resulting in impact. We've developed a fix for this issue which is currently being validated in a test environment. We expect that the validation and safe rollout process for this fix will take several weeks. We'll provide a better estimate to deployment to your environment at our next scheduled update. Scope of Impact: This issue may intermittently impact a subset of your users and our telemetry indicates that only a small number of organizations may be impacted by this issue. Root cause: A recent update to resolve an unrelated issue has resulted in a configuration issue on a component that facilitates some Microsoft Teams meeting video tasks, resulting in impact. Next update by: Monday, September 27, 2021, at 5:00 PMUTC; PublishedTime=2021-09-23T17:57:19.917Next update by: Monday, September 27, 2021, at 5:00 PMUTC; PublishedTime=2021-09-23T17:57:19.917Z

Status: Our fix has been validated in our test environment and committed for safe deployment, a process that is expected to take several weeks to complete. We're unable to provide an accurate estimation for completing the safe deployment of the fix at this juncture; however, we're monitoring this process to ensure that the deployment stays on track and will provide an updated estimate of the fix completion when available. Scope of Impact: This issue may intermittently impact a subset of your users and our telemetry indicates that only a small number of organizations may be impacted by this issue. Root cause: A recent update to resolve an unrelated issue has resulted in a configuration issue on a component that facilitates some Microsoft Teams meeting video tasks, resulting in impact. Next update by: Monday, October 4, 2021, at 5:00 PMUTC; PublishedTime=2021-09-27T16:16:27.937Next update by: Monday, October 4, 2021, at 5:00 PMUTC; PublishedTime=2021-09-27T16:16:27.937Z

Status: We estimate that the fix will completely deploy by Tuesday, October 12, 2021. We'll continue monitoring the deployment process to ensure that the deployment stays on track and completes as expected. Scope of Impact: This issue may intermittently impact a subset of your users and our telemetry indicates that only a small number of organizations may be impacted by this issue. Root cause: A recent update to resolve an unrelated issue has resulted in a configuration issue on a component that facilitates some Microsoft Teams meeting video tasks, resulting in impact. Next update by: Friday, October 8, 2021, at 5:00 PMUTC; PublishedTime=2021-10-04T16:26:46.83Next update by: Friday, October 8, 2021, at 5:00 PMUTC; PublishedTime=2021-10-04T16:26:46.83Z

Status: The fix is still on track to be fully deployed and remediate impact by Tuesday, October 12, 2021. We'll continue monitoring until this deployment completes to confirm that impact is remediated. Scope of Impact: This issue may intermittently impact a subset of your users and our telemetry indicates that only a small number of organizations may be impacted by this issue. Estimated time to resolve: We expect that the issue will be fully resolved by Tuesday, October 12, 2021. Root cause: A recent update to resolve an unrelated issue has resulted in a configuration issue on a component that facilitates some Microsoft Teams meeting video tasks, resulting in impact. Next update by: Tuesday, October 12, 2021, at 5:00 PMUTC; PublishedTime=2021-10-08T15:39:20.143Next update by: Tuesday, October 12, 2021, at 5:00 PMUTC; PublishedTime=2021-10-08T15:39:20.143Z

Final status: The fix has completed deploying to all formerly affected environments and we've confirmed via monitoring telemetry that mitigation has been achieved for all affected users. Next update by: Tuesday, October 12, 2021, at 5:00 PMUTC; PublishedTime=2021-10-08T15:39:20.143Z

--------------------------------------------------------------------------------------------------
Title Users in Canada may have degraded service functionality across all Microsoft 365 services
ID MO263732
Service Microsoft 365 suite
Feature Portal
Status Post-incident report published
Impact Users may have been unable to access Microsoft 365 services or had degraded functionality.
Severity
Start time 2021-06-22T14:27:00Z
End time 2021-06-22T16:12:00Z
Last update 2021-07-06T15:38:45.673Z
More info Examples of Impacted scenarios included users being unable to connect to Exchange Online, open documents in SharePoint via Office for the web, create or start Skype Meeting Broadcasts, use live event Q&A in Microsoft Teams, or perform Create, Read, Update and Delete (CRUD) operations in Power Automate. Final status: We've completed our rebalancing efforts and monitored the environments to confirm system health, validating mitigation via telemetry. Scope of impact: Any user serviced by the affected infrastructure in the Canada region may have been impacted. Start time: Tuesday, June 22, 2021, at 2:27 PM'UTC End time: Tuesday, June 22, 2021, at 4:12 PM'UTC Root cause: Between 2:27 PM and 4:12 PM UTC on 22 Jun 2021, a subset of users may have experienced intermittent connectivity failures or latency when attempting to access Microsoft 365 services in Canada East from Canada Central. Recovery time varied by service, and most services fully recovered by 4:12 UTC. Microsoft leverages a local carrier's network for connectivity to Canada East from Canada Central data centers. During a routine fiber relocation in the carrier's network, an unexpected change to the physical path was made, which cross-connected our routers. This type of error usually prevents routing from occurring over the bad path, and we seamlessly fail-over to redundant paths. However, this specific error resulted in a rare and unexpected topology configuration, which partially activated routing over the bad path and subsequently caused a subset of IPv4 traffic to be dropped on networking links between Canada East and Canada Central. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. This is the final update for the event.; PublishedTime=2021-06-22T17:21:18.21Z
Root cause Between 2:27 PM and 4:12 PM UTC on 22 Jun 2021, a subset of users may have experienced intermittent connectivity failures or latency when attempting to access Microsoft 365 services in Canada East from Canada Central. Recovery time varied by service, and most services fully recovered by 4:12 UTC. Microsoft leverages a local carrier's network for connectivity to Canada East from Canada Central data centers. During a routine fiber relocation in the carrier's network, an unexpected change to the physical path was made, which cross-connected our routers. This type of error usually prevents routing from occurring over the bad path, and we seamlessly fail-over to redundant paths. However, this specific error resulted in a rare and unexpected topology configuration, which partially activated routing over the bad path and subsequently caused a subset of IPv4 traffic to be dropped on networking links between Canada East and Canada Central. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. This is the final update for the event.; PublishedTime=2021-06-22T17:21:18.21Z

MO263732 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-06-22T15:28:31.753

Status: We've identified a networking issue within a portion of datacenter infrastructure, resulting in impact. We're reviewing network traces and logs to isolate the underlying cause and develop mitigation actions. Next update by: Tuesday, June 22, 2021, at 5:00 PMUTC; PublishedTime=2021-06-22T15:46:25.747Z

Status: Our review of network logs has confirmed that a subset of IPv4 networking links between Canada region datacenters is dropping packets. We're rebalancing load to alternate links to mitigate impact while isolating the underlying cause of impact. Next update by: Tuesday, June 22, 2021, at 6:00 PMUTC; PublishedTime=2021-06-22T16:33:10.447Z

Final status: We've completed our rebalancing efforts and monitored the environments to confirm system health, validating mitigation via telemetry. Next update by: Tuesday, June 22, 2021, at 6:00 PMUTC; PublishedTime=2021-06-22T16:33:10.447Z

--------------------------------------------------------------------------------------------------
Title Some users may be intermittently unable to complete searches in Outlook on the web and Outlook desktop
ID EX266081
Service Exchange Online
Feature E-Mail and calendar access
Status Service restored
Impact Users may have been intermittently unable to complete searches in Outlook on the web and the Outlook desktop client.
Severity
Start time 2021-06-22T08:00:00Z
End time 2021-07-01T23:00:00Z
Last update 2021-07-02T22:23:11.663Z
More info Affected users would likely see an error message when searching that states: "We're having trouble fetching results from the server..." Final status: We've confirmed that disabling the synthetic traffic has reduced traffic and confirmed mitigation via telemetry. Scope of impact: Impact was specific to some users who were served through the affected infrastructure. Start time: Tuesday, June 22, 2021, at 8:00 AM'UTC End time: Thursday, July 1, 2021, at 11:00 PM'UTC Root cause: An unexpected increase in synthetic traffic resulted in impact. Next steps: - We're continuing our investigation of system logs as to the cause of the traffic and working to develop potential automated mitigation actions and prevent future occurrences. This is the final update for the event.; PublishedTime=2021-07-02T22:23:11.663Z
Root cause An unexpected increase in synthetic traffic resulted in impact. Next steps: - We're continuing our investigation of system logs as to the cause of the traffic and working to develop potential automated mitigation actions and prevent future occurrences. This is the final update for the event.; PublishedTime=2021-07-02T22:23:11.663Z

EX266081 Details

Status: We're reviewing a potential authentication issue that may be resulting in the search failures. Next update by: Thursday, July 1, 2021, at 2:00 AMUTC; PublishedTime=2021-07-01T00:44:54.06Z

Status: We're continuing our review of a potential authentication issue that may be causing the search failures. Next update by: Thursday, July 1, 2021, at 4:00 AMUTC; PublishedTime=2021-07-01T01:55:01.663Z

Status: Based on our monitoring, we can see that failures rates are decreasing at the moment. Our code review for a potential authentication issue is progressing, albeit at a slower rate than we initially anticipated. Initial analysis indicates that there may be an issue within the tokens, and we're working to validate if this is the case or a symptom of the underlying issue. Next update by: Thursday, July 1, 2021, at 6:00 AMUTC; PublishedTime=2021-07-01T03:59:36.687Z

Status: Our investigation into the authentication issue is taking time to progress. We're continuing to validate a potential token issue that may be contributing to the failures. Next update by: Thursday, July 1, 2021, at 3:00 PMUTC; PublishedTime=2021-07-01T05:53:22.16Z

Status: We've determined that there is a latency issue occurring within the authentication pipeline which we believe is causing the impact. We're analyzing the authentication pipeline, to determine the next mitigation steps. Next update by: Thursday, July 1, 2021, at 5:00 PM UTC; PublishedTime=2021-07-01T14:33:04.137Z

Status: We're continuing to review our authentication pipeline and a group of delayed authentication requests as we determine our next steps for remediating impact. Next update by: Friday, July 2, 2021, at 4:00 PMUTC; PublishedTime=2021-07-01T15:17:32.227Z

Status: Our investigation suggests that unusually high traffic may have caused the affected infrastructure to stop operating as efficiently as expected, impacting users' ability to conduct Outlook searches. We've disabled synthetic traffic to reduce machine load, and are continuing to monitor telemetry to ensure this provides alleviation. Next update by: Friday, July 2, 2021, at 11:00 PMUTC; PublishedTime=2021-07-02T15:57:11.897Z

Final status: We've confirmed that disabling the synthetic traffic has reduced traffic and confirmed mitigation via telemetry. Next update by: Friday, July 2, 2021, at 11:00 PMUTC; PublishedTime=2021-07-02T15:57:11.897Z

--------------------------------------------------------------------------------------------------
Title Some Exchange Online and hybrid Exchange Online users were unable to sign in to the service from any connection method
ID EX268014
Service Exchange Online
Feature Sign-in
Status Service restored
Impact Users were unable to sign in to Exchange Online from any connection method.
Severity Sev2
Start time 2021-06-22T07:00:00Z
End time 2021-07-14T15:00:00Z
Last update 2021-09-23T21:17:41.57Z
More info This issue was specific to Exchange Online and hybrid Exchange users hosted on infrastructure in North and West Europe, as well as East United States. Final status: We've confirmed the deployed fix has completely saturated within the infrastructure and determined impact has been remediated after monitoring the environment. Scope of impact: This issue impacted a subset of Exchange Online and hybrid Exchange Online users hosted on infrastructure in North and West Europe, as well as East United States, whose mailboxes were unable to be detected correctly by the AutoDetect service. Start time: Tuesday, June 22, 2021, at 7:00 AM UTC End time: Wednesday, July 14, 2021, at 3:00 PM UTC Root cause: A code regression during a platform migration resulted in our AutoDetect system to perform protocol actions, overloading Domain Name System (DNS) queries and causing requests to timeout. Next steps: - We're reviewing our platform migration procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-07-16T11:26:49.683Z
Root cause A code regression during a platform migration resulted in our AutoDetect system to perform protocol actions, overloading Domain Name System (DNS) queries and causing requests to timeout. Next steps: - We're reviewing our platform migration procedures to avoid similar impact in the future. This is the final update for the event.; PublishedTime=2021-07-16T11:26:49.683Z

EX268014 Details

Status: We've received reports that a subset of Microsoft 365 and hybrid Exchange Online users hosted on infrastructure in North and West Europe, as well as East United States are unable to sign in to the service through any connection method. Our investigation indicates that the affected users' mailboxes are unable to be detected correctly by the AutoDetect service. We're reviewing provided logs and recent updates to determine our mitigation plan. Next update by: Wednesday, July 14, 2021, at 5:00 AMUTC; PublishedTime=2021-07-09T03:05:45.017Z

Status: We've identified that Domain Name System (DNS) queries are timing out when AutoDetect fails to detect users' mailboxes, resulting in impact. We suspect that a code regression is causing the problem. We've developed a script to scan for affected domains based on a specific set of rules which would result in impact, and apply new configurations to query the database for undetectable mailboxes to avoid the timeout issue and mitigate impact for all users. We anticipate this script will be run and mitigation complete by our next scheduled update. Next update by: Monday, July 12, 2021, at 7:30 PM UTC; PublishedTime=2021-07-09T21:01:35.473Z

Status: We've identified the domains that have been affected by this issue and have completed mitigation efforts on approximately ten percent of affected domains. We're continuing to resolve the remaining domains and some users may experience progressive remediation as the script progresses. We anticipate that this process will complete by our next scheduled update. Next update by: Monday, July 12, 2021, at 7:30 PMUTC; PublishedTime=2021-07-11T02:13:25.227Z

Status: Our deployed fix to alleviate impact for the affected domains is taking longer than expected; however, we've confirmed via service telemetry that 53 percent of impacted domains have received our mitigation efforts. We're continuing to monitor the deployment process to ensure it is completes as expected, and we'll provide more information regarding an anticipated ETA to full remediation by our next scheduled update. Next update by: Friday, July 16, 2021, at 1:00 PMUTC; PublishedTime=2021-07-12T18:18:32.827Z

Final status: We've confirmed the deployed fix has completely saturated within the infrastructure and determined impact has been remediated after monitoring the environment. Next update by: Friday, July 16, 2021, at 1:00 PMUTC; PublishedTime=2021-07-12T18:18:32.827Z

--------------------------------------------------------------------------------------------------
Title Some users may have been unable to post messages or load their Home Feed in Yammer
ID YA263514
Service Yammer Enterprise
Feature Yammer Components
Status Service restored
Impact Users may have been unable to post messages or load their Home Feed in Yammer.
Severity
Start time 2021-06-21T18:54:00Z
End time 2021-06-21T19:44:00Z
Last update 2021-06-21T21:55:18.107Z
More info
Root cause Routing components that facilitate message posting and Home Feed loading stopped processing traffic as expected in the affected environment, resulting in impact. Next steps: - We're analyzing performance data and trends on the affected systems to identify the root cause of impact and help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-06-21T21:55:18.107Z

YA263514 Details

Status: We've investigated a problem with the Yammer service in which some users may have been unable to post messages or load their Home Feed in Yammer. We've confirmed that service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-06-21T21:18:03.42

Final status: We identified that routing components that facilitate message posting and Home Feed loading stopped processing traffic as expected in the affected environment, resulting in impact. We routed traffic away from the affected infrastructure and restarted each degraded component to remediate impact.

--------------------------------------------------------------------------------------------------
Title Some admins may have experienced delayed data in Productivity score reports from the Microsoft 365 admin center
ID MO265231
Service Microsoft 365 suite
Feature Portal
Status Service restored
Impact Admins may have experienced delayed data in Productivity score reports from the Microsoft 365 admin center.
Severity
Start time 2021-06-20T00:00:00Z
End time 2021-06-29T20:00:00Z
Last update 2021-06-30T07:32:01.393Z
More info Admins may have seen delayed data beginning from the 20th June, 2021. Final status: Whilst analyzing source logs, our automated recovery system restored missing profiles in the downstream source path. We've confirmed that all the data from the 20th June 2021 has been backfilled and the service has been restored. Scope of impact: Impact was specific to any admin running the Productivity score report. Start time: Sunday, June 20, 2021, at 12:00 AM UTC End time: Tuesday, June 29, 2021, at 8:00 PM UTC Root cause: A downstream source path was causing Productivity score report data to not update after Sunday, 20th June, 2021. Next steps: - We're reviewing our data source path procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-06-30T07:32:01.393Z
Root cause A downstream source path was causing Productivity score report data to not update after Sunday, 20th June, 2021. Next steps: - We're reviewing our data source path procedures to find ways to prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-06-30T07:32:01.393Z

MO265231 Details

Status: We determined a downstream source path is causing Productivity score report data to not update after Sunday, 20th June, 2021. We're analyzing data source logs to identify the next steps to mitigate the impact. Next update by: Wednesday, June 30, 2021, at 9:00 AM UTC; PublishedTime=2021-06-28T09:32:53.417Z

Final status: Whilst analyzing source logs, our automated recovery system restored missing profiles in the downstream source path. We've confirmed that all the data from the 20th June 2021 has been backfilled and the service has been restored. Next update by: Wednesday, June 30, 2021, at 9:00 AM UTC; PublishedTime=2021-06-28T09:32:53.417Z

--------------------------------------------------------------------------------------------------
Title Admins may see missing results after June 17, 2021, from some Exchange Online usage reports
ID MO263432
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Admins may have seen missing results after June 17, 2021, from some Exchange Online usage reports.
Severity
Start time 2021-06-19T00:00:00Z
End time 2021-06-22T22:00:00Z
Last update 2021-06-23T06:15:19.517Z
More info Affected reports included: - Email activity report. - Email app usage report. - Mailbox usage report. Final status: We determined that a portion of the infrastructure that hosts historic Exchange Online data was performing below acceptable thresholds, resulting in a backlog of data. We processed the backlog and backfilled the relevant data to mitigate the issue. Scope of impact: This issue may have affected any admin attempting to view the affected usage reports. Start time: Saturday, June 19, 2021, at 12:00 AM'UTC End time: Tuesday, June 22, 2021, at 10:00 PM'UTC Root cause: A portion of the infrastructure that hosts historic Exchange Online data was performing below acceptable thresholds, resulting in a backlog of data. Next steps: - We're investigating the cause of the degraded performance issue to find ways of preventing this issue from reoccurring. This is the final update for the event.; PublishedTime=2021-06-23T06:15:19.517Z
Root cause A portion of the infrastructure that hosts historic Exchange Online data was performing below acceptable thresholds, resulting in a backlog of data. Next steps: - We're investigating the cause of the degraded performance issue to find ways of preventing this issue from reoccurring. This is the final update for the event.; PublishedTime=2021-06-23T06:15:19.517Z

MO263432 Details

Status: We've been processing a backlog of historic Exchange Online usage data, which has resulted in the usage data after June 17, 2021, not being included in reports. We're developing a mitigation strategy to resolve this issue. Next update by: Wednesday, June 23, 2021, at 8:00 AM UTC; PublishedTime=2021-06-21T13:11:59.79Z

Final status: We determined that a portion of the infrastructure that hosts historic Exchange Online data was performing below acceptable thresholds, resulting in a backlog of data. We processed the backlog and backfilled the relevant data to mitigate the issue. Next update by: Wednesday, June 23, 2021, at 8:00 AM UTC; PublishedTime=2021-06-21T13:11:59.79Z

--------------------------------------------------------------------------------------------------
Title Users may not receive Microsoft 365 mobile apps configuration policies applied via the Microsoft 365 Apps admin center
ID IT262630
Service Microsoft Intune
Feature Microsoft Intune
Status Service restored
Impact Users may not have received Microsoft 365 mobile apps configuration policies via the Microsoft 365 Apps admin center.
Severity
Start time 2021-06-16T23:28:00Z
End time 2021-06-17T02:30:00Z
Last update 2021-06-17T04:04:18.323Z
More info Additionally, admins that configured the Microsoft 365 mobile apps policies via the Microsoft 365 Apps admin center would have been unable to see, add, or update the policies. Final status: We've completed the process to redeploy the service with the configuration fix and our telemetry shows that the issue has been mitigated. Scope of impact: This issue could have potentially affected any of your users' devices that receive Microsoft 365 mobile configuration policies applied via the Microsoft 365 Apps admin center. Start time: Wednesday, June 16, 2021, at 11:28 PM'UTC End time: Thursday, June 17, 2021, at 2:30 AM'UTC Root cause: An infrastructure configuration change was resulting in impact. Next steps: - We're investigating the configuration update to determine way of preventing this from happening again. This is the final update for the event.; PublishedTime=2021-06-17T04:04:18.323Z
Root cause An infrastructure configuration change was resulting in impact. Next steps: - We're investigating the configuration update to determine way of preventing this from happening again. This is the final update for the event.; PublishedTime=2021-06-17T04:04:18.323Z

IT262630 Details

Status: We've determined that devices that receive Microsoft 365 mobile configuration policies applied via the Microsoft 365 apps admin center may not see them take effect. We've found that an infrastructure configuration change is resulting in impact. We're redeploying the service with a configuration update, which will be complete in approximately three hours. Users and admins may see progressive mitigation as the deployment saturates. Next update by: Thursday, June 17, 2021, at 4:00 AM UTC; PublishedTime=2021-06-17T01:17:44.647Z

Status: The process to redeploy the service is taking longer than expected; however, our telemetry indicates that the service is healthy and no users are currently impacted. We now estimate that the redeployment will be done within the next two hours. Next update by: Thursday, June 17, 2021, at 6:00 AMUTC; PublishedTime=2021-06-17T03:53:50.17Z

Final status: We've completed the process to redeploy the service with the configuration fix and our telemetry shows that the issue has been mitigated. Next update by: Thursday, June 17, 2021, at 6:00 AMUTC; PublishedTime=2021-06-17T03:53:50.17Z

--------------------------------------------------------------------------------------------------
Title Users are unable to use the video sharing feature within Microsoft Teams desktop client on certain Mac devices
ID TM262707
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users were unable to use the video sharing feature within Microsoft Teams desktop client on certain Mac devices.
Severity
Start time 2021-06-16T23:20:00Z
End time 2021-06-17T14:42:00Z
Last update 2021-06-17T16:10:41.277Z
More info Users may have been able to access the video sharing feature via the Microsoft Team web client and mobile app. Final status: We've identified a recent configuration change meant to improve the video sharing resolution for the Microsoft Teams desktop client on Mac M1 devices contained a code error that caused the configuration to become invalid, resulting in impact. We've corrected this code issue and confirmed via internal testing that impact was remediated. Scope of impact: Impact was specific to users attempting to use video sharing from the Microsoft Teams desktop client on Mac M1 devices. Start time: Wednesday, June 16, 2021, at 11:20 PM UTC End time: Thursday, June 17, 2021, at 2:42 PM UTC Root cause: A recent configuration change meant to improve the video sharing resolution for the Microsoft Teams desktop client on Mac M1 devices contained a code issue that caused the configuration to become invalid, resulting in the degradation of the video sharing feature. Next steps: - We're reviewing our update procedures to better identify similar code issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-06-17T16:03:23.407Z
Root cause A recent configuration change meant to improve the video sharing resolution for the Microsoft Teams desktop client on Mac M1 devices contained a code issue that caused the configuration to become invalid, resulting in the degradation of the video sharing feature. Next steps: - We're reviewing our update procedures to better identify similar code issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-06-17T16:03:23.407Z

TM262707 Details

Status: We're investigating a potential issue with video sharing. We'll provide an update within 30 minutes.; PublishedTime=2021-06-17T14:20:23.55

Status: We're reviewing recent configuration changes to isolate the source of the issue. Next update by: Thursday, June 17, 2021, at 4:30 PM UTC; PublishedTime=2021-06-17T14:36:25.487Z

Final status: We've identified a recent configuration change meant to improve the video sharing resolution for the Microsoft Teams desktop client on Mac M1 devices contained a code error that caused the configuration to become invalid, resulting in impact. We've corrected this code issue and confirmed via internal testing that impact was remediated. Next update by: Thursday, June 17, 2021, at 4:30 PM UTC; PublishedTime=2021-06-17T14:36:25.487Z

--------------------------------------------------------------------------------------------------
Title Some users are unable to utilize sensitivity labels within the Power BI service
ID PB262677
Service Power BI
Feature PowerBI.com
Status Service restored
Impact Users were unable to utilize sensitivity labels within the Power BI service.
Severity
Start time 2021-06-16T22:00:00Z
End time 2021-06-17T14:35:00Z
Last update 2021-06-17T15:50:19.11Z
More info
Root cause A code regression in a recent deployment was causing issues with the authentication of tokens used to get the labels for multiple Microsoft applications. Next steps: - We're reviewing this recent deployment to understand why the code regression was missed during our testing and validation processes. This is the final update for the event.; PublishedTime=2021-06-17T15:50:19.11Z

PB262677 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-06-17T10:00:40.31

Status: We've identified a recent deployment to the affected infrastructure is causing issues with the authentication of tokens, used to get the labels for Microsoft applications. We're halting the deployment to prevent further impact. Additionally, we're preparing to roll back the deployment on the affected infrastructure to mitigate impact. Next update by: Thursday, June 17, 2021, at 12:00 PM UTC; PublishedTime=2021-06-17T10:11:48.97Z

Status: We've halted the deployment so no further impact will occur. We're identifying the changes made in the recent deployment that need to be rolled back to mitigate impact. Next update by: Thursday, June 17, 2021, at 2:00 PM UTC; PublishedTime=2021-06-17T11:35:51.233Z

Status: We're deploying a patch to the affected infrastructure, which we anticipate will correct the changes made from the impacting change. We're monitoring the progress of the patch and expect complete saturation in approximately two hours. Next update by: Thursday, June 17, 2021, at 4:00 PM UTC; PublishedTime=2021-06-17T13:53:20.117Z

Final status: We've confirmed the deployment of the patch to the affected infrastructure has completed, and after monitoring telemetry we've validated that it has successfully mitigated impact. Next update by: Thursday, June 17, 2021, at 4:00 PM UTC; PublishedTime=2021-06-17T13:53:20.117Z

--------------------------------------------------------------------------------------------------
Title Some users are unable to utilize sensitivity labels within the Power BI service
ID PB262677
Service Power BI
Feature PowerBI.com
Status Service restored
Impact Users were unable to utilize sensitivity labels within the Power BI service.
Severity
Start time 2021-06-16T22:00:00Z
End time 2021-06-17T14:35:00Z
Last update 2021-06-17T15:50:19.11Z
More info
Root cause A code regression in a recent deployment was causing issues with the authentication of tokens used to get the labels for multiple Microsoft applications. Next steps: - We're reviewing this recent deployment to understand why the code regression was missed during our testing and validation processes. This is the final update for the event.; PublishedTime=2021-06-17T15:50:19.11Z

PB262677 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-06-17T10:00:40.31

Status: We've identified a recent deployment to the affected infrastructure is causing issues with the authentication of tokens, used to get the labels for Microsoft applications. We're halting the deployment to prevent further impact. Additionally, we're preparing to roll back the deployment on the affected infrastructure to mitigate impact. Next update by: Thursday, June 17, 2021, at 12:00 PM UTC; PublishedTime=2021-06-17T10:11:48.97Z

Status: We've halted the deployment so no further impact will occur. We're identifying the changes made in the recent deployment that need to be rolled back to mitigate impact. Next update by: Thursday, June 17, 2021, at 2:00 PM UTC; PublishedTime=2021-06-17T11:35:51.233Z

Status: We're deploying a patch to the affected infrastructure, which we anticipate will correct the changes made from the impacting change. We're monitoring the progress of the patch and expect complete saturation in approximately two hours. Next update by: Thursday, June 17, 2021, at 4:00 PM UTC; PublishedTime=2021-06-17T13:53:20.117Z

Final status: We've confirmed the deployment of the patch to the affected infrastructure has completed, and after monitoring telemetry we've validated that it has successfully mitigated impact. Next update by: Thursday, June 17, 2021, at 4:00 PM UTC; PublishedTime=2021-06-17T13:53:20.117Z

--------------------------------------------------------------------------------------------------
Title Users noticing that calls through Microsoft Teams are going to voicemail
ID TM261472
Service Microsoft Teams
Feature Teams Components
Status Post-incident report published
Impact Users may notice that calls through Microsoft Teams are going to voicemail.
Severity
Start time 2021-06-11T13:50:00Z
End time 2021-06-13T19:00:00Z
Last update 2021-06-18T14:42:47.33Z
More info Affected users with access to the Microsoft Teams web app may have been able to use that platform as an alternate solution. Some mobile device users may have not received push notifications as expected during this event. Final status: We've deployed our solution and have confirmed via monitoring our service telemetry that impact has been remediated. Scope of impact: This issue may have impacted any of your Microsoft Teams desktop client or mobile app users. Start time: Friday, June 11, 2021, at 1:50 PM'UTC End time: Sunday, June 13, 2021, at 7:00 PM'UTC Root cause: A recent configuration change prevented the infrastructure responsible for generating call notifications from confirming an active session for end users, resulting in calls being sent directly to voicemail. When a call is placed, the notification service will query a central repository (Registrar), to gather a list of the call recipient's active endpoints, and then rings those active endpoint devices. In a situation where the notification service queries Registrar for active endpoints and a 404 error is returned, it is assumed that there are no active endpoints, and therefore the call is routed to voicemail. For service security, each active endpoint uses a hashing method which has a custom salt for hashing the information. If the Registrar is unable to find the primary key calculated from the hash, a 404 error is generated. The issue was triggered after a regular hashing key renewal resulted in device registrations being stored under the incorrect primary key that was used to query a user's device list. This resulted in empty device lists for users and the service assuming there were no active endpoints, therefore sending calls to voicemail. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. We'll publish a post-incident report within five business days.; PublishedTime=2021-06-13T19:30:25.443Z
Root cause A recent configuration change prevented the infrastructure responsible for generating call notifications from confirming an active session for end users, resulting in calls being sent directly to voicemail. When a call is placed, the notification service will query a central repository (Registrar), to gather a list of the call recipient's active endpoints, and then rings those active endpoint devices. In a situation where the notification service queries Registrar for active endpoints and a 404 error is returned, it is assumed that there are no active endpoints, and therefore the call is routed to voicemail. For service security, each active endpoint uses a hashing method which has a custom salt for hashing the information. If the Registrar is unable to find the primary key calculated from the hash, a 404 error is generated. The issue was triggered after a regular hashing key renewal resulted in device registrations being stored under the incorrect primary key that was used to query a user's device list. This resulted in empty device lists for users and the service assuming there were no active endpoints, therefore sending calls to voicemail. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. We'll publish a post-incident report within five business days.; PublishedTime=2021-06-13T19:30:25.443Z

TM261472 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 60 minutes.

Status: We're reviewing Microsoft Teams desktop diagnostics and call logs for impacted users to understand the source of the problem and the appropriate scope of impact. Next update by: Friday, June 11, 2021, at 8:00 PMUTC; PublishedTime=2021-06-11T18:58:28.943Z

Status: After additional investigation we've determined that the infrastructure responsible for generating call notifications is unable to confirm an active session for the end user, resulting in calls being sent directly to voicemail. We're investigating further to isolate the source of this issue and determine the steps necessary to address the problem. Next update by: Friday, June 11, 2021, at 9:30 PM UTC; PublishedTime=2021-06-11T19:53:46.537Z

Status: We've identified a recent configuration change that may potentially be the source of this issue. We're in the process of completing a targeted rollback to validate the effectiveness of this mitigation strategy Next update by: Friday, June 11, 2021, at 9:30 PM UTC; PublishedTime=2021-06-11T20:15:05.113Z

Status: We're continuing to monitor the rollback to determine if this action has remediated impact. Next update by: Friday, June 11, 2021, at 11:30 PM UTC; PublishedTime=2021-06-11T21:32:45.9Z

Status: Our telemetry indicates that the rollback is providing relief in some circumstances. Further investigation is underway to determine if additional action is required to fully remediate impact. Next update by: Saturday, June 12, 2021, at 1:30 AM UTC; PublishedTime=2021-06-11T23:28:59.313Z

Status: Our telemetry indicates that the rollback is nearly complete and we're seeing resolution for customers. While we're closely monitoring the service to determine if additional actions are required, we recommend users restart their Teams client or app for more immediate relief. Next update by: Saturday, June 12, 2021, at 2:00 AM UTC; PublishedTime=2021-06-12T00:12:33.05Z

Status: We're continuing to monitor service recovery and expect most Desktop client users are no longer experiencing impact. We're investigating some residual impact to mobile users to determine if additional actions are required. Next update by: Saturday, June 12, 2021, at 4:00 AM UTC; PublishedTime=2021-06-12T02:01:12.937Z

Status: We've confirmed via telemetry that Teams Desktop client users should no longer be experiencing impact. For mobile app users, we have determined the change can take longer depending on your device. For Android devices, we recommend users restart their session to expedite recovery. For iOS devices, we expect the change to complete within the next 24 to 48 hours. Next update by: Sunday, June 13, 2021, at 6:00 PM UTC; PublishedTime=2021-06-12T02:29:48.16Z

Status: We've determined that our fix for mobile app users has partially remediated the issue and have prepared an additional solution that we anticipate will fully mitigate impact. We're deploying that solution now and monitoring our service telemetry as it propagates throughout the affected environments. Next update by: Sunday, June 13, 2021, at 10:00 PM UTC; PublishedTime=2021-06-13T16:58:54.377Z

Final status: We've deployed our solution and have confirmed via monitoring our service telemetry that impact has been remediated. Next update by: Sunday, June 13, 2021, at 10:00 PM UTC; PublishedTime=2021-06-13T16:58:54.377Z

--------------------------------------------------------------------------------------------------
Title Users may have been unable to manually enter dates into date fields within the modern UI of SharePoint Online
ID SP262674
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users may have been unable to manually enter dates into date fields within the modern UI of SharePoint Online.
Severity
Start time 2021-06-11T06:42:00Z
End time 2021-06-17T10:52:00Z
Last update 2021-06-17T11:27:28.09Z
More info Datetime fields were unaffected by this issue. As a workaround, users could enter the dates using the picker. We'd like to get your feedback on the workaround we provided. From a non-mobile device, please rate this post and submit it in the textbox. Final status: We've validated that rolling back the build to a small section of the environment mitigated the issue. We've finished rolling back the impacting build to the remaining environment and received confirmation from users that the issue has been resolved. Scope of impact: Impact was specific to any user with modern UI. Start time: Friday, June 11, 2021, at 6:42 AM UTC End time: Thursday, June 17, 2021, at 10:52 AM UTC Root cause: A regular service update was unexpectedly causing impact to users being able to manually enter data in date fields. Next steps: - We're reviewing our deployment and provisioning procedures to help prevent similar problems in the future. This is the final update for the event.; PublishedTime=2021-06-17T11:27:28.09Z
Root cause A regular service update was unexpectedly causing impact to users being able to manually enter data in date fields. Next steps: - We're reviewing our deployment and provisioning procedures to help prevent similar problems in the future. This is the final update for the event.; PublishedTime=2021-06-17T11:27:28.09Z

SP262674 Details

Status: We're reviewing logs provided by affected users to determine the next troubleshooting steps. Next update by: Thursday, June 17, 2021, at 12:00 PM UTC; PublishedTime=2021-06-17T09:49:56.787Z

Status: We've identified a regular service update is unexpectedly causing impact. We're rolling back the change to a small section of the affected environment to validate that it resolves the issue. Next update by: Thursday, June 17, 2021, at 1:00 PM UTC; PublishedTime=2021-06-17T10:47:29.117Z

Final status: We've validated that rolling back the build to a small section of the environment mitigated the issue. We've finished rolling back the impacting build to the remaining environment and received confirmation from users that the issue has been resolved. Next update by: Thursday, June 17, 2021, at 1:00 PM UTC; PublishedTime=2021-06-17T10:47:29.117Z

--------------------------------------------------------------------------------------------------
Title Some users may encounter a prompt to "select a certificate" when attempting to sign in to the Microsoft Teams service
ID TM261228
Service Microsoft Teams
Feature Teams Components
Status Post-incident report published
Impact Users may have encountered a prompt to "select a certificate" when attempting to sign in to the Microsoft Teams service.
Severity
Start time 2021-06-10T06:30:00Z
End time 2021-06-10T14:30:00Z
Last update 2021-06-17T16:06:48.19Z
More info This issue may have affected users attempting to access the service from the desktop client, web app, and mobile devices. Subsequent retries to sign in may have been successful. Final status: We've completed reverting the update and confirmed by monitoring system telemetry that the issue is now fully resolved. Scope of impact: Any user routed through the affected infrastructure may have been impacted. Start time: Thursday, June 10, 2021, at 6:30 AM'UTC End time: Thursday, June 10, 2021, at 2:30 PM'UTC Root cause: A recent update moved the AMS to a shared library, which was intended to improve overall performance and reduce latency within Microsoft Teams. The library was misconfigured and contained a latent code bug. The incorrect configuration requested a client certificate during a Transport Layer Security (TLS) handshake between services, causing users to be prompted to provide a certificate. Furthermore, Microsoft Teams is currently migrating to a new underlying service platform, designed to improve overall service stability. At the current migration stage, a large majority of traffic remains on the old platform, and only a small section of traffic is served through the new platform. As the regression occurred within the new platform, impact was limited to 3% of traffic served through the regional infrastructure. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. We'll publish a post-incident report within five business days.; PublishedTime=2021-06-10T16:54:26.673Z
Root cause A recent update moved the AMS to a shared library, which was intended to improve overall performance and reduce latency within Microsoft Teams. The library was misconfigured and contained a latent code bug. The incorrect configuration requested a client certificate during a Transport Layer Security (TLS) handshake between services, causing users to be prompted to provide a certificate. Furthermore, Microsoft Teams is currently migrating to a new underlying service platform, designed to improve overall service stability. At the current migration stage, a large majority of traffic remains on the old platform, and only a small section of traffic is served through the new platform. As the regression occurred within the new platform, impact was limited to 3% of traffic served through the regional infrastructure. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. We'll publish a post-incident report within five business days.; PublishedTime=2021-06-10T16:54:26.673Z

TM261228 Details

Status: We're analyzing system logs to determine the source of the issue. Next update by: Thursday, June 10, 2021, at 2:00 PM UTC; PublishedTime=2021-06-10T11:56:26.567Z

Status: We've determined a recent deployment, designed to improve service performance, is inadvertently causing impact. We're reverting the update while we monitor the affected environment to validate that it resolves the issue. Next update by: Thursday, June 10, 2021, at 5:30 PM UTC; PublishedTime=2021-06-10T13:20:41.88Z

Final status: We've completed reverting the update and confirmed by monitoring system telemetry that the issue is now fully resolved. Next update by: Thursday, June 10, 2021, at 5:30 PM UTC; PublishedTime=2021-06-10T13:20:41.88Z

--------------------------------------------------------------------------------------------------
Title Admins are unable to upload and save user photos in the Microsoft 365 admin center
ID MO261286
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Admins were unable to upload and save user photos in the Microsoft 365 admin center.
Severity
Start time 2021-06-09T14:00:00Z
End time 2021-06-11T21:50:00Z
Last update 2021-06-12T00:40:59.97Z
More info
Root cause A recent service configuration change inadvertently prevented admins from uploading and saving user photos in the Microsoft 365 admin center. Next steps: - We're reviewing service configurations to explore options to prevent similar impact from recurring in the future. This is the final update for the event.; PublishedTime=2021-06-12T00:40:59.97Z

MO261286 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-06-10T18:45:57.307

Status: We determined that a recent service configuration change inadvertently caused the impact. We've completed development of a fix which is undergoing final validation and will subsequently begin deploying it in phases to the affected environments. We anticipate deployment of the fix to complete in full by the next scheduled update. Next update by: Saturday, June 12, 2021, at 4:00 AMUTC; PublishedTime=2021-06-10T19:32:29.803Z

Final status: We've confirmed that our fix has completed deployment, remediating impact. Next update by: Saturday, June 12, 2021, at 4:00 AMUTC; PublishedTime=2021-06-10T19:32:29.803Z

--------------------------------------------------------------------------------------------------
Title Some users are unable to add comments to major version documents during publication within the SharePoint Online service
ID SP263418
Service SharePoint Online
Feature SharePoint Features
Status Service restored
Impact Users were unable to add comments to major version documents during publication within the SharePoint Online service.
Severity
Start time 2021-06-09T04:01:00Z
End time 2021-06-21T17:20:00Z
Last update 2021-06-21T18:03:42.933Z
More info
Root cause A recent service update meant to improve the SharePoint Online publishing experience uncovered a code regression, blocking users from adding comments to major versions of documents during publication. Next steps: - We're reviewing our update procedures to better identify similar code regressions during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-06-21T18:03:42.933Z

SP263418 Details

Status: We're reviewing system logs along with a reproduction of the issue to determine our next troubleshooting steps. Next update by: Monday, June 21, 2021, at 11:30 AM UTC; PublishedTime=2021-06-21T09:22:54.757Z

Status: We're continuing to investigate system logs while also analyzing recently gathered trace logs to help identify the root cause of this issue. Next update by: Monday, June 21, 2021, at 7:00 PM UTC; PublishedTime=2021-06-21T10:03:51.77Z

Final status: We've identified that a recent service update meant to improve the SharePoint Online publishing experience uncovered a code regression, blocking users from adding comments to major versions of documents during publication. We've reverted this service update and confirmed via internal monitoring that impact was remediated. Next update by: Monday, June 21, 2021, at 7:00 PM UTC; PublishedTime=2021-06-21T10:03:51.77Z

--------------------------------------------------------------------------------------------------
Title Some users may have been unable to use their bot sign-in message extension with the Microsoft Teams service
ID TM261230
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may have been unable to use their bot sign-in message extension with the Microsoft Teams service.
Severity
Start time 2021-06-08T23:52:00Z
End time 2021-06-10T14:06:00Z
Last update 2021-06-10T15:03:48.893Z
More info
Root cause A recent code change in a deployment caused bot sign-ins to have the wrong tokens. Next steps: - We're reviewing our deployment procedures to help prevent similar problems in the future. This is the final update for the event.; PublishedTime=2021-06-10T14:30:10.98Z

TM261230 Details

Status: We've identified a recent code change in a deployment that caused bot sign-ins to have the wrong tokens. We're reverting this deployment to mitigate the impact. Next update by: Thursday, June 10, 2021, at 3:00 PM UTC; PublishedTime=2021-06-10T12:49:28.143Z

Final status: We've reverted the deployment change and from further monitoring confirmed that this has resolved the issue. Next update by: Thursday, June 10, 2021, at 3:00 PM UTC; PublishedTime=2021-06-10T12:49:28.143Z

--------------------------------------------------------------------------------------------------
Title Users are intermittently unable to navigate within Outlook on Intune-managed Android devices
ID IT262943
Service Microsoft Intune
Feature Microsoft Intune
Status Service restored
Impact Users' Outlook mobile apps were sometimes freezing on Intune-managed Android devices when navigating between features.
Severity
Start time 2021-06-08T17:00:00Z
End time 2021-06-18T19:10:00Z
Last update 2021-06-18T19:41:16.627Z
More info This affected users with Microsoft Intune Mobile Application Management (MAM) policies applied on Android devices only. Users could restore Outlook mobile functionality by either rotating the app orientation on their device or restarting the app. Final status: Deployment of our fix is complete and users are able to update their Company Portal app manually via the Google Play store at this time. Any users who do not manually update their Company Portal app should receive the fix with their next automatic update. Scope of impact: Impact was specific to Intune-managed Android device users who were served through the affected infrastructure. Start time: Tuesday, June 8, 2021, at 5:00 PM'UTC End time: Friday, June 18, 2021, at 7:10 PM'UTC Root cause: A recent Company Portal app update introduced a code regression causing a race condition resulting in impact. Next steps: - We're reviewing our validation and update procedures to detect issues like this prior to deployment and prevent these problems from occurring in the future. This is the final update for the event.; PublishedTime=2021-06-18T19:41:16.627Z
Root cause A recent Company Portal app update introduced a code regression causing a race condition resulting in impact. Next steps: - We're reviewing our validation and update procedures to detect issues like this prior to deployment and prevent these problems from occurring in the future. This is the final update for the event.; PublishedTime=2021-06-18T19:41:16.627Z

IT262943 Details

Status: We're analyzing diagnostic data to determine the source of the issue. Next update by: Friday, June 25, 2021, at 5:00 PM UTC; PublishedTime=2021-06-18T14:42:41.997Z

Status: Our analysis of diagnostic data has determined that a recent Company Portal app update introduced a code regression causing a race condition resulting in impact. We've developed and validated a fix and are beginning deployment. More details regarding deployment timeline will be provided via our next scheduled update. Next update by: Friday, June 18, 2021, at 9:00 PMUTC; PublishedTime=2021-06-18T15:55:18.577Z

Final status: Deployment of our fix is complete and users are able to update their Company Portal app manually via the Google Play store at this time. Any users who do not manually update their Company Portal app should receive the fix with their next automatic update. Next update by: Friday, June 18, 2021, at 9:00 PMUTC; PublishedTime=2021-06-18T15:55:18.577Z

--------------------------------------------------------------------------------------------------
Title Users are intermittently unable to send email messages in Outlook on the web using Internet Explorer 11
ID EX261459
Service Exchange Online
Feature E-Mail timely delivery
Status Service restored
Impact Users were intermittently unable to send email messages in Outlook on the web using Internet Explorer 11.
Severity
Start time 2021-06-08T13:00:00Z
End time 2021-06-11T17:40:00Z
Last update 2021-06-11T19:01:09.087Z
More info Users could use other internet browsers, such as Microsoft Edge, to send email messages via Outlook on the web. Users would see the following error message when attempting to send email. 'This message can't be sent right now. Please try again later.' Final status: We've identified that the latest Outlook on the web JavaScript file downloaded from the Content Delivery Network contained a code issue, resulting in impact. We've rolled back this file and confirmed with affected users that impact was remediated. Scope of impact: All users attempting to send email in Outlook on the web using Internet Explorer 11 may have intermittently experienced impact. Start time: Tuesday, June 8, 2021, at 1:00 PM UTC End time: Friday, June 11, 2021, at 5:40 PM UTC Root cause: The latest Outlook on the web JavaScript file downloaded from the Content Delivery Network contained a code issue, resulting in an error when attempting to send email messages in Outlook on the web using Internet Explorer 11. Next steps: - We're further reviewing the source of the code issue in the JavaScript file to prevent similar issues from happening in the future. This is the final update for the event.; PublishedTime=2021-06-11T18:59:58.407Z
Root cause The latest Outlook on the web JavaScript file downloaded from the Content Delivery Network contained a code issue, resulting in an error when attempting to send email messages in Outlook on the web using Internet Explorer 11. Next steps: - We're further reviewing the source of the code issue in the JavaScript file to prevent similar issues from happening in the future. This is the final update for the event.; PublishedTime=2021-06-11T18:59:58.407Z

EX261459 Details

Status: We're analyzing network trace logs and support case data to determine our next steps. Next update by: Friday, June 11, 2021, at 5:30 PMUTC; PublishedTime=2021-06-11T16:33:07.137Z

Status: We've created an internal reproduction of this error and are investigating data gathered from this reproduction to identify the root cause of impact and determine our next steps. Next update by: Friday, June 11, 2021, at 7:30 PMUTC; PublishedTime=2021-06-11T17:13:55.19Z

Final status: We've identified that the latest Outlook on the web JavaScript file downloaded from the Content Delivery Network contained a code issue, resulting in impact. We've rolled back this file and confirmed with affected users that impact was remediated. Next update by: Friday, June 11, 2021, at 7:30 PMUTC; PublishedTime=2021-06-11T17:13:55.19Z

--------------------------------------------------------------------------------------------------
Title Some users may be unable to view image previews in chats and channels within Microsoft Teams
ID TM260665
Service Microsoft Teams
Feature Teams Components
Status Service restored
Impact Users may be unable to view image previews in chats and channels within Microsoft Teams.
Severity
Start time 2021-06-08T11:03:17Z
End time 2021-06-08T11:45:00Z
Last update 2021-06-08T12:03:08.493Z
More info
Root cause

TM260665 Details

Status: We're analyzing system logs to determine the source of the issue. Next update by: Tuesday, June 8, 2021, at 1:30 PM UTC; PublishedTime=2021-06-08T11:09:18.073Z

Final status: While we were in the process of analyzing system logs to determine the source of the issue, system monitoring indicated that the service had returned to normal health. We've confirmed that the problem is no longer occurring, and we'll continue to monitor the service to ensure that the problem does not happen again. Next update by: Tuesday, June 8, 2021, at 1:30 PM UTC; PublishedTime=2021-06-08T11:09:18.073Z

--------------------------------------------------------------------------------------------------
Title Some users are intermittently disconnected from or can't connect to Microsoft Teams meetings
ID TM277577
Service Microsoft Teams
Feature Teams Components
Status Post-incident report published
Impact Users were intermittently disconnected from or couldn't connect to Microsoft Teams meetings.
Severity Sev2
Start time 2021-06-07T07:00:00Z
End time 2021-09-29T16:04:00Z
Last update 2021-10-07T01:49:00.83Z
More info This issue was specific to the Microsoft Teams Windows desktop client. Final status: The fix has deployed to 100 percent of the affected environment, and our internal monitoring, along with affected users, confirms that impact was remediated. Scope of impact: This issue intermittently impacted some users attempting to join or already connected to Microsoft Teams meetings. Start time: Monday, June 7, 2021, at 7:00 AM'UTC End time: Wednesday, September 29, 2021, at 4:04 PM'UTC Root cause: A recent update introduced a regression that caused thread leaking on Native Render components, which caused some threads to remain open. Once the number of open unused threads reached a specific threshold, the Teams client User Interface (UI) would become unresponsive and crash, resulting in impact. As such, if a user joined and left meetings multiple times between Teams client restarts, the number of open unused threads would build up and increased the likelihood of impact occurring. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. We'll publish a post-incident report within five business days.; PublishedTime=2021-09-29T20:34:48.48Z
Root cause A recent update introduced a regression that caused thread leaking on Native Render components, which caused some threads to remain open. Once the number of open unused threads reached a specific threshold, the Teams client User Interface (UI) would become unresponsive and crash, resulting in impact. As such, if a user joined and left meetings multiple times between Teams client restarts, the number of open unused threads would build up and increased the likelihood of impact occurring. Next steps: - For a more comprehensive list of next steps and actions, please refer to the Post Incident Review document. We'll publish a post-incident report within five business days.; PublishedTime=2021-09-29T20:34:48.48Z

TM277577 Details

Status: We're investigating a potential issue with the Microsoft Teams desktop Native Render component and we're disabling the component to confirm that this is the cause of impact. Next update by: Friday, August 13, 2021, at 12:00 AM UTC; PublishedTime=2021-08-12T14:09:07.137Z

Status: We're continuing our work with affected users to disable the native renderer to isolate the underlying cause of impact and develop mitigation actions. Next update by: Friday, August 13, 2021, at 2:00 PMUTC; PublishedTime=2021-08-12T22:30:49.233Z

Status: We've disabled the native renderer for a subset of impacted users and we're monitoring the results to isolate the underlying cause of impact in order to develop a mitigation plan. Next update by: Wednesday, August 18, 2021, at 9:00 PM UTC; PublishedTime=2021-08-13T13:16:31.25Z

Status: Our disabling of the native renderer for a subset of impacted users appears to be having the desired effect, with reports of fewer users experiencing impact. We're continuing to disable the native renderer for affected users and confirm if it will be a viable solution for remediating impact. Next update by: Friday, August 20, 2021, at 9:30 PMUTC; PublishedTime=2021-08-18T20:13:35.02Z

Status: We've disabled the native renderer for some affected users and we're analyzing log data for these users to confirm that this action has mitigated impact. Next update by: Monday, August 23, 2021, at 10:00 PMUTC; PublishedTime=2021-08-20T21:17:39.617Z

Status: We're confirming with users that had the native render disabled, as well as confirming through monitoring service telemetry for these users, that crash rate has decreased. This feedback will assist in confirming the root of the issue. If confirmed we expect to have a fix ready and deployed by our next update. Next update by: Friday, August 27, 2021, at 10:00 PMUTC; PublishedTime=2021-08-23T20:53:42.02Z

Status: We've received confirmation from some users for which we disabled the native renderer that the issue is no longer occurring; however, we're verifying with some additional users as well as through our service telemetry that this action is successful. This likely indicates that a recent update introduced a code issue or other regression that impacted the functionality of the native renderer, and we're investigating to further narrow this down. We will provide an update on what we expect to be a continually decreasing rates of crashes by the next update time. Next update by: Saturday, August 28, 2021, at 12:00 AMUTC; PublishedTime=2021-08-27T21:54:35.17Z

Status: Disabling the native renderer appears to be a viable solution as we haven't noticed an increase in the rate of crashes, although we're still reaching out to additional users to verify that impact is remediated after disabling the native renderer. In parallel, we've determined that a recent service update inadvertently caused a hang when processing threads, resulting in impact. We've developed a long-term fix that's currently undergoing validation and we'll provide a timeline to mitigation once it's closer to deployment. Next update by: Wednesday, September 1, 2021, at 10:00 PMUTC; PublishedTime=2021-08-27T23:47:38.403Z

Status: We've confirmed with your representatives that disabling native rendering has effectively provided short-term mitigation, and users are no longer disconnected intermittently from Microsoft Teams meetings. Addionally, our fix developed to provide long-term mitigation has completed validation, is being deployed, and is expected to begin providing alleviation by late September. We're monitoring the deployment process and we'll provide more precise details regarding its progress and estimated time to saturation via our next scheduled update. Next update by: Wednesday, September 15, 2021, at 10:00 PMUTC; PublishedTime=2021-09-01T22:21:07.51Z

Status: Deployment of the long-term fix is in the initial stages of deployment to our internal environment for further vetting prior to it's broader deployment. We expect that the long-term fix should be ready for deployment to the impacted environment by our next update. Next update by: Wednesday, September 29, 2021, at 10:00 PMUTC; PublishedTime=2021-09-15T21:24:41.877Z

Status: Our fix continues through the internal validation stage prior to the phased rollout; however, in parallel, we're evaluating ways to expedite this fix to all impacted environments. We'll provide more information on our expedited deployment and potential timeline in our next scheduled update. Next update by: Monday, September 20, 2021, at 10:00 PMUTC; PublishedTime=2021-09-17T22:58:16.88Z

Status: Our fix has completed the internal validation stage, and we're initiating deployment of the fix to a subset of affected users for further validation of mitigation. We anticipate this phase of the rollout will take approximately three days to complete, including receiving feedback from affected users on mitigation efficacy. Next update by: Wednesday, September 22, 2021, at 10:00 PMUTC; PublishedTime=2021-09-20T20:35:13.153Z

Status: We're continuing to verify the efficacy of our fix via targeted release to a portion of affected users. We anticipate this validation phase to culminate within the next two days, with subsequent widespread deployment beginning as initially projected on Friday, September 24, 2021. We'll provide additional information regarding the fix's deployment status as well as estimated time to resolution via our next scheduled update. Next update by: Friday, September 24, 2021, at 10:00 PMUTC; PublishedTime=2021-09-22T21:58:42.957Z

Status: We've completed verification of our fix via targeted release to a portion of affected users and are proceeding with broad deployment of the fix after receiving positive feedback. Currently the fix has deployed to approximately 25 percent of the affected environment, and we're planning a staged rollout which, barring setbacks, will push deployment to 50 percent by Monday, September 27, 75 percent by Tuesday, September 28, and 100 percent by Wednesday, September 29. We're monitoring the progress of the fix as it continues to saturate throughout the affected environment. Next update by: Tuesday, September 28, 2021, at 9:00 PM UTC; PublishedTime=2021-09-24T20:26:44.5Z

Status: Currently the fix has deployed to approximately 75 percent of the affected environment, and we expect it to be fully deployed by our next scheduled update. We're continuing to monitor the affected environment to ensure deployment of the fix is successful and impact is mitigated. Next update by: Wednesday, September 29, 2021, at 9:30 PMUTC; PublishedTime=2021-09-28T20:30:32.867Z

Final status: The fix has deployed to 100 percent of the affected environment, and our internal monitoring, along with affected users, confirms that impact was remediated. Next update by: Wednesday, September 29, 2021, at 9:30 PMUTC; PublishedTime=2021-09-28T20:30:32.867Z

--------------------------------------------------------------------------------------------------
Title Users see failures when generating any embedded reports for Microsoft Power Automate flows
ID MF259739
Service Microsoft Power Automate in Microsoft 365
Feature Service and web access issues
Status Service restored
Impact Users experienced failures when generating any embedded reports for Microsoft Power Automate flows.
Severity
Start time 2021-06-03T20:13:00Z
End time 2021-06-04T04:28:00Z
Last update 2021-06-04T04:37:16.407Z
More info
Root cause A recent service update intended to enhance authentication component functionality inadvertently contained a configuration issue, resulting in impact. Next steps: - We're reviewing the impacting service update to better understand how the configuration issue was introduced and how we can prevent similar issues from making it past our testing and validation procedures. This is the final update for the event.; PublishedTime=2021-06-04T04:37:16.407Z

MF259739 Details

Status: We're investigating a potential issue and checking for impact to your organization. We'll provide an update within 30 minutes.; PublishedTime=2021-06-04T03:10:17.503

Status: We're reviewing available system data to investigate our suspicion that impact is related to a recent service update. Next update by: Friday, June 4, 2021, at 4:30 AMUTC; PublishedTime=2021-06-04T03:24:57.023Z

Status: Our investigation indicates that a recent service update intended to enhance authentication component functionality inadvertently contained a configuration issue, resulting in authentication components not functioning as expected. We're working to correct that configuration issue to remediate impact. Next update by: Friday, June 4, 2021, at 6:30 AMUTC; PublishedTime=2021-06-04T04:18:49.477Z

Final status: We've completed our configuration changes and confirmed via system monitoring that this has resolved impact. Next update by: Friday, June 4, 2021, at 6:30 AMUTC; PublishedTime=2021-06-04T04:18:49.477Z

--------------------------------------------------------------------------------------------------
Title Some admins may have been unable to see usernames within usage reports extracted from the Microsoft Admin Center
ID MO259572
Service Microsoft 365 suite
Feature Administration
Status Service restored
Impact Admins may have been unable to see usernames within usage reports extracted from the Microsoft Admin Center.
Severity
Start time 2021-06-03T00:00:00Z
End time 2021-06-05T12:00:00Z
Last update 2021-06-05T12:46:33.65Z
More info
Root cause A recent update intended to reduce data storage inadvertently prevented usernames from appearing within usage reports. Next steps: - We're reviewing our update procedures to better identify similar issues during our development and testing cycles. This is the final update for the event.; PublishedTime=2021-06-05T12:46:33.65Z

MO259572 Details

Status: We're reviewing details provided in the support case along with analyzing our internal reproduction of the issue to determine the root cause of impact. Next update by: Thursday, June 3, 2021, at 10:00 AM UTC; PublishedTime=2021-06-03T08:11:57.27Z

Status: Following our investigation using support case details and our internal reproduction, we've determined a recent flight change inadvertently resulted in a configuration issue. We're disabling the flight and reverting the change on the affected infrastructure to mitigate impact. Next update by: Friday, June 4, 2021, at 12:00 PM UTC; PublishedTime=2021-06-03T09:57:06.32Z

Status: We've completed disabling the flight and changes on the affected infrastructure which has mitigated impact, however, users may still be unable to see previous reports as we're currently processing the backlog, which could take up to 24 hours to complete. Next update by: Saturday, June 5, 2021, at 12:00 PM UTC; PublishedTime=2021-06-04T11:04:41.283Z

Status: We're continuing to process the backlog of usage reports, which we estimate could take up to 4 hours to complete. Next update by: Sunday, June 6, 2021, at 12:00 PM UTC; PublishedTime=2021-06-05T11:05:36.797Z

Final status: We've completed processing the backlog of usage reports and can confirm that impact has been mitigated. Next update by: Sunday, June 6, 2021, at 12:00 PM UTC; PublishedTime=2021-06-05T11:05:36.797Z

--------------------------------------------------------------------------------------------------
Title Users may encounter a generic error when attempting to download sign-in and activity logs, or accessing certain portals
ID IS259461
Service Identity Service
Feature Sign-In
Status Service restored
Impact Users may have encountered an error when attempting to download sign-in and activity logs, or accessing certain portals.
Severity
Start time 2021-06-02T17:05:00Z
End time 2021-06-02T17:55:00Z
Last update 2021-06-02T18:48:51.973Z
More info Users may have encountered an error when attempting to access the My Sign-ins, My Apps, and My Account portals. Final status: We've identified that a portion of infrastructure responsible for routing the impacted requests stopped functioning as intended, resulting in impact. We've applied a configuration change to the affected infrastructure as a mitigation action and after monitoring the service, we've confirmed the issue is resolved. Scope of impact: Your organization was impacted by this event, and all users may have experienced impact. Start time: Wednesday, June 2, 2021, at 5:05 PM'UTC End time: Wednesday, June 2, 2021, at 5:55 PM'UTC Root cause: A portion of infrastructure responsible for routing the impacted requests stopped functioning as intended, resulting in impact. Next steps: - We're analyzing performance data and trends on the affected infrastructure to better understand why it stopped functioning as intended and help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-06-02T18:48:51.973Z
Root cause A portion of infrastructure responsible for routing the impacted requests stopped functioning as intended, resulting in impact. Next steps: - We're analyzing performance data and trends on the affected infrastructure to better understand why it stopped functioning as intended and help prevent this problem from happening again. This is the final update for the event.; PublishedTime=2021-06-02T18:48:51.973Z

IS259461 Details

Status: We've investigated a problem and confirmed that the service is now restored. We'll provide additional information in a closure summary within the 'History' section of the 'Service health' dashboard shortly.; PublishedTime=2021-06-02T18:41:07.607

Final status: We've identified that a portion of infrastructure responsible for routing the impacted requests stopped functioning as intended, resulting in impact. We've applied a configuration change to the affected infrastructure as a mitigation action and after monitoring the service, we've confirmed the issue is resolved.

--------------------------------------------------------------------------------------------------
Title Dynamics 365 Marketing ' General slowness for event registration and emails
ID CR259310
Service Dynamics 365 Apps
Feature Components/Features
Status Service restored