Feed aggregator

Incident began at 2024-07-24 22:36 and ended at 2024-07-25 16:27 (times are in Coordinated Universal Time (UTC)).

Based on post incident analysis, we are downgrading the severity of this incident to ‘Service information’, as the issue affected significantly fewer users than initially thought.

Affected products: Chrome Browser

Incident began at 2024-07-23 07:00 and ended at 2024-07-25 23:00 (times are in Coordinated Universal Time (UTC)).

Summary: Poly Studio X devices experienced unexpected deprovisioning and might require manual re-enrollment.

Description: Poly Studio X devices running Google Meet experienced unexpected deprovisioning between Tuesday, 2024-11-23 to Thursday, 2024-11-25.

Google engineers rolled out a software fix to mitigate the issue, however if the device had already got impacted and got to the activation code screen, they would require manual re-enrollment. Please refer to the workaround section for more information.

Diagnosis: The affected customer may encounter Poly Studio X devices offline/deprovisioned.

Workaround: Customer Action required: Each affected device has to be manually re-enrolled in the Admin Console using the displayed activation code, as part of the “Enroll your device (non-ChromeOS devices) steps. Following the re-enrollment, the devices will be reconfigured to restore their previous settings and its functionality.

Affected products: Google Meet

Incident began at 2024-07-23 07:00 and ended at 2024-07-25 23:00 (times are in Coordinated Universal Time (UTC)).

Summary:

Poly Studio X devices experienced unexpected deprovisioning and might require manual re-enrollment.

Description:

Poly Studio X devices running Google Meet experienced unexpected deprovisioning between Tuesday, 2024-11-23 to Thursday, 2024-11-25.

Google engineers rolled out a software fix to mitigate the issue, however if the device had already got impacted and got to the activation code screen, they would require manual re-enrollment. Please refer to the workaround section for more information.

Diagnosis: The affected customer may encounter Poly Studio X devices offline/deprovisioned.

Workaround:

Customer Action required: Each affected device has to be manually re-enrolled in the Admin Console using the displayed activation code, as part of the “Enroll your device (non-ChromeOS devices) steps. Following the re-enrollment, the devices will be reconfigured to restore their previous settings and its functionality.

Affected products: Google Meet

Incident began at 2024-07-24 22:36 and ended at 2024-07-25 16:27 (times are in Coordinated Universal Time (UTC)).

Mini Incident Report

We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced an impact outside of what is listed below, please reach out to Google Workspace Support using the help article https://support.google.com/a/answer/1047213.

(All Times US/Pacific)

Incident Start: 24 July, 2024 15:36

Incident End: 25 July, 2024 09:27

Duration: 17 hours, 51 minutes

Affected Services and Features:

Chrome Browser

Regions/Zones: Global

Description:

A significant number of users were unable to find or save passwords in the password manager for Chrome browser, for a period of 17 hours, 51 minutes.

From the preliminary analysis the root cause of the issue is a change in product behavior without proper feature guard. Google engineers mitigated the issue by deploying a fix.

Google will complete a full IR in the following days that will provide a full root cause.

Customer Impact:

Impacted users were unable to find passwords in Chrome's password manager. Users can save passwords, however it was not visible to them. The impact was limited to the M127 version of Chrome Browser on the Windows platform.

Approximately 2% of users out of the 25% of the entire user base where the configuration change was rolled out, experienced this issue.

Additional details:

While an interim workaround was provided during the incident, after the fix was fully rolled out, users are being advised to restart their Chrome browser to ensure that the fix takes effect.

Affected products: Chrome Browser

Incident began at 2024-07-17 17:53 and ended at 2024-07-25 16:27 (times are in Coordinated Universal Time (UTC)).

The issue with Chrome Browser has been mitigated for all affected users as of Thursday, 2024-07-25 09:27 US/Pacific.

Users are advised to restart their Chrome Browser to ensure that the fix takes effect.

We thank you for your patience while we worked on resolving the issue.

Affected products: Chrome Browser

Incident began at 2024-07-17 17:53 (times are in Coordinated Universal Time (UTC)).

SUMMARY: Chrome Browser Password Manager Issue

We are experiencing an issue with Chrome Browser where impacted users may be unable to find or save passwords in Chrome's Password Manager. This impacts users on M127 version of Chrome Browser.

Our Engineering team has identified the cause of the issue and are actively working on deploying a fix for all users.

We will continue to monitor the situation as the fix is rolled out to ensure a smooth resolution.

We will provide more information by Thursday, 2024-07-25 09:45 US/Pacific.

WORKAROUND:

Users currently impacted by this can try the following:

Launch Chrome with " --enable-features=SkipUndecryptablePasswords" command line flag.

1. Locate Chrome shortcut on desktop. If you don't have it, go to chrome://settings/manageProfile and at the bottom enable "Create desktop shortcut".

2. Fully exit Chrome

3. Go to desktop and right click the shortcut then click Properties.

4. To the target field paste the following string to the end " --enable-features=SkipUndecryptablePasswords".

5. Close the dialog with OK and launch Chrome using this shortcut.

Affected products: Chrome Browser

Incident began at 2024-07-16 17:40 and ended at 2024-07-16 18:34 (times are in Coordinated Universal Time (UTC)).

Summary:

We have received reports of an issue with Gmail.

Description:

Upon further investigation, our engineering teams believe that the scope is very limited and/or no customers were impacted.

If you have questions or feel that you may be impacted, please open a case with the Support Team and we will work with you until the issue is resolved. No further updates will be provided here.

Affected products: Gmail

Incident began at 2024-07-16 17:40 and ended at 2024-07-16 18:34 (times are in Coordinated Universal Time (UTC)).

Summary:

We have received reports of an issue with Gmail.

Description:

Upon further investigation, our engineering teams believe that the scope is very limited and/or no customers were impacted.

If you have questions or feel that you may be impacted, please open a case with the Support Team and we will work with you until the issue is resolved. No further updates will be provided here.

Affected products: Gmail

Incident began at 2024-07-16 18:18 (times are in Coordinated Universal Time (UTC)).

We're investigating reports of an issue with Gmail. We will provide more information shortly.

Affected products: Gmail