I would like to address the recent alerts generated for our Singapore servers. It appears that these alerts, which were intended to be suppressed during the migration (where similar alerts were suppressed for the Global estate but left for alerting in the specific Singapore estate), have continued to trigger for over six months without action from the DBA team. As I recall, these alerts were specifically designated for the Singapore servers due to their critical nature, yet they were suppressed for the Global estate during the migration.
Over the past 11 days, we have observed approximately 300 alerts, and it seems we have not taken the necessary steps to suppress them despite various notifications. As discussed with ABC, I have raised a ticket with the appropriate team to address this issue, as we cannot afford to wait for a resolution given the volume of alerts being generated daily.
Additionally, it’s important to recognize that these false positive alerts can negatively impact our incident metrics. I believe it would be beneficial for us to review our alert management processes to ensure we respond more effectively in the future. If there are any obstacles that have prevented action on these alerts, please feel free to share your insights.
Thank you for your attention to this matter. I look forward to our collaborative efforts in improving our alert management system.
0 comments:
Post a Comment