If you no longer need a notification setting for alerting a user, you can delete it from Control Center. If you only want to temporarily stop notifications (e.g., if the person is on vacation), you can edit the notification setting to disable it and then activate it again later.
Control Center lists all notification settings set up for your organization. (To view your personal notification settings, click your username (in the upper-right corner) then click Notification Settings.)
On the row for the notification setting you want to delete, click the icon.
In the confirmation window that appears, click OK.