To remove a group, click Remove Group button.
Note, the Default Group "All List Contacts" could not be removed and if you remove a static group, all contacts under the group will be also removed.