If you are the owner of the album or the photos & videos, you have the right to delete all comments.
As a recipient of a shared album or photos & videos, you can only delete your own comments.
Deleting the comments works as follows in both cases:
Click on the three dots in the upper right corner of the comment and then select "Delete". The comment will then disappear.