Google have just released a ‘Delete’ button that now resides next to the ‘Report Spam’ button:
When you’re positive you won’t need a message, use the Delete button to send it to Trash. Deleted messages and conversations are permanently removed from your Gmail account 30 days after you send them to Trash. Deleting unimportant mail is a great way to free up some of your storage, although Google recommends using the Archive feature so that its possible to read the message/conversation in the future.
Didn't they already have that feature in the 'More Options' menu?
Yes they did but they've now added the actual button to make it easier.