How do I make this go away? It shows up in Preview and the published page.
Welcome to our forum.
This is only visible to admin, don't worry, this is useful for quick edit.
you can hide this from settings
Options > Accordion Settings > Hide edit link on front-end.
Please see the screenshot