Every widget carries three controls while you are arranging: a grip to drag it, a cog for its settings, and an X to remove it.
Dragging a widget into the tray on the left takes it off the page but keeps it, with its title and settings, ready to put back. The X is for one that should not exist at all.