Arranging happens on the real page rather than on an admin screen, so you can see the result as you work.
Every widget you can add is listed down the left, grouped by what it belongs to. Drag one onto any outlined area and drop it. Nothing changes for anybody else until you press Save & publish, and Discard undoes everything since you started.
Each widget has a cog for its settings and an X to remove it. Removing is undoable until you save.