The piezo beeper is between a positive rail and ground. Normally the positive rail leading to it is switched off, so no current flows through the beeper. When it’s supposed to beep, somewhere on the circuit a relay or transistor (or a pin on a microcontroller, but that just means a transistor inside the microcontroller) provides current to the rail, which takes the path of least resistance through the beeper into ground, which causes it to beep. The current then flows to ground where it effectively dissipates.

When the beeper is removed, all that happens when it’s supposed to beep is that the rail is switched on, but since there’s no path to ground through it, (practically) no electrons flow into it. While it is theoretically possible to design a circuit that would blow up when this happens, in practice it won’t ever be the case for low voltage electronics like this.

A helpful analogy is to think of electricity as water flowing through pipes, the piezo puzzer as a little water wheel, and the buzzer toggle switch as a valve. If you open the valve, water flows through the pipe and spins the water wheel. Remove the water wheel and plug the pipe next to it, and when the valve is opened all that happens is that the pipe fills with water up to the plug and then stops flowing.