Sounds like you're picking up an ac hum from your tape recorder. Probably have a bad ground in your tape recorder or an impedance mismatch with your microphone. Make sure the microphone is matched to the input of your amplifier. I would suggest not using a tape recorder, they usually have poor amplifiers in them, but using an audio amplifier designed for what you are using the mic for, such as singing or speaking.
Peter