Correct answer is (d) convolution
The best explanation: The convolution of out-of-band signal with bandpass characteristic of the receiver gives the interference in the receiver owing to out-of-band energy at the transmitter.This type of interference can affect the roll-offs. Frequency separation is employed to suppress this type of interference.