[This subject justifies a whole book, or two; the following is not going
to make sense to people who haven't got some background on modulation
I'm not convinced this is a productive directions, but the most
complex modulation scheme that you will probably find is your local
broadcast TV station, especially if it has closed captions (teletext)
and digital sound. Unfortunately the bandwidth is too high for the
normal Argus hardware, so what you will probably see is the main carrier
and sidebands for line and frame rates all at once, and the various
subcarriers independently - because they are amplitude modulated, on
the main carrier, they will appear as distinct signals when put through
a spectrum analyzer.
(Analogue TV is vestigial sideband amplitude modulation with a frequency
modulated sound subcarrier and a suppressed carrier, quadrature
amplitude modulated colour subcarrier, time division multiplexed with a
pilot carrier for the colour. (The European, PAL system, modifies the
subcarrier modulation at half the line rate.) If there are captions,
this is likely to be an amplitude keyed sub-carrier time division
multiplexed with the video. Digital sound will be another sub-carrier,
although I've forgotten the detailed structure.)
For quadrature amplitude phase modulation, look at the audio from your
modem (carrier is around 18 to 20 something kHz (V.21 and V.23 excluded,
as these are frequency shift keyed; X.90 downlink is effectively just
amplitude modulated, but with a very large number of levels).
For phase modulated direct sequence spread spectrum, try GPS. These
signals are of the order of a MHz wide, so you will only see the out of
balance carrier and maybe sidebands for the chipping code repeat interval.
Note that a lot of newer systems are spread spectrum and all are much
too wide to be seen all at once by Argus type systems.
For gaussian minimum shift keying, try your GSM phone (anyone else's is
illegal in both the UK and USA).
For another example of direct sequence spread spectrum (not sure of the
lower level modulation) do the same for your CDMA mobile phone. I think
some cordless phones use these two techniques.
For phase (narrow band frequency) modulation, look at the local amateur
radio repeater, or the speech channel of your analogue mobile phone, or
even an analogue cordless phone.
For multi-frequency narrow band data (various phase and amplitude
combinations on multiple sub-carriers) look at the HF amateur bands'
teleprinter sections for Clover transmissions. You will probably find
more simple frequency shift keying, though.
For a frequency modulated sub-carrier phase modulated onto the main
carrier, try the 1200 bps access channel for your local amateur data
Even FM broadcast is complex (pre-emphasised frequency modulation with
a pilot frequency and suppressed carrier double side band sub-carrier).
There may be another sub-carrier if there is traffic or program data.
These are two wide for the typical Argus configuration to see in one go.
For single sideband, suppressed carrier, look at the HF amateur radio