Does anyone have any tips on increasing sensitivity for a medium wave
receiver trying to distinguish navtex 170Hz phase shifts on a low
power carrier at 518kHz?!

Ive been on and off trying to do this project for my boat for a year.
I say off and on, its become most frustrating! I can't use a loop
antenna because its just too big and too directional for use on a
boat. So I'm currently using a Nasa Marine active aerial (it is just
a small plate attached to a standard fet common source driver
circuit). I have that going down some coax through capacitive
impedance transformer and inductor creating some extra front end
selectivity, to the input of a 612 mixer, getting 6kHz IF (after being
mixed with a lo). I'm driving the 612 differentially, as its a slight
improvement over single ended. From here Ive tried using several op
amp filter circuits, and they all perform adequately.

Heres my problem, the receiver just picks up too much noise.
Everything interferes with it, even the scan on my oscilloscope. Even
without this, taking the aerial out of the room, the actual noise in
the system is too high to pick up anything but the strongest navtex
signals (which are decoded). Of course AM radio broadcasts are way
above the background noise, and navtex signals are much lower power.
And AM radio has the help of our ears to tune out the noise.

It works better when closer to the transmitter of course, but the
navtex spec says you should be able to pick up stations 400 miles
away, and i'm only just decoding the local one 100 miles away.

So my question is, does anyone know of any special techniques on the
RF side that can improve my noise handling/signal integrity? (my
expensive sony worldband receiver seems pretty good at it on SSB).
And....does anyone know how the upper market receivers get such a good
signal? My receiver just samples the input frequency (as does the
Nasa low end I think), so is very susceptible to any interference.

Grateful and very interested to hear views on this.

Many Thanks
Andy