*** <NAME> <VERSION> Help ***

* <NAME> <VERSION> is copyright © 2010, P. Lutus, http://www.arachnoid.com, and is released under the GPL (http://www.gnu.org/licenses/gpl.html)

* Be sure to visit the <NAME> home page at http:///www.arachnoid.com/python/<NAMELC>.

*** Introduction

People who live in cities can just turn on their TV sets or go online and get an instant weather report, but people who really need weather information are often in a remote location, or at a ship at sea, out of reach of beautiful, multicolored weather charts. For the latter group, there is a service called weatherfax (a.k.a. HF radiofax). In the U.S., this service is administered by the National Weather Service (http://www.nws.noaa.gov/om/marine/home.htm).

Weatherfax is a rather old-fashioned technology that transmits weather charts and satellite images by way of shortwave radio, at what may be regarded as a rather pokey data rate by modern Internet standards -- a typical radiofax weather chart might require seven minutes to transmit and receive.

During my around-the-world solo sail (http://arachnoid.com/sailbook) I arranged to receive these weather charts using a pretty baroque setup -- an antenna rigged to my boat's mast, a shortwave radio, a fax demodulator box, and a printer that would slowly print the received chart, line by line.

In more recent times, in my Alaska travels (http://arachnoid.com/alaska2010) I still use the weatherfax service, because it's still the most reliable way to get an overview of the weather when one is in a remote location. And until recently I was using the original fax demodulator box, over twenty years old, even though it is rather out-of-date and not a particularly reliable way to receive weather charts.

I recently decided to retire the fax demodulator box and write a program that could listen to the shortwave radio directly by way of a typical computer's line input jack. The program would need to decode the shortwave signal, turn it into a weather chart, display it and save it as a graphic image file. After a bit of struggle, this program is the result.

*** Very Briefly:

* To use <NAME> <VERSION> to receive weather charts, all you need are a shortwave radio and a cord to connect the radio to your computer's line input or microphone input jack, preferably the former.

* Radiofax broadcast schedules and frequencies are available at the NOAA NWS marine forecast site (http://www.nws.noaa.gov/om/marine/home.htm).

* The above site is the best resource for specific, up-to-date information and details, but here are some broadcasting locations, times in UTC, and frequencies (winter 2010):

        Boston, MA (NMF) 4235(02z, 08z), 6340.5, 9110, 12750(14z) kHz
        New Orleans, LA (NMG) 4317.9, 8503.9, 12789.9, 17146.4(12,18Z) kHz
        Kodiak, AK (NOJ) 2054(10z, 18z), 4298, 8459, 12412.5(4z, 22z) kHz
        Pt.Reyes, CA (NMC) 4346(except 19z,23Z), 8682, 12786, 17151.2, 22527(19z,23Z) kHz
        Honolulu, HI (KVM70) 9982.5(0519-1556z), 11090, 16135(1719-0356z) kHz

* To receive weatherfax broadcasts, set your receiver to upper-sideband mode and tune 1.9 KHz below the above listed frequencies. For example, to receive the NMC 4346 KHz broadcast, you would tune to (4346-1.9) 4344.1 KHz.

* Tuning is rather critical, so if you have an older, non-digital radio, it may be difficult to get acceptable results (and read about Automatic Frequency Control below).

*** Installation and Basic Instruction

To function as intended, <NAME> <VERSION> requires some system resources, including Python, Gstreamer, GTK and a handful of other packages. On most platforms, acquiring these resources is trivial -- just use your package manager to download them from the Internet (and be sure to test the program fully before getting beyond Internet access). If you run the program before a resource is acquired, the program should tell you what is missing. To be sure to see such messages, run the program from the command line the first few times.

After you have acquired the needed system resources, and if you want to run <NAME> without typing anything, for example by clicking a desktop icon, you can write a shell script that should do two things:

    1. Change directories to the <NAME> program directory.
    2. Launch the <NAME> Python script.

Here is an example shell script that will work on Linux:

    #!/bin/sh
    
    cd (<NAME> program directory)
    exec ./<NAMELC>.py

I am sure there is a similar approach that will work on ... that other operating system.

In the bricks-and-mortar world outside the computer, you only need an audio cord to connect your shortwave radio to your computer's sound card. If you have a choice of shortwave radios, choose one that is modern, digitally tuned, and stable.

Once things have been set up as described above, and assuming you have a stable digital radio and a radiofax station is tuned in, <NAME> <VERSION> will commence receiving and saving any number of weather charts on its own. But there are some details that should be addressed to optimize the quality of the charts.

*** Clock Calibration

Radiofax charts are transmitted at a specific clock frequency that must be exactly matched at the receiving end. If your clock, and the transmitter's clock, were exactly synchronized, every received chart would look like this:

    #:::::::::::::***********:::::::::::::#
    #:::::::::***:::::::::::::***:::::::::#
    #:::::::**:::::::::::::::::::**:::::::#
    #:::::**:::::::::::::::::::::::**:::::#
    #::::**:::::::::::::::::::::::::**::::#
    #::::**:::::::::::::::::::::::::**::::#
    #::::**:::::::::::::::::::::::::**::::#
    #::::**:::::::::::::::::::::::::**::::#
    #:::::**:::::::::::::::::::::::**:::::#
    #:::::::**:::::::::::::::::::**:::::::#
    #:::::::::***:::::::::::::***:::::::::#
    #:::::::::::::***********:::::::::::::#

But this won't be your outcome at first, because perfect synchronization between a fax sender and receiver requires a much higher level of timing accuracy than computer clocks tend to have. Here is an example chart that shows that the receiving computer's clock is slightly faster than the transmitter's clock:

    #:::::::::::::***********:::::::::::::#
    ##:::::::::***:::::::::::::***:::::::::
    :##:::::::**:::::::::::::::::::**::::::
    ::##:::::**:::::::::::::::::::::::**:::
    :::##::::**:::::::::::::::::::::::::**:
    ::::##::::**:::::::::::::::::::::::::**
    *::::##::::**:::::::::::::::::::::::::*
    **::::##::::**:::::::::::::::::::::::::
    **:::::##:::::**:::::::::::::::::::::::
    *:::::::##:::::::**:::::::::::::::::::*
    :::::::::##:::::::::***:::::::::::::***
    ::::::::::##:::::::::::::***********:::

See the diagonal bar at the left? It should be vertical. You could solve this by clicking the <NAME> program "Calibrate" button and typing in numbers one after another, then receiving new charts, until you managed to make the bar vertical, but there is a much easier way:

* Receive a complete chart -- wait until reception is complete and the indicator at the upper right says "Receive|WAIT".

* Point your mouse cursor at the location marked "A" below, then press the right mouse button:

    #A::::::::::::***********:::::::::::::#
    ##:::::::::***:::::::::::::***:::::::::
    :##:::::::**:::::::::::::::::::**::::::
    ::##:::::**:::::::::::::::::::::::**:::
    :::##::::**:::::::::::::::::::::::::**:
    ::::##::::**:::::::::::::::::::::::::**
    *::::##::::**:::::::::::::::::::::::::*
    **::::##::::**:::::::::::::::::::::::::
    **:::::##:::::**:::::::::::::::::::::::
    *:::::::##:::::::**:::::::::::::::::::*
    :::::::::##:::::::::***:::::::::::::***
    ::::::::::##:::::::::::::***********:::

* You will see a message at the top of the <NAME> window: "Clock calibrate mode". This message alerts you that you have taken the first of two calibration steps.

* Now click the right mouse cursor at the location marked "B" below:

    #A::::::::::::***********:::::::::::::#
    ##:::::::::***:::::::::::::***:::::::::
    :##:::::::**:::::::::::::::::::**::::::
    ::##:::::**:::::::::::::::::::::::**:::
    :::##::::**:::::::::::::::::::::::::**:
    ::::##::::**:::::::::::::::::::::::::**
    *::::##::::**:::::::::::::::::::::::::*
    **::::##::::**:::::::::::::::::::::::::
    **:::::##:::::**:::::::::::::::::::::::
    *:::::::##:::::::**:::::::::::::::::::*
    :::::::::##:::::::::***:::::::::::::***
    ::::::::::##B::::::::::::***********:::

* Now <NAME> will ask whether you want to save this calibration for future charts. Respond "Yes".

* If the existing caibration value is not zero, <NAME> will ask whether to add the new calibration value to the old or replace the old number with the new. If the present chart was just received, add the new number to the old (this allows repeated refinements in calibration).

* <NAME> will then ask whether you want to apply the calibration to the present chart. Respond "Yes" to this as well.

If you have been careful about pointing your mouse cursor, the result should look like the first example chart above -- the black bar should be perfectly vertical. But if it isn't, just perform another calibration as described above, keeping the present calibration number. At the end of the new procedure, after the questions asked above, <NAME> will again ask whether you want to add the new calibration value to the old, or replace the old adjustment. If the chart has been adjusted using the existing correction number, add the new correction to the old. This process can be repeated until you are satisfied with the alignment, and the final result will be reflected in the calibration number.

There is a variation on the above procedure when your local clock is slightly slower than the transmitter's clock. In such a case, the received chart might look like this:

    #:::::::::::::***********:::::::::::::#
    :::::::::***:::::::::::::***:::::::::##
    ::::::**:::::::::::::::::::**:::::::##:
    :::**:::::::::::::::::::::::**:::::##::
    :**:::::::::::::::::::::::::**::::##:::
    **:::::::::::::::::::::::::**::::##::::
    *:::::::::::::::::::::::::**::::##::::*
    :::::::::::::::::::::::::**::::##::::**
    :::::::::::::::::::::::**:::::##:::::**
    *:::::::::::::::::::**:::::::##:::::::*
    ***:::::::::::::***:::::::::##:::::::::
    :::***********:::::::::::::##::::::::::

Or this (the worst case):

    ::::##:::::::::::::***********:::::::::
    :::##:::::::::***:::::::::::::***::::::
    ::##:::::::**:::::::::::::::::::**:::::
    :##:::::**:::::::::::::::::::::::**::::
    ##::::**:::::::::::::::::::::::::**::::
    #::::**:::::::::::::::::::::::::**::::#
    ::::**:::::::::::::::::::::::::**::::##
    :::**:::::::::::::::::::::::::**::::##:
    :::**:::::::::::::::::::::::**:::::##::
    ::::**:::::::::::::::::::**:::::::##:::
    :::::***:::::::::::::***:::::::::##::::
    ::::::::***********:::::::::::::##:::::

To calibrate <NAME> using such an image, first "translate" it -- click the image with your left mouse button. This "translates" the image, meaning the image moves left or right without changing its angle. Ideally, you would translate the above image so it looks like this:

    **:::::::::::::##:::::::::::::*********
    ::***:::::::::##:::::::::***:::::::::::
    ::::**:::::::##:::::::**:::::::::::::::
    :::::**:::::##:::::**::::::::::::::::::
    :::::**::::##::::**::::::::::::::::::::
    ::::**::::##::::**:::::::::::::::::::::
    :::**::::##::::**::::::::::::::::::::::
    ::**::::##::::**:::::::::::::::::::::::
    **:::::##:::::**:::::::::::::::::::::::
    ::::::##:::::::**:::::::::::::::::::**:
    :::::##:::::::::***:::::::::::::***::::
    ::::##:::::::::::::***********:::::::::

Then, having brought the entire vertical bar into view, you can perform the above calibration procedure.

Here are some additional notes about calibation:

    * The order in which you click locations in the image is not important (top first, or bottom first), and you don't need a complete image to perform the procedure -- a partial image will do for a rough calibration.

    * Any vertical feature may be used for calibration, not just the synchronization bar. Lines of longitude in weather charts are particularly useful becuse of their distincness and small width.

    * Many aspects of <NAME>'s operation are improved by a good calibration, so be sure to perform one in advance of serious chart reception.

    * Once you have acceptable timing calibration, it might be wise to record the number in the calibration window for future reference (<NAME> will save this number too, of course). The number can always be manually re-entered to avoid going through the calibration procedure in the event that the user-specific <NAME> configuration values are lost.

    * The calibration number is unique to the computer on which it is acquired. Because computer clocks differ, each computer that runs <NAME> will need a separate calibration procedure.

*** A Quick Review

* To enter a previously acquired calibration number, click the "Calibrate" button, enter the number in the calibration window, and press "Enter".

* To begin the chart-based clock calibration procedure, press the right mouse button on a vertical feature or click "Calibrate".

* To translate charts (move them right or left), press the left mouse button on the image feature you want to be located at the margin.

* To undo an image editing action you have just taken, press "Undo".

* The above options are only available for charts that have been completely received, not for those in the process of being received.

*** Weak Signal Problems and Solutions

* Out of Sync

High-frequency radio reception varies from day to day, even minute to minute. This means <NAME> may have to deal with marginal signals at times, which may prevent it from properly synchronizing itself with the timing of the received chart. This issue is separate from the clock issue described above -- it has to do with the position of the image. An improperly synchronized image might look like this:

    **:::::::::::::##:::::::::::::*********
    :::***:::::::::##:::::::::***::::::::::
    ::::::**:::::::##:::::::**:::::::::::::
    ::::::::**:::::##:::::**:::::::::::::::
    :::::::::**::::##::::**::::::::::::::::
    :::::::::**::::##::::**::::::::::::::::
    :::::::::**::::##::::**::::::::::::::::
    :::::::::**::::##::::**::::::::::::::::
    ::::::::**:::::##:::::**:::::::::::::::
    ::::::**:::::::##:::::::**:::::::::::::
    :::***:::::::::##:::::::::***::::::::::
    **:::::::::::::##:::::::::::::*********

Such an image means <NAME> was unable to receive the necessary synchronization signals before receiving the chart itself. This won't always be true, only during times of poor reception. To correct this in a chart that has already been received, just place your mouse cursor at the position marked "AA" below and press the left mouse button:

    **:::::::::::::##:::::::::::::*********
    :::***:::::::::##:::::::::***::::::::::
    ::::::**:::::::##:::::::**:::::::::::::
    ::::::::**:::::##:::::**:::::::::::::::
    :::::::::**::::##::::**::::::::::::::::
    :::::::::**::::##::::**::::::::::::::::
    :::::::::**::::AA::::**::::::::::::::::
    :::::::::**::::##::::**::::::::::::::::
    ::::::::**:::::##:::::**:::::::::::::::
    ::::::**:::::::##:::::::**:::::::::::::
    :::***:::::::::##:::::::::***::::::::::
    **:::::::::::::##:::::::::::::*********

The result should look like this:

    #:::::::::::::***********:::::::::::::#
    #:::::::::***:::::::::::::***:::::::::#
    #:::::::**:::::::::::::::::::**:::::::#
    #:::::**:::::::::::::::::::::::**:::::#
    #::::**:::::::::::::::::::::::::**::::#
    #::::**:::::::::::::::::::::::::**::::#
    #::::**:::::::::::::::::::::::::**::::#
    #::::**:::::::::::::::::::::::::**::::#
    #:::::**:::::::::::::::::::::::**:::::#
    #:::::::**:::::::::::::::::::**:::::::#
    #:::::::::***:::::::::::::***:::::::::#
    #:::::::::::::***********:::::::::::::#

If the result isn't aligned as you would like, just click the image near its edge to make small corrections. Clicking near the left edge pushes the image slightly to the left, clicking near the right pushes to the right. With practice, one learns to align a chart with a single mouse click. 

* Start/stop tone threshold adjustment

<NAME> begins and ends fax reception based on two special tones, one to start reception, one to stop, that are transmitted along with the fax information. The default threshold for tone detection is 50%, which in tests using clear, strong signals is an optimal level. But in noisy or weak-signal conditions, this level may not work. Here are signs that the default setting may need to be changed:

    * If <NAME> fails to begin fax reception when the start tone is received, decrease the tone detection threshold setting.
    
    * If <NAME> ends reception prematurely before completing the chart and receiving the stop tone, and returns to the WAITSTB state (indicated at the upper right), increase the tone detection threshold setting.

*** Other Features

* Grayscale

The grayscale option allows reception of charts with shades of gray, for example satellite images of cloud cover. This feature will produce good results only with the best quality radios, radios able to remain tuned to a specific frequency for long periods. When using the grayscale mode, be aware that the AFC mode (described below) will degrade image quality if enabled.

The quality of grayscale images improves with high data rates (see "Date Rate Selector List" below).

* Tuning Problems

Radiofax reception is best with precise radio tuning, ideally that provided by a modern digital radio with an accurate frequency readout and a stable local oscillator. Older radios often cannot be tuned precisely, and this may prevent proper fax reception. To deal with this problem, <NAME> has an Automatic Frequency Control feature that automatically corrects for inaccurate radio tuning or drift. But this feature, enabled with the <NAME> "AFC" checkbox, degrades the quality of grayscale charts, so it should only be used when needed to correct for tuning errors not resolvable another way. AFC can be used to produce acceptable charts received with substandard radios or radios that drift in frequency over time, but for the best quality results using a modern digital radio, it should be disabled.

* Lock/Unlock Buttons

During times of poor reception, <NAME> may not receive the radiofax "start" signal, which will prevent it from receiving the chart that follows. To force reception, press the "Lock" button, which overrides the <NAME> synchronization procedure described below (see "State Machine States") and commences reception. This step prevents <NAME> from receiving synchronization signals, as a result, the received chart will need to be adjusted after reception using the methods described above.

If you want to stop reception, or if you want to leave <NAME> while a chart is being received, just press the "Unlock" button. This forces an ongoing reception to stop.

* Receive/Standby Buttons

To automatically process and save charts as they are received off the air, click the "Receive" button (this is the default <NAME> mode). To prevent automatic reception, for example while editing chart files, click the "Standby" button.

* Date Rate Selector List

Changes in data rate can only be made in standby mode. The quality of grayscale images improves at high data rates, but so does the workload on your computer. Sometimes at high data rates, other computer activities will cause an image to jump out of sync -- like this:

    #:::::::::::::***********:::::::::::::#
    #:::::::::***:::::::::::::***:::::::::#
    #:::::::**:::::::::::::::::::**:::::::#
    #:::::**:::::::::::::::::::::::**:::::#
    #::::**:::::::::::::::::::::::::**::::#
    #::::**:::::::::::::::::::::::::**::::#
    #::::**:::::::::::::::::::::::::**::::#
    :**::::##::::**::::::::::::::::::::::::
    **:::::##:::::**:::::::::::::::::::::::
    :::::::##:::::::**:::::::::::::::::::**
    :::::::##:::::::::***:::::::::::::***::
    :::::::##:::::::::::::***********::::::

To prevent this, reduce the data rate, and/or apply one or more strategies to reduce the <NAME> workload, described below under the topic "Computer workload".

* Input Monitoring / Monitor Volume

<NAME> provide an audio monitor output of its own input, which is useful when the line input or microphone input jack is used for the signal source. If you want to hear the incoming signal but cannot, enable this feature and increase the monitor volume setting. But there are some circumstances when the monitor output will be fed right back into the input, sort of like microphone feedback in an auditorium. In this case, turn the monitor volume down to zero.

To reduce the computer workload, disable the input monitor when it's not in use.

* Load Button

To load previously received charts into <NAME> for viewing or processing, click "Load" and choose one or more charts. These charts can be processed using the clock-correction and translation tools described above. Any charts that have been modified will be saved when you exit <NAME>.

* Defaults button

To reset nearly all <NAME> options to their default values, click the "defaults" button. The only program option that is not reset is the calibration value, sometimes the result of much effort -- this value can be manually reset if needed.

* Quit Button

Users will discover that they can't quit <NAME> while a chart is being received, This is a safeguard to prevent loss of a chart if the "Quit" button should accidentally be pressed during reception. To exit while a chart is being received, press "Unlock", then "Quit".

*** Computer Workload

When configured for high data rates, <NAME> can produce a heavy workload for even a modern, fast computer. The default data rate of 22,050 samples per second produces excellent images, but monopolizes much computer time and resources because of the processing required to receive a fax image. To solve any problems this workload might cause, choose a lower data rate consistent with the chart quality you need to have. It turns out that acceptable monochrome charts, even marginally acceptable grayscale satellite images, can be received at the lowest available data rate of 6,000 samples per second, which is a small fraction of the workload required by the default rate.

Here are more suggestions to reduce computer workload:

    * Disable the input monitoring function when it's not in use.

    * Disable Automatic Frequency Control mode and rely on precise receiver tuning instead.

    * To the degree possible, suspend other computer activities while receiving weather charts at high data rates.

*** Image Editing Options

Each image tab has its own function buttons to change properties of a received image:

    * The "Rot CCW" button rotates the image 90° counterclockwise, while the "Rot CW" button does the opposite. These are used to reorient weather charts that have been transmitted in a rotated orientation to allow larger charts than the default weatherfax protocol would permit.

    * The "Invert B/W" button inverts black, white and all shades of gray. This can be used to improve the appearance of certain satellite images.

*** State Machine States

This is a bit technical. To assure correct reception, the radiofax chart protocol has a number of phases in a time sequence, and <NAME>'s internal state machine has corresponding states. Here are the phases and the <NAME> names that are displayed at the upper right during reception:

    WAITSIG : wait for audio signal at 2300 Hz
    WAITSTB : wait for beginning of 300 Hz FM start-image tone (duration 5 seconds)
    WAITSTE : wait for ending of start tone
    WAITLS1 : wait for local clock line-start mark at initial clock rate
    SYNC    : sync local frame rate with incoming signal (20 seconds)
    WAITLS2 : wait for local clock line-start mark at new clock rate
    PROC    : process image lines (duration depends on number of image lines)
            : and watch for 450 Hz FM end-image tone (duration 5 seconds)
    END     : end image, save chart

*** Technical details

This is even more technical. A radiofax signal originates with an FM transmitter whose output is centered at the station's assigned frequency and frequency-modulated with "mark" and "space" frequencies at +400 and -400 Hz away from the center frequency. If the receiver is tuned 1900 Hz below the carrier frequency, this means the mark and space frequencies are located at 2300 and 1500 Hz in the receiver's output.

In the fax protocol, mark corresponds to white and space to black. Gray levels occupy intermediate frequencies between mark and space, between 2300 and 1500 Hz.

To produce a signal suitable for decoding (and to work with <NAME>), a radio receiver is normally tuned 1900 Hz below the transmitter's assigned frequency, which correctly places the mark and space frequencies and assures correct gray-level registration in the grayscale mode. This is why inaccurate tuning can prevent reception, and may require use of the above-described AFC mode to get readable results from a mistuned receiver.

The Nyquist-Shannon Sampling Theorem specifies that a sampling rate must be at least twice the highest frequency of interest. The lowest data rate provided by <NAME>, 6,000 samples per second, is at this lower limit -- it is a bit more than twice the highest frequency that must be decoded by <NAME> and turned into a readable chart. Nevertheless, this data rate produces acceptable monochrome charts and marginally acceptable grayscale charts.

<NAME> uses a number of internal strategies to maximize the quality of its output, among which are:

    * Automatic gain control, to compensate for slow amplitude changes caused by receiver signal fading.

    * Frequency-error tracking, which (after calibration) automatically compensates for computer clock errors.

    * Independent narrow-band tone detectors for the start-image and end-image tone frequencies (300 and 450 Hz).

    * AFC (Automatic Frequency Control) that can be used to correct for inaccurate tuning or receiver drift, with an unavoidable side effect of streaks in grayscale images (therefore this mode is only recommended for monochrome images).

*** Signal Routing

<NAME> can receive audio signals from many sources -- files, generator programs and the system line input and microphone input jacks among others. But the routing and configuration for these choices is primarily managed outside <NAME>. I recommend that Linux users run the PulseAudio Volume Control (pavucontrol) program to manage this routing activity. Pavucontrol is normally located under the Multimedia program directory, but if it is not easily located, users can launch it from the command line like this:

    $ pavucontrol &

Users can use this program to control signal sources and destinations, and volume levels. In this connection, it's important to say that many newer systems have built-in microphones, and pavucontrol can and should be used to disable this microphone to prevent room noises from interfering with the operation of <NAME>.

*** Program Storage

<NAME> saves your choices, and its charts, in a directory it creates located at (user home directory)/.<NAME>. Within this directory is a configuration file named <NAME>.ini with your option choices, plus any charts that have been received. Remember this location to gain access to your weather charts after reception.

*** Coda

Be sure to visit the <NAME> Home Page at http://www.arachnoid.com/python/<NAMELC> to check for updates and further technical information.
