.. _flags:

Flagging the data
============================

.. note::

    The flags rely on the past data collected. Depending on the :py:attr:`station.orientation` parameter, either the data in the N/S orientation (:py:attr:`station.orientation` = 1), the E/W (:py:attr:`station.orientation` = 0) or a combination of both (:py:attr:`station.orientation` = 2) will be analysed (see :ref:`station_class`)

To use correcly the data, it is necessary to flag it appropriately. 4 cases can arise:

#. The antenna is not working anymore (\  ``flag``\  = 1).
#. The transmitter is off (\  ``flag``\  = 2).
#. It is nightime (\  ``flag``\  = 3).
#. Nothing particular is happening, the data can be used (\  ``flag``\  = 0).

.. _flag_antenna_down:

Antenna shutdown
----------------------

The first case is the antenna shutting down. First of all, in this case, an email alert can be sent (see :ref:`emails`).

This is detected if the number of points in the file being read stays the same. Since for a single station the file is read every minute,
there should be around 60 new points. If the antenna has stopped, then the file is still a\  ``.mau``\  file, and it is still the last updated
file, ensuring that this method works.

.. note::
    Because there may be short antenna shutdowns, and since normally the antenna starts again on its own, we choose not to send an
    alert at each interruption. Instead, an alert is sent after five times reading the files without any new points.

.. _flag_tx_off:

Transmitter off
----------------------

This situation doesn't really warrant an alert, as transmitters are regularly stopped for maintenance. However, when a transmitter
is not transmitting, the amplitude becomes very low and the phase random. The detection script, and all the data analysis will not
work in this case, thus the data needs to be flagged.

The flagging for this case works by saying that the amplitude is very low (below the antenna sensitivity, which is specified by the :py:attr:`threshold` parameter of the :py:class:`~vlf4ions.class_definition.receiver` class).
Because there may be punctual noise, this is checked on the median amplitude recorded for the past minute.

Nighttime
----------------------

Nighttime is detected by computing a high solar zenith angle at either the receiver or the transmitter site. In practice, the solar zenith angle is computed through the :py:func:`~vlf4ions.sunrise_sunset.get_sza` function. The solar zenith angle threshold is set as an attribute of the :py:class:`~vlf4ions.class_definition.station` class, and is set to 85° by default (see figure below).

.. figure:: ../Figures/Sza_threshold.png

    Solar zenith angle at sunrise and sunset (at the minimum of amplitude) for the GQD transmitter recorded with a VLF receiver in Nançay in 2024. The path is a short path, almost entirely in the N/S direction.

Since nighttime doesn't necessitates an alert, all that happens in this case is that the flag is set to 3.

Phase not well demodulated
---------------------------------

Sometimes the phase doesn't seem to be well demodulated, resulting in apparently random phase variations. This only happens very rarely, and may be due to operator actions at the transmitter site. In this case, the\ ``flag``\ value is set to 2. This is detected by noting jumps of more than 20° between consecutive phase values.

.. important::

    Sometimes, all of the above fail, but we may have false detection of very strong flares due to either bad phase correction or very noisy data (see figure below). In this case, if :py:attr:`DP` is above 1000° we still flag the data, in order to avoid 'detecting' an X flare due to this.

    Note that an alert for strong flare will only be sent if more than half of the stations detect the flare.

    If all else fails, if a station detects a breakpoint below the detection threshold but it has been three hours since the last quiet breakpoint, it is forced to be in quiet time again

.. image:: ../Figures/Example_bad_data.png

.. important::

    If there is a problem with one station (:py:obj:`flag` is 1 or 2), then the data for this station will not be considered for 15 min, to limit the false positive due to transmitters rapidly switching on and off