Is it possible to detect the presence of a superweak signal in some trendless sequence?

Full Text

Introduction and formulation of the problem

Under super weak signal (SWS) the author understands small perturbations when their signal to noise ratio defined as

$\begin{array}{l}SNR=10\cdot \lg \left(\frac{Rg\left(Sgn\right)}{Rg\left(Ts\right)}\right)\le -100dB,\\ Rg\left(f\right)=\max \left(f\right)-\min \left(f\right).\end{array}$ (1)

In this case, the range of the trendless sequence, where SWS can be found, is defined by Rg(Ts), but the range of the given deterministic signal is determined by Rg(Sgn) in frequency range . Because equation (1) implies the ratio of amplitudes of their quadratic values if the matching ratio of powers is taken into consideration, this formula can be regarded as preferable to conventional ones. It is clear that the detection of weak signals is a major issue in many natural and technical sciences, and it is actually a "friend-foe" general problem solution also. In medical, this issue involves converting cardiograms and other recorded data into other (…) grams in order to detect minor deviations. In chemistry, it is closely linked to the detection of hidden traces in the given solutes. No sense to innumerate of different radio-electronics and photonic/laser applications where this problem plays a central role.

          Analysis of different publications devoted to this subject shows that the SNR=0 serves as a specific "red line" dividing the devices with SNR > 0 where the noise influence is suppressed from the devices with SNR < 0 when the noise influence becomes predominant. To the best author's knowledge, he did not find in literature the papers where SNR < –60 dB detection of a small signal inside noisy data is considered (that implies the presence of a signal with the range of amplitudes of the order 10^-6 to the range of the noise having the unit order). Highly likely, these types of publications are not available in open Internet access. However, papers that are close to the goal stated in the paper's title need to be reminded of this. It is preferable to group these studies according to the SNR value that was attained. The author of the paper [1] based it on Karhunen–Loève transformation yielded an SNR of -3dB. The authors of the research [2] assert that if the SNR is -5dB, they may extract the signal using the DOA (direction of arrival) approximated approach. By using unique experimental techniques, the authors of the works [3, 5] improved the sensitivity of common commercial receivers if the SNR falls between 5 and 20 dB. The authors of the research [4] asserted that they could get a distinct signal for the range of the SNR in the interval [–6.5, 18.5] dB by using the various locations of satellite signals sensors. The receiver's relative location in relation to the satellite position has a significant impact on this ratio. Using techniques identical to those described in paper [4], Chinese scientists obtained acceptable SNR from the several satellites in the interval [8.7, 19.5] dB in paper [6]. As can be seen from these studies, the authors of the aforementioned papers were able to detect the presence of weak signals down to -6.5 dB by employing experimental and signal processing techniques. In this paper the author proposes the completely new signal processing method that can be used by many researchers working in this area. It is instructive to formulate the following problem.

Is it possible to detect quantitatively the presence of the SWS when the influence of the random fluctuations (RF) becomes essential and SNR achieves the unbelievable values –100dB or even less?

           By another words, it means that when the given noise background normalized to the unit value the level of the signal should have range of amplitudes of the order 10^-10 or even less. Attentive analysis shows that this problem can be solved and the positive answer will be found. When a SWS can be discovered, the Md-POLS approach allows the starting ratio to be increased up to acceptable ratios (SNR [-2,2]). This indicates that the Md-POLS method's "power" allows for a sensitivity increase of 10–12 orders of magnitude from its original SNR level!

It is preferable to provide a straightforward response to the question posed above in order to wrap up this section. Under what circumstances might a SWS be detected? Assume that the level of the initial TS is Rg(Ts)=1 and that the range of a detected signal is initially Rg(Sgn)= $a\cdot {10}^{-s}\equiv {10}^{-\overline{s}}$. When we enter these values into (1), we get $SNR=-10\cdot \overline{s}$. In this instance, it is impossible to detect the intended signal. The following minimal value, Rg(T_S)=$b\cdot {10}^{-n}\equiv {10}^{-\overline{n}}$  can be reached by lowering the original TS limit, using expression (2) shown in next section. When we enter these estimates into (1), we get $SNR=10\cdot \left(\overline{n}-\overline{s}\right)$. Consequently, we arrive to a straightforward conclusion. If the embedded SWS satisfies to relationship $SNR=10\cdot \left(\overline{n}-\overline{s}\right)>0$ it is detectable; if $SNR=10\cdot \left(\overline{n}-\overline{s}\right)\le 0$ it cannot be detected. These straightforward inequalities eliminate any uncertainty regarding the supplied SWS's detection potential.

The solution of the problem

In order to solve this problem, we use the POLS (procedure of the optimal linear smoothing) that it was used earlier [7, 8] for elimination of a possible trend located inside a "confidence tube". The original and previously proposed formula that will be used in this paper is modernized and can be rewritten in the form:

$\begin{array}{l}{Y}_j\left(w,p\right)=\frac{{\displaystyle \sum _{i=1}^{N}K\left(\frac{{\Delta }_{ji}}{w}\right)y_i}}{{\displaystyle \sum _{i=1}^{N}K\left(\frac{{\Delta }_{ji}}{w}\right)}},\\ K\left(u\right)=F\left(-{\left|u\right|}^p\right),{\Delta }_{ji}=x_j-x{}_i.\end{array}$ (2)

Here w- is the parameter of the window smoothing, p is defined as the acceleration parameter. The function K(u) determines the smoothing kernel. In this paper we associate the kernel K(u) with exponential function that was used earlier in papers [7, 8]. For the best author's knowledge expression (2) is original and never appeared in papers used by other authors. The verification of expression (2) on available data confirms the following observation, namely; if p lies in the interval [0, 1] then the smoothing process becomes slow, while for the interval [1, 6] the smoothing process accelerates. It is interesting to underscore the limiting values of the window smoothing parameter w. For w >> 1 the smoothed function Y(w>>1,p) » mean(y), while for w®0 Y(w®0,p) » y and the smoothing property disappears. Despite its precision, the final limiting case highlights some noteworthy characteristics that were overlooked in the first place. These characteristics are linked to the discrete nature of the functions Y_j and y_j (j=1,2,…,N), which in turn dictates how expression (2) behaves specifically for p > 1 and at small values of w (near zero), where a specific "competition" between small distance D_ji and w takes place. Consequently, expression (2) can be used to verify the suggested challenge (1) and find the desired answer for detection of a SWS.

Verification of (2) on available data

          We use the available meteorological data as a noise background. A detailed discussion of these facts is not required; readers who are curious about them can find it in the study [9]. The only thing that matters to us about the registration of gases CH₄, CO₂, and H₂O streams in the atmosphere is that there are enough digits (close to 10) following the dividing point. Fig. 1 demonstrates the initial TS and SWS that is embedded inside. We distinguish the chosen stream, which is shown in Fig. 1(a), in order to remove a concealed trend by differentiation procedure. Fig. 1(b) displays the little signal's particular form.

Fig. 1. Demonstration the initial TS (а) and the form of the SWS (b) taken in the form of the wave packet: (а) we show the trendless sequence TS_0(x) that is considered as initial TS. It has 358 data points with range of amplitudes Rg=146.7. (b) then we add to the TS_0(x) a small signal that is depicted as SWS

The functional form of this wave packet is the following:

$SWS\left(x\right)=\left[A\cos \left(\omega x\right)+B\sin \left(\omega x\right)\right]x(1-x)$ (3)

This wave packet has the following parameters (A=5.21×10^-16, B= 5.55×10^-16, w=100). We add this wave packet to initial signal that is shown on Fig. 1 (a). In order to avoid a simple additive signal, we add this small supplement by nonlinear way

$TS\_1\left(x\right)=\left[SWS\left(x\right)+\frac{TS\_0\left(x\right)\cdot SWS\left(x\right)}{Range(TS\_0(x\left)\right)}\right]+TS\_0\left(x\right)$ (4)

A tiny additive is present in the trendless sequence provided by expression (4), but it is entirely concealed within it. Since expression (4) graphically coincides with Fig. 1 (a), it is pointless to illustrate it. The NOCFASS (NOnorthogonal Combined Fourier Analysis of the Smoothed Signals), which was successfully used in research [10], makes sense since, in our opinion, it is crucial to improve the informative presentation of the initial signal in Fig. 1 (a). One can utilize only a portion of the modes situated to the right of the F-spectrum central axis by moving the traditional F-spectrum to the left p-angle. The shifted F-transformation and the truncated spectrum are shown by Fig.2. It incorporates on two parts that are shown below.

Fig. 2. The shifted F-transformation and the truncated spectrum: (a) we demonstrate the F-spectrum in terms of sizeless modes (0<Vect<358). The central mode is located at K/2=179. For the fitting of the TS-0(x) with maximal accuracy (Relative error = 10^-14 %) it is sufficient to use practically all (237) modes located on the right-hand side of the shifted F-spectrum; (b) we demonstrate the truncated spectrum 6.32<Wc(k)<1499.15 that provides the desired fit.

In Fig.3 we demonstrate the fit of the initial signal, realized in the frame of the NOCFASS. It includes also the amplitudes modulus and phases distributions, respectively.

 Fig. 3. The fit of the initial signal TS_0(x), realized in the frame of the NOCFASS: (a) we demonstrate the fit of TS_0(x) realized with exceptional accuracy (Rel.err = 10^-14 %); (b) we show the modulus amplitude distribution ${\mathbf{Amd}}_{\mathbf{k}}\mathbf{=}\sqrt{{\mathbf{Ac}}_{\mathbf{k}}^{\mathbf{2}}\mathbf{+}{\mathbf{As}}_{\mathbf{k}}^{\mathbf{2}}}$ for this TS; c) we show the phase distribution Ph_k=tan^-1(As_k/Ac_k). The solid red line shows the sequence of the range amplitudes (SRA). As one can see from this figure it is expressed by the segment of the stright line. It means that the phase distribution Ph_k coincides practically with uniform distribution

The fitting function is given by conventional expression

$\begin{array}{l}F\left(x\right)=A_0+\sum _{k=1}^K\left[A{c}_k\cos \left(\Omega c\left(k\right)\cdot x\right)+A{s}_k\sin \left(\Omega c\right(k)\cdot x)\right]=\\ \equiv A_0+\sum _{k=1}^K\left[Am{d}_k\cos \left(\Omega c\left(k\right)\cdot x-P{h}_k\right)\right],\\ Am{d}_k=\sqrt{A{c}_k^2+A{s}_k^2},P{h}_k={\tan }^{-1}\left(\frac{A{s}_k}{A{c}_k}\right),\backslash \\ \Omega c\left(k\right)=\Omega c\left(0\right)+\frac{k}{K}\left(\Omega c\left(K\right)-\Omega c\left(0\right)\right),\\ \Omega c\left(0\right)=\mathrm{6.32,}\Omega c\left(K\right)=1499.15.\end{array}$ (5)

In the same manner we treat the TS-1(x) with small supplement. The details are shown by Fig.4.

In Fig.4 below we demonstrate the fit of the initial signal, realized in the frame of the NOCFASS. It includes also the amplitudes modulus $Am{d}_k=\sqrt{A{c}_k^2+A{s}_k^2}$ and phases distribution Ph_k=tan^-1(As_k/Ac_k), respectively. All these three figures contain the deformed SWS that is shown in Fig.1(b) above.

Fig. 4. The fit of the initial signal TS_1(x), realized in the frame of the NOCFASS: (a) we demonstrate the fit of TS_1(x) realized with same exceptional accuracy (Rel.err = 10^-14 %);  (b) we show the modulus amplitude distribution ${\mathbf{Amd}}_{\mathbf{k}}\mathbf{=}\sqrt{{\mathbf{Ac}}_{\mathbf{k}}^{\mathbf{2}}\mathbf{+}{\mathbf{As}}_{\mathbf{k}}^{\mathbf{2}}}$ for this TS;  (c) we show the phase distribution Ph_k=tan^-1(As_k/Ac_k). The solid red line demonstrates the sequence of the range amplitudes (SRA). As one can notice again that it is expressed by the segment of the stright line. It means that the phase distribution coincides practically with uniform distribution

Now we have two important informative plots Amd_k and Ph_k (depicted in fig. 3 and fig. 4) that should be compared with each other in terms of expression (2). As it has been mentioned above the principle of application (2) can be expressed by the following algorithm: "smooth – subtract".

Let us consider firstly the comparison of the modulus $Am{d}_k=\sqrt{A{c}_k^2+A{s}_k^2}$ and Ph_k distributions, accordingly, in the frame of expression (2). For certainty, we put p=2, F coincides with exponential function and will change the value of the smoothing window in the interval w Î[0, 10]. The corresponding ranges behavior of the smoothed Amd_k and Ph_k are shown in Fig.5 a), b).

Fig.5 demonstrates the ranges distributions vs. the smoothing parameter w for modulus amplitudes and phases, respectively; for the curves with SWS (connected red balls) and without SWS (marked by the connected black squared)

Fig. 5. The ranges distributions vs the smoothing parameter w for modulus amplitudes and phases: (a) It is shows the ranges distributions behavior with respect to the value of the smoothing window that is located in the interval w Î[1, 10]. The black connected squared refers to the TS_0(x) and red connected balls to the TS_1(x). The differences between these curves are noticeable;  (b) The same behavior is similar for the Ranges distributions of phases. However, the most interesting region for detection of the SWS is located in the interval [0, 2]. It is separated on the figure (a) by two bolded vertical lines

Let us put w=1.01. Figure 6 demonstrates this result at small values of the w=1.01.

Fig. 6. Demonstration of operation result at small values of the w=1.01: (a) we put together the smoothed replicas of Amd_k. (w=1.01); (b) the difference between them is shown. It is convenient to compare the result of Md-POLS method in terms of SNR (initial (as a priori estimated) and final one) that is given by expression (1). The successive increasing of the SNR is given in Table 1

Table 1. The values of SNR achieved in the result of Md-POLS application

Comments. As one can notice from these results the value of the smoothing window w=1.01 is the critical one. Slightly changing of this value shows the limit for SNR resolution related to amplitudes modules and demonstrates the limit (w=1.0) for SNR (Ph_k). This table demonstrates also a specific "digital" limit (w=0.9) for detection of the presence SWS in the given TS.

In Fig.7 we demonstrate the phase distributions for the TS with SWS and without one together with the differences between them.

Fig. 7. Phase distributions for the TS with SWS and without one together with the differences between them: (a) We put together the smoothed replicas for the Phases distribution Ph_k (w=1.01); (b) The difference between is convenient to compare the result of Md-POLS method in terms of SNR (initial (as a priori estimated) and final one) that is given by expression (1). The successive increasing of the SNR is given in Table 1 shown above

We chose the TS with expected SNR (after embedding a small signal) into TS_0(x) equaled to -169.87. The corresponding TS is shown in Fig. 1 (a).

The following statement is worth emphasizing. Every TS with an identical incorporated SWS has a unique sensitivity. As an illustration, we use TS with SNR (expected)= -166.01.

The calculated results for another TS are shown below in Table 2.

Table 2. The values of SNR achieved in the result of Md-POLS application for another TS

Results and Discussion

In this section we should emphasize some attractive features of the Md-POLS method.

1. It makes to identify a possible SWS that could be hidden inside the designated TS. It is sensitive to the number of fixed digits after the division point.
2. It must be detected by raising the initial SNR (anticipated value) by many orders of magnitude, which corresponds to the SWS (expected value). One can surpass the 16-order-of-magnitude limit in the instance under consideration. It indicates that for the specified TS, the SNR may be increased from -160 dB up to +3 dB or even more. The Md-POLS technique becomes an essential tool for recognizing a broad class of disguised SWS as a result of this real data.
3. It is necessary to emphasize the following peculiarity of this method that was found during these numeric calculations. Each TS has its own sensitivity for detection of the SWS. In order to "percept" it we demonstrate the following pictures.

Fig. 8 demonstrates some interesting features in their behavior at the small values of the smoothing parameter w.

Fig. 8. Behavior of expected SNR ratio. TS is near –160 dB: (a) one can see the behavior of the final SNR with respect to the value of the smoothing window in the small interval [0.9-1.2] close to zero. This behavior was calculated for the Amd_k; (b) similar plot is given for the phases Ph_k ratios calculated for the same interval [1.0-1.2]. One can notice that this behavior is not uniform. There are regions where SNR achieves maximal value as 6 dB (on the left) and 1.7 dB (for the right plot).

Let us take another TS. This TS is randomly taken from the given data. Initial expected SNR ratio is – 166 dB. Similar plots for this chosen sequence are shown on Fig. 9. Fig. 9 demonstrates some interesting features in their behavior at the small values of the smoothing parameter w. It is taken another TS for comparison with Fig.8.

Figs. 9: Behavior of expected SNR. TS is near –166 dB: (a) one can see the behavior of the final SNR with respect to the value of the smoothing window w in the same small interval [1.0-1.2]. This behavior was calculated for the Amd_k; (b) Similar plot is given for the phases Ph_k distribution ratios. One can notice that this behavior is not uniform and does not coincide with Fig. 8 (a, b). There are own regions where SNR achieves maximal value as 3 dB (a) and 1.9 dB (b)

The final sensitivity of any chosen TS to detect the SWS embedded within it is determined by the particular distributions of the close amplitudes and phases, as can be seen from a comparison of the last figures.

These obtained results prompt to make the following basic conclusion. Special attention should be paid to obtaining a stable background (reproducibility of an experiment plays a crucial role!) serving as a particular detector to record any small signal that may appear outside the confidence "tube" of the background due to an uncounted SWS. This recorded perturbation can assist in detecting the desired "signal" that separates the new "quality" from the random fluctuations contained within the "reference" background.

The author sincerely hopes that this knowledge on this intriguing technique will lead to new opportunities for the detection of a broad class of different SWS in many natural disciplines.

Acknowledgements. The author would like to respectfully thank Prof. Morozov O.G. for his insightful views and comments. He focused on the articles [1]–[6], which examined the latest developments in lowering the SNR minimum limit using various techniques.

About the authors

R. R. Nigmatullin

Author for correspondence.
Email: vskhayrova@kai.ru
Russian Federation, Kazan

References

Supplementary files

Supplementary Files

Action

1. JATS XML

Download

2. Fig. 1. Demonstration the initial TS (а) and the form of the SWS (b) taken in the form of the wave packet: (а) we show the trendless sequence TS_0(x) that is considered as initial TS. It has 358 data points with range of amplitudes Rg=146.7. (b) then we add to the TS_0(x) a small signal that is depicted as SWS

Download (336KB)

Indexing metadata

3. Fig. 2. The shifted F-transformation and the truncated spectrum: (a) we demonstrate the F-spectrum in terms of sizeless modes (0

Download (337KB)

Indexing metadata

4. Fig. 3. The fit of the initial signal TS_0(x), realized in the frame of the NOCFASS: (a) we demonstrate the fit of TS_0(x) realized with exceptional accuracy (Rel.err = 10-14 %); (b) we show the modulus amplitude distribution for this TS; c) we show the phase distribution Phk=tan-1(Ask/Ack). The solid red line shows the sequence of the range amplitudes (SRA). As one can see from this figure it is expressed by the segment of the stright line. It means that the phase distribution Phk coincides practically with uniform distributionНеозаглавлен

Download (324KB)

Indexing metadata

5. Fig. 4. The fit of the initial signal TS_1(x), realized in the frame of the NOCFASS: (a) we demonstrate the fit of TS_1(x) realized with same exceptional accuracy (Rel.err = 10-14 %); (b) we show the modulus amplitude distribution for this TS; (c) we show the phase distribution Phk=tan-1(Ask/Ack). The solid red line demonstrates the sequence of the range amplitudes (SRA). As one can notice again that it is expressed by the segment of the stright line. It means that the phase distribution coincides practically with uniform distribution

Download (313KB)

Indexing metadata

6. Fig. 5. The ranges distributions vs the smoothing parameter w for modulus amplitudes and phases: (a) It is shows the ranges distributions behavior with respect to the value of the smoothing window that is located in the interval w Î[1,10]. The black connected squared refers to the TS_0(x) and red connected balls to the TS_1(x). The differences between these curves are noticeable; (b) The same behavior is similar for the Ranges distributions of phases. However, the most interesting region for detection of the SWS is located in the interval [0,2]. It is separated on the figure (a) by two bolded vertical lines Let us put w=1.01. Figure 6 demonstrates this result at small values of the w=1.01.

Download (280KB)

Indexing metadata

7. Fig. 6. Demonstration of operation result at small values of the w=1.01: (a) we put together the smoothed replicas of Amdk. (w=1.01); (b) the difference between them is shown. It is convenient to compare the result of Md-POLS method in terms of SNR (initial (as a priori estimated) and final one) that is given by expression (1). The successive increasing of the SNR is given in Table 1

Download (254KB)

Indexing metadata

8. Fig. 7. Phase distributions for the TS with SWS and without one together with the differences between them: (a) We put together the smoothed replicas for the Phases distribution Phk (w=1.01); (b) The difference between is convenient to compare the result of Md-POLS method in terms of SNR (initial (as a priori estimated) and final one) that is given by expression (1). The successive increasing of the SNR is given in Table 1 shown above

Download (291KB)

Indexing metadata

9. Fig. 8. Behavior of expected SNR ratio. TS is near –160 dB: (a) one can see the behavior of the final SNR with respect to the value of the smoothing window in the small interval [0.9-1.2] close to zero. This behavior was calculated for the Amdk; (b) similar plot is given for the phases Phk ratios calculated for the same interval [1.0-1.2]. One can notice that this behavior is not uniform. There are regions where SNR achieves maximal value as 6 dB (on the left) and 1.7 dB (for the right plot).

Download (214KB)

Indexing metadata

10. Figs. 9: Behavior of expected SNR. TS is near –166 dB: (a) one can see the behavior of the final SNR with respect to the value of the smoothing window w in the same small interval [1.0-1.2]. This behavior was calculated for the Amdk; (b) Similar plot is given for the phases Phk distribution ratios. One can notice that this behavior is not uniform and does not coincide with Fig. 8 (a, b). There are own regions where SNR achieves maximal value as 3 dB (a) and 1.9 dB (b)

Download (238KB)

Indexing metadata