AhmadPouradabiDepartmentof Electrical EngineeringShahidRajaee Teacher Training UniversityTehran,[email protected] mohammadsharahiDepartmentof Electrical EngineeringShahidRajaee Teacher TrainingUniversityTehran,[email protected] Mohammad Shams Esfand AbadiDepartment of Electrical EngineeringShahid Rajaee Teacher Training UniversityTehran, Iran [email protected] Abstract— Thispaper presents a convex combination of diffusion LMS algorithm for the distributedestimation problem over diffusion networks in order to improve trackingcapabilities in non-stationary environments.

Two different step-sizes inadaptation stage have been used, A fast DLMS for fast convergence and slow DLMSto attain lower steady-state error. In other hand the proposed algorithm is asolution to compromise between step-size and rate of convergence in diffusionnetworks and brings the ability to have fast convergence and low steady-stateerror at price of a little increase of complexity.Keywords- diffusion LMS; convex combination; diffusionnetwork; non-stationary environments.

I. IntroductionAdaptive filters have been applied to wide variety ofsignal processing problems, including noise cancelation 1, systemidentification 2 or channel equalization 3. Choosing right adaptive filterfor an application is very depend on situation and as it’s obvious there is atrade-off between rate of convergence and final misadjustment for selectingparameters. As a solution to this problem combination of several adaptivefilters has been proposed 4. These filters are combined in a way that theadvantages of both component filters are kept: the fast convergence from thefast filter, and the reduced steady-state error from the slow filter.

Adaptive networks includesof distributed nodes with a connection topology. Distributed networks have beenused an efficient data processing technology for network applications. The globaltarget is to make the nodes able to estimate a vector of parameters of interestfrom the observed data. In a centralized network, the data orlocal estimates from all nodes would be transmitted to a central processorwhere they would be combined and the vector of parameters estimated. This methodneeds efficient communication waysto transmit and receive data between the nodes and the central processor.inthese kind of networks Failure of the central node disrupt the whole network.

Besidesa centralized solution limits the ability of the nodes to adapt in real-time totime varying statistical profiles.In recent literatures limited cooperation and incrementallike techniques 5, 6, has proposed to solve distributed estimation problemsadaptively. When sufficientcommunication resources are available, distributed adaptive algorithms can beused that employ more network connectivity and increase the degree ofcooperation among nodes. In contrast to classical centralized techniques,distributed processing exploits local computations at each node andcommunications among neighboring nodes to solve problems over the entirenetwork. This useful capability extends the scalability and flexibility of thenetwork and leads to a wide range of applications.

In the diffusion mode ofcooperation, the nodes interchange their estimates with neighbor nodes andincorporate the collected estimates via linear combinations.Various adaptive algorithm rules in adaptation step ofdiffusional cooperation have been implemented. These include DAPA families78, DNSAF families 9 10, DSAF 11, DSSAF 1213, DLMS 14, DNLMS15, DRLS 16, VS-DSAF 17. Among these algorithms, Diffusion LMS is asimple, robust and low complexity algorithm. But like always there is acompromise between rate of convergence and steady-state error when we have toadjust appropriate step-size.

To solve this problem in this paper convexcombination of two diffusion LMS is proposed.This paper is organized as follows. In section II, convexcombination of single adaptive filters and estimation problem over distributednetworks based on diffusion LMS strategies have been reviewed.

In section III,formulation of convex combination of DLMS are introduced. Section IV presentssimulation results.Throughoutthe paper, the following notations are used ? .

? Norm of a scalar. Squared Euclidean norm of a vector. L1-norm of a vector. Transpose of a vector or a matrix. II. ReviewA. Convex combination of simple adaptive filtersThe most simple combination scheme incorporates two adaptivefilters 18.

This configuration is illustrated in Figure 1. Both filters haveaccess to the same input and reference signal. It has two adaptation layer:single adaptive filters and combination layer. Overall output of thiscombination filter is given by (1) Where are the outputs of the two adaptive filtersdefines by weights , and is a mixing parameter. The estimatedweight vector and the error of the combination scheme are given by (2) (3) , are the errors of theadaptive filters components.

In Convex combination schemes, activationfunctions utilized to keep the mixing parameter in the range of interest. 12proposed an activation function using an auxiliary parameter that is related to via the sigmoid function (4) Regarding to this activation function, automaticallywill have values between 0 and 1, and can beadapted without constraints. Using a gradient descent method to minimize the squarederror of the overall filter 19, from (z), we have following equation to updatea(n): (5) Figure 1. Simple combinationof two adaptive filtersDiffusionLMSIn order to optimize MSE in adistributed manner there is two approach, cooperative and non-cooperative mode 20. Diffusion strategies 21 enable the solution of optimizingin a distributed and adaptive manner.

Compared to the non-cooperative solution 22, diffusion strategies introducea useful aggregation step that brings the ability to collect information fromlocal neighborhoods and Participate them in adaptation step. Two diffusion scheme canbe derived 23; one is the adapt-then-combine(ATC) structure, which is described by the following update: (6) (7) where denotes the estimator for atagent k at time i. The first operation in (6) is anadaptation step where agent k uses its data {} to obtain its intermediateestimator. The second operation is acombination step where agent k fuse the estimators from its neighbors toupdate the intermediate estimator to All other agents in the networkare simultaneously performing a similar operation and aggregating theestimators of their neighbors and updating their own estimator. Here, diffusionhave the meaning when in (6) data from theneighborhood have effect on the location k and the information diffuse throughthe network. The reason for the qualification “diffusion” isthat the intermediate state in (6)allows information to diffuse through the network by bringing into location kthe effect of data beyond the neighborhood of k.

III. Convex Combination of DLMSIn the matter of combination of two different step-sizeDLMS, it is necessary to define two adaptation rule, one for slow and one forfast DLMS. (8) (9) (10) Next we have adaptivemixing layer that uses global error of each node to compute mixing parameter: (11) denotes error of node k for each DLMS atiteration i. similar to convex combination we define an auxiliary functionwhich adaptively update to obtain optimum mixing parameter. Mixing parameter isdefined via a sigmoid activation function: (12) is mixing parameter of node k atiteration i. is being update adaptively tominimize the error of combined filter. By means of stochastic gradientalgorithm we have following update equation: (13) (14) Where is a priori error of combined filters of every iteration at node k. Auxiliary function is different for each node so every node have a uniquemixing parameter that is better for our cause and brings the ability to applyvarious step-sizes in nodes.

Global weights is derived from following equationwhich is familiar within the convex combination: (15) IV. Simulation ResultsIn this section,we show the performance of the proposed combination and compare our experimentswith other estimation solutions. Two different network topology have been used. Figure 2, 3,4 shows the network topology and the node profiles of and in 20 Nodes topology and Figure 5, 6, 7 the sameway in 10 Nodes network topology. Figure2. Network topology (J=20)Figure 3.

Noise variance for each Node (J = 20)Figure 4. Correlationindex for each Node (J = 20)Figure 5. Networktopology (J = 10)Figure 6. Noisevariance for each Node (J = 10)Figure 7. Correlationindex for each Node (J = 10)A. Convergence We obtained the performance of the proposedalgorithms in diffusion network with J = 20 nodes in a system identificationsetup. The impulse response of the random unknown system has M = 16 taps.

The noise sequence of each node is a whiteGaussian process with variance ? 0, 0.1). We use the Metropolis rule forcombination weights () in theadapt-then-combine (ATC) diffusion strategy without data exchange. Theperformance of the algorithms are compared by the network normalized meansquare deviation (NMSD). All simulated learning curves are obtained byaveraging over 100 independent trials. Figure 8 and Figure 10 and Figure 11illustrate the comparison between two single DLMS and proposed combination ofthem. As it shows the convergence follows the fast DLMS up to near its steadystate and then follows the slow DLMS. Figure 9 and Figure 12 shows the NMSDlearning curve of each node for both topologies.

Figure8. TheNLMS learning curves of CC-DLMS with = 0.002 and = 0.008, =300 and two single DLMS with µ = 0.002 and µ = 0.008.

Figure 9. NMSDfor each Node (J = 10) with CC-DLMS AlgorithmFigure 10. The NLMS learning curves of CC-DLMS with = 0.005 and = 0.01, =300 and two single DLMS with µ = 0.005 and µ = 0.01 (J = 20).Figure11.

TheNLMS learning curves of CC-DLMS with = 0.005 and = 0.01, =300 and two single DLMS with µ = 0.005 and µ = 0.01 (J = 20).Figure 12.

NMSD for each Node(J = 10) with CC-DLMS (= 0.002, = 0.008, = 300) and two single DLMS (µ = 0.002 and µ =0.008).A. Tracking Figure 13 andFigure 14 illustrate tracking capabilities improvement of single DLMS inproposed algorithm.

In this manner we changed the optimum weights of networkafter 15000 iteration in both topologies.Figure13. Trackingperformance of CC-DLMS (= 0.002, = 0.008, =300) and two single DLMS (µ = 0.002 and µ = 0.01) in 20Nodes topology.Figure14.

Trackingperformance of CC-DLMS (= 0.002, = 0.008, =300) and two single DLMS (µ = 0.002 and µ = 0.

01) in 10 Nodes topology.References 1 B. Widrow et al.

, “Adaptive NoiseCancelling: Principles and Applications,” Proc. IEEE, vol. 63, no. 12,pp. 1692–1716, 1975.2 J.Arenas-García, M. Martínez-Ramón, Á.

Navia-Vázquez, and A. R. Figueiras-Vidal,”Plant identification via adaptive combination of transversal filters,” SignalProcessing, vol. 86, no. 9, pp. 2430–2438, Sep. 2006.3 S.

U. H. Qureshi, “Adaptive Equalization,” Proc. IEEE, vol. 73, no.

9, pp.1349–1387, 1985.4 J.Arenas-Garcia, L. A. Azpicueta-Ruiz, M. T. M.

Silva, V. H. Nascimento, and A.H. Sayed, “Combinations of Adaptive Filters: Performance and convergenceproperties,” IEEE Signal Process. Mag., vol. 33, no.

1, pp. 120–140,Jan. 2016.5 A.H. Sayed and C. G. Lopes, “Distributed processing over adaptive networks,” in 20079th International Symposium on Signal Processing and Its Applications,2007, vol.

2, no. 3, pp. 1–3.6 C.G. Lopes and A.

H. Sayed, “Distributed Adaptive Incremental Strategies:Formulation and Performance Analysis,” 2006 IEEE Int. Conf. Acoust. SpeedSignal Process.

Proc., vol. 3, no. 4, p. III-584-III-587, 2006.7 M.S.

E. Abadi and Z. Saffari, “Distributed estimation over an adaptive diffusionnetwork based on the family of affine projection algorithms,” in 6thInternational Symposium on Telecommunications (IST), 2012, vol. 58, no. 1,pp.

607–611.8 L.Li and J. A.

Chambers, “Distributed adaptive estimation based on the APAalgorithm over diffusion networks with changing topology,” in 2009 IEEE/SP15th Workshop on Statistical Signal Processing, 2009, vol. 1, no. 5, pp.757–760.

9 M.Shams Esfand Abadi and M. S.

Shafiee, “A family of diffusion normalized subbandadaptive filter algorithms over distributed networks,” Int. J. Commun. Syst.,vol. 30, no. 17, p.

e3348, Nov. 2017.10 M. Shams Esfand Abadi and M. S.

Shafiee, “Diffusion normalizedsubband adaptive algorithm for distributed estimation employing signedregressor of input signal,” Digit. Signal Process., vol. 70, pp. 73–83,2017.11 J. Seo, S.

M. Jung, and P. Park, “Acoustic echo cancellation indistributed network using improved diffusion subband adaptive filteringalgorithm,” in 2016 10th International Symposium on Communication Systems,Networks and Digital Signal Processing (CSNDSP), 2016, pp. 1–6.12 W.

Wang, H. Zhao, and Q. Liu, “Diffusion Sign Subband AdaptiveFiltering Algorithm with Individual Weighting Factors for Distributed Estimation,”Circuits, Syst. Signal Process.

, vol. 36, no. 6, pp. 2605–2621, 2017.

13 J. Ni, “Diffusion Sign Subband Adaptive Filtering Algorithm forDistributed Estimation,” IEEE Signal Process. Lett.

, vol. 22, no. 11,pp. 2029–2033, Nov. 2015.14 C. G. Lopes and A.

H. Sayed, “Diffusion Least-Mean Squares OverAdaptive Networks: Formulation and Performance Analysis,” IEEE Trans. SignalProcess., vol. 56, no. 7, pp. 3122–3136, Jul.

2008.15 S. A. Baqi, A.

Zerguine, and M. O. Bin Saeed, “Diffusion normalizedleast mean squares over wireless sensor networks,” in 2013 9th InternationalWireless Communications and Mobile Computing Conference (IWCMC), 2013, pp.1454–1457.16 F. S. Cattivelli, C. G.

Lopes, and A. H. Sayed, “Diffusionrecursive least-squares for distributed estimation over adaptive networks,” IEEETrans. Signal Process.

, vol. 56, no. 5, pp. 1865–1877, May 2008.17 J.-H. Seo, S. M.

Jung, and P. Park, “A diffusion subband adaptivefiltering algorithm for distributed estimation using variable step size and newcombination method based on the MSD,” Digit. Signal Process., vol. 48,pp. 361–369, Jan.

2016.18 M. Martinez-Ramon, J. Arenas-Garcia, A. Navia-Vazquez, and A. R.Figueiras-Vidal, “An adaptive combination of adaptive filters for plant identification,”in 2002 14th International Conference on Digital Signal ProcessingProceedings. DSP 2002 (Cat.

No.02TH8628), 2002, vol. 2, pp. 1195–1198.19 J.

Arenas-García, A. R. Figueiras-Vidal, and A.

H. Sayed,”Mean-square performance of a convex combination of two adaptive filters,” IEEETrans. Signal Process., vol. 54, no. 3, pp.

1078–1090, 2006.20 M. H. Degroot, “Reaching a consensus,” J. Am.

Stat. Assoc.,vol. 69, no.

345, pp. 118–121, Mar. 1974.

21 A. H. Sayed, “Diffusion Adaptation Over Networks,” in AcademicPress Library in Signal Processing: Volume 3 Array and Statistical SignalProcessing, vol. 3, Elsevier Masson SAS, 2014, pp.

323–453.22 S. Tu and A. H. Sayed, “Diffusion Strategies Outperform ConsensusStrategies for Distributed Estimation Over Adaptive Networks,” IEEE Trans.

Signal Process., vol. 60, no. 12, pp. 6217–6234, Dec. 2012.23 F.

S. Cattivelli and A. H. Sayed, “Diffusion LMS Strategies forDistributed Estimation,” IEEE Trans. Signal Process.

, vol. 58, no. 3,pp. 1035–1048, Mar. 2010.