Volume 13 Number 7 (Jul. 2018)
Home > Archive > 2018 > Volume 13 Number 7 (Jul. 2018) >
JCP 2018 Vol.13(7): 805-822 ISSN: 1796-203X
doi: 10.17706/jcp.13.7.805-822

Impact of Different Random Initializations on Generalization Performance of Extreme Learning Machine

Xiaofang Zhang1, Xiaoli Lin1, Rana Aamir Raza Ashfaq2
1Information Engineering Department, City College, Wuhan University of Science and Technology, Wuhan 430083, China.
2Department of Computer Science, Bahauddin Zakariya University, Multan, Pakistan.

Abstract—The generalization performance of extreme learning machine (ELM) is influenced by the random initializations to input-layer weights and hidden-layer biases. In this paper, we demonstrate this conclusion through testing the classification accuracies of ELMs corresponding to different random initializations. 30 UCI data sets and 24 continuous probability distributions are employed in this experimental study. The final results present the following important and valuable observations and conclusions, i.e., (1) the probability distributions with symmetrical and bell-shaped probability density functions (e.g., Hyperbolic Secant, Student's-t, Laplace and Normal) always bring about the higher training accuracies and easily cause the over-fitting of ELM; (2) ELMs with random input-layer weights and hidden-layer biases chosen from heavy-tailed distributions (e.g., Gamma, Rayleigh and Frechet) have the better generalization performances; and (3) the light-tailed distributions (e.g., Central Chi-Squared, Erlang, F, Gumbel and Logistic) are usually unsuited to initialize the input-layer weights and hidden-layer biases for ELM. All these provide the useful enlightenments for practical applications of ELMs in different fields.

Index Terms—Extreme learning machine, ELM, generalization performance, random initialization.

[PDF]

Cite: Xiaofang Zhang, Xiaoli Lin, Rana Aamir Raza Ashfaq, "Impact of Different Random Initializations on Generalization Performance of Extreme Learning Machine," Journal of Computers vol. 13, no. 7, pp. 805-822, 2018.

General Information

ISSN: 1796-203X
Frequency: Monthly (2006-2014); Bimonthly (Since 2015)
Editor-in-Chief: Prof. Liansheng Tan
Executive Editor: Ms. Nina Lee
Abstracting/ Indexing: DBLP, EBSCO,  ProQuest, INSPEC, ULRICH's Periodicals Directory, WorldCat, CNKI,etc
E-mail: jcp@iap.org
  • Dec 26, 2017 News!

    Vol 12, No 1-N0 5 has been indexed by EI (Inspec)     [Click]

  • Dec 26, 2017 News!

    Vol 11, No 4-N0 6 has been indexed by EI (Inspec)     [Click]

  • Dec 21, 2017 News!

    Vol 13, No 7 has been published with online version 12 papers are published in this issue after peer review   [Click]

  • Sep 26, 2017 News!

    Papers published in JCP Volume 12 have all been indexed by DBLP   [Click]

  • Sep 22, 2017 News!

    Vol 13, No 6 has been published with online version 11 papers are published in this issue after peer review   [Click]

  • Read more>>