Volume 13 Number 7 (Jul. 2018)
Home > Archive > 2018 > Volume 13 Number 7 (Jul. 2018) >
JCP 2018 Vol.13(7): 805-822 ISSN: 1796-203X
doi: 10.17706/jcp.13.7.805-822

Impact of Different Random Initializations on Generalization Performance of Extreme Learning Machine

Xiaofang Zhang1, Xiaoli Lin1, Rana Aamir Raza Ashfaq2
1Information Engineering Department, City College, Wuhan University of Science and Technology, Wuhan 430083, China.
2Department of Computer Science, Bahauddin Zakariya University, Multan, Pakistan.

Abstract—The generalization performance of extreme learning machine (ELM) is influenced by the random initializations to input-layer weights and hidden-layer biases. In this paper, we demonstrate this conclusion through testing the classification accuracies of ELMs corresponding to different random initializations. 30 UCI data sets and 24 continuous probability distributions are employed in this experimental study. The final results present the following important and valuable observations and conclusions, i.e., (1) the probability distributions with symmetrical and bell-shaped probability density functions (e.g., Hyperbolic Secant, Student's-t, Laplace and Normal) always bring about the higher training accuracies and easily cause the over-fitting of ELM; (2) ELMs with random input-layer weights and hidden-layer biases chosen from heavy-tailed distributions (e.g., Gamma, Rayleigh and Frechet) have the better generalization performances; and (3) the light-tailed distributions (e.g., Central Chi-Squared, Erlang, F, Gumbel and Logistic) are usually unsuited to initialize the input-layer weights and hidden-layer biases for ELM. All these provide the useful enlightenments for practical applications of ELMs in different fields.

Index Terms—Extreme learning machine, ELM, generalization performance, random initialization.

[PDF]

Cite: Xiaofang Zhang, Xiaoli Lin, Rana Aamir Raza Ashfaq, "Impact of Different Random Initializations on Generalization Performance of Extreme Learning Machine," Journal of Computers vol. 13, no. 7, pp. 805-822, 2018.

General Information

ISSN: 1796-203X
Frequency: Monthly
Editor-in-Chief: Prof. Liansheng Tan
Executive Editor: Ms. Nina Lee
Abstracting/ Indexing: DBLP, EBSCO,  ProQuest, INSPEC, ULRICH's Periodicals Directory, WorldCat, CNKI,etc
E-mail: jcp@iap.org
  • Sep 13, 2018 News!

    Vol 13, No 10 has been published with online version   [Click]

  • Apr 28, 2019 News!

    Vol 14, No 4 has been published with online version 8 papers are published in this issue after peer review   [Click]

  • Mar 20, 2019 News!

    Vol 14, No 3 has been published with online version   [Click]

  • Feb 22, 2019 News!

    Vol 14, No 2 has been published with online version 8 papers are published in this issue after peer review   [Click]

  • Jan 04, 2019 News!

    Vol 14, No 1 has been published with online version   [Click]

  • Read more>>