Volume 15 Number 3 (May 2020)
Home > Archive > 2020 > Volume 15 Number 3 (May 2020) >
JCP 2020 Vol.15(3): 85-97 ISSN: 1796-203X
doi: 10.17706/jcp.15.3.85-97

Image Augmentation for Eye Contact Detection Based on Combination of Pre-trained Alex-Net CNN and SVM

Yuki Omori, Yoshihiro Shima
Meisei University, 2-1-1, Hodokubo, Hino, Tokyo, Japan 191-8506.
Abstract—Making eye contact is the most powerful mode of establishing a communicative link between humans. We propose a method for detecting eye contact (mutual gaze) from images of both eyes through the combined usage of a pre-trained convolutional neural network (CNN) and a support vector machine (SVM). Neural networks are a powerful technology for classifying object images. When it comes to classification accuracy, a huge number of training samples is the key to success. The training samples are augmented by image perturbation, namely, shifting the cropping regions. A pre-trained CNN, Alex-Net, is used as the image feature extractor after being pre-trained for large-scale object image datasets. An SVM is used as the trainable classifier. Original both-eyes samples of two classes on the Columbia Gaze Data Set CAVE-DB are divided in five-fold cross-validation. Manually cropped images and automatically augmented images on the CAVE-DB are trained by the SVM. The feature vectors of the eye images are then passed to the SVM from Alex-Net. We performed 5-fold t-testing on 77 images and found that the average error rate was 16.44%, and the lowest error rate of images without glasses was 8.96% with 7,850 training images of perturbation. These results demonstrate that the proposed method is effective in detecting eye contact.

Index Terms—CNN, eye contact, gaze direction, head pose, image augmentation, SVM.

[PDF]

Cite: Yuki Omori, Yoshihiro Shima, "Image Augmentation for Eye Contact Detection Based on Combination of Pre-trained Alex-Net CNN and SVM," Journal of Computers vol. 15, no. 3, pp. 85-97, 2020.

Copyright © 2020 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).

General Information

ISSN: 1796-203X
Abbreviated Title: J.Comput.
Frequency: Bimonthly
Editor-in-Chief: Prof. Liansheng Tan
Executive Editor: Ms. Nina Lee
Abstracting/ Indexing: DBLP, EBSCO,  ProQuest, INSPEC, ULRICH's Periodicals Directory, WorldCat,etc
E-mail: jcp@iap.org
  • Nov 14, 2019 News!

    Vol 14, No 11 has been published with online version   [Click]

  • Mar 20, 2020 News!

    Vol 15, No 2 has been published with online version   [Click]

  • Dec 16, 2019 News!

    Vol 14, No 12 has been published with online version   [Click]

  • Sep 16, 2019 News!

    Vol 14, No 9 has been published with online version   [Click]

  • Aug 16, 2019 News!

    Vol 14, No 8 has been published with online version   [Click]

  • Read more>>