Copyright and Licensing
Articles accepted for publication will be licensed under the Creative Commons BY-NC-SA. Authors must sign a non-exclusive distribution agreement after article acceptance.
Model and implementation of body movement recognition using Support Vector Machines and Finite State Machines with cartesian coordinates input for gesture-based interaction
Keywords:
Gesture, SVM, FSM, Kinect, ModelAbstract
The growth in the use of gesture-based interaction in video games has highlighted the potential for the use of such interaction method for a wide range of applications. This paper presents the implementation of an enhanced model for gesture recognition as input method for software applications. The model uses Support Vector Machines (SVM) and Finite State Machines (FSM) and the implementation was based on a Kinect R device. The model uses data input based on Cartesian coordinates. The use of Cartesian coordinates enables more flexibility to generalise the use of the model to different applications, when compared to related work encountered in the literature based on accelerometer devices for data input. The results showed that the use of SVM and FSM with Cartesian coordinates as input for gesture-based interaction is very promising. The success rate in gesture recognition was 98%, from a training corpus of 9 sets obtained by recording real users’ gestures. A proof-of-concept implementation of the gesture recognition interaction was performed using the application Google Earth(R). A preliminary acceptance evaluation with users indicated that the interaction with the system via the implementation reported was satisfactory.
Downloads
Download data is not yet available.
References
[1] S. Mitra, and T. Acharya, Gesture Recognition: A Survey, IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, May, 2007, vol. 37, no. 3, pp. 311–324.
[2] M. Weiser, The Computer for the 21st Century, Scientific American – Special Issue on Communications, Computers, and Networks, September, 1991, vol. 265, no. 3, pp. 94–104.
[3] M. Weiser, Some Computer Science Issues in Ubiquitous Computing, Communications of the ACM – Special Issue on Computer Augmented Environments: Back to the Real World, July, 1993, vol. 36, no. 7, pp. 75–84.
[4] S. S. Rautaray, and A. Agrawal, Real Time Multiple Hand Gesture Recognition System for Human Computer Interaction, International Journal of Intelligent Systems and Applications (IJISA), May, 2012, vol. 4, no. 5, pp. 56–64.
[5] K. A. Yuksel, and S. H. Adali, Prototyping Input Controller for Touch-less Interaction with Ubiquitous Environments, Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI’11), Stockholm, Sweeden, August-September, 2011, pp. 635–640.
[6] P. M. Yanik, J. Manganelli, J. Merino, A. L. Threatt, J. O. Brooks, K. E. Green, and I. D. Walker, Use of Kinect Depth Data and Growing Neural Gas for Gesture based Robot Control, Proceedings of the 6th International Conference on Pervasive Computing Technologies for Health-care (PervasiveHealth), San Diego, California, USA, 21-24 May, 2012, pp. 283–290.
[7] M. F. Shiratuddin, and K. W. Wong, Non-contact Multi-hand Gestures Interaction Techniques for Architectural Design in a Virtual Environment, International Conference on Information Technology and Multimedia (ICIM), Kajang, Malaysia, 14-16 November, 2011, pp. 1–6.
[8] M. Roccetti, and G. Marfia, Recognizing Intuitive Predefined Gestures for Cultural Specific Interactions: An Image-based Approach, Proceedings of the IEEE Consumer Communications and Networking Conference, Las Vegas, Nevada, USA, 8-11 January, 2011, pp. 172–176.
[9] L. R. Rabiner, A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition, Proceedings of the IEEE, February, 1989, vol. 77, no. 2, pp. 257–286.
[10] J. Yamato, J. Ohya, and K. Ishii, Recognizing Human Action in Time-sequential Images using Hidden Markov Model, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Illinois, 15-18 June, 1992, pp. 379–385.
[11] F. Samaria, and S. Young, HMM-based Architecture for Face Identification, Image and Vision Computing, October, 1994, vol. 12, no. 8, pp. 537–543.
[12] F. Bevilacqua, B. Zamborlin, A. Sypniewski, N. Schnell, F. Guédy, and N. Rasamimanana, Continuous Realtime Gesture Following and Recognition, Proceedings of the 8th International Conference on Gesture in Embodied Communication and Human-Computer Interaction, Bielefeld, Germany, 2010, pp. 73–84.
[13] J. Davis, and M. Shah, Visual Gesture Recognition, IEEE Proceedings - Vision, Image and Signal Processing, April, 1994, vol. 141, no. 2, pp. 101–106.
[14] P. Hong, M. Turk, and T. S. Huang, Gesture Modeling and Recognition using Finite State Machines, Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition: IEEE Computer Society, Grenoble, France, 28-30 March, 2000, pp. 410–415.
[15] A. F. Bobick, and A. D. Wilson, A State-based Approach to the Representation and Recognition of Gesture, IEEE Transactions on Pattern Analysis and Machine Intelligence, December, 1997, vol. 19, no. 12, pp. 1325–1337.
[16] M. Oshita, and T. Matsunaga, Automatic Learning of Gesture Recognition Model using SOM and SVM, Proceedings of the 6th International Conference on Advances in Visual Computin (ISVC’10) - Volume Part I , Las Vegas, Nevada, USA, September, 2010, pp. 751–759.
[17] M. Lech, and B. Kostek, Gesture-based Computer Control System Applied to the Interactive Whiteboard, Proceedings of the 2nd International Conference on Information Technology (ICIT), Gdansk, Poland, 28-30 June, 2010, pp. 75–78.
[18] T. Matsunaga, and M. Oshita, Recognition of Walking Motion Using Support Vector Machines, Proceedings of the 1st International Symposium on Information and Computer Elements (ISICE), 2007, pp. 337–342.
[19] Microsoft Research, Programming Guide: Getting Started with the Kinect for Windows SDK Beta from Microsoft Research, Microsoft Corporation, Technical Report, 2011.
[20] K. K. Biswas, and S. K. Basu, Gesture Recognition using Microsoft Kinect, Proceedings of the 5th International Conference on Automation, Robotics and Applications (ICARA), Wellington, New Zealand, 6-8 December, 2011, pp. 100–103.
[21] Z. Ren, J. Meng, J. Yuan, and Z. Zhang, Robust Hand Gesture Recognition with Kinect Sensor, Proceedings of the 19th ACM International Conference on Multimedia, Scottsdale, Arizona, USA, November-December, 2011, pp. 759–760.
[22] Y. Song, D. Demirdjian, and R. Davis, Continuous Body and Hand Gesture Recognition for Natural Human-Computer Interaction, ACM Transactions on Interactive Intelligent Systems (TiiS) – Special Issue on Affective Interaction in Natural Environments, March, 2012, vol. 2, no 1, pp.1–28.
[23] M. A. Hearst, Support Vector Machines, IEEE Intelligent Systems and their Applications – Issue: Computing & Processing (Hardware/Software), July-August, 1998, vol. 18, no. 4, pp. 18–28.
[24] A. C. Lorena, and A. C. P. L. F. Carvalho, An Introduction to Support Vector Machines (in Portuguese), Journal of Applied and Theoretical Informatics (RITA), 2007, vol. 14, no. 2, pp. 43–67.
[25] M. Behzad, K. Asghari, M. Eazi, and M. Palhang, Generalization Performance of Support Vector Machines and Neural Networks in Runoff Modeling, Expert Systems with Applications: An International Journal, May, 2009, vol. 36, no. 4, pp. 7624–7629.
[26] C. C. Chang, and C. J. Lin, LIBSVM: A Library for Support Vector Machines, 2012. Available in: http://www.csie.ntu.edu.tw/ cjlin/libsvm. Accessed in July, 2012.
[27] C. Hsu, C. C. Chang, and C. J. Lin, A Practical Guide to Support Vector Classification, Department of Computer Science National Taiwan University, Taipei 106, Taiwan, 2010. Available in: http://www.csie.ntu.edu.tw/ ̃cjlin/papers/guide/guide.pdf
[28] S. Haykin, Neural Networks: A Comprehensive Foundation, Upper Saddle River, NJ, USA, Prentice Hall PTR, Second Edition, March, 1999.
[29] K. C. Louden, Compiler Construction: Principles and Practice, Boston, MA, USA: PWS Publishing Co., First Edition, January 1997.
[2] M. Weiser, The Computer for the 21st Century, Scientific American – Special Issue on Communications, Computers, and Networks, September, 1991, vol. 265, no. 3, pp. 94–104.
[3] M. Weiser, Some Computer Science Issues in Ubiquitous Computing, Communications of the ACM – Special Issue on Computer Augmented Environments: Back to the Real World, July, 1993, vol. 36, no. 7, pp. 75–84.
[4] S. S. Rautaray, and A. Agrawal, Real Time Multiple Hand Gesture Recognition System for Human Computer Interaction, International Journal of Intelligent Systems and Applications (IJISA), May, 2012, vol. 4, no. 5, pp. 56–64.
[5] K. A. Yuksel, and S. H. Adali, Prototyping Input Controller for Touch-less Interaction with Ubiquitous Environments, Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI’11), Stockholm, Sweeden, August-September, 2011, pp. 635–640.
[6] P. M. Yanik, J. Manganelli, J. Merino, A. L. Threatt, J. O. Brooks, K. E. Green, and I. D. Walker, Use of Kinect Depth Data and Growing Neural Gas for Gesture based Robot Control, Proceedings of the 6th International Conference on Pervasive Computing Technologies for Health-care (PervasiveHealth), San Diego, California, USA, 21-24 May, 2012, pp. 283–290.
[7] M. F. Shiratuddin, and K. W. Wong, Non-contact Multi-hand Gestures Interaction Techniques for Architectural Design in a Virtual Environment, International Conference on Information Technology and Multimedia (ICIM), Kajang, Malaysia, 14-16 November, 2011, pp. 1–6.
[8] M. Roccetti, and G. Marfia, Recognizing Intuitive Predefined Gestures for Cultural Specific Interactions: An Image-based Approach, Proceedings of the IEEE Consumer Communications and Networking Conference, Las Vegas, Nevada, USA, 8-11 January, 2011, pp. 172–176.
[9] L. R. Rabiner, A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition, Proceedings of the IEEE, February, 1989, vol. 77, no. 2, pp. 257–286.
[10] J. Yamato, J. Ohya, and K. Ishii, Recognizing Human Action in Time-sequential Images using Hidden Markov Model, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Illinois, 15-18 June, 1992, pp. 379–385.
[11] F. Samaria, and S. Young, HMM-based Architecture for Face Identification, Image and Vision Computing, October, 1994, vol. 12, no. 8, pp. 537–543.
[12] F. Bevilacqua, B. Zamborlin, A. Sypniewski, N. Schnell, F. Guédy, and N. Rasamimanana, Continuous Realtime Gesture Following and Recognition, Proceedings of the 8th International Conference on Gesture in Embodied Communication and Human-Computer Interaction, Bielefeld, Germany, 2010, pp. 73–84.
[13] J. Davis, and M. Shah, Visual Gesture Recognition, IEEE Proceedings - Vision, Image and Signal Processing, April, 1994, vol. 141, no. 2, pp. 101–106.
[14] P. Hong, M. Turk, and T. S. Huang, Gesture Modeling and Recognition using Finite State Machines, Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition: IEEE Computer Society, Grenoble, France, 28-30 March, 2000, pp. 410–415.
[15] A. F. Bobick, and A. D. Wilson, A State-based Approach to the Representation and Recognition of Gesture, IEEE Transactions on Pattern Analysis and Machine Intelligence, December, 1997, vol. 19, no. 12, pp. 1325–1337.
[16] M. Oshita, and T. Matsunaga, Automatic Learning of Gesture Recognition Model using SOM and SVM, Proceedings of the 6th International Conference on Advances in Visual Computin (ISVC’10) - Volume Part I , Las Vegas, Nevada, USA, September, 2010, pp. 751–759.
[17] M. Lech, and B. Kostek, Gesture-based Computer Control System Applied to the Interactive Whiteboard, Proceedings of the 2nd International Conference on Information Technology (ICIT), Gdansk, Poland, 28-30 June, 2010, pp. 75–78.
[18] T. Matsunaga, and M. Oshita, Recognition of Walking Motion Using Support Vector Machines, Proceedings of the 1st International Symposium on Information and Computer Elements (ISICE), 2007, pp. 337–342.
[19] Microsoft Research, Programming Guide: Getting Started with the Kinect for Windows SDK Beta from Microsoft Research, Microsoft Corporation, Technical Report, 2011.
[20] K. K. Biswas, and S. K. Basu, Gesture Recognition using Microsoft Kinect, Proceedings of the 5th International Conference on Automation, Robotics and Applications (ICARA), Wellington, New Zealand, 6-8 December, 2011, pp. 100–103.
[21] Z. Ren, J. Meng, J. Yuan, and Z. Zhang, Robust Hand Gesture Recognition with Kinect Sensor, Proceedings of the 19th ACM International Conference on Multimedia, Scottsdale, Arizona, USA, November-December, 2011, pp. 759–760.
[22] Y. Song, D. Demirdjian, and R. Davis, Continuous Body and Hand Gesture Recognition for Natural Human-Computer Interaction, ACM Transactions on Interactive Intelligent Systems (TiiS) – Special Issue on Affective Interaction in Natural Environments, March, 2012, vol. 2, no 1, pp.1–28.
[23] M. A. Hearst, Support Vector Machines, IEEE Intelligent Systems and their Applications – Issue: Computing & Processing (Hardware/Software), July-August, 1998, vol. 18, no. 4, pp. 18–28.
[24] A. C. Lorena, and A. C. P. L. F. Carvalho, An Introduction to Support Vector Machines (in Portuguese), Journal of Applied and Theoretical Informatics (RITA), 2007, vol. 14, no. 2, pp. 43–67.
[25] M. Behzad, K. Asghari, M. Eazi, and M. Palhang, Generalization Performance of Support Vector Machines and Neural Networks in Runoff Modeling, Expert Systems with Applications: An International Journal, May, 2009, vol. 36, no. 4, pp. 7624–7629.
[26] C. C. Chang, and C. J. Lin, LIBSVM: A Library for Support Vector Machines, 2012. Available in: http://www.csie.ntu.edu.tw/ cjlin/libsvm. Accessed in July, 2012.
[27] C. Hsu, C. C. Chang, and C. J. Lin, A Practical Guide to Support Vector Classification, Department of Computer Science National Taiwan University, Taipei 106, Taiwan, 2010. Available in: http://www.csie.ntu.edu.tw/ ̃cjlin/papers/guide/guide.pdf
[28] S. Haykin, Neural Networks: A Comprehensive Foundation, Upper Saddle River, NJ, USA, Prentice Hall PTR, Second Edition, March, 1999.
[29] K. C. Louden, Compiler Construction: Principles and Practice, Boston, MA, USA: PWS Publishing Co., First Edition, January 1997.
Downloads
Published
2013-10-01
How to Cite
Bettio, R. W. de, Silva, A. H. C., Heimfarth, T., Freire, A. P., & Sá, A. G. C. de. (2013). Model and implementation of body movement recognition using Support Vector Machines and Finite State Machines with cartesian coordinates input for gesture-based interaction. Journal of Computer Science and Technology, 13(02), p. 69–75. Retrieved from https://journal.info.unlp.edu.ar/JCST/article/view/617
Issue
Section
Original Articles
