Dog behavior recognition and tracking based on faster R-CNN

Abstract

Recently, detection and recognition of animal faces, body postures, behaviors, and physical movements is became an interdisciplinary field. Computer vision methods can contribute to determine behaviors of animals and predict the following behavior of animals. Moreover, these methods would contribute to domesticate animals. In this study, a deep learning based system is proposed for the detection and classification of dog's behaviour. In the study, firstly, a dataset is created by collecting videos containing the behavior of dogs which don't avoid contact with people. After the necessary analysis on the obtained videos, a customized data set consisting of more meaningful sections is developed by extracting determined behaviors in videos. It is recognized the behavior with the Faster R-CNN (Faster Regional-Convolutional Neural Networks) by selecting key frames from these customized video sections. In the last stage, the related behaviors in videos are followed by video tracker after the behavior of the dog is recognized. As a result of experimental studies, the behaviors of dog such as opening the mouth, sticking out the tongue, sniffing, rearing the ear, swinging the tail and playing are examined and accuracy rates 94.00%, 98.00%, 99.33%, 99.33%, 98.00% and 98.67% are obtained for these behaviors, respectively. With the results obtained in the study, it is seen that our proposed method based on key frame selection and determination of regions of interest are successful in recognition the behavior of dogs.