Combining Spatio-Temporal Appearance Descriptors and Optical Flow for Human Action Recognition in Video Data

Abstract

This paper proposes combining spatio-temporal appearance (STA) descriptors with optical flow for human action recognition. The STA descriptors are local histogram-based descriptors of space-time, suitable for building a partial representation of arbitrary spatio-temporal phenomena. Because of the possibility of iterative refinement, they are interesting in the context of online human action recognition.We investigate the use of dense optical flow as the image function of the STA descriptor for human action recognition, using two different algorithms for computing the flow: the Farnebäck algorithm and the TVL1 algorithm. We provide a detailed analysis of the influencing optical flow algorithm parameters on the produced optical flow fields. An extensive experimental validation of optical flow-based STA descriptors in human action recognition is performed on the KTH human action dataset. The encouraging experimental results suggest the potential of our approach in online human action recognition.

Files

Full Paper as PDF

BibTeX Citation

DOI

10.20532/ccvw.2013.0007

https://doi.org/10.20532/ccvw.2013.0007

BibTeX

@InProceedings{10.20532/ccvw.2013.0007,
  author =       {Karla Brki{\' c} and Sr{\dj}an Ra{\v s}i{\' c} and
                  Axel Pinz and Sini{\v s}a {\v S}egvi{\' c} and Zoran
                  Kalafati{\' c}},
  title =        {Combining Spatio-Temporal Appearance Descriptors and
                  Optical Flow for Human Action Recognition in Video
                  Data},
  booktitle =    {Proceedings of the Croatian Compter Vision Workshop,
                  Year 1},
  pages =        {9-14},
  year =         2013,
  editor =       {Lon{\v c}ari{\' c}, Sven and {\v S}egvi{\' c},
                  Sini{\v s}a},
  address =      {Zagreb},
  month =        {September},
  organization = {Center of Excellence for Computer Vision},
  publisher =    {University of Zagreb},
  abstract =     {This paper proposes combining spatio-temporal
                  appearance (STA) descriptors with optical flow for
                  human action recognition. The STA descriptors are
                  local histogram-based descriptors of space-time,
                  suitable for building a partial representation of
                  arbitrary spatio-temporal phenomena. Because of the
                  possibility of iterative refinement, they are
                  interesting in the context of online human action
                  recognition.We investigate the use of dense optical
                  flow as the image function of the STA descriptor for
                  human action recognition, using two different
                  algorithms for computing the flow: the Farnebäck
                  algorithm and the TVL1 algorithm. We provide a
                  detailed analysis of the influencing optical flow
                  algorithm parameters on the produced optical flow
                  fields. An extensive experimental validation of
                  optical flow-based STA descriptors in human action
                  recognition is performed on the KTH human action
                  dataset. The encouraging experimental results
                  suggest the potential of our approach in online
                  human action recognition.},
  doi =          {10.20532/ccvw.2013.0007},
  url =          {https://doi.org/10.20532/ccvw.2013.0007}
}