Privacy Mitigating Algorithms for Medical Action Recognition

Abstract

Video-based human action recognition has become a crucial component in recent years for many applications, such as human machine interaction, video surveillance and healthcarerelated systems. The primary task for human action recognition is to analyse the human behavior based on the given action data. However, different from the general action recognition, medical action recognition is more challenging due to the data limitation, privacy protection and noisy annotations issues. In this thesis, in order to improve the medical action recognition performance and the robustness of system by addressing the aforementioned issue, a variety of enhanced approaches are proposed. The first contribution aims to focus on human multiple fall events classification using a deep neural network framework by reducing the redundant information and presenting a two-stage framework. The proposed redundant reducing theory is developed to remove the unimportant part, including the redundant empty frames from the video and the redundant body parts from the processed privacy-mitigated human skeleton data. In addition, the proposed two-stage framework is designed for addressing the imbalanced data issue from the data limitation. To improve the classification performance, the gating parameter is utilized along with the proposed structure. The second contribution relates to address the noisy annotation issue for multiple fall events classification, since the quality of the annotations plays a key role in the data-driven methods. The proposed noisy annotation managing system includes two parts: cascaded noisy annotation purification and noisy annotation learning framework, which is called JoCoT. The purification theory is based on the principle of the joint distribution probability density function to identify and prune the incorrect annotations. JoCoT is proposed for fully exploiting the potential of the noisy instances with a trinity network. The small loss theory is utilized for selecting the clean instances. Moreover, both the co-regularization and contrastive learning with joint loss function are applied for enhancing the performance. The third contribution focuses on extracting the novel direction-level features by the proposed signal image generation (SIG) to further protect the privacy information, which could assist the position-level feature to improve the performance by investigating their complementary benefits in different stages for medical action recognition. A one-shot learning framework is developed to address the medical data limitation issue, and together with the cross-attention mechanism (CsA) is used to reduce the misclassification bias for the similar medical action issue. Moreover, dynamic time warping (DTW) module is proposed to minimize the temporal mismatching issue between the instances from the same category, thereby improving the performance. The proposed contributions are evaluated on the UP-Fall, NTU RGB+D 60, NTU RGB+D 120 and PKU-MMD benchmark datasets, which are widely used for medical action recognition. Detailed evaluations on the benchmarks, along with the comparisons with the recent state-of-the-art methods, confirm the effectiveness of the proposed approaches on medical action recognition.