Inspired by the ICME 2024 Grand Challenge
on Multi-Modal Video Reasoning and Analyzing Competition (MMVRAC)
AJA-Pose Model Accuracy on the Validation Set
Species for which the model is able to accurately produce poses
Built on PyPI which is accessible anytime, anywhere
Computer vision allows for non-intrusive monitoring of
animal behavior without the disruptions and potential harm
caused by traditional methods like GPS trackers.
Consequently, Animal pose estimation focuses on joint
localisation requiring a robust model architecture due to
the diverse and intricate skeletal structures of animals.
VHR-BirdPose model, initially used for birds, was trained on
the Animal Kingdom dataset, which includes 33,000 frames of
over 850 species. VHR uses HRNet and attention branch to process input images
for spatial and contextual feature extraction.
Model performance was evaluated using Percentage of Correct
Keypoints (PCK)@0.05.
The loss function used was Mean Squared Error (MSE) between
predicted and ground truth heatmaps.
Comparing our model performance trained on all animals and
evaluating the performance of birds against when trained on birds as of VHR-BirdPose alone, we see that our model performs better.
This indicates that the model generalises better given that
birds in the dataset share similar attributes with other
animals, as they are all vertebrates.
The framework simplifies distribution and installation, fostering community contributions and continuous improvement.
The model struggled with frames containing multiple animals, increasing the likelihood of errors in joint fitting.
A top down approach was shown to be useful for estimating pose in uncontrolled, cluttered images, as it fixes the scale and reduces the (x, y) search region to an enlarged area around the detection window.
We propose using object detection to dynamically localise regions of interest, reducing noise and improving robustness for better general performance.
Interested in this project? Reach out to us with your comments, queries or suggestions.
We look forward to hearing from you
