Solving the Human Activity Recognition Problem in Sports

Wyatt Marvil
Thomas Jefferson High School for Science and Technology

This article was included in the 2019-2020 Teknos Science Journal

Legs pumping, heart racing, sweat streaming down your neck, you can only focus on one thing as adrenaline takes over and pushes you across the finish line. To perform at an elite level, athletes need to seize advantages wherever possible, be that in action on the field or in training sessions off the court. Thanks to the rise of assistive technology over the past decades, the training aid industry is constantly growing. From fitness trackers to health monitoring systems, it is all about identifying an athlete’s weakness and providing a statistically observable track to increase performance. 

The human activity recognition problem has been of growing interest in the computer science industry over the last decade, with an expanding number of practical applications in the real world, including surveillance, intelligent prosthetic limbs, monitoring of the elderly and disabled, and many others. Advances in both hardware technology and machine learning techniques make these possible, and create an exponentially expanding frontier for research and development of new technological solutions to our everyday lives. Along with my research team in the Computer Systems Research Lab, I am developing a data-tracking lacrosse stick, equipped with an array of gyroscope accelerometer sensors. The sensors collect acceleration and rotation data from several locations along the shaft and relay that information to the LaxBit mobile application, where an LSTM neural network classifies the movement data. The research into employing human activity recognition in sports is unprecedented and will prove to be valuable as the applications of artificial intelligence continue to grow within the realm of sports.

On a basic level, human activity recognition boils down to collecting raw data, which represents the nuances and complexities of human movement, and then sending that data through a set of machine learning algorithms to create a mathematical representation of human activity. This mathematical representation can then be utilized in recognizing the movement, typically by classifying it based on a subset of previously defined categories. For example, an algorithm trained on data collected from a group of track and field athletes might be equipped to identify walking, running, or jumping from future activity data. 

There are several ways to perform human activity recognition, including but not limited to movement data, observational data, and electromyography. My team focused on the use of movement data based on acceleration and rotation along the axis of a lacrosse shaft. Some applications such as those of the surveillance industry are better suited for observation data, while others may benefit from electromyography, a technique for recording electrical activity in the skeletal muscles [1].

When asked about the benefits of electromyography, Professor Manfredo Atzori, a senior researcher at the University of Applied Sciences Western Switzerland, says, “Electromyography data describe[s] quite well the inner activity of the muscles, thus it has quite a long history in being used for research in prosthetics” (personal communication, Jan. 15, 2020). He goes on to explain that classifiers have trouble generalizing acceleration data from movements. Professor Atzori also mentions extensions to his research, listing multimodal data fusion for hand-eye coordination, 3D printed prosthesis development, and muscular synergies as the most promising advancements in recent publications (personal communication, Jan. 15, 2020).

Moving on from the collection of human activity data, we must discuss the methods for processing the massive amount of accumulated information. A common approach is to utilize a concurrent neural network. Concurrent neural networks were originally applied to image data to recognize differences in pixel data, but have since been well generalized to distinguish patterns in other types of data, such as acceleration or rotation [2]. In addition to concurrent neural networks, many approaches use the k-Nearest Neighbors algorithm. This algorithm categorizes the data locally on a scatter plot and uses clusters of neighboring values to draw polydimensional decision boundaries among the dataset, which can be used to classify new data as belonging to a certain cluster. These clusters are then interpreted as classifications [3]. 

Another technique for sharpening the effectiveness of a machine learning algorithm is principal component analysis. To remove an axis of complexity in a dataset, developers will perform a principal component analysis to identify features in the dataset with little bearing on the outcome of the data. By eliminating these unnecessary features, we can reduce a common machine learning error called overfitting, which occurs when the data model is too specific to the training data and fails to properly generalize to other data [5].

As for classifications of the data itself, many studies would primarily identify a chunk of data as either static or nonstatic. From there, the static data could be broken down into other categories, such as walking versus running, while nonstatic data could be identified as either sitting or laying down. This approach greatly simplifies the work required by a single algorithm, perhaps analogous to a binary tree structure [2]. 

A final note for the collection of data pertains to the placement of sensors and choice of machine learning techniques. In previous studies, it was found that higher numbers of sensor modules had little positive effect on model accuracy, especially in areas such as walking or running [4]. However, my team chose to utilize three sensors in our shaft to account for the rapid nature of movements in lacrosse, which would be more easily identifiable with three times as much data recorded. 

Ultimately, advancements in machine learning and hardware technology are paving the way to a brighter future for sports enhancement. Tackling huge technological challenges like the human activity recognition problem is an inevitable requirement to continue furthering the assistive technology and training aid industries. With the completion of our LaxBit data tracking device, my team and I contribute our research to the human activity recognition field and plan to expand it to other sports and athletic activities.

References

[1] Atzori, M., Cognolato, M., & Muller, H. (2016). Deep learning with convolutional neural networks applied to electromyography data: A resource for the classification of movements for prosthetic hands. Neurorobotics.  https://doi.org/10.3389/fnbot.2016.00009 

[2] Brownlee, J. (2018, September). Deep learning models for human activity recognition [Blog post]. Retrieved from Machine Learning Mastery website: https://machinelearningmastery.com/ deep-learning-models-for-human-activity-recognition/ 

[3] Chin, Z., Ng, H., Yap, T., Tong, H., Ho, C., & Goh, V. (2018). Daily activities classification on human motion primitives detection dataset. Computational Science and Technology, 481, 117-125. https://doi.org/10.1007/978-981-13-2622-6_12 

[4] Sheng, B., Moosman, O., Pozo-Cruz, B., Pozo-Cruz, J., Alfonso-Rosa, R., & Zhang, Y. (2020). A comparison of different machine learning algorithms, types and placements of activity monitors for physical activity classification.  Measurement, 154. https://doi.org/10.1016/j.measurement.2020.107480 

[5] Wang, K., Shi, X., Pei, A., Goh, X., & Qian, S. (2019). A machine learning based study on pedestrian movement dynamics under emergency evacuation. Fire Safety Journal, 106, 163-176. https://doi.org/10.1016/j.firesaf.2019.04.008