Transfer Learning for Pickleball Ball Tracking Using TrackNetV2

Introduction

Pickleball is rapidly growing in the United States, combining elements of badminton, table tennis, and tennis into an accessible game. With increased popularity comes interest in sports analytics, particularly ball tracking. Tennis uses Hawk-Eye technology, but its cost and proprietary nature make it impractical for pickleball.

TrackNetV2, designed for tracking shuttlecocks in badminton, can handle high-speed objects with complex trajectories, making it a strong candidate for pickleball tracking. My project focused on adapting TrackNetV2 to track the pickleball using transfer learning. This solution is cost-effective, requiring only one camera compared to Hawk-Eye's multiple cameras, and could significantly advance analytics for pickleball games.

TrackNetV2 Model Architecture

Approach

To adapt TrackNetV2, I created a dataset using footage from a PPA Tour match between Ben Johns and Quong Duong. After trimming the footage to include only active play segments, I manually labeled each frame to mark the pickleball's coordinates and bounce status.

Heat MapAll Data PointsAll Bounces

Heat Map Representation of All Data Points

The data underwent several preprocessing steps, including frame extraction, label modification, and batch splitting. Using Google Colab Pro's GPU, I trained the model with adjusted parameters, such as learning rate and training epochs. Given the limited scope of the project, the results were promising.

Performance and Findings

The modified TrackNetV2 model was validated using metrics like accuracy, true positive rate (TPR), and precision:

These metrics suggest the model can reliably detect the pickleball, but high values also indicate potential overfitting due to limited dataset diversity. During visual inspections, the model performed well in direct views but struggled with angled shots and occlusions. (See the video below for an in/out detection demo)

Lessons Learned and Challenges

Overfitting was a key challenge, with the model performing well on training data but less effectively on new data. It also struggled with occlusions and angle variations. Future iterations should use more diverse training videos and preprocessing steps like color separation to improve performance.

Key Takeaways

This project demonstrates that transfer learning can effectively adapt a shuttlecock tracking model for pickleball. While promising, the model's real-world usability depends on reducing overfitting through more diverse data and improved preprocessing techniques.

Future improvements could include predicting shot types and ball paths, enhancing pickleball broadcasts and player analytics. Ultimately, models like this could add depth for viewers and provide players with valuable insights.

Acknowledgment

A portion of the code used in training my model was sourced from the following GitHub repository: TrackNet-Pickleball by Andrew Dettor.

For questions or further information, please feel free to contact me at [email protected].