Top Skills Used
- Computer vision
- Real-time stereo vision
- Predictive algorithms
- C++ GUI application development
- Motor control
Project Overview
I built an automated beer pong opponent for my senior year capstone project (ENPH 479) as part of a four-person team. Our objective was to design and build a system that can predict the trajectory of a ball thrown toward a “solo cup” and intercept it mid-flight by shooting pellets from an airsoft gun. The basic operation is shown in the system diagram.
This project had two main parts: a stereo computer vision system built with two FLIR cameras, allowing for real-time 3D trajectory estimation, and the launcher itself, actuated by two stepper encoder motors. The main challenge of this project was the integration between the two. How do the motors know where to move to based on what the cameras are seeing? This is known as the hand-eye calibration problem in robotics, and it was one of the most fun puzzles I've ever solved.
This is was a very complex project and there is A LOT to talk about, so I encourage you to check out our landing-page GitHub and our main code repository for more (perhaps too much) detail. Nevertheless, here is a brief summary that I have done in this project:
- Created the mechanical design in CAD (OnShape) and built it. Used 3D-printed parts and off-the-shelf components from McMaster-Carr to build a very rigid design optimized for high-torque loads.
- Built a high-precision stereo computer vision system using FLIR cameras, Spinnaker SDK, and OpenCV, controlled by a fully-featured C++ Qt6 GUI application.
- Implemented a unique calibration to convert stereo coordinates to motor orientation coordinates, i.e. solved the hand-eye calibration problem for our project.
- Trained and integrated a YOLOv8 model for object detection and accelerated image processing x10 using CUDA to allow for real-time performance.
