Challenges

Perception

• Ball detection was unreliable with either HSV or YOLO at first. Iterated between both before settling on HSV as default.
• Recovering the camera to robot transform required an ArUco tag on the base as a localization anchor.
• Shifted away from trajectory prediction entirely, replacing it with visual servoing for more robust real-time tracking.
• Realsense RGB was maximum at 30 HZ which was the beginning of our pipeline which would influence the speed of our later pipeline. This made it hard to get good results with the faster-moving ball.

Motion Planning

• Went through strike planner, rhythmic bumping, horizontal paddle, and balloon-bounce mode before finding something that worked.
• Dropped full vertical tracking late in the project. Final strategy: hold position and pop when ball is roughly 6 inches above the paddle.

Inverse Kinematics & Arm Control

• Had to find appropriate PID tunings to allow the arm to move within a safe range, avoiding collisions or joint constraitns
• Tuning tool offset was tricky to get the balloon to fall not on the center of the paddle, but farther, to avoid the end effector
• Needed heavy PID K-value tuning to get a smooth and consistent bounce.