08/27/2024 update: this would be a really nice thing to try (https://sites.google.com/view/hand-eye-kalib?pli=1). If this works well, then you don’t need to do the following hand-eye calibration.

Installation and follow this guide: https://github.com/moveit/moveit_tutorials/blob/master/doc/hand_eye_calibration/hand_eye_calibration_tutorial.rst

Frames:

• We should use base instead of base_link because the red line (x-axis) should be pointed to the right, and the green line (y-axis) should be pointed away from us.

• We should use ee_link instead of tool0 because it’s more accurate.

• Click “Add” to add TF :

  • Only show base and ee_link. The rest should be toggled off.

• Click “Add” to add HandEye Calibration.

  • Set the appropriate target and context information.

• Go to Calibration tab in the HandEye Calibration window.

  • Check Target detection under HandEyeCalibration to determine if the object has been detected.

  • Move the robot arm through various poses and press “Take sample” to record them.

    • I’ve found using a bigger Aruco board and getting the board close to the camera while going through various robot poses (maximize the joint angle differences between each sample) to work well. Following these techniques, I was able to get a ~0.02m projection error in translations with only 5 samples. If I didn’t get a ~0.02m projection error with 5 samples, I would clear all samples and re-take the samples until I got a projection error close to 0.02m.

  • This is what it looks like after I’ve taken 10 samples:

    

    Note that a good calibration would result in the handeye_target being close to ee_link in the visualization.

  • Go to TF and under camera_color_optical_frame to find the transformation information (relative position and relative orientation):

    

• The entire calibration process can be found in the video below (this is the calibration result):

  https://youtu.be/KnW53IM5QGs