Recognize Color Blocks

This case uses the eye-to-hand mode, uses the camera, combines Python+OpenCV, performs color positioning through OpenCV, and frames the color blocks that meet the situation, and calculates the relative position of the blocks through the relevant points. at the spatial coordinate position of the manipulator. Set up a set of related actions for the robotic arm, and place them in different areas according to the color of the identified objects.

Supported end effectors: myCobot Pro Single Head Suction Pump, myCobot Pro Adaptive Gripper，myGripper F100 force-controlled gripper.

1 Camera Adjustment

First, you need to use python to run OpenVideo.py under the aikit_V2 package. If the enabled camera is a computer camera, cap_num needs to be modified. For details, please refer to: Precautions. Make sure that the camera completely covers the entire recognition area, and the recognition area is square in the video, as shown in the figure below. If the recognition area does not meet the requirements in the video, the camera position needs to be adjusted.

M5 version:

  • Open a command prompt terminal (shortcut Win+R, enter "cmd" to access the terminal). Switch to the directory path where "aikit_V2" is located, then enter the target folder.
cd Desktop/caikit_V2/AiKit_320M5/
  • Enter the following command to open the camera for adjustment
python scripts/OpenVideo.py

Raspberry Pi version:

  • Open a console terminal (shortcut key Ctrl+Alt+T), enter the following command to enter the target folder:
cd ~/aikit_V2/AiKit_320PI/
  • Enter the following command to open the camera for adjustment
python scripts/OpenVideo.py

2 HSV color value adjustment

2.1 Why do we need to adjust the HSV color value range?

  1. Adapt to different lighting conditions:

    • Different ambient light can affect the results of color detection: sunlight, indoor lighting, and objects in the shadow may cause the color to change in the HSV space.

    • Adjusting the upper and lower limits of S (saturation) and V (brightness) can improve the adaptability of the algorithm under different light conditions.

  2. Dealing with color interference:

    • In real scenes, the background may contain areas of similar colors.

    • By adjusting the range of H (hue), the target color can be filtered more accurately to reduce misidentification.

  3. The impact of object material:

    • Different materials (such as reflective surfaces, matte surfaces) will cause the color to appear differently in the image.

    • Adjusting the HSV value range can adapt to objects of different materials.

2.2 Specific HSV adjustment strategy

  1. Adjust hue (H):

    • Determine the range of the target color:

      • Red: H = 0~8 and 170~180.

      • Blue: H = 78~110.

      • Green: H = 35~85.

    • Fine-tune the upper and lower limits according to the actual situation to ensure that the target color can be captured.

  2. Adjust saturation (S):

    • Saturation indicates the purity of the color:

      • High purity: bright colors.

      • Low purity: colors are close to gray or white.

    • Adjust the lower limit according to the ambient light intensity:

      • Sunlight: S > 100.

      • Indoor lighting: S > 50.

  3. Adjust brightness (V):

    • Brightness indicates the brightness of the color:

      • High brightness: bright colors.

      • Low brightness: darker colors.

    • Adjust the lower and upper limits to accommodate shadows or reflective areas:

      • In shadows: lower the lower limit of V.
      • In highlight areas: lower the upper limit of V to prevent detection of reflective areas.

2.3 Adjustment Example

Note: Here we take the adjustment of the red wooden block as an example. If you want to adjust other colors, you can refer to the sample script code for modification.

M5 version:

  • Open a console terminal (shortcut key Win+R, enter cmd to enter the terminal), first switch to the folder path where aikit_V2 is located, and then enter the target folder
cd Desktop/aikit_V2/AiKit_320M5/
  • Enter the following command to start the HSV color detection program.
python scripts/hsv_color_test.py

Raspberry Pi version:

  • Open a console terminal (shortcut key Ctrl+Alt+T), enter the following command to enter the target folder:
cd ~/aikit_V2/AiKit_320PI
  • Enter the following command to start the HSV color detection program.
python scripts/hsv_color_test.py

Place the red block in the camera recognition area, and you will see three screen windows:

hsv image

Among them:

  • Window 1 displays the original image captured by the camera (unprocessed).
  • Window 2 displays the color mask. The white area indicates the detected target color, and the black area indicates the filtered non-target color. It can be used to verify whether the HSV parameters are set correctly. If the target object in the mask is not recognized or the detection area is incomplete, it means that the HSV color value range needs to be adjusted.
  • Window 3 displays the image after filtering by the color mask. It is used to observe whether the target color area is correctly extracted.

3 Case Reproduction-Suction Pump Model

M5 version:

  • Open a command prompt terminal (shortcut Win+R, enter "cmd" to access the terminal). Switch to the directory path where "aikit_V2" is located, then enter the target folder.
cd Desktop/aikit_V2/AiKit_320M5/
  • Enter the following command to open the camera for adjustment
python scripts/aikit_color.py

Raspberry Pi version：

  • Open a console terminal (shortcut key Ctrl+Alt+T), enter the following command to enter the target folder:
cd ~/aikit_V2/AiKit_320PI
  • Then enter the following command to start the color recognition program.
python scripts/aikit_color.py

4 Case Reproduction-Gripper Model

M5 version:

  • Open a command prompt terminal (shortcut Win+R, enter "cmd" to access the terminal). Switch to the directory path where "aikit_V2" is located, then enter the target folder.
cd Desktop/aikit_V2/AiKit_320M5/
  • End assembly Pro adaptive gripper, can be run:
python scripts/aikit_gripper_color.py
  • End assembly myGripper F100 force-controlled gripper, can be run:
python scripts/aikit_force_gripper_color.py

Raspberry Pi version：

  • Open a console terminal (shortcut key Ctrl+Alt+T), enter the following command to enter the target folder:
cd ~/aikit_V2/AiKit_320PI
  • End assembly Pro adaptive gripper, can be run:
python scripts/aikit_gripper_color.py
  • End assembly myGripper F100 force-controlled gripper, can be run:
python scripts/aikit_force_gripper_color.py
  • When the command terminal shows ok and the camera window can be opened normally, it means that the program runs successfully. At this time, the identifiable objects can be placed in the recognition area for recognition and capture.

Gripper effect demonstration (similar to the suction pump type, but the end effector is different):

Precautions

  1. When the camera does not automatically frame the recognition area correctly, you need to close the program, adjust the position of the camera, and move the camera to the left or right.
  2. If the command terminal does not show ok, and the color cannot be recognized, you need to move the camera slightly backward or forward, and the program can run normally when the command terminal shows ok.
  3. OpenCV color recognition will be affected by the environment, and the recognition effect will be greatly reduced in a darker environment.

results matching ""

    No results matching ""