Interactive Drawing with Hand Gestures: A Fun Project Using OpenCV and Mediapipe

Have you ever wanted to bring your drawing to life with just the movement of your hands? In this project, we're turning it into reality using the magic of OpenCV and Mediapipe.

Key Takeaways:

• Turn your hand gestures into a digital paintbrush, creating art on the screen in real-time

• Explore the power of OpenCV and Mediapipe for computer vision and hand tracking, making interactive drawing possible.

• Customize your brush color with a convenient color selector

• Effortlessly erase elements by closing your hand

Hand Gestures + Paint

Let's start by turning your hand gestures into a digital paintbrush. Take a look at this code snippet:

# Set up video capture
cap = cv2.VideoCapture(0)
cv2.namedWindow("Drawing App", cv2.WINDOW_NORMAL)  # Make the window resizable


mpHands = mp.solutions.hands
hands = mpHands.Hands(static_image_mode=False,max_num_hands=2,
        min_detection_confidence=0.5, min_tracking_confidence=0.5)
mpDraw = mp.solutions.drawing_utils

We set up the video capture and created a hand-tracking instance using Mediapipe. This allows us to directly detect and track hand movements in real-time.

OpenCV + Mediapipe

We leverage OpenCV and Mediapipe for computer vision and handtracking respectively. The next snippet demonstrates the hand-tracking process:

# Process each frame for hand landmarks
while True:
    success, img = cap.read()
    imgRGB = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    results = hands.process(imgRGB)

    if results.multi_hand_landmarks:
        for handLms in results.multi_hand_landmarks:
            # Extract hand landmarks and perform actions
            # ...

    # Display the result
    cv2.imshow("Interactive Drawing", img)
    cv2.waitKey(1)

We continuously process each frame, extracting hand landmarks and allowing us to perform actions based on the movements of the hands.

Brush + Colour + Finger

You can let your creativity run loose on the virtual paintbrush by changing colors.

# Create trackbars for color selection
cv2.namedWindow('Color Selector')
cv2.createTrackbar('Blue', 'Color Selector', 0, 255, on_color_change)
cv2.createTrackbar('Green', 'Color Selector', 0, 255, on_color_change)
cv2.createTrackbar('Red', 'Color Selector', 0, 255, on_color_change)

This snippet sets up trackbars for adjusting the blue, green, and red components of the color, providing a dynamic color selection experience.

Closed Hand = Erase

Make erasing as easy as closing your hand. Check out this code snippet:

# Erase elements when hand is closed
if is_hand_closed:
    erase_elements()

By detecting when the hand is closed, we trigger the erasing function, creating an intuitive and seamless experience

What's next?

In the next part of our blog series, we'll use this code to make a Pong Game where the platform will literally be in the power of your hands.

Check Out the Project on GitHub

Ready to dive deeper into the code and contribute to the project? The complete source code, along with detailed documentation, is available on my GitHub repository. Feel free to explore, star the project, and contribute to the ever-growing creative community.

Stay connected

Follow me on GitHub for updates, and join the community discussions. Share your creations, feedback, and ideas. Let's continue to build and innovate together!

Your curiosity and insights are valuable! If you have questions, suggestions, or spot anything that could use improvement, consider yourself my VIP invitee. This project is a canvas for creativity, and your input adds the brushstrokes that make it unique.

Feel free to open an issue on GitHub, ask questions, or contribute your ideas. Together, we can make this project even more vibrant and enjoyable for everyone.