OPERATION MIMIC: Bionic Hand
by ybandarkar772 in Workshop > 3D Printing
547 Views, 8 Favorites, 0 Comments
OPERATION MIMIC: Bionic Hand
This Instructable will cover how to CAD, assemble, and code a hand that will mimic your own! Some of the software we used includes Autodesk Fusion 360, Prusa Slicer, and Visual Studio Code.
We made this as our SIDE project in our Principles of Engineering class at Irvington High School with Ms. Berbawy.
Supplies
Tools/Materials:
Parts:
3D Model
We referred to The Viral Science - The home of Creativity's video to CAD and design the fingers. We designed the pointer finger and scaled them up and down to create the other fingers.
We used our hands as a reference for the size of the palm, and created slots for the servo motors.
We SLA printed custom servo horns for our project (Image 2).
We printed the ball and socket joint in order to create the rotation of the hand. However, we glued the ball into the socket and used it as a stand for our hand.
Python Code
The Python script's job is to use the webcam to track the user's hand and find the angles of the finger and wrist. Then, it would send the information to the Arduino via Serial Communication.
First, we imported these libraries:
import cv2
import mediapipe as mp
import numpy as np
import time
import serial
from google.protobuf.json_format import MessageToDict
Then, we created some objects and variables that are necessary for using the webcam, hand tracking model, and serial port.
cap = cv2.VideoCapture(0) //creates the cv2 object (uses my main webcam)
mpHands = mp.solutions.hands
hands = mpHands.Hands(max_num_hands=1, min_detection_confidence=0.5) //only allows one hand to be
mpDraw = mp.solutions.drawing_utils //detected at a time
mySerial = serial.Serial(port='COM5') //creates the serial object
We then created functions to generalize the positions of the fingers (because the tracking model is not always entirely accurate)
def findquadrant(angle):
if angle >=0 and angle<= 90:
return 0
if angle >=91 and angle<= 180:
return 2
def findthumbquadrant(angle):
if angle >=0 and angle<= 155:
return 0
if angle >=156 and angle<= 180:
return 2
def findwristangle(angle):
if len(str(angle)) == 1:
angle = '00' + str(angle)
return(angle)
elif len(str(angle)) == 2:
angle = '0' + str(angle)
return(angle)
else:
return(angle)
next, we created a while loop, in which most of the code will run:
while True:
success, img = cap.read()
img = cv2.flip(img, 1)
imgRGB = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
results = hands.process(imgRGB)
if results.multi_hand_landmarks:
for i in results.multi_handedness:
# Return whether it is Right or Left Hand
label = MessageToDict(i)[
'classification'][0]['label']
if label == 'Right':
for handLMS in results.multi_hand_landmarks: //draws the landmarks on the users hand
mpDraw.draw_landmarks(img, handLMS, mpHands.HAND_CONNECTIONS)
fingerpositions = "$"
inside the loop, we calculated the angles of the fingers
# calculating angle for index finger
indexA = np.array([float(wristLMK.x), float(wristLMK.y)]) # First coord
indexB = np.array([float(bottomIndexLMK.x), float(bottomIndexLMK.y)]) # Second coord
indexC = np.array([float(topIndexLMK.x), float(topIndexLMK.y)]) # Third coord
indexRadians = np.arctan2(indexC[1] - indexB[1], indexC[0] - indexB[0]) - np.arctan2(indexA[1] - indexB[1], indexA[0] - indexB[0])
indexAngle = float(np.abs(indexRadians * 180.0 / np.pi))
if indexAngle > 180:
indexAngle = 360 - indexAngle
#print("index finger angle:", int(indexAngle))
fingerpositions = fingerpositions + str(findquadrant(int(indexAngle)))
else:
#print("index finger angle:", int(indexAngle))
fingerpositions = fingerpositions + str(findquadrant(int(indexAngle)))
this logic then repeats for all the fingers and the wrist.
Lastly, we sent all data to the Arduino:
print(str(fingerpositions))
mySerial.write(str(fingerpositions).encode())
time.sleep(0.5) //delay the code running to save memory
//as the arduino processes the information
cv2.imshow("Image", img) //display the camera feed
cv2.waitKey(1)
Arduino Code
The Arduino code's job is to receive the data from the python and control the servos accordingly.
First, we imported these libraries.
#include <Servo.h>
#include <LiquidCrystal.h>
#include <Chrono.h>
- Serial is used to communicate with the Python script.
- Liquid Crystal is used to control the LCD display(we used an LCD to verify that the Arduino was receiving data correctly from the serial port.
- Chrono is used to calculate the time between starting a servo and when it should stop. This allowed us to control the servos' time spinning without delaying the entire program.
Next, we initialized the servo objects.
Servo servoThumb;
Servo servoIndex;
Servo servoMiddle;
Servo servoRing;
Servo servoPinky;
Servo servoWrist;
I named them according to what they will control, using camel casing.
Next, we needed some way to receive the serial data, so we created some variables that will we will be using to do so.
Huge shoutout to @Murtaza'sWorkshop on YouTube for helping us out with the serial communication.
#define numOfValsRec 8
#define digitsPerValRec 1
int valsRec[numOfValsRec];
int stringLength = numOfValsRec * digitsPerValRec + 1; //$00000000
int counter = 0;
bool counterStart = false;
String receivedString;
int prevVals[numOfValsRec] = {2,2,2,2,2,0,0,0};
This defines how many digits we need to receive, and how many digits there are in each value.
All the fingers will get one digit; either a 2(open finger), or a 0(closed finger).
The last three digits are for the wrist servo.
We also created a counter so that we know how many digits have been received. When a digit is added to the received string, the counter starts and adds one. This repeats until the counter reaches 8, at which point it knows that we have received all the characters we need.
Then, we created a function called receiveData()
void receiveData() {
while(Serial.available())
{
char c = Serial.read();
if (c=='$') { //only starts receiving characters if the first character is '$'
counterStart = true; //this makes sure that the script is reading the right characters first
}
if (counterStart = true) { //adds a new digit from the serial buffer to the string
if (counter < stringLength) { //as long as the counter is on and the string is less
receivedString = String(receivedString+c); //than 8
counter++; //counter increases for every digit added to the string
}
if (counter>= stringLength){ //adds received characters into an array
int num = (i*digitsPerValRec)+1;
valsRec[i] = receivedString.substring(num,num+digitsPerValRec).toInt();
}
valsRec[5] = (valsRec[5] *100) + (valsRec[6]*10) + (valsRec[7]); //combines the three digits of t
//he wrist angle into one value
receivedString = "";
counter = 0; //resets the string and counter
counterStart = false;
}
}
}
}
Then we could just call the function in void loop() to use the values to control the servos:
void loop() {
// put your main code here, to run repeatedly:
receiveData();
This is the logic we wrote to control the index finger:
if (valsRec[0] - prevVals[0] > 0) //runs the following block if the difference between the new
{ //value and the last value is greater than 0
servoIndex.attach(8);
servoIndex.write(180); //servo starts spinning counter clockwise to open the finger
startIndex.restart(); //starts the timer
indexShouldSpinFor = (indexOpenSpinTime*abs(valsRec[0] - prevVals[0])); //calculates how long the //servo has to spin
prevVals[0] = valsRec[0]; //sets the last value as the new value
}
if (valsRec[0] - prevVals[0] < 0) //runs the following block if the finger is supposed to close
{
servoIndex.attach(8);
servoIndex.write(0); //servo starts spinning clockwise to close the finger
startIndex.restart();
indexShouldSpinFor = (indexSpinTime*abs(valsRec[0] - prevVals[0]));
prevVals[0] = valsRec[0];
}
And then to close the fingers:
while (servoIndex.attached() | servoMiddle.attached() | servoRing.attached() | servoPinky.attached() | servoThumb.attached() ) //runs the code only if any of the servos are spinning
{
if (startIndex.hasPassed(indexShouldSpinFor)) //runs the block if the servo has spun for its set
{ //time or longer
servoIndex.write(90); //sets the servo to stop
servoIndex.detach();
}
}
The reason why we detach the servos after every movement is because the servos would sometimes become uncalibrated and start spinning when they were not supposed to.
Then we just copied that logic for all the other fingers
And lastly, we set the wrist in motion with:
servoWrist.write(valsRec[5]);
Assemble
- Loop the elastic bands through the 3d printed fingers and connect them to the palm by tying it from the bottom.
- Attach the 3d printed servo horns to the servo motor, and screw the servo motors into its slots.
- Attach fishing line to the servo horns of all the fingers (Tie the fishing line through the hole on the servo horn).
- Attach the other end of the fishing line by putting them through the holes in the finger and tying them through the elastic band at the top of the fingers (Image 2). Make sure the strings are on tight. Adjust the servos if needed.
- Note: You will have to attach the fishing line to the thumb after you put on the cover which will be done at the end.
- Attach the ball and socket wrist to the bottom of the hand and super glue it on.
- Laser cut a box 8in by 4 in by 4in (Image 3).
- Note: When gluing the box together, don't glue the top and side on until the end.
- Put the mg995 servo motor into its 3d printed case and glue it to the bottom of the wooden box and wire it to the Arduino. Attach the 6-arm servo horn to the servo motor.
- Attach the bottom of the arm to the mg995 servo motor
- Wire the Arduino as shown in the picture above and connect them to the servo motors through the rectangular hole from the top (Image 1). In image 6, we connected the wires before putting on the lid and we had to rewire it after putting the lid on.
- Attach the fishing line to the thumb and screw on the cover (Image 4).
- Connect the Arduino to the laptop and use the hand tracker.
- ENJOY!!!
Test It Out!
Have fun messing around with your new robotic hand!