How to Track Your Robot With OpenCV

UPDATE: Lets Make Robots, my home digital hackerspace, was purchased by RobotShop. I don't want to go into, but the new owners banned myself and most of the veteran members. That stated, most of the links there will be broken. Luckily, I was able to make a copy of my content (the reason they banned me) and moved it to a Github site--it was a hurried move, so the content maybe a little rough right now.

UPDATE: I have tweaked the code in these instructions into a Python module. Muuuch easier to use.

Friendly Overlord


These are redneck instructions on how to control a robot with a static webcam for under 50 USD.

I'm a robot builder and I got tired of seeing universities play robot soccer or something with computer vision guiding their players, and no matter how much I begged, darn Ivy Leagues wouldn't share.

So, I wrote my own. And while I did it, I swore I'd try to help anyone trying something similar.

1. Webcam sends images of its field-of-view.
2. OpenCV looks for the largest red blob.
3. It begins tracking the red blob's X, Y.
4. The PC averages these X, Y positions for around 150 camera frames.
5. If the blob hasn't moved much, the PC assumes the red blob is the robot.
6. The PC gets frisky and gives our robot a random target within the webcam's field-of-view.
7. The PC calculates the angle between the bot and the target.
8. Meanwhile, the robot's microcontroller is taking readings from a magnetometer on the robot.
9. The robot, with a one time human calibration, translates true North to "video-game north," aka, top of PC's screen.
10. The microcontroller transmits this code to the PC.
11. The PC compares the angle of the bot from the target with the robots angle.
12. The PC sends a code to the bot telling it to turn left, right, or move forward (closer to the target).
13. When the robot has made it within an acceptable distance from the target he "Munches the Dot."
14. A new random dot appears. Rinse repeat. (For real though, don't rinse the bot. Consider Asimov's Third Law.)

I'm a homeless outreach worker. The job's amazing. But I'll say, emotionally taxing. Skipping the politics and the sermon on harm-reduction, I decided at the start I needed something far from the job to allow my mind rest and prevent compassion fatigue. Something that consumed my brain-power so I'd not be stressing over the 6 months pregnant 17 year-old, shooting up under a bridge on I-35. Something to protect my down-time so I'd be frosty for the next day.

Well, I saw that TED talk about the Sand Flea and I told Bek, "That's awesome, think I could build one?"
"Oh crap," she said, "new obsession?"

Now, robots are my relief. My way to prevent white-matter from becoming dark-matter as I rake through sludge looking for those who want out.

I started reading a lot. I discovered, Arduino, Sparkfun, eBay, Raspberry Pi, ferric chloride, Python, hackaday, HC-SR04, Eagle, OSHPark, and the list goes on. But every time I Googled something about robots, I'd end up at the same place.

Lets Make Robots (edit: sadly, RobotShop bought them and banned the member who helped me the most).

These guys are brilliant. They are a college education from collaboration, I swear.

Soon, I ended up with my first bot. A piece of sh...short-circuits. Although, I did learn a lot interfacing the bot with the Raspberry Pi. Also, while I was working with a Raspberry Pi, I played with OpenCV, and was considering adding a face tracker to my bot before I got distracted. But before I quit, I created a proof-of-concept.

So, all these experiences began to culminate.

Meanwhile, I was taking a graduate Research Methods class at UTA and my professor disappeared. The university was amazing; good professors filled in and made sure our education didn't suffer. But we wondered for many months. Sadly, it was discovered he had killed himself.

It shook me. I deal with suicidality every other day, but it's usually on the street. Why a successful research professor? My thoughts got dark for a bit, which meant I sunk into robots even more. Yet, now, a question sat at the front of my mind: Will robots one day kill themselves?



This may sound silly. But I believe the formula for self-termination can be expressed in Boolean logic, and therefore coded.

Pseudo-code would be:

if painOfExistence > senseOfPurpose then:

self_terminate()

Derived from work and life experience I genuinely believe the root-motive for suicide is existential-anxiety, which seems to me, entangled within both constructs.

Ok. Skipping the Time bit.

Someday, I'd like to delve into swarm robotics. Or, at least, attempt to replicate organic group behavior within a robot group. And I thought it might be possible to control a group of robots with a setup similar to those universities or research groups keep showing off. (Jockish Ivy Leagues :P)

Well, I found these desires, information, and tools synergized into a passion. After two days, I was able to write a basic OpenCV Python script that could control a robot using a static webcam looking down on it. Let me clarify, I'm of average intelligence, simply obsessive, so when I mentioned "two-days" I'm trying to convey the utter feasibility of this project, for anyone. Python, Arduino, and OpenCV make it so very easy; anyidiot like me can hack it out.

Of course, my purpose for this platform is to control robot groups. The group being the second social collection (one-to-eight) and social interaction seems to be the essential in developing a positronic brain. The white-mattered brained being necessary for me to test the above mentioned self-termination formula. So, maybe, I'll learn if robots will commit suicide, or perhaps, have a better understanding of why humans do.

Dark and depressing! I know, right? Who writes this crap!?

A robot

It doesn't matter what sort of robot you use, it only needs:

1. A microncontroller (e.g., Arduino, PicAxe, etc.)
2. Built from material of a bold, solid color.
3. The ability to change directions and move.
4. A magnetometer. I used the HMC5883L. They're like 2 USD on eBay.
5. A wireless serial connection. Bluetooth, Xbee, and nRF24L01 would be my recommendation since all are well documented creating a bridge between PC and microcontroller.

I personally built my own using red cutting-board I stole from Bek (shh). For my serial connection I used two $10 Bluetooth 4.0 modules, which I've written an instructable on setting up a Bluetooth 4.0 module to work with an Arduino and PC: Bluetooth 4.0 and Arduino.

A PC

Probably something less than 10 years old. It could be running Linux or Windows;though, I'll be using Windows Vista (hey, I'm first-world poor and can't afford Windows 7 :P).

1. The PC will need to be running Python 2.7
2. It'll need OpenCV 2.4.4
3. It will need a wireless serial connection that pairs with your bot. Again, I used my BT 4.0 modules.

A Webcam

It's really up to you. I'm not going to lie, I went with the cheapest webcam I saw, which costs 6.87 USD. But I would not recommend this webcam. It didn't like my PC, so every time my Python script stopped I had to unplug the webcam and re-plug it in. A real annoyance for debugging.

1. I'd suggest a high-resolution webcam. Maybe even a IP cam, if you're rich? If you are, would you buy me one too?
2. Long male-to-female USB cable. Again, I got two 15' USB cables on eBay for around 4.50 USD. If you get everything setup and you notice problems with the webcam at the end of the cable, you can put a powered hub at the end of the cable with an extension cord and it'll take care of the issue. Though, I didn't have this problem at 15'.
3. A wife that'll let you screw your webcam into the ceiling. Or...don't ask...

So, I made my robot, Dot Muncher, using an Arduino Uno, Motor Shield, and a Bluetooth 4.0 module. The chassis was made from HDPE, a cutting board I stole from my wife. The motors and tires were from eBay.

Now, about any robot will work, like I've stated, so Google away and select a robot build you like.

Of course, everything you'd every want to know can be found at

www.letsmakerobots.com

I'm just sayin'.

But the code, that's the part we want to focus on. Really, our robot only has a nerves and muscles, the brain will actually be in the PC, all the robot does is,

1. Calculates the compass info.
2. Sends the compass info to the PC.
3. Reads the movement codes from the PC.
4. Translates the movement code received into a motor activation.

That's it. Pretty simple.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210

<span style="color: rgb(136,136,136);">//I've been using Zombie_3_6_RC in Processing to interact.</span>


<span style="color: rgb(136,136,136);">// Reference the I2C Library</span>
<span style="color: rgb(85,119,153);">#include </span><span style="color: rgb(85,119,153);"><Wire.h></span>
<span style="color: rgb(136,136,136);">// Reference the HMC5883L Compass Library</span>
<span style="color: rgb(85,119,153);">#include </span><span style="color: rgb(85,119,153);"><HMC5883L.h></span>

<span style="color: rgb(136,136,136);">// Store our compass as a variable.</span>
HMC5883L compass;

<span style="color: rgb(136,136,136);">// Record any errors that may occur in the compass.</span>
<span style="color: rgb(51,51,153);font-weight: bold;">int</span> error <span style="color: rgb(51,51,51);">=</span> <span style="color: rgb(0,0,221);font-weight: bold;">0</span>;

<span style="color: rgb(136,136,136);">//int pwm_a = 10; //PWM control for motor outputs 1 and 2 is on digital pin 10</span>
<span style="color: rgb(51,51,153);font-weight: bold;">int</span> pwm_a <span style="color: rgb(51,51,51);">=</span> <span style="color: rgb(0,0,221);font-weight: bold;">3</span>;  <span style="color: rgb(136,136,136);">//PWM control for motor outputs 1 and 2 is on digital pin 3</span>
<span style="color: rgb(51,51,153);font-weight: bold;">int</span> pwm_b <span style="color: rgb(51,51,51);">=</span> <span style="color: rgb(0,0,221);font-weight: bold;">11</span>;  <span style="color: rgb(136,136,136);">//PWM control for motor outputs 3 and 4 is on digital pin 11</span>
<span style="color: rgb(51,51,153);font-weight: bold;">int</span> dir_a <span style="color: rgb(51,51,51);">=</span> <span style="color: rgb(0,0,221);font-weight: bold;">12</span>;  <span style="color: rgb(136,136,136);">//dir control for motor outputs 1 and 2 is on digital pin 12</span>
<span style="color: rgb(51,51,153);font-weight: bold;">int</span> dir_b <span style="color: rgb(51,51,51);">=</span> <span style="color: rgb(0,0,221);font-weight: bold;">13</span>;  <span style="color: rgb(136,136,136);">//dir control for motor outputs 3 and 4 is on digital pin 13</span>

<span style="color: rgb(51,51,153);font-weight: bold;">int</span> lowspeed <span style="color: rgb(51,51,51);">=</span> <span style="color: rgb(0,0,221);font-weight: bold;">120</span>;
<span style="color: rgb(51,51,153);font-weight: bold;">int</span> highspeed <span style="color: rgb(51,51,51);">=</span> <span style="color: rgb(0,0,221);font-weight: bold;">140</span>;

<span style="color: rgb(136,136,136);">//Distance away</span>
<span style="color: rgb(51,51,153);font-weight: bold;">int</span> distance;

<span style="color: rgb(136,136,136);">//Sets the duration each keystroke captures the motors.</span>
<span style="color: rgb(51,51,153);font-weight: bold;">int</span> keyDuration <span style="color: rgb(51,51,51);">=</span> <span style="color: rgb(0,0,221);font-weight: bold;">10</span>;

<span style="color: rgb(51,51,153);font-weight: bold;">int</span> iComp;

<span style="color: rgb(51,51,153);font-weight: bold;">void</span> <span style="color: rgb(0,102,187);font-weight: bold;">setup</span>()
{
  Serial.begin(<span style="color: rgb(0,0,221);font-weight: bold;">9600</span>);
  
  Wire.begin(); <span style="color: rgb(136,136,136);">// Start the I2C interface.</span>

  Serial.println(<span style="background-color: rgb(255,240,240);">"Constructing new HMC5883L"</span>);
  compass <span style="color: rgb(51,51,51);">=</span> HMC5883L(); <span style="color: rgb(136,136,136);">// Construct a new HMC5883 compass.</span>
    
  Serial.println(<span style="background-color: rgb(255,240,240);">"Setting scale to +/- 1.3 Ga"</span>);
  error <span style="color: rgb(51,51,51);">=</span> compass.SetScale(<span style="color: rgb(102,0,238);font-weight: bold;">1.3</span>); <span style="color: rgb(136,136,136);">// Set the scale of the compass</span>
  error <span style="color: rgb(51,51,51);">=</span> compass.SetMeasurementMode(Measurement_Continuous); <span style="color: rgb(136,136,136);">// Set the measurement mode to Continuous</span>
  
  pinMode(pwm_a, OUTPUT);  <span style="color: rgb(136,136,136);">//Set control pins to be outputs</span>
  pinMode(pwm_b, OUTPUT);
  pinMode(dir_a, OUTPUT);
  pinMode(dir_b, OUTPUT);
  
  analogWrite(pwm_a, <span style="color: rgb(0,0,221);font-weight: bold;">0</span>);        
  <span style="color: rgb(136,136,136);">//set both motors to run at (100/255 = 39)% duty cycle (slow)  </span>
  analogWrite(pwm_b, <span style="color: rgb(0,0,221);font-weight: bold;">0</span>);

  pinMode (<span style="color: rgb(0,0,221);font-weight: bold;">2</span>,OUTPUT);<span style="color: rgb(136,136,136);">//attach pin 2 to vcc</span>
  pinMode (<span style="color: rgb(0,0,221);font-weight: bold;">5</span>,OUTPUT);<span style="color: rgb(136,136,136);">//attach pin 5 to GND</span>
  <span style="color: rgb(136,136,136);">// initialize serial communication:</span>
  Serial.begin(<span style="color: rgb(0,0,221);font-weight: bold;">9600</span>);
  
}

<span style="color: rgb(51,51,153);font-weight: bold;">void</span> <span style="color: rgb(0,102,187);font-weight: bold;">loop</span>()
{


  
  <span style="color: rgb(136,136,136);">// Retrive the raw values from the compass (not scaled).</span>
  MagnetometerRaw raw <span style="color: rgb(51,51,51);">=</span> compass.ReadRawAxis();
  
  <span style="color: rgb(136,136,136);">// Retrived the scaled values from the compass (scaled to the configured scale).</span>
  MagnetometerScaled scaled <span style="color: rgb(51,51,51);">=</span> compass.ReadScaledAxis();
  
  <span style="color: rgb(136,136,136);">// Values are accessed like so:</span>
  <span style="color: rgb(51,51,153);font-weight: bold;">int</span> MilliGauss_OnThe_XAxis <span style="color: rgb(51,51,51);">=</span> scaled.XAxis;<span style="color: rgb(136,136,136);">// (or YAxis, or ZAxis)</span>

  <span style="color: rgb(136,136,136);">// Calculate heading when the magnetometer is level, then correct for signs of axis.</span>
  <span style="color: rgb(51,51,153);font-weight: bold;">float</span> heading <span style="color: rgb(51,51,51);">=</span> atan2(scaled.YAxis, scaled.XAxis);
  
  <span style="color: rgb(136,136,136);">// Once you have your heading, you must then add your 'Declination Angle', which is the 'Error' of the magnetic field in your location.</span>
  <span style="color: rgb(136,136,136);">// Find yours here: http://www.magnetic-declination.com/</span>
  <span style="color: rgb(136,136,136);">// Mine is: 2� 37' W, which is 2.617 Degrees, or (which we need) 0.0456752665 radians, I will use 0.0457</span>
  <span style="color: rgb(136,136,136);">// If you cannot find your Declination, comment out these two lines, your compass will be slightly off.</span>
  <span style="color: rgb(51,51,153);font-weight: bold;">float</span> declinationAngle <span style="color: rgb(51,51,51);">=</span> <span style="color: rgb(102,0,238);font-weight: bold;">0.0457</span>;
  heading <span style="color: rgb(51,51,51);">+=</span> declinationAngle;
  
  <span style="color: rgb(136,136,136);">// Correct for when signs are reversed.</span>
  <span style="color: rgb(0,136,0);font-weight: bold;">if</span>(heading <span style="color: rgb(51,51,51);"><</span> <span style="color: rgb(0,0,221);font-weight: bold;">0</span>)
    heading <span style="color: rgb(51,51,51);">+=</span> <span style="color: rgb(0,0,221);font-weight: bold;">2</span><span style="color: rgb(51,51,51);">*</span>PI;
    
  <span style="color: rgb(136,136,136);">// Check for wrap due to addition of declination.</span>
  <span style="color: rgb(0,136,0);font-weight: bold;">if</span>(heading <span style="color: rgb(51,51,51);">></span> <span style="color: rgb(0,0,221);font-weight: bold;">2</span><span style="color: rgb(51,51,51);">*</span>PI)
    heading <span style="color: rgb(51,51,51);">-=</span> <span style="color: rgb(0,0,221);font-weight: bold;">2</span><span style="color: rgb(51,51,51);">*</span>PI;
   
  <span style="color: rgb(136,136,136);">// Convert radians to degrees for readability.</span>
  <span style="color: rgb(51,51,153);font-weight: bold;">float</span> headingDegrees <span style="color: rgb(51,51,51);">=</span> heading <span style="color: rgb(51,51,51);">*</span> <span style="color: rgb(0,0,221);font-weight: bold;">180</span><span style="color: rgb(51,51,51);">/</span>M_PI; 

  <span style="color: rgb(136,136,136);">// Normally we would delay the application by 66ms to allow the loop</span>
  <span style="color: rgb(136,136,136);">// to run at 15Hz (default bandwidth for the HMC5883L).</span>
  <span style="color: rgb(136,136,136);">// However since we have a long serial out (104ms at 9600) we will let</span>
  <span style="color: rgb(136,136,136);">// it run at its natural speed.</span>
  <span style="color: rgb(136,136,136);">// delay(66);</span>

  <span style="color: rgb(136,136,136);">//This throttles how much data is sent to Python code.  </span>
  <span style="color: rgb(136,136,136);">//Basically, it updates every second (10 microsecond delay X 100 iComps)</span>
  <span style="color: rgb(0,136,0);font-weight: bold;">if</span> (iComp <span style="color: rgb(51,51,51);">>=</span> <span style="color: rgb(0,0,221);font-weight: bold;">30</span>){
    
    <span style="color: rgb(51,51,153);font-weight: bold;">int</span> adjHeading <span style="color: rgb(51,51,51);">=</span> <span style="color: rgb(0,0,221);font-weight: bold;">0</span>;    
    <span style="color: rgb(136,136,136);">//The "floor" part makes the float into an integer, rounds it up.</span>
    headingDegrees <span style="color: rgb(51,51,51);">=</span> floor(headingDegrees);
    <span style="color: rgb(0,136,0);font-weight: bold;">if</span> (headingDegrees <span style="color: rgb(51,51,51);">>=</span> <span style="color: rgb(0,0,221);font-weight: bold;">280</span>){
        adjHeading <span style="color: rgb(51,51,51);">=</span> map(headingDegrees, <span style="color: rgb(0,0,221);font-weight: bold;">280</span>, <span style="color: rgb(0,0,221);font-weight: bold;">360</span>, <span style="color: rgb(0,0,221);font-weight: bold;">0</span>, <span style="color: rgb(0,0,221);font-weight: bold;">79</span>);
    }
    <span style="color: rgb(0,136,0);font-weight: bold;">else</span> <span style="color: rgb(0,136,0);font-weight: bold;">if</span> (headingDegrees <span style="color: rgb(51,51,51);"><=</span> <span style="color: rgb(0,0,221);font-weight: bold;">279</span>) {
        adjHeading <span style="color: rgb(51,51,51);">=</span> map(headingDegrees, <span style="color: rgb(0,0,221);font-weight: bold;">0</span>, <span style="color: rgb(0,0,221);font-weight: bold;">279</span>, <span style="color: rgb(0,0,221);font-weight: bold;">80</span>, <span style="color: rgb(0,0,221);font-weight: bold;">360</span>);
    }

    Serial.println(adjHeading);
    iComp<span style="color: rgb(51,51,51);">=</span><span style="color: rgb(0,0,221);font-weight: bold;">0</span>;
  }
  iComp<span style="color: rgb(51,51,51);">++</span>;

  delay(<span style="color: rgb(0,0,221);font-weight: bold;">10</span>); <span style="color: rgb(136,136,136);">//For serial stability.</span>
  
  
   
  <span style="color: rgb(51,51,153);font-weight: bold;">int</span> val <span style="color: rgb(51,51,51);">=</span> Serial.read() <span style="color: rgb(51,51,51);">-</span> <span style="color: rgb(0,68,221);">'0'</span>;
    
  <span style="color: rgb(0,136,0);font-weight: bold;">if</span> (val <span style="color: rgb(51,51,51);">==</span> <span style="color: rgb(0,0,221);font-weight: bold;">1</span>)
  {
    Back();
  }
   
  <span style="color: rgb(0,136,0);font-weight: bold;">else</span> <span style="color: rgb(0,136,0);font-weight: bold;">if</span> (val <span style="color: rgb(51,51,51);">==</span> <span style="color: rgb(0,0,221);font-weight: bold;">2</span>)
  {
    Right();
  } 
  
  <span style="color: rgb(0,136,0);font-weight: bold;">else</span> <span style="color: rgb(0,136,0);font-weight: bold;">if</span> (val <span style="color: rgb(51,51,51);">==</span> <span style="color: rgb(0,0,221);font-weight: bold;">3</span>)
  {
    Forward();
  } 
   
  <span style="color: rgb(0,136,0);font-weight: bold;">else</span> <span style="color: rgb(0,136,0);font-weight: bold;">if</span> (val <span style="color: rgb(51,51,51);">==</span> <span style="color: rgb(0,0,221);font-weight: bold;">4</span>)
  {
    Left();
  }   
   
  <span style="color: rgb(0,136,0);font-weight: bold;">else</span> <span style="color: rgb(0,136,0);font-weight: bold;">if</span> (val <span style="color: rgb(51,51,51);">==</span> <span style="color: rgb(0,0,221);font-weight: bold;">5</span>)
  {
    Stop();
  }
}

<span style="color: rgb(51,51,153);font-weight: bold;">void</span> <span style="color: rgb(0,102,187);font-weight: bold;">Back</span>(){
<span style="color: rgb(136,136,136);">//Straight back</span>
      analogWrite(pwm_a, highspeed);      
      analogWrite(pwm_b, highspeed);
  
      digitalWrite(dir_a, HIGH);  <span style="color: rgb(136,136,136);">//Reverse motor direction, 1 high, 2 low</span>
      digitalWrite(dir_b, LOW);  <span style="color: rgb(136,136,136);">//Reverse motor direction, 3 low, 4 high</span>
      
delay(keyDuration);
}

<span style="color: rgb(51,51,153);font-weight: bold;">void</span> <span style="color: rgb(0,102,187);font-weight: bold;">Left</span>(){
      <span style="color: rgb(136,136,136);">//Left</span>
      analogWrite(pwm_a, lowspeed);      
      analogWrite(pwm_b, lowspeed);
      
      digitalWrite(dir_a, HIGH);  <span style="color: rgb(136,136,136);">//Reverse motor direction, 1 high, 2 low</span>
      digitalWrite(dir_b, HIGH);  <span style="color: rgb(136,136,136);">//Reverse motor direction, 3 low, 4 high</span>
    
delay(keyDuration);
}

<span style="color: rgb(51,51,153);font-weight: bold;">void</span> <span style="color: rgb(0,102,187);font-weight: bold;">Right</span>(){
      <span style="color: rgb(136,136,136);">//Right</span>
      analogWrite(pwm_a, lowspeed);      
      analogWrite(pwm_b, lowspeed);
      
      digitalWrite(dir_a, LOW);  <span style="color: rgb(136,136,136);">//Reverse motor direction, 1 high, 2 low</span>
      digitalWrite(dir_b, LOW);  <span style="color: rgb(136,136,136);">//Reverse motor direction, 3 low, 4 high</span>
    
delay(keyDuration);
}
 
<span style="color: rgb(51,51,153);font-weight: bold;">void</span> <span style="color: rgb(0,102,187);font-weight: bold;">Forward</span>(){
  <span style="color: rgb(136,136,136);">//set both motors to run at 100% duty cycle (fast)</span>
  analogWrite(pwm_a, highspeed);      
  analogWrite(pwm_b, highspeed);
  
  <span style="color: rgb(136,136,136);">//Straight forward</span>
  digitalWrite(dir_a, LOW);  <span style="color: rgb(136,136,136);">//Set motor direction, 1 low, 2 high</span>
  digitalWrite(dir_b, HIGH);  <span style="color: rgb(136,136,136);">//Set motor direction, 3 high, 4 low</span>
  
  delay(keyDuration);
}

<span style="color: rgb(51,51,153);font-weight: bold;">void</span> <span style="color: rgb(0,102,187);font-weight: bold;">Stop</span>(){
  <span style="color: rgb(136,136,136);">//set both motors to run at 100% duty cycle (fast)</span>
  analogWrite(pwm_a, <span style="color: rgb(0,0,221);font-weight: bold;">0</span>);      
  analogWrite(pwm_b, <span style="color: rgb(0,0,221);font-weight: bold;">0</span>);
  
  <span style="color: rgb(136,136,136);">//Straight forward</span>
  digitalWrite(dir_a, LOW);  <span style="color: rgb(136,136,136);">//Set motor direction, 1 low, 2 high</span>
  digitalWrite(dir_b, HIGH);  <span style="color: rgb(136,136,136);">//Set motor direction, 3 high, 4 low</span>
  
  delay(keyDuration);
}

 

1. OpenCV (which we'll call cv2).
2. Numpy
3. Serial
4. Time
5. Math
6. Random
7. Threading

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

import cv2.cv as cv
import time

cv.NamedWindow("camera", 1)

capture = cv.CaptureFromCAM(0)

while True:
    img = cv.QueryFrame(capture)
    cv.ShowImage("camera", img)
    if cv.WaitKey(10) == 27:
        break

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

#Written by the pathos filled hack: C. Thomas Brittain

import cv2
import numpy as np
import serial
from time import sleep
import threading
import math 
from math import atan2, degrees, pi 
import random

• cv2 is the OpenCV module.
• Numpy, which we'll call "np" throughout the code, is used for higher number functions needed for OpenCV to do her magic.
• Serial is the module which will allow us to establish a serial connection between the PC and the robot, via whichever wireless device you've chosen.
• Time allows us to basically idle the code.  This is important in controlling many things, for instance, how far the robot moves.  We tell the motors to turn on, wait 10 secs, then turn off.  Because the sleep function actually puts the code into an idle state, we must have the threading module, since our code requires the PC to do several things at once.
• Math.  From the math module we get the code to help us simplify the trigonometry calculations, like the angle between the robot and target.
• The random module is only used to gives us a random target. 
• Threading.  Important module.  Basically, threading allows the computers to do two tasks at the same time.  This becomes important when we are both trying to track the robot and receive his position.  Throughout this code we will have three threads
  1. The thread running the OpenCV stuff.  This tracks the robot and is also the largest.
  2. A thread controlling the serial connection between the robot and PC.
  3. And a thread with the small job of telling the motors how long to be on, thereby controlling how far the robot will move.

98

        thresh = cv2.inRange(hsv,np.array((130, 70, 110)), np.array((190, 190, 200)))

1. Setup your webcam the in the area you'll be using, just like you're ready to control him.
2. Run the webcam program attached in step 10.
   
    
3. While the webcam program is watching your robot, hit Ctrl + Print Screen
4. Open Gimp.
5. Hit Ctrl + V to paste the screen capture into gimp.
   
    
6. Now, find the Color Selector tool.
   
    
7. Select the main color of your robot.
8. Now double click on the color square on the toolbar.
   
    
9. A window should pop open with color information regarding the color you selected, your robot.
   
10. Now, the three numbers listed should be close to what we need.  Sadly, we have to convert from Gimp's HSV number range to OpenCV's HSV number range.  You see, HSV value range in Gimp is H = 0- 360, S = 0-100, and V = 0-100.  In OpenCV, H = 0-180, S = 0-255, V = 0-255.  So, some conversion needs to take place.
11. From my selection I ended with Gimp numbers of, H: 355, S:50, and V:61.  I could get all fancy and calculate the right numbers, but I figure 180 (OpenCV) is half of 360, so for my H I just divided by two: 177. The other two I kinda guess at a little.  I doubled and added 25, S: 125 and V: 147.
12. In the end, this gave me middle numbers.  But I wanted an upper and lower threshold, so I took each number and subtracted 20 to give me a lower, and added 20 to give me an upper. 
13. The result for my robot was:

1

        thresh = cv2.inRange(hsv,np.array((152, 105, 127)), np.array((180, 145, 167)))

1. Choosing HSV
2. Finding HSV in image.