Chess Pieces Object Detection in 15 Minutes

So today I'm gonna show you how to infer and train YOLOv5 for the purpose of detecting chess pieces in under 15 minutes.

So I'm sure by now most of you have been watching the Queen's Gambit on Netflix. You can definitely see evidence of this on Google trends for the keyword chess.

So the reason you would want to detect chess pieces is if you noticed how players always record their movements on paper next to them. Now imagine having an AI that's able to look at the Chessboard, to detect, map, and record the player's moves. Now if you are a novice player like me, you may also want the AI to help coach and provide simulations of the match to help better your game.

The process that we're going to follow is with the Roboflow Platform, which is free to sign up by the way, and essentially this will help you do the following:

• Obtain, manage, and augment
• import our custom dataset
• set the parameters
• train a model using either Cloud ML as well as Google Colab
• test our model
• download the weights

What's the first thing that we need? A dataset, right? So you can either annotate your own or you can drag it into raw flow otherwise you can use any one of these computer vision datasets that's already on ROBOFLOW. I'll leave a link down below for you to access this page directly. You can even check out public.roboflow.com

https://roboflow.com/as-chess-dataset

So for the chess piece detection, we are going to look for a specific data set over here called Chess Pieces Dataset. Click on it and you'll find the data set has around 289 images. Now what I want you to do is to click fork dataset. Click it again and this is so that we can pre-process and augment our data set that may yield better results for our chess piece detection model. If you have not signed up as yet please do so now and then fork the data set.

Just to familiarize everyone as to what's going on here, we have 289 Images and you can view the images here. You can see that we have each piece labeled of each of the images in the dataset. If you want to add in more images you can just click on Add More Images, and then add in the files or folder with respect to your annotations. You can specify the train validation and test split to any amount that you wish. If you click on edit splits, you can play around with the trainer or valid splits as well as the test and valid split.

We have the pre-processing options, so you have the option to correct any images that are not in a practical orientation. What I mean by this is that your chess pieces will normally be showing upwards instead of upside down. In this case, all of our data sets are in the proper orientation. You can also resize your dataset and specify the resolution that you want. In case you want to modify your classes or your class names you can do it, for our example, we have a total of 12 classes: 6 for white and 6 for black.

As seen with the previous video/image, we have a data augmentation tool where we can accumulate more data. I'm just going to add in a crop stage so we can crop. You can decide how much more you want to crop into. That's practical, I'm also going to add in a little bit of a rotation - make that around six percent so then we can check at different angles, and then let's look at our exposure by brightening or darkening images we can get some variability to our dataset. Let's also drop in a grayscale transformation, so in the case where we have different color schemes in our dataset, you can specify the percentage of output images that would need to be greyscaled, then click apply.

The benefit of this is, it helps our model not to memorize our data set and thus allowing it to generalize better. So I challenge you to choose an experiment with any amount of data augmentation options that you want. In the augmentation output box, we can specify how many augmented versions that we want to create, I'm just going to stick with the default three(3). If you change this to four(4) for example, you can see that our images multiply accordingly. Let's just keep it to 3 for now, then click save.

One last thing before we generate our data set, I want you to click on Dataset health check tab. Here you can check the quality of our dataset. You can see that it is now a very biased dataset, we just have too many pawns, and our royal classes are heavily under represented. As mentioned previously, it's best to have a balanced data set to prevent data bias. This gives us an action to say that we need to acquire and label more of the other classes. I have the link up here, I'll show the dimension insides and we can see the average distribution of our dataset. Our data set is pretty wide as you can also see in the annotation heat maps, you can also see that you can even crop out datasets on the side to speed up training.

So let's go back to our dataset, see that everything is as it's supposed to be and then let's click generate. We're gonna call this just v1, then click generate. Cool, now we just sit back and wait for our dataset to augment to 709 images after augmentation.

Now we have a choice, as to how we want to export our dataset. You can see that we have a variety of yellow formats, TensorFlow formats, as well as XML, and JSON. Which you can download either as a zip file for training natively but I'm going to show you a little secret on how to quickly train and test your model using AutoML within the Roboflow platform. I'm just going to leave it here as the download code continues and we'll use this code in a bit.

Now that we have our data set, go down below. We see it says, use cloud-managed training. Some of my students mentioned that they don't have Nvidia GPU that is a requirement for most of my courses but with Roboflow you can train your models in the cloud using your favorite services either AWS, Google Cloud, or Microsoft Azure.

Now, just note that there are training costs involved should you wish to train on these platforms. If you click this link over here it'll take you to this page which mentions the trade-offs and costs for each of the cloud ML platforms. It may seem pricey but hey it's cheaper than buying a brand new GPU right and this is depending on your usage of course. For now, I'm going to train using the Roboflow Platform by clicking, Use Roboflow Train. So what Roboflow train is? It's a one-click model training service. You'll need some training credits for this to work, so click Start Training once you have credits.

So as I mentioned here, training is in progress and it could take anyway up to 24 hours. I suggest maybe working on something else whilst the training is happening.

After training, in this box, you can get the results of your trained models in terms of mean average precision or map precision as well as recall. You can also check out the detailed results as well as the training graphs right, and this is really great if you want rapid prototyping of your dataset.

Moving on, we have trained our dataset, now what? Well, to see if our model is working, we can open up this web example. Now what you're seeing is a production-ready Web API with all of the source code which you can customize for your own applications and clients. So for example, I can customize this to my own Augmented Startup branding very easily.

First up what we'll do is we'll locate our HTML, we see this header logo over here, I'm going to type in Augmented Startups, search for images, look for my logo and then copy the image link. I'll put it right over "header-logo src", paste and let's see what happens. So I got my logo over here and we'll just customize it now to my branding colors. So we're gonna look for a button, I'll put in my hex code. Mine is #ff9600 and does the same for the borders. For the primary button, we'll do the same, and voila. So like I said you can take all this code and deploy it on your own servers.

And now, you must be asking yourself so okay Rits, what does this app do? I'm glad you asked because now we've trained our model. We can select a file, I'm going to look for this image and upload this image and we can fold it out for specific classes if we need to. Leave the inference to resolve this image and turn on our labels. Run inference, and voila. I don't know about you but I mean that was really easy to deploy and test our custom models like if you zoom into here you can see that we have king, we have pawn, we have our queen over here. We've detected another king over there with our bishop and all looks good.

So if you exclude the train time, the time it took for us to go from importing the dataset all the way to a trained model took us under 15 minutes to do all of this, and like I said this is really great for fast prototyping, getting your production-ready API and in the future, we'll talk about auto labeling of your dataset. If those lines are looking a little bit thin you can change it to 10 pixels from inference and you can see we have thicker lines.

So as you can see obviously our model is not perfect because we have red boxes that are just borderlined but with more data and better training parameters we can get better detections. You can also play around with the confidence, as well as the max overlap in terms of percentage.

Now you're probably asking, okay Ritz, this is all great but I prefer to do my own training and tweak my own parameters for training. Well, there is also an option for using Google Colab. So the benefit of this is that for this small chest dataset your training will take around 12 to 15 minutes instead of the 24 hours specified for AutoML. To get started, go to:

https://augmentedstartups.info/YOLOv5Colab

So this call-up is quite straightforward with a lot of information explaining how it all works and how it's all put together. You'll start off by first ensuring that your runtime has been set to GPU. So go to runtime, change runtime type, and that the hardware accelerator is on GPU, click save. So first up, we'll install our dependencies and for this, we'll have a git clone and then we'll clone our YOLOv5 repository. Change our directory to YOLOv5 and then the next step.

We can install our dependencies as necessary using the command pop install requirements.txt, import our torch libraries as well as some other google drive libraries as well and then print out the statement that setup is complete once that's done.

So once we have done that, here's the only tricky part. Actually, I wouldn't call it tricky it's actually quite easy and this involves downloading the correctly formatted custom dataset. Now, if you remember we already created our dataset augmented it and this needs to be in a YOLOv5 python-format where did we see this before YOLOv5 PyTorch. Let's go back to our dataset, get the link, and copy this code. Make sure not to share your link with anyone else. Copy the link, click done, and just ensure that you were on your YOLOv5 PyTorch. Go back to our collab and what we're going to do, we're going to copy and paste our link. We'll select this line ctrl-v and there we have it, click play and it will import our dataset.

So first up, you can see that it's extracting all of the images as well as the txt files. Those are our annotations and now to simplify things further, we have this file called data.yml and this is the yml file that Roboflow wrote for us while we are loading our data into this notebook. So you click play and you can see that we have the number of classes which is six and over here we're only focusing on the type of piece and not the color. So you can see we have the bishop class, king, knight, pawn, queen, and rook classes. Great, next we need to define our model configuration and architecture so we can import our yml file as well as our model configuration for this tutorial.

We're going to be looking at YOLOv5s, so you can see initially it says our number of classes is 80 but we actually have 6. So we need to change that, you can also look at our YOLOv5 backbone and head. This part over here just customizes our python right file. So we can write in our variables. So as you click play, you can see that our YOLOv5s model has been customized for our new data set. We also can see a number of classes. If you hover a little, it says six, which is correct and you can play around with your model. Should you wish to get a little deep into the YOLOv5 architecture as well as its parameters.

Now over here we have an optional training fix, so if your training didn't go as expected you can run these commands but for me, it all worked the first time, so I'm just going to hide this. So if everything went fine let's train our custom YOLOv5 detector. So if you're familiar with training your CNNS you'll recognize most of these parameters. So these parameters are the image, the number of epochs, we can specify this as maybe 100. Putting our data.yml file, our config files as well as our weights and the name that we want to save our YOLOv5s results to, click play and let the training begin. Now just note that this may take some time.

Great and so we are done, so let's go and check how we did. So in this section it'll say evaluate custom YOLOv5 detector performance, so using tensorboard we can check out our training losses and perform metrics. Now if you remember we spoke about YOLOv5s results that we specified and these results will be plotted as a png after the training completes. Now that we have it completed just click play and let's check out our results. Great, so we can check out our min average position. Ideally, we'd like for our graph to reach 1 as soon as possible, so we can still do some improvements.

Cool so moving on, we can visualize our training dataset. So if we click play you'll see that we have our ground truth for each of the images. We can also do this for the augmented dataset just to look at what that looks like. So you can see what happened here, we skewed some images, we had some of the images rotated and this helps our models generalize better. And now for the main part, so let's test our model performs when we run an inference with drain weights. So just click the corg here then you go to the second one. We will get our weights, click play. And you can see that we are performing inference on all of our images in our test dataset.

Okay, so once our data set has been inferred we can visualize it and you can see it does quite well by detecting the majority of just pieces. Obviously, there's more that we can do to make this better like maybe adjusting our detection threshold, training with better quality data adjusting our data augmentation parameters, and much much more. So for now, this will suffice given our very small dataset. Now if you're happy with your dataset you can always export your weights for future use or future inference, if you click the button you can download it to your google drive. You will just need to enter in your authorization code which you can get from this link.

So that is it for this tutorial, if you think the Roboflow Platform is awesome and easy for training, testing, and deploying your models then let me know in the comments. For those of you who are interested in object detection, object tracking, post estimation courses. Buying your own oak kits or accessing the GitHub code, all the links will be down below. We also have a membership area on Youtube and Patreon where you can access all of our Premium tutorials and source code as well as get early access to all of the videos and tutorials.

Click the join button to become an elite member and a supporter of this channel. In the next video, we'll be dealing with mask detection to control a boom gate using Raspberry PI. Lastly, if you'd like to see more tutorials like this please hit the like, share, and subscribe buttons. Put that bell icon as it will really help me out. Thank you for watching and we'll see you the next.