Introduction

There are currently various attempts to apply Deep Reinforcement to actual robot. The characteristics of Reinforcment Learning require over hundred of the initialization of the environment and many data for Deep Learning making it impossible to train the robot in the actual environment.

To overcome these difficulties, people first create a simulation that looks exactly like the real world. In this simulation, training is advanced for the virtual robot. Since then, many researches try to use the tranined model for an actual robot.

In this post, I’ll try to confirm these methods are actually working. The test uses a robot called Jetbot that has a GPU on its main board. And the environment is a soccer that is familiar to many people.

Code for that post can be found on the Github Jetbot Soccer Github.

All the products used in the tests are inexpensive and can be purchased in Amazon. Thus, you can easily try to reproduce them.

Table of Contents

  1. What is Jetbot?
  2. Soccer game using Jetbot
  3. URDF file for Jetbot
  4. Simulating Jetbot at Rviz
  5. Simulating Jetbot at Gazebo
  6. Install soccer object at Gazebo
  7. Connect Gazebo and Python
  8. Get object location in Python
  9. Place multiple Jetbot in field
  10. Physical parameter of Jetbot
  11. Soccer ball detection
  12. Customize soccer field for Jetbot
  13. Convert continuous action to discrete
  14. Set real soccer environment
  15. Object location change in Python
  16. Reinforcement Learning test
    1. Network Structure
    2. Reward Setting
    3. Training result
    4. Changing reward setting for accurate control
    5. Training result after revision
  17. Real Jetbot test
    1. Controlling original Jetbot
    2. Deep Learning model for Jetbot
    3. Transfering trained Tensorflow weight to Jetbot
    4. Controlling wheel of Jetbot Soccer version
  18. Soccer robot design
    1. Soccer robot design(Wheel)
    2. Soccer robot design(Ball)
    3. Soccer robot design(Siumlation)

What is Jetbot?

Recently, the Jetson Nano board is released by NVIDIA. And various robots that using that board is selling on the market. One of them is Jetbot that is a kind of line tracer robot. I purchased it at Amazon Jetbot Purchasing Link. Designing a customizing robot is a difficult and time-consuming task. Thus, I choose a Jetbot.

Furthermore, ROS nodes and Gazebo model of Jetbot is released together. Thus, it became easier to install and running.

Jetbot ROS test Click to Watch!

I install ROS on the purchased jetbot and test the basic operation. Installation ROS is very easy and Jupyter Notebook work environment is also easy.

However, the 18650 battery of 65mm length and 2600mAH capacity installed on Jetbot can not be purchased in Korea local shop. That’s why I try to buy from an overseas online shop. It looks difficult importing battery from another country because of customs clearance. However, I find one battery seller in the secondhand market.

Soccer game using Jetbot

First of all, I plan to use a Jetbot for playing soccer. I can use a Gazebo of ROS, Deep Learning for this project.

Gazebo reduces a inconvenience of having to test a robot in a real environment by simulating robot in virtual environment. And Deep Learning technology can make agent play soccer like a human. My goal in this project is combining these two skill for Jetbot playing a soccer like a real human.

URDF file for Jetbot

The URDF file is a format that describes the appearance, physical characteristics, mounting sensor, etc of a ROS robot. Also, to simulate a robot with Gazebo, that file is needed first .

There is also another format called SDF file that is a more general than URDF. But, that format does not support the simulation in Gazebo yet. Thus, it is unavoidable to change SDF file of Jetbot to URDF format. Usaually, Gazebo does not use URDF files directly. It uses a similar format called XACRO that is very similar to URDF. Changing format from URDF to SDF is very easy.

File must also contain information about characteristics of sensor such as camera for Gazebo simulation. This is created separately in a file with Gazebo extension and loaded from Xacro file. It makes a XACRO file is not too complicated.

You can download model file of Jetbot at Jetbot URDF file link.

Simulating Jetbot at Rviz

After creating the URDF file, you can verify it in ROS Rviz. If there is a joing part that can move like a wheel, you can use the GUI to move it in Rviz.

Simulating Jetbot at Gazebo

Unlike RviZ, Gazebo work with the physical elements of the actual robot and the surrounding environment. Thus, you can get the same effect as testing the robot in the actual environment.

Install soccer object at Gazebo

A Jetbot model was created and tested using Rviz and Gazebo. Now you can add soccer fields, goals, and soccer balls around the robot and control the robot by applying torque to each wheel. That kind of testing method is provided in Gazabo without making additional code.

Jetbot soccer test1 Click to Watch!

As you can see in the video, the physical conditions of Gazebo are similar to the actual conditions. Thus, it is not easy to control the robot than pure simulated environmnet.

Connect Gazebo and Python

The jetbot will receive the camera and other sensor information as input and will use it to adjust the speed of the left and right wheels to play soccer. For this, we need Python code that performs the above steps. ROS can input and output these data using a package called rospy.

Jetbot soccer test2 Click to Watch!

As you can see in the video above, you can use Python code to output the camera sensor image and adjust the speed of the left and right wheels.

Get object location in Python

The most important goal in soccer will be to put a soccer ball into the opponent’s goal. To do this, we should know the location of the soccer ball and goal in real time.

football_pose: 
position: 
  x: 0.0
  y: 0.0
  z: 0.119999999992
  
left_goal_pose: 
position: 
  x: -9.0
  y: 0.0
  z: 0.01
  
right_goal_pose: 
position: 
  x: 9.0
  y: 0.0
  z: 0.01

If you know the position of each object in the above way, we could use a method that gives a score when it is close to the soccer ball goal or reset a field.

# maybe do some 'wait for service' here
reset_simulation = rospy.ServiceProxy('/gazebo/reset_simulation', Empty)
def state_callback(msg):
    # print("msg.name: " + str(msg.name))
    # msg.name: ['ground_plane', 'field', 'left_goal', 'right_goal', 'football', 'jetbot']

    football_index = (msg.name).index("football")
    left_goal_index = (msg.name).index("left_goal")
    right_goal_index = (msg.name).index("right_goal")
    #print("football_index: " + str(football_index))

    football_pose = (msg.pose)[football_index]
    #print("football_pose.position.x: " + str(football_pose.position.x))

    football_pose_x = football_pose.position.x
    print("football_pose_x: " + str(football_pose_x))

    if (football_pose_x > 5):
        reset_simulation()

    left_goal_pose = (msg.pose)[left_goal_index]
    #print("left_goal_pose: " + str(left_goal_pose))

    right_goal_pose = (msg.pose)[right_goal_index]
    #print("right_goal_pose: " + str(right_goal_pose))

    global x
    global y
    global theta

    x = msg.pose[1].position.x
    y = msg.pose[1].position.y
    rot_q = msg.pose[1].orientation
    (roll, pitch, theta) = euler_from_quaternion([rot_q.x, rot_q.y, rot_q.z, rot_q.w])

The above code is part of https://github.com/kimbring2/jetbot_soccer/blob/master/jetbot/jetbot_control/src/main.py. After running Gazebo and controller, give a control signal to move the soccer ball using Jetbot. After that, when the soccer ball goal is reached, the simulation is initialized.

Place multiple Jetbot in field

For the simplest soccer game, Jetbot that kicks a soccer ball and Jetbot that prevents it are placed in the field, and an environment is established that can get the image of the camera sensor and control wheel speed of each robot.

Jetbot soccer test3 Click to Watch!

As described above, two Jetbots are placed in the field, one jetbot tries to make a goal, and one jetbot tries to defend the goal. In the case of multiplt agent such as that case, the Self-play Reinforcement Learning looks good for using.

Physical parameter of Jetbot

A Jetbot consists of a chassis and two wheels connected to it. In order to simulate a real environment with Gazebo, these physical parameters are set in the same way as in the actual case. Since Jetbot’s exact gazebo physics parameters is not asked now, I temporarily use the parameters of existing robots called iRobot Gazebo Repository.

<gazebo reference="chassis">
  <mu1>0.0</mu1>
  <mu2>0.0</mu2>
  <slip1>1.0</slip1>
  <slip2>1.0</slip2>
  <kp>10000000</kp>
  <kd>1</kd>
  <fdir1>1 0 0</fdir1>
  <minDepth>0.0001</minDepth>
  <maxContacts>1</maxContacts>
</gazebo>
 
 <gazebo reference="left_wheel">
  <mu1>50</mu1>
  <mu2>10</mu2>
  <slip1>0.0</slip1>
  <slip2>0.0</slip2>
  <kp>10000000</kp> 
  <kd>1</kd>
  <fdir1>1 0 0</fdir1>
  <minDepth>0.0001</minDepth>
  <maxContacts>1</maxContacts>
</gazebo>

<gazebo reference="right_wheel">
  <mu1>50</mu1>
  <mu2>10</mu2>
  <slip1>0.0</slip1>
  <slip2>0.0</slip2>
  <kp>10000000</kp> 
  <kd>1</kd>
  <fdir1>1 0 0</fdir1>
  <minDepth>0.0001</minDepth>
  <maxContacts>1</maxContacts>
</gazebo>

Jetbot soccer test4 Click to Watch!

In previous tests, the proper physical parameters are not found, and the Jetbot moved abnormally slowly. However, after applying the appropriate parameters, the robot is able to confirm the movement similar to the actual situation.

Soccer ball detection

To play soccer, Jetbot must first find a soccer ball. Jetbot do it using the camera sensor and object detection Deep Learning library.

import cvlib as cv

bbox, label, conf = cv.detect_common_objects(cv_image, confidence=0.05)
for obj in label:
   if obj == "sports ball":
       index = label.index(obj)
       x1 = int(bbox[index][0])
       y1 = int(bbox[index][1])
       x2 = int(bbox[index][2])
       y2 = int(bbox[index][3])
       cv2.rectangle(cv_image, (x1,y1), (x2,y2), (0,255,0), 2)

The Jetbot camera sensor currently in use is a fixed type. Thus, there is a problem that it is not recognized when the soccer ball is too close. However, it is correctly recognized when the entire soccer ball is on the screen.

Customize soccer field for Jetbot

In the case of Jetbot, unlike other soccer-only robots, there is no arm to fix the soccer ball. Therefore, there are restrictions on various actions. So I devise a way to set up walls on all sides.

Creating a wall in this way will prevent the soccer ball from leaving too far from robot, even if Jetbot doesn’t hole the soccer ball.

Convert continuous action to discrete

The most basic action of Jetbot is the speed of both wheels. However, since this speed value is a continuos value, the range is too wide for direct use. Thus, I combine this value and turned it into the six most used actions in soccer.

stop_action = [0, 0]
forward_action = [40, 40]
left_action = [40, -40]
right_action = [-40, 40]
bacward_action = [-40, -40]
kick_action = [100, 100]
robot_action_list = [stop_action, forward_action, left_action, right_action, bacward_action, kick_action]

Then, I select the generated discrete type action as random and confirm the action of Jetbot.

robot1_action_index = random.randint(0,5)
robot2_action_index = random.randint(0,5)
robot1_action = robot_action_list[robot1_action_index]
robot2_action = robot_action_list[robot2_action_index]
    
pub_vel_left_1.publish(robot1_action[0])
pub_vel_right_1.publish(robot1_action[1])

pub_vel_left_2.publish(robot2_action[0])
pub_vel_right_2.publish(robot2_action[1])

Now that everything is ready, we can use Deep Reinforcement Learning for Jetbot playing soccer.

Set real soccer environment

After Jetbot can play soccer using Reinforcement Learning in Gazebo. We should transfer the trained model to a real Jetbot and make sure it works well in a real environment.

First, I pick an appropriate goal and a soccer ball on Amazon Soccer post and ball Amazon link. The shape of the simulation are a little different from these things. Thus, shapd of goal and goal post in simulation should be changed. Also, I select green soccer field rug for floor Soccer field rug Amazon link.

Object location change in Python

When performing Reinforcement Learning, position changed of soccer ball when episode is initialized is needed to confirm traning is done correct. In Gazebo, we can change the position of an object in the world using “/ gazebo / set_model_state” topic.

from gazebo_msgs.msg import ModelState

set_state = rospy.ServiceProxy('/gazebo/set_model_state', SetModelState)
pose = Pose() 
pose.position.x = np.random.randint(0,6)
pose.position.y = np.random.randint(0,6)
pose.position.z = 0.12
  
pose.orientation.x = 0
pose.orientation.y = 0
pose.orientation.z = 0
pose.orientation.w = 0
    
state_model = ModelState()   
state_model.model_name = "football"
state_model.pose = pose
resp = set_state(state_model)

When one episode is completed, the initialization position of the Jetbot is fixed. However, the position of the soccer ball can be changed every time as can be seen in the video,

Reinforcement Learning test

For this project, I chose Deep Reinforcement Learning as the control method for Jetbot playing soccer. This method train robot control neural networks via reward.

Network Structure

For Reinforcement Learning, network input and output are required. In the case of Jetbot, the input is an image obtained by changing the size of the image taken by the camera to 84x84x3. And the output is an action consisting of a ball kick, stopping, moving forward, moving backward, turning left, turning right.

Reward Setting

Moreover, it is necessary to define a reward for the training of Reinforcement Learning. At the very beginning, the camera is set to give a reward value of 1 when a soccer ball is detected by the camera.

r = 0
for obj in label:
    if obj == "sports ball":
        r = 1
        index = label.index(obj)

In the initial stage of Reinforcement Learning, training data is collected while performing random actions so that the agent can experience various situation. This part is for exploration of Reinforcement Learning.

Jetbot soccer test5 Click to Watch!

After trarning, I can compare the behavior of Jetbot with the previous behavior to check Reinforcement Learning is effective.

Training result

When Reinforcement Learning works normally, after setting as above, Jetbot must see the direction where the soccer ball is located after a certain episode.I observe the behavior of Jetbot after Reinforcement Learning as a condition to receive a reward of 1 when soccer ball is detected.

First Jetbot Reinforcement Learning test1 Click to Watch!

Changing ball size for well detection

I am able to confirm that there is a case where the total sum of reward do not increase even though the behavior of Jetbot is like finding a soccer ball.

Thus, I reconfirm the part that detects the soccer ball. The minimum distance for detection is confirmed around 1.3m. I think soccer ball size is needed bigger than now.

Reflecting these circumstances, the size of the soccer field has been reduced to 2/3 due to the limited distance of the Jetbot camera. And the size of the soccer ball is made slightly larger so that it can be taken well by the camera.

Changing reward setting for accurate control

After adjusting the size of the field and the size of the ball to match Jetbot, the conditions for reward for Reinforcmenet Learning are slightly raised. When soccer ball enter the screen, Jetbot get a reward, but this time ball should be located around the center of screen.

r = 0
for obj in label:
  if obj == "sports ball":
    index = label.index(obj)
    ball_bbox_x1 = bbox[index][0]
    ball_bbox_x2 = bbox[index][2]
    ball_bbox_middle = (ball_bbox_x1 + ball_bbox_x2) / 2.0
    #print("ball_bbox_middle: " + str(ball_bbox_middle))
            
    if ( (ball_bbox_middle > 330) & (ball_bbox_middle < 390) ):
      r = 1

Second Jetbot Reinforcement Learning Click to Watch!

Training result after revision

After trarning about 12 hours, when I check a behavior of Jetbot, it seems to try to pick up a soccer ball inside the camera angle. Therefore, I can confirm that there is no problem with the Reinforcement Learning algorithm itself.

However, Jetbot do not take a soccer ball exactly in the middle of the camera. Thus, I check a the conditions that give reward and find it is too wide and decide to narrow the scope.

Third Jetbot Reinforcement Learning Click to Watch!

I train after reducing the range displayed on the soccer ball camera that gives the reward. The results confirm that Jetbot shows the action to pick up the soccer ball in the middle of the camera.

However, sometimes there is a phenomenon where Jetbot was standing in a place that was too long. Such a phenomenon is considered to be a phenomenon that occurs because there is no effect on the reward even if only the time has passed without doing anything.

Real Jetbot test

In addition to trarning Reinforcement Learning by simulation, I also test a the actual Jetbot to reduce the difference between simulation and actual.

Controlling original Jetbot

While trarning the simulation, the actual environment is constructed for using the trained model. I use various basic examples of Jetbot to check the operation of the robot in this environment. The carpet for soccer field looks like a little small. It is necessary to use the entire area including the white line.

First Jetbot real soccer test Click to Watch!

While creating the actual environment directly, I am able to discover difference between simulation and actual. The biggest one is that Jetbot should not collide with each other because the camera is exposed to the outside. It seems that camera will be broken at the time of collision.

Deep Learning model for Jetbot

In addition to creating a soccer field for Jetbot, I should also consider how to use the Tensorflow model trained in Gazebo. First of all, it is necessary to implement the input and output of the network of the neural network used in Gazebo as it is in the actual Jetbot. Thus, it will be possible to use the trained network without change.

In particular, embedded boards such as Jetson Nano may not use common libraries can be used on general research PCs. Thus, this kind of library also needs to be changed. For example, I use cvlib when detecting a soccer ball in Gazebo, but this library correctly can not be installed in Jetson Nano.

import jetson.inference
import jetson.utils

b_channel, g_channel, r_channel = cv2.split(cv_image)
alpha_channel = np.ones(b_channel.shape, dtype=b_channel.dtype) * 50 # creating a dummy alpha channel image.
img_BGRA = cv2.merge((b_channel, g_channel, r_channel, alpha_channel))
in_arr = jetson.utils.cudaFromNumpy(img_BGRA)

overlay = "box,labels,conf"

net = jetson.inference.detectNet("ssd-mobilenet-v2", threshold=0.3)
detections = net.Detect(in_arr, width, height, overlay)

First, jetson.inference and jetson.utils is needed to be imported in the python file. Next, change a format of camera image from numpy array to tensor using utils.cudaFromNumpy. Overlay parameter should be given to inference.detectNet function for boxing, labeling and getting confidence of detection.

In fact, if you look at the results of a test using Jetbot, you can see that a fairly wide range of objects are detected and displayed with high accuracy.

Transfering trained Tensorflow weight to Jetbot

And considering the limited Jetson Nano board GPU compared to a normal PC, it is necessary to use a trained model to optimize when inference. In the Jetson series, this can be done mainly using TensorRT. First the model is trained and saved using Gazebo and Reinforcmenet Learning on a general PC with GPU. Next, convert the weight of the saved model to TensorRT inference graph format, and use this to execute inference in session.

# Import TensorFlow and TensorRT
import tensorflow as tf
import tensorflow.contrib.tensorrt as trt
import numpy as np

# Inference with TF-TRT `MetaGraph` and checkpoint files workflow:
graph = tf.Graph()
with graph.as_default():
    with tf.Session() as sess:
        # First create a `Saver` object (for saving and rebuilding a
        # model) and import your `MetaGraphDef` protocol buffer into it:
        saver = tf.train.import_meta_graph("model-1.cptk.meta")
        # Then restore your training data from checkpoint files:
        
        checkpoint = tf.train.get_checkpoint_state("/home/jetbot/Notebooks/")
        input_checkpoint = checkpoint.model_checkpoint_path
        saver.restore(sess, input_checkpoint)
        
        #for op in graph.get_operations():
        #    print(op.name)
            
        # Finally, freeze the graph:
        your_outputs = ["main/ArgMax"]
        frozen_graph = tf.graph_util.convert_variables_to_constants(
            sess,
            tf.get_default_graph().as_graph_def(),
            output_node_names=["main/ArgMax"])
        
        # Now you can create a TensorRT inference graph from your
        # frozen graph:
        trt_graph = trt.create_inference_graph(
            input_graph_def=frozen_graph,
            outputs=["main/ArgMax"],
            max_batch_size=1,
            max_workspace_size_bytes=50000,
            precision_mode="FP16")
        
        tf.import_graph_def(trt_graph, name='')
    
        x = graph.get_tensor_by_name('main/scalarInput:0')
        y = graph.get_tensor_by_name('main/ArgMax:0')
        trainLength = graph.get_tensor_by_name('main/trainLength:0')
        batch_size = graph.get_tensor_by_name('main/batch_size:0')
    
        input_test = np.empty([84, 84, 3], dtype=float)
        input_test_flatten = input_test.flatten()
        
        y_out = sess.run(y, feed_dict={
            x: [input_test_flatten],
            trainLength: 1,
            batch_size: 1
        })
        
        print(y_out)

Controlling wheel of Jetbot soccer version

I use the dynamixel SDK and ROS together to control wheel of the Jetbot soccer version. Detailed code can be found at https://github.com/kimbring2/DeepSoccer/blob/master/jetbot_soccer_motors.py.

:warning: This code is used in actual Jetson Nano, not in simulation.: Don’t get confused!

These files must be built using ‘catkin_make’ first, usually like a ROS project. Then run ‘roscore’ and then run that file with ‘python jetbot_soccer_motors.py’. ‘rostopic list’ command can be used to see the topic for each wheel published.

To send a command to each topic, “rostopic pub -1 / jetbot_soccer_motors / cmd_str_wheel1 std_msgs / String” data: “100” “’ command is used.

Jetbot soccer field test 1 Click to Watch!

Robot can move well in real soccer field like a video. Not only the wheel but also controlling of roller and solenoid can be done in the same way.

Soccer robot design

I conclude that no matter how much football fields I changed, Jetbot need to take a soccer ball or kick it. Thus, I decided to design a robot exclusively for soccer and using the Jetson Nano.

Fortunately, 3d model of NVIDIA Kaya robots and robot participating in Robocup are available online. By using both of these as a reference and utilizing 3d printer, I can easily create a Jetbot for football. And recently, the price of 3D printers has dropped sharply. Thus, I manage to find one printer to create a soccer robot at a very affordable price.

  1. Kaya robot model
  2. Robocup robot model
  3. 3D printer

Jetbot’s soccer robot design has two primary focuses. Kaya robot’s motors, screws, batteries, electronics, etc., initially offered by NVIDIA, are the latest parts sold today and should be used wherever possible. Second, the overall design of the robot should be as similar as possible to the design of the robot that entered Robotbup.

To achieve such a goal, two robot parts are once separated and measured to confirm what parts are needed to be newly designed or modified.

The new robot will have a solenoid in the center for a kicking and the required parts will be produced using a 3D printer.

Soccer robot design(Wheel)

First Ender 3D printer test Second Ender 3D printer test Click to Watch!

After a little trial and error, I am able to produce a parts designed to fix the Dynamixel in good quality.

After creating a suspention for each wheel, I also produce a the body of the robot for supporting a suspention together.

Third Ender 3D printer test Click to Watch!

After constructing the first layer of the robot body, I attache four motors to the body with the suspension I created earlier using 3D printer.

OmniWheel test Click to Watch!

The power supply to the dynamixel is 12V, which utilizes the Jetbot of WaveShares main board which has 3 18560 battery.

The U2D2 board and U2D2 power hub can be purchased at the Robotis shopping mall. However, if you have existing one, you can change only the 12V power supply method and use the rest as it is.

  1. U2D2
  2. U2D2 Power Hub

It is judged that the size omniwheel is too large. Thus, I decided to replace it with a slightly smaller wheel. Basically, onmiwheel product which provides a 3D model is selected for using in Gazebo simualtion.

  1. Omniwheel shop(Korea local shop)
  2. Omniwheel 3D model

Dynamixel MX-12W is not changed.

Dynamixel test 1 Click to Watch!

ID and communication method and firmware version of of Dynamixel can given via a program provided by ROBOTIS. I test a motor seperately before installing it to robot body. A power supply and TTL communication can be done by using a U2D2 board and power hub.

After confirming only the operation of the motor separately, the motor and the control, power board are combined to the robot body.

In the case of dynamixel, the initial mode is the joint mode. Mode is needed to be changed to the wheel mode for using soccer robot. This can be achieved by setting the CCW Angle Limit of motor to 0. To rotate the motor set in the wheel mode at a specific speed, you just need to give a specific value to Moving Speed.

Dynamixel test 2 Click to Watch!

Next, I test adjusting the speed of dynamixel using rostopic, as in the previous Jetbot.

Dynamixel test 3 Click to Watch!

The Jetbot Soccer version uses an omniwheel that has a many sub wheel. In order to properly simulate this with Gazebo, we must make sure that each sub wheel rotates correctly. First, I check it using RViz in the same way as a main wheel.

Omniwheel RVIz test Click to Watch!

After completing the test with RVIz, the test is similarly performed with Gazebo. It is confirmed that when the friction with the floor is large, the phenomenon that the sub wheel do not rotate properly is occurred. Finding the optimal friction parameters will be an important task.

Omniwheel Gazebo test Click to Watch!

Soccer robot design(Ball)

The mechanism for controlling the ball is composed of a rubber roller for fixing and a solenoid electromagnet for kicking.

  1. Engraving rubber roller(Made in Korea)

For the part for grabiing the soccer ball, I use a part of engraving roller. The core of the roller is made by 3D printer and connected to a DC motor which is included in origin Jetbot.

Roller test 1 Click to Watch!

After completing the operation test with the roller alone, it is mounted on the of the robot second body layer. Since then, I test whether robot can actually hold the ball.

Roller test 2 Click to Watch!

For the part for kicking the soccer ball, I use a solenoid electromagnet and motor driver for easy control ot it.

Solenoid test 1 Click to Watch!

When the solenoid electromagnet is applied a voltage of 12V, iron bar back of middle of motor is pulled. When the power supply is removed after that, the iron bar hit the ball of front by the elastic force of the spring.

  1. Solenoid electromagnet and driver(Made in Korea)

After testing the operation of a single solenoid motor, it is insalled to the robot body. Then, I perform an test it with Roller, which catches the ball to confirm that robot can do a catch and kick operation simultaneously.

Solenoid test 2 Click to Watch!

As a result of the test, the robot can hold and kick a ball without any problem in stop situaiton. However, there are two minor problem that robot can not kick a ball far because of ball is little heavy and hold perfectly because of absence of side wall around roller.

To control a solenoid electromagnet through the Jetson Nano, a driver that relays control signals and power between the 12V power supply and the motor is required. I used drivers sold together with a solenoid electromagnet. First, connect the motor and 12V power supply to driver as can be seen in the schematic diagram. Thereafter, the solenoid electromagnet can be controlled with the sixth wire of the CON1 pin group of the driver.

Jetson Nano GPIO control is performed by referring to the https://www.jetsonhacks.com/2019/06/07/jetson-nano-gpio/ blog post.

Solenoid test 3 Click to Watch!

You can control the solenoid electromagnet by setting the GPIO to a value of 0 or 1 by terminal command as you can see it in the video.

Solenoid test 4 Click to Watch!

Finally, to control the solenoid electromagnet in the ROS, we should control GPIO using code. I use a GPIO library of Jetson (https://github.com/NVIDIA/jetson-gpio) provided by NVIDIA.

Soccer robot design(Simulation)

Due to the late readiness of the soccer ball catching and car parts, I decide to first create the control code for the assembled omniwheel. The bodies produced for the test are assembled together and power is supplied to Jetson Nana and Dynamixel.

Next, I controll the motor by using a Dynamixel Wizard and check the image of the camera already tested with previous Jetbot to confirm that everything is working properly.

Protoype test 1 Click to Watch!

When parts of the mechanical section are added in the future, it will be possible to add about one body here and add parts and circuits.

After designing the hardware of the robot, I generate a 3D model for training it in Gazebo simulation. After that, the operating of the wheel, roller, solenoid motor is first confirmed using RViz in the same manner like a original Jetbot simulation.

Protoype RViz Click to Watch!

The operation of each part is tested using the ‘rostopic pub’ command. The wheel, roller, solenoid motor which are three parts of the robot are tested in order.

1. Wheel test

Gazebo wheel test Click to Watch!

2. Holding test

Gazebo ball holding test Click to Watch!

3. Kick test

Gazebo ball kicking test Click to Watch!

Combining all the above actions, robot can dribble the ball and kick towards the goal.

4. Dribble test

Gazebo ball dribbling test Click to Watch!

Now we have all the basic robot functions to play soccer. Now, let’s train the robot for playing soccer using reinforcement learning like the previous Jetbot.