Brain Pong Walkthrough
In this tutorial, we are going to walk through how to operate Neurorobotics Studio, using an autonomous and manual brain pong example. If you are new to Neurorobotics Studio, you will be presented 2 options: view the tutorials or create an experiment. Select make a new experiment. If you have previously created an NRS experiment, your screen will look like this. Click the same button in the upper right corner. 1. Select the environment for your robot. Physical environments typically involve an experiment using tangible robots. Virtual environments are used for computer generated characters. In the case you don’t have a physical robot, select virtual. 2. Select your embodiment. You are able to filter the embodiments by different features, such as in this case, where they require a virtual environment. You’re also able to sort the robots according to when they were added or alphabetically. If you want something specific, you can search up that exact embodiment as well. For this walkthrough, we are going to use the brain pong embodiment. 3. Choose a genome that evolves to become your robot brain. You can filter and sort these options as well. You’re able to see the complexity of each genome by viewing the neuron and synapse count. You’re also able to see its original genealogy by clicking on the branching icon. For this pong simulation, we are going to choose the essential genome it’s important to note that you are able to easily change your selected genome once you enter the brain visualizer. 4. Name and launch your experiment! Please wait a moment for your brain to load.
Making a New Experiment
In this tutorial, we are going to walk through how to operate Neurorobotics Studio, using an autonomous and manual brain pong example. If you are new to Neurorobotics Studio, you will be presented 2 options: view the tutorials or create an experiment. Select make a new experiment. If you have previously created an NRS experiment, your screen will look like this. Click the same button in the upper right corner. 1. Select the environment for your robot. Physical environments typically involve an experiment using tangible robots. Virtual environments are used for computer generated characters. In the case you don’t have a physical robot, select virtual. 2. Select your embodiment. You are able to filter the embodiments by different features, such as in this case, where they require a virtual environment. You’re also able to sort the robots according to when they were added or alphabetically. If you want something specific, you can search up that exact embodiment as well. For this walkthrough, we are going to use the brain pong embodiment. 3. Choose a genome that evolves to become your robot brain. You can filter and sort these options as well. You’re able to see the complexity of each genome by viewing the neuron and synapse count. You’re also able to see its original genealogy by clicking on the branching icon. For this pong simulation, we are going to choose the essential genome it’s important to note that you are able to easily change your selected genome once you enter the brain visualizer. 4. Name and launch your experiment! Please wait a moment for your brain to load.
Brain Visualizer
Upon entering the brain visualizer, you will be presented with the Quickplay screen. It breaks down the purpose of each button in NRS and how to start an experiment. To navigate the brain visualizer: Right click and hold to rotate Scroll or pinch in to zoom in and scroll or punch out to zoom out, according to whether you have a mouse or touchpad, respectively Left click and hold to move left and right or up and down You can increase or decrease the font size with the big and little uppercase A’s in the “create and edit” toolbar. This is the circuit builder view, in which you can drag outputs to inputs to establish connections. Navigate by holding the center square in the bottom left control pad You can also put the brain visualizer and circuit builder views side by side. You can switch views, by clicking the neighboring tab in the same toolbar. After getting comfortable with the BV layout, exit out of that tab to view the start of your robot brain based on the essential genome. Here you can see: The brain power and axis indicator located at the origin. If the axis is red, that means the 3D brain monitoring environment is connected with the brain. The axis marks the origin within the 3D environment and will be your reference point when adding and changing the position of any object within the 3D environment. The brain power is a special brain region that is always on and indicates brain is capable of firing neurons. The teal meshed structures are called cortical areas and can consist of one or many blocks where each block can contain one or more neurons. -The central vision cortical area that feeds the visual cues to the brain. To view the details of any of these neuron blocks, click on it to open the QuickView and click on the magnifying glass to open the cortical area details. As you can see, this input cortical area has dimensions of 64x64x3. It has a depth of 3 to allow it to see in color where neurons in each layer are sensitive to Red, Green, and Blue intensity. This topic is covered in depth in the vision cortical areas tutorial. The 9 cortical areas surrounding the central vision capture activations from peripheral vision. The motor_output acts like motor neurons enabling the control of a rotary motor. It will not be applicable to the pong game.
Motion Control
To make the pong paddle move, we will need to create a motion control cortical area. 1. Click the add a cortical area button in the top left. 2. Select the output type because we want to use the motion control to send signals (otherwise known as the left to right movement) to our actuator (which is the paddle.) 3. Then select the motioncontrol option. 4. By default, the dimensions are 4x3x10, The width of four corresponds to left, up, down, and right The height of 3 corresponds to cardinal, yaw, roll, and pitch directions, but this will be covered more in the actuators cortical areas tutorial. The depth of 10 corresponds to the speed of each movement, based on a % ranging from 0-100. The coordinate 0,0,0 represents the power, so the motor will not move even when it fires, but 0,0,1 will result in a very subtle movement, while 0,0,9 is the fastest speed. To make things simple, we are going to change the height to 1. The 0,0,0 position results in a left movement, while 3,0,0 makes the embodiment move right 1,0,0 corresponds to up and 2,0,0 to down, but since the pong paddle only moves left to right, we do not need to use those blocks. You are also able to adjust its location. Its position does not affect its performance, but rather makes it easier to organize and make visible. To learn more about motion control cortical area please refer to the actuators tutorial.*
Accelerometer
To track the current position of the ball, you can build an accelerometer sensor. 1. Once again, add a cortical area, but this time select input and choose the accelerometer option. 2. By default, the accelerometer has dimensions of 3x1x21. The width of 3 corresponds to the tracking along the x, y, and z positions. Since we are only having the paddle respond to what side the ball is on, we can make the width 1. The depth of 21 also responds to the % of intensity, but the middle number of 0,0,10 is the reset position. 0,0,21 is a position all the way to the right, while 0,0,0 is a position all the way to left To connect the pong game to your experiment, click the embodiment tab and press the “launch button.” From this tab you are also able to replace your embodiment with another type or integrate it as well. Follow the directions in the pop-up window to copy and paste the link to the brain pong embodiment in a new tab.
Connectivity Rules
There are 2 different ways to play the game via FEAGI, which are manual activation and circuit design. To build a circuit so FEAGI can play the game on its own, we first need to come up with a strategy. One option is to use information about the ball distance from the wall to control paddle position. We have repurposed the accelerometer sensor to convey such information. Next, we need to come up with a connectivity rule that would help us map the accelerometer sensor to motion control area so neuronal activations related to ball distance from the wall can drive the paddle. We are going to use circuit builder, but if you were to make a connectivity rule in the brain visualizer: 1. Click on the source cortical area, then select the branching icon to open quick connect. 2. Then select your destination cortical area. 3. Click the pencil to edit the connectivity rule and finally establish the connection. In the circuit builder, 1. First, click the add connectivity rules button in the top left. 2. Select the pattern type 3. We are going to start with a right movement. Select add a pattern pair. 4. From here, you will be able to edit source and destination coordinates. As we previously discussed, 0,0,21 indicates the rightmost position of the ball and 0,0,10 is in the middle, so we are going to map every number over 11 to motion control’s right position 3,0,5. 5. Next, we are going to create a connectivity rule for a left movement. 6. Add a pattern pair. As we previously discussed, 0,0,0 indicates the leftmost position of the ball and 0,0,10 is in the middle, so we are going to map every number under 9 to motion control’s right position 0,0,5. Let’s connect these rules from the accelerometer to the motion control. In the circuit builder tab, drag a line from the accelerometer sensor source to the motion control input. The edit mappings pop-up will appear. You can then add connectivity rules and customize their properties, such as making it plastic or excitatory, but this is covered in more detail in the connectivity rules tutorial. Hit update to initiate the changes. Then navigate back to the brain visualizer to see your pong game in action!
Manual Pong
Time to create a manual game of pong. 1. Go back into the circuit builder view to disconnect the previously established rules. The trash icon deletes the connectivity rules. 2. To manually activate the relevant cortical area so you can control the robot, shift click a box. This same box should turn green, when you hit the space bar the box should flash red to indicate activation. 3. To deselect the box, shift click again. You could activate both directions at the same time, but it would result in the paddle staying in place in the center.
