Developing Imitation Learning with FEAGI

Introduction

While this biologically informed approach allows FEAGI to handle complex tasks intuitively by utilizing separate cortical areas (e.g., visual fields and motor connections), it also presents challenges. The system is still in its infancy, with ongoing development to address bugs and implement new features. Despite these limitations, the framework shows great promise.

The goal of our project was to develop a model capable of mimicking human players and learning to play games by understanding human actions. Due to technical challenges and the complexity of designing such circuits, we successfully implemented a human-mimicking model in a controlled environment. While progress toward achieving a fully functional learning model was limited, the foundational work establishes a strong basis for future improvements.

The implications of this project extend beyond gaming. FEAGI's potential applications include robotics, where robots could learn tasks independently rather than relying on preprogrammed behaviors. For instance, instead of a robot designed solely to make toast by walking to the bread box, taking bread and putting it in a toaster; a learning-capable robot could adapt to walk a dog, clean a yard, or care for a sick individual, significantly increasing its utility.

The below pictures showcase FEAGI's capabilities for modelling complex biological principles.

This first image shows simple neuron firing mechanics that can be tweaked inside of FEAGI. To a neuroscientist these things are common, but for computer scientists they provide extra helpful tools in tweaking and developing ai models.

Another set of neuron parameters, this controls how a neruon acts with its downstream neighbors. These parameters allow certain cortical areas to immediately have more or less weight in a model than would regularly be expected.

This third image shows, in part, how connections are formed between different areas. We can see the areas that are inputs and outputs to this cortical area, and by clicking any of these areas you would also see the rules that define these connections.

The final image is of part of the connection view of our final product and shows how complex these interactions can become. Every green line is a positive static input, red is negative, and the dashed lines that begin to appear at the right are plastic connections that are tuned over time in accordance with spike time dependent plasticity.

Development

Motor Babbling

The motor babbling phase is designed to teach the AI agent how its inputs affect its environment. Humans develop expectations about actions, such as pressing an "up" button to move upward, through experience. The AI agent replicates this learning process by randomly pressing inputs and observing their effects on the virtual world.

Motor babbling enables the agent to develop an understanding of its potential actions within minutes. However, one challenge arises from differentiating dual inputs. For example, if pressing "up" and "left" simultaneously moves the character diagonally, the agent initially struggles to associate each button with its respective direction. Additional training is required to resolve this ambiguity. Currently, initiating and stopping motor babbling is manual, but future iterations could automate this process using degrading neural connections between the randomizer and the rest of the circuit.

Mimicry

The mimicry component is designed to interpret human actions and replicate them. We developed two versions of this circuit:

1. Stimulus Decoding: The initial version analyzed the human visual field and decoded the specific inputs used by the human to perform actions. While this method produced discrete instructions for the AI agent, it was computationally intensive, prone to over/underestimation, and highly susceptible to noisy environments.
2. Positional Mimicry: The improved version focuses on determining the human player's position relative to the AI agent. Instead of decoding exact inputs, the AI agent learns to mimic the human's positioning. For example, if the human player moves left to reach a target, the agent mirrors this movement. This approach is faster, more efficient, and mimics how humans naturally adapt to one another's actions. However, to refine its understanding, the agent must also interpret the human player's environment, such as recognizing that the target object was to the left of the human player.

Future Work

The ultimate goal remains to create a model capable of mimicking and learning from human actions. The next steps for this project include:

1. Object Discrimination: The AI agent must differentiate between objects on the screen. During the motor babbling stage, it would first identify itself within the environment. Subsequently, it would track other objects, learning to determine their significance and adjust its actions based on human interactions with those objects.

1. Enhanced Environmental Awareness: Building on the mimicry component, the agent needs to develop a deeper understanding of object interactions. For instance, if a ball is to the left of the human player and the player moves left to interact with it, the agent must understand both the human's movement and the reason behind it.
2. Autonomous Learning: Future iterations will aim to make the AI agent fully autonomous, enabling it to learn and adapt without manual intervention. This involves further refining circuits for motor babbling and mimicry, as well as integrating additional biologically inspired learning mechanisms.

Conclusion

FEAGI represents a significant step toward biologically inspired artificial intelligence, offering the potential for more intuitive and adaptive learning systems. While challenges remain, our foundational work on human mimicry demonstrates the feasibility of using FEAGI to replicate human actions in virtual environments. With continued development, this approach holds promise for applications in robotics and beyond.