The MindViewer is a utility class that allows users to visualize information about the contents of an agent’s mind during execution. It is easily integrated into an existing code, facilitates debugging during development and provides an interface for monitoring and presenting results.
Basic Notions
The visualization interface of MindViwer is divided into two main panels. The panel above displays all of the mind’s entities (memories and codelets) as well as the activation levels of a list of codelets defined during class instantiation. The panel below focuses on codelets for the motivational system, displaying drives activations and codelets.
Figure 1 - MindViwer's main interface.
In the Memories tab of the Mind’s Entities panel it is possible to visualize all memory objects present on the mind alongside its associated information and evaluation in parenthesis. By right-clicking on a memory item on the tree, options for inspecting the memory are available in the form of a tree or table, where detailed information about the memory can be accessed, such as latest timestamp, object parameters and the information stored.
Figure 2 - Panel for structuresd visualization of memories.
The Codelets tab shows all codelets of the agent’s mind organized by the codelets groups defined in code. Expanding the tree is possible to visualize the memories assigned as input, output or broadcast for a codelet, represented by a left arrow, right arrow and yellow circle respectively.
Figure 3 - Panel for structuresd visualization of codelets.
On the CodeRack Inspection tab a graph shows the activation values of a list of codelets in real time. The update can be paused by unchecking the auto-refresh box and the slide bar on the bottom allows the inspection of the logged activations values.
Figure 4 - Code Rack Inspection graph.
The Motivational Subsystem panel is divided into various tabs for displaying specific codelets and its associated memories and activations.
Figure 5 - Drives tab of motivational subsytem panel.
Usage
The following example demonstrates how to integrate the MindViewer class with a test application. The source code can be accessed on the MindViwerTest github repository and requires cst-desktop repository as dependency, which can be set following this instructions.
The code consists of two basic classes Main and TestCodelet used to initialize a test architecture containing codelets, that varies its activation between 0 and 1, and memories, that store float numbers and are incrementally updated. The focus of the example is on the prepareMind and createAndShowGUI functions present on the Main class, responsible for setting up the agent’s mind and mind viewer.
For better debugging and architecture organization is used codelet groups and memory groups, which separates the agent’s codelets and memories into lists indexed by a string name. To initialize the groups the commands createCodeletGroup and createMemoryGroup from Mind are used and must receive a string with the group name. After that codelets and memories can be assigned to groups with the registerCodelet or registerMemory commands from Mind, passing the object and the group name. Additionally, codelets can be assigned during insertion in the agent’s mind using the command insertCodelet and passing the codelet and group name.
Below is shown the code from the Mind Viewer Test application responsible for setting the agent’s mind. It creates 5 codelet groups (sensory, perception, behavioral, motivational and motor) and 2 memory groups (standard memories and containers), to which are added some codelets and memories to demonstrate the MindViewer.
public Mind prepareMind() {
m = new Mind();
m.createCodeletGroup("Sensory");
m.createCodeletGroup("Perception");
m.createCodeletGroup("Behavioral");
m.createCodeletGroup("Motivational");
m.createCodeletGroup("Motor");
m.createMemoryGroup("StandardMemories");
m.createMemoryGroup("Containers");
MemoryObject m1 = m.createMemoryObject("M1", 1.12);
m.registerMemory(m1,"StandardMemories");
MemoryObject m2 = m.createMemoryObject("M2", 2.32);
m.registerMemory(m2,"StandardMemories");
MemoryObject m3 = m.createMemoryObject("M3", 3.44);
m.registerMemory(m3,"StandardMemories");
MemoryObject m4 = m.createMemoryObject("M4", 4.52);
m.registerMemory(m4,"StandardMemories");
MemoryObject m5 = m.createMemoryObject("M5", 5.12);
m.registerMemory(m5,"StandardMemories");
MemoryContainer m6 = m.createMemoryContainer("C1");
m.registerMemory(m6,"Containers");
MemoryContainer m7 = m.createMemoryContainer("C2");
m.registerMemory(m7,"Containers");
int mc1 = m7.setI(7.55, 0.23);
int mc2 = m6.setI(6.33, 0.22);
int mc3 = m6.setI(6.12, 0.13);
int mc4 = m6.add(m7);
Codelet c = new TestCodelet("Sensor1");
c.addInput(m1);
c.addInput(m2);
c.addOutput(m3);
c.addOutput(m4);
c.addBroadcast(m5);
m.insertCodelet(c,"Sensory");
Codelet c2 = new TestCodelet("Motor1");
c2.addInput(m4);
c2.addInput(m5);
c2.addOutput(m6);
c2.addOutput(m3);
c2.addBroadcast(m5);
m.insertCodelet(c2,"Motor");
Codelet mot1 = new TestCodelet("Curiosity");
mot1.addInput(m7);
mot1.addOutput(m5);
m.insertCodelet(mot1,"Motivational");
Codelet mot2 = new TestCodelet("Fear");
mot2.addInput(m3);
mot2.addOutput(m4);
try {mot2.setActivation(1.0);} catch(Exception e){}
m.insertCodelet(mot2,"Motivational");
Codelet mot3 = new TestCodelet("Anger");
mot3.addInput(m1);
mot3.addOutput(m2);
try {mot3.setActivation(0.5);} catch(Exception e){}
m.insertCodelet(mot3,"Motivational");
m.start();
return(m);
}
After organizing the codelets and memories into groups the MindViewer class is easily set up by calling its constructor method and passing the mind instance, a name for the window that will be displayed and a list of codelets to be plotted in the CodeRack Inspection. Below is shown the code of createAndShowGUI function, where the MindViewer is instantiated and set visible.
private void createAndShowGUI(Mind m)
MindViewer mv = new MindViewer(m,"MindViewer",m.getCodeletGroupList("Motor"));
mv.setVisible(true);
}
When executing the application the window shown below will appear, where the agent’s mind can be inspected.
Figure 6 - MindViwer operation for the test application.
Custom Views
A custom view for inspecting a codelet or memory may also be added programatically. Codelets or memories that implement the Inspectable interface from cst-desktop package can be discovered by the MindViewer class and an option to inspect the object is added to the dropdown menu shown when right-clicking it from the MindViwer panel.
The Inspectable interface has only one function to be implemented: inspect(). It is expected that the implementation of this function will create a JForm object containing the desired custom view of the codelet or memory data. It is good practice to create a class inheriting the JForm class and define the custom view in it. Then, an instance of this class can be created on the constructor method from the codelet or memory that will use it. Therefore, the implementation of the function inspect() will consist simply of the setVisible(true) call from the custom view instance.
As example, a simple custom view implementation for the above MindViwerTest application to display a text indicating if the activation of a codelet is increasing or decreasing is shown below.
First a MyCustomView class, inheriting JForm, is created. This class contains the necessary code for creating a window displaying a text and, since it is not the focus of this tutorial, the inspection and understanding of this is left to the reader. Additionally, the class contains a helper function that allows to update the text being displayed.
public class CustomView extends javax.swing.JFrame {
public CustomView() {
initComponents();
}
private void initComponents() {
jLabel1 = new javax.swing.JLabel();
setDefaultCloseOperation(javax.swing.WindowConstants.EXIT_ON_CLOSE);
jLabel1.setFont(new java.awt.Font("Liberation Sans", 0, 20)); // NOI18N
jLabel1.setHorizontalAlignment(javax.swing.SwingConstants.CENTER);
jLabel1.setText("Ascending");
javax.swing.GroupLayout layout = new javax.swing.GroupLayout(getContentPane());
getContentPane().setLayout(layout);
layout.setHorizontalGroup(
layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
.addGroup(layout.createSequentialGroup()
.addGap(124, 124, 124)
.addComponent(jLabel1, javax.swing.GroupLayout.PREFERRED_SIZE, 139, javax.swing.GroupLayout.PREFERRED_SIZE)
.addContainerGap(137, Short.MAX_VALUE))
);
layout.setVerticalGroup(
layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
.addGroup(layout.createSequentialGroup()
.addGap(121, 121, 121)
.addComponent(jLabel1, javax.swing.GroupLayout.PREFERRED_SIZE, 49, javax.swing.GroupLayout.PREFERRED_SIZE)
.addContainerGap(130, Short.MAX_VALUE))
);
pack();
}
public void updateInfo(boolean ascending) {
if (ascending) {
jLabel1.setText("Ascending");
} else {
jLabel1.setText("Descending");
}
}
private javax.swing.JLabel jLabel1;
}After that, the codelet class is set to implement the Inspectable interface. A private variable of type MyCustomView is added and initialized inside the constructor method. Inside the proc function, an update of the custom view text is made whenever the value of the ascending variable changes. Finally, the inspect method is implemented with a call to customView.setVisible(True).
public class TestCodelet extends Codelet implements Inspectable{
boolean ascending = true;
CustomView customView;
public TestCodelet(String name) {
setName(name);
customView = new CustomView();
}
@Override
public void accessMemoryObjects() {
}
@Override
public void calculateActivation() {
double currentactivation = getActivation();
if (ascending) {
try {
setActivation(getActivation()+0.1);
} catch (CodeletActivationBoundsException e) {
ascending = !ascending;
customView.updateInfo(ascending);
}
}
else {
try {
setActivation(getActivation()-0.1);
} catch (CodeletActivationBoundsException e) {
ascending = !ascending;
customView.updateInfo(ascending);
}
}
}
@Override
public void proc() {
}
@Override
public void inspect() {
customView.setVisible(true);
}
}Exercise
As an exercise, try to extend the CoreModelTrail to use the MindViwer. Start by modifying the agent mind to structure codelets and memories into groups. Also create custom views for some codelets, for example maps with the known apples.