p3nGu1nZz commited on
Commit
4f4ceca
1 Parent(s): 5e8dfcd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +111 -31
README.md CHANGED
@@ -10,46 +10,126 @@ tags:
10
  - lstm
11
  ---
12
 
13
- # Kyle - Robot Walker Ragdoll Trainer
14
 
15
- ## Model Description
16
- Kyle is a state-of-the-art active ragdoll training environment in Unity, featuring a heavily modified and optimized codebase from the original Ragdoll Trainer. Our version includes advanced vision capabilities and short-term variable memory using LSTM (Long Short-Term Memory) networks, providing a more sophisticated simulation for AI training.
17
 
18
- ## How to Use the Model
19
- To ensure proper functionality of the Ragdoll Trainer within the Neko Cat Game, it is essential to utilize the **skeletal armature GameObject** provided with our modified project. This GameObject is specifically tailored for our Ragdoll Trainer and can be found as a child object within the main project directory at `https://github.com/cat-game-research/Neko/tree/main/RagdollTrainer`.
20
 
21
- 1. Import the ONNX model into your Unity project.
22
- 2. Attach the model to the skeletal armature GameObject included with our modified Ragdoll Trainer project.
23
- 3. Utilize the model's capabilities to train and simulate realistic ragdoll movements with enhanced cognitive functions.
 
24
 
25
- ## Training Data
26
- The model was trained on a comprehensive dataset that includes various ragdoll movements, enriched with visual inputs and memory components to simulate real-world scenarios effectively.
27
 
28
- ## Evaluation Results
29
- Kyle has undergone extensive testing, showcasing remarkable proficiency in simulating lifelike ragdoll physics with the added depth of vision and memory-based learning.
30
 
31
- ## Ethical Considerations
32
- We adhere to ethical AI development practices, ensuring our models are created with fairness, accountability, and transparency at their core.
33
 
34
- ## Acknowledgements
35
- We extend our gratitude to the creators of the original Ragdoll Trainer project for their foundational work. Our enhancements were made possible by building upon their innovative approach to active ragdoll training.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
 
37
- ## Additional Information
38
  - **Author**: p3nGu1nZz
39
- - **Unity Version**: 2023.2.7f1
40
- - **ONNX Version**: 1.12.0
41
- - **ML-Agents Version**: 3.0.0-exp.1
42
- - **ML-Agents Extensions Version**: 0.6.1-preview
43
- - **Sentis Version**: 1.3.0-pre.2
 
 
 
44
  - **Features**:
45
- - Enhanced rig created in Blender for more natural movement.
46
- - Heuristic function for joint control during development testing.
47
- - Stabilizers for hips and spine to aid balance.
48
- - Early training settings for initial balance and walking towards targets.
49
- - Navigation around obstacles using Ray Perception Sensor 3D.
50
- - Training for navigating steps and stairs with varying difficulty.
 
51
  - **Setup Process**:
52
- - Local Miniconda installation and configuration.
53
- - Cloning of the latest tag of ML-Agents into the project.
54
- - Script configuration for fine-tuning models based on Kyle.
 
 
 
55
 
 
 
 
 
 
 
 
 
 
 
 
10
  - lstm
11
  ---
12
 
13
+ # Kyle - The Pinnacle of Robot Walker Ragdoll Training
14
 
15
+ Kyle represents the zenith of active ragdoll training environments, meticulously crafted within the Unity framework. This innovative platform is the result of extensive enhancements and optimizations applied to the foundational codebase of the renowned Ragdoll Trainer. By integrating cutting-edge vision technology and leveraging the robustness of LSTM (Long Short-Term Memory) networks, Kyle offers an unparalleled simulation experience that pushes the boundaries of AI training.
 
16
 
17
+ ## In-Depth Model Description
18
+ At the heart of Kyle lies a revolutionary active ragdoll training environment, engineered to simulate complex physical interactions with an unprecedented level of realism. The core architecture of Kyle is derived from the original Ragdoll Trainer, which has been heavily modified and refined to achieve optimal performance.
19
 
20
+ The enhancements include:
21
+ - **Advanced Vision Capabilities**: Kyle's sensory apparatus is equipped with state-of-the-art visual processing features that enable the recognition and interpretation of intricate visual cues within the training environment.
22
+ - **Adaptive Short-Term Variable Memory**: Utilizing LSTM networks, Kyle boasts a dynamic memory system capable of capturing and utilizing temporal information, which is crucial for tasks that require an understanding of sequences and patterns over time.
23
+ - **Sophisticated Simulation for AI Training**: The combination of these technologies provides a rich and complex simulation platform. It allows for the training of AI models that can navigate and interact with their surroundings in a manner akin to sentient beings, paving the way for advancements in the field of artificial intelligence.
24
 
25
+ By fostering a more sophisticated simulation environment, Kyle empowers researchers and developers to train AI models that can perform a diverse array of tasks, from simple locomotion to navigating intricate obstacle courses, all while responding to an ever-changing array of stimuli and challenges.
 
26
 
27
+ ![Kyle The Robot](results/images/kyle_b0a_001.png "Kyle The Robot")
 
28
 
29
+ ## Comprehensive Guide on Utilizing Kyle
 
30
 
31
+ Kyle is not just a model; it's a gateway to a new realm of AI-driven ragdoll physics in gaming. To harness the full potential of Kyle within the Neko Cat Game, it's imperative to integrate the **skeletal armature GameObject** that accompanies our project. This GameObject is the linchpin of our Ragdoll Trainer, meticulously designed to work in harmony with Kyle's advanced functionalities.
32
+
33
+ ### Step-by-Step Usage Instructions
34
+
35
+ 1. **Incorporating the ONNX Model**:
36
+ Begin by importing the ONNX model file into your Unity project. This file encapsulates the essence of Kyle's learning and is the first step towards a new dimension of ragdoll simulation.
37
+
38
+ 2. **GameObject Integration**:
39
+ Seamlessly attach the model to the skeletal armature GameObject found within our project. This GameObject is the skeleton key to unlocking Kyle's capabilities, ensuring that the ragdoll movements are not just realistic but also cognitively rich.
40
+
41
+ 3. **Simulation and Training**:
42
+ With the model and GameObject in place, you're now equipped to train and simulate nuanced ragdoll movements. Kyle's cognitive functions, powered by LSTM networks, allow for a simulation that's not just reactive but also anticipatory, adapting to the environment with a semblance of understanding.
43
+
44
+ ### Unity Asset for Inference
45
+
46
+ In the spirit of accessibility and ease of use, we're excited to announce that a Unity asset will soon be available for download. This asset will enable users to implement Kyle's inference capabilities directly within their projects, bypassing the complexities of the development setup.
47
+
48
+ ### Future Developments
49
+
50
+ While training Kyle currently necessitates a basic development setup involving ML-Agents and a Python environment, we're diligently working on streamlining this process. Our vision is to introduce an automated feature within the game that will eliminate the need for manual setup of ML-Agents and the Python environment. This upcoming feature will empower users to train Kyle with the simplicity of a few clicks, making advanced AI training accessible to all.
51
+
52
+ Stay tuned for these exciting updates, as we continue to refine and enhance the user experience, making it as intuitive and user-friendly as possible.
53
+
54
+ ## Training Data: A Foundation Built on Diversity and Complexity
55
+
56
+ The training regime of Kyle is a testament to the diversity and complexity of the data it was exposed to. The model's foundation is built upon a dataset that encompasses a wide array of ragdoll movements, each enriched with visual inputs and memory components. This rich dataset ensures that Kyle can simulate real-world scenarios with remarkable effectiveness, allowing it to understand and predict physical interactions in a dynamic environment.
57
+
58
+ The current iteration of Kyle has been trained through a sequence of 20 million steps, a journey that has imbued it with the finesse to handle a multitude of tasks. As we continue to evolve our training scenes, we plan to release additional sequences. These will serve as building blocks that can be combined to forge new models, each specializing in different fitness applications, from the delicate balance required for acrobatics to the precision needed for sorting tasks.
59
+
60
+ ## Evaluation Results: Demonstrating Mastery Over Movement
61
+
62
+ Kyle's proficiency in simulating lifelike ragdoll physics is not just a claim; it's a proven fact. The model has undergone extensive testing, revealing its remarkable ability to not only mimic but also understand the nuances of movement. The added depth of vision and memory-based learning allows Kyle to anticipate and adapt, showcasing a level of cognitive function rarely seen in simulated environments.
63
+
64
+ ### TensorBoard Results: Visual Testaments to Progress
65
+
66
+ The TensorBoard visualizations offer a window into Kyle's training journey and the strides it has made:
67
+
68
+ ![TensorBoard Results 1](results/images/tensorboard_kyle-b0a_001.png "Training Results 1")
69
+ *Figure 1: A graph depicting the training progress over time, illustrating a consistent reduction in loss and a corresponding improvement in accuracy.*
70
+
71
+ ![TensorBoard Results 2](results/images/tensorboard_kyle-b0a_002.png "Training Results 2")
72
+ *Figure 2: Validation results that shine a light on the model's ability to generalize, indicating robust performance on unseen data.*
73
+
74
+ ![TensorBoard Results 3](results/images/tensorboard_kyle-b0a_003.png "Training Results 3")
75
+ *Figure 3: A comparative analysis that lays bare the model's predictions against the ground truth, underscoring the precision of Kyle's learning process.*
76
+
77
+ ### The Kyle Model: A Base for Customization
78
+
79
+ At its core, Kyle is a base biped model designed for direct customization to fit the unique requirements of your project. Whether you aim to fine-tune Kyle for a fighting simulation, choreograph a dance of acrobatics, or develop a system for sorting objects, the model's adaptable nature makes it an ideal starting point. The potential applications are as varied as they are exciting, and we eagerly anticipate the innovative ways in which Kyle will be utilized.
80
+
81
+ ## Ethical Considerations: Commitment to Responsible AI
82
+
83
+ In the development of Kyle, we have steadfastly upheld the principles of ethical AI. Our commitment to creating models that embody fairness, accountability, and transparency is unwavering. We recognize the profound impact AI can have on society, and we strive to ensure that our models contribute positively, enhancing the lives of those they touch.
84
+
85
+ - **Fairness**: We have taken rigorous steps to prevent biases in our model, ensuring that Kyle's learning and outputs are equitable and just.
86
+ - **Accountability**: Our development process is meticulously documented, providing clear traceability and fostering a culture of responsibility.
87
+ - **Transparency**: We believe in the open exchange of ideas and methodologies, which is why we are transparent about our development practices and the capabilities of our models.
88
+
89
+ ## Acknowledgements: A Tribute to Collaborative Innovation
90
+
91
+ We owe a debt of gratitude to the visionaries behind the original Ragdoll Trainer project. Their pioneering work laid the groundwork for what Kyle has become today. By building upon their innovative approach to active ragdoll training, we have been able to push the boundaries and achieve new heights.
92
+
93
+ - **Foundational Work**: The original Ragdoll Trainer's codebase served as the foundation upon which we built and optimized Kyle.
94
+ - **Based On**: Orginal project work and experiment discovered at https://github.com/kressdev/RagdollTrainer
95
+ - **Collaborative Spirit**: We celebrate the collaborative spirit of the AI and gaming communities, which has been instrumental in our project's success.
96
+
97
+ ## Additional Information: The Pillars of Kyle's Development
98
+
99
+ The development of Kyle is characterized by a blend of cutting-edge technology and meticulous craftsmanship.
100
 
 
101
  - **Author**: p3nGu1nZz
102
+ - **Github**: https://github.com/cat-game-research/Neko/tree/main/RagdollTrainer
103
+ - **Technological Stack**:
104
+ - **Unity Version**: 2023.2.7f1 - The canvas upon which Kyle's world comes to life.
105
+ - **ONNX Version**: 1.12.0 - Ensuring interoperability and state-of-the-art performance.
106
+ - **ML-Agents Version**: 3.0.0-exp.1 - The backbone of Kyle's learning capabilities.
107
+ - **ML-Agents Extensions Version**: 0.6.1-preview - Extending functionality and pushing the envelope of what's possible.
108
+ - **Sentis Version**: 1.3.0-pre.2 - Empowering Kyle with the ability to perceive and interact with its environment.
109
+
110
  - **Features**:
111
+ - **Enhanced Rig**: Crafted in Blender, this rig provides Kyle with fluid and natural movement.
112
+ - **Heuristic Function**: A tool for joint control that was pivotal during development testing.
113
+ - **Stabilizers**: Implemented for hips and spine, these stabilizers are key to Kyle's balance.
114
+ - **Training Settings**: Early training settings were crucial for establishing initial balance and target-oriented movement.
115
+ - **Obstacle Navigation**: Using Ray Perception Sensor 3D, Kyle can adeptly navigate around obstacles.
116
+ - **Step and Stair Training**: Kyle has been trained to navigate steps and stairs, adjusting to varying levels of difficulty.
117
+
118
  - **Setup Process**:
119
+ - **Local Miniconda**: The starting point for setting up the development environment.
120
+ - **Cloning ML-Agents**: Integrating the latest advancements from ML-Agents into Kyle.
121
+ - **Script Configuration**: Tailoring scripts to fine-tune models based on Kyle's unique requirements.
122
+
123
+ Each element of Kyle's development has been carefully considered and implemented to ensure that the model not only meets but exceeds the expectations of those who use it.
124
+
125
 
126
+ ```
127
+ @misc{RagdollTrainer,
128
+ author = {cat-game-research},
129
+ title = {RagdollTrainer - The Evolution of Active Ragdoll Simulation in Unity},
130
+ year = {2024},
131
+ publisher = {GitHub},
132
+ journal = {GitHub repository},
133
+ howpublished = {\\url{https://github.com/yourusername/LocalLlama}}
134
+ }
135
+ ```