update

README.md

@@ -7,9 +7,11 @@ tags:
 - Reinforcement Learning
 - Vehicle Routing Problem
 ---
-![LOGO.png](./images/GREEDRL-Logo-Original-320x320.png)
+<div align="center">
+    <img src="./images/GREEDRL-Logo-Original-640.png" width = "515" height = "380"/>
+</div>
 
-# GreedRL
+# ✊‍GreedRL
 
 # Introduction
 
@@ -17,32 +19,130 @@ tags:
 ## Architecture design
 The entire architecture is divided into three layers:
 
-* High-performance Env framework
+* **High-performance Env framework**
 
-The constraints and optimization objectives for the problems to be solved are defined in the RL Env.
-Based on performance and ease of use considerations, the Env framework provides two implementations: one based on pytorch and one based on CUDA C++.
-To facilitate the definition of problems for developers, the framework abstracts multiple variables to represent the environment's state, which are automatically generated after being declared by the user. When defining constraints and optimization objectives, developers can directly refer to the declared variables.
-Currently, various VRP variants such as CVRP, VRPTW and PDPTW, as well as problems such as Batching and Online Assignment, are supported.
+    The constraints and optimization objectives for the problems to be solved are defined in the Reinforcement Learning(RL) Environment(Env).
+Based on performance and ease of use considerations, the Env framework provides two implementations:one based on **pytorch** and one based on **CUDA C++**.
 
-* Pluggable NN components
+    To facilitate the definition of problems for developers, the framework abstracts multiple variables to represent the environment's state, which are automatically generated after being declared by the user. When defining constraints and optimization objectives, developers can directly refer to the declared variables.
+Currently, various VRP variants such as CVRP, VRPTW and PDPTW, as well as problems such as Batching, are supported.
 
-The framework provides certain neural network components, and developers can also implement custom neural network components.
 
-* High-performance NN operators
+* **Pluggable NN components**
 
-In order to achieve the ultimate performance, the framework implements some high-performance operators specifically for OR scenarios to replace pytorch operators, such as masked addition attention and masked softmax sampling."
+    The framework provides certain neural network(NN) components, and developers can also implement custom neural network components.
 
-![Architecture](./images/GREEDRL-Framwork.png)
+
+* **High-performance NN operators**
+
+    In order to achieve the ultimate performance, the framework implements some high-performance operators specifically for Combinatorial Optimization(CO) problems to replace pytorch operators, such as the Masked Addition Attention and Masked Softmax Sampling."
+
+<div align="center">
+    <img src="./images/GREEDRL-Framwork.png" width = "515" height = "380"/>
+</div>
 
 ## Network design
-The network structure adopts the seq2seq architecture commonly used in NLP, with the Transformer used in the encoding part and RNN used in the decoding part, as shown in the diagram below.
+The neural network adopts the Seq2Seq architecture commonly used in Natural Language Processing(NLP), with the Transformer used in the Encoding part and RNN used in the decoding part, as shown in the diagram below.
+
+<div align="center">
+    <img src="./images/GREEDRL-Network.png" width = "515" height = "380"/>
+</div>
+
+## Modeling examples
+
+### VRP with Time Windows(VRPTW)
+<details>
+    <summary>VRPTW</summary>
+
+```python
+from greedrl import Problem, Solution, Solver
+from greedrl.feature import *
+from greedrl.variable import *
+from greedrl.function import *
+from greedrl.model import runner
+from greedrl.myenv import VrptwEnv
+
+features = [continuous_feature('worker_weight_limit'),
+            continuous_feature('worker_ready_time'),
+            continuous_feature('worker_due_time'),
+            continuous_feature('worker_basic_cost'),
+            continuous_feature('worker_distance_cost'),
+            continuous_feature('task_demand'),
+            continuous_feature('task_weight'),
+            continuous_feature('task_ready_time'),
+            continuous_feature('task_due_time'),
+            continuous_feature('task_service_time'),
+            continuous_feature('distance_matrix')]
+```
+</details>
+<details>
+
+```python
+variables = [task_demand_now('task_demand_now', feature='task_demand'),
+             task_demand_now('task_demand_this', feature='task_demand', only_this=True),
+             feature_variable('task_weight'),
+             feature_variable('task_due_time'),
+             feature_variable('task_ready_time'),
+             feature_variable('task_service_time'),
+             worker_variable('worker_weight_limit'),
+             worker_variable('worker_due_time'),
+             worker_variable('worker_basic_cost'),
+             worker_variable('worker_distance_cost'),
+             worker_used_resource('worker_used_weight', task_require='task_weight'),
+             worker_used_resource('worker_used_time', 'distance_matrix', 'task_service_time', 'task_ready_time',
+                'worker_ready_time'),
+             edge_variable('distance_last_to_this', feature='distance_matrix', last_to_this=True),
+             edge_variable('distance_this_to_task', feature='distance_matrix', this_to_task=True),
+             edge_variable('distance_task_to_end', feature='distance_matrix', task_to_end=True)]
+
+
+class Constraint:
+
+    def do_task(self):
+        return self.task_demand_this
+
+    def mask_task(self):
+        # 已经完成的任务
+        mask = self.task_demand_now <= 0
+        # 车辆容量限制
+        worker_weight_limit = self.worker_weight_limit - self.worker_used_weight
+        mask |= self.task_demand_now * self.task_weight > worker_weight_limit[:, None]
+
+        worker_used_time = self.worker_used_time[:, None] + self.distance_this_to_task
+        mask |= worker_used_time > self.task_due_time
+
+        worker_used_time = torch.max(worker_used_time, self.task_ready_time)
+        worker_used_time += self.task_service_time
+        worker_used_time += self.distance_task_to_end
+        mask |= worker_used_time > self.worker_due_time[:, None]
+
+        return mask
+
+    def finished(self):
+        return torch.all(self.task_demand_now <= 0, 1)
+
+
+class Objective:
+
+    def step_worker_start(self):
+        return self.worker_basic_cost
+
+    def step_worker_end(self):
+        return self.distance_last_to_this * self.worker_distance_cost
+
+    def step_task(self):
+        return self.distance_last_to_this * self.worker_distance_cost
+```
+
+</details>
+
+### Pickup and Delivery Problem with Time Windows(PDPTW)
 
-![network.png](./images/GREEDRL-Network.png)
 
 ## 🏆Award
 
 
-# GreedRL-VRP-pretrained model
+# 🤠GreedRL-VRP-pretrained model
 
 ## Model description
 
@@ -53,9 +153,7 @@ The network structure adopts the seq2seq architecture commonly used in NLP, with
 
 You can use these model for solving the vehicle routing problems (VRPs) with reinforcement learning (RL).
 
-## Usage
-
-You can use this model directly with a pipeline for masked language modeling:
+## How to use
 
 ### Requirements
 This library requires Python == 3.8. [Miniconda](https://docs.conda.io/en/latest/miniconda.html#system-requirements) / [Anaconda](https://docs.anaconda.com/anaconda/install/) is our recommended Python distribution.
@@ -64,25 +162,41 @@ This library requires Python == 3.8. [Miniconda](https://docs.conda.io/en/latest
 pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113
 ```
 
+You need to compile first and add the resulting library `greedrl_c` to the `PYTHONPATH`
+
+```aidl
+python setup.py build
+
+export PYTHONPATH={root_path}/greedrl/build/lib.linux-x86_64-cpython-38/
+```
+
 
 ### Training
 
+We provide examples of Capacitated VRP(CVRP) for training and inference.
+
 1. Training data
 
 We use the generated data for the training phase, the customers and depot locations are randomly generated in the unit square [0,1] X [0,1].
 
-For the Capacitated VRP(CVRP), we assume that the demand of each node is a discrete number in {1,...,9}, chosen uniformly at random.
+For the CVRP, we assume that the demand of each node is a discrete number in {1,...,9}, chosen uniformly at random.
 
 
 2. Start training
 ```python
-python train.py 
+cd examples/cvrp
+
+python train.py --model_filename cvrp_5000.pt --problem_size 5000
 ```
 
 ### Evaluation
 
+We provide some pretrained models for different CVRP problem sizes, such as `cvrp_100`, `cvrp_1000`, `cvrp_2000` and `cvrp_5000`, that you can directly use for inference.
+
 ```python
-python solve.py
+cd examples/cvrp
+
+python solve.py --device cuda --model_name cvrp_5000.pt --problem_size 5000
 ```
 
 ## Support