Shapes and predictions questions

kefahB: <p>Hello Community,</p>
<p>I am a brand new to the AI world and I am trying to lean an Agent to play a card game for 4 player, and I thought that I can maybe share how I do thinks to know if am I on a good path <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>Suits:<br>
Since all suite has same power I use here one-hot-encoding:$</p>
<pre data-code-wrap="py"><code class="lang-py">diamonds = (1,0,0,0)
spades = (0,1,0,0)
hearts = (0,0,1,0)
clubs = (0,0,0,1)
</code></pre>
<p><strong>Card encoding</strong>:<br>
the game has 52 cards with 4 suits (diamonds, spades, hearts, clubs) since the cards has a power from its number I thought that I shouldn’t use the one-hot-encoding<br>
1st digit represent if the card in the agent hand<br>
2nd digit represent if the card has been played<br>
3d to 6th the card suite<br>
7th and last the card number</p>
<pre data-code-wrap="py"><code class="lang-py">
cards = [
   (1,0,1,0,0,0,12), 
   ...
]

</code></pre>
<p><strong>About the states:</strong><br>
I have a NN with this shape  <code>Linear_QNet(7, 1024, 1)</code> so the state for instance are all the agent cards and all the other players cards, and some information about the length of each suit on the agent hand and all the played cards</p>
<p><strong>Question 1:</strong> If I did not add 6 zeros to <code>game.get_player_clubs_count()</code> to unify the size as the card tuple is of length of 7, I will get and <code>inhomogeneous parts</code> is this a good practice to add zero like that ? or there are some other strategy</p>
<p><strong>Question 2:</strong> it is about the loop below the states, I do same as Question 1 but for the rows this times, tensor throw an error if the shape of each row is not equal to the first one. If you wonder why the shape change, it is because when a card is played I remove it from the player card. Also <code>(game.get_player_clubs_count(self.current_player),0,0,0,0,0,0)</code> has a row shape = 1</p>
<pre data-code-wrap="py"><code class="lang-py">
state = [
            *player_1_cards, # 13 tuples as initial state
            *(*player_2_cards, # 13 tuples as initial state
            *player_3_cards, # 13 tuples as initial state
            *player_4_cards), # 13 tuples as initial state
            (game.get_player_clubs_count(self.current_player),0,0,0,0,0,0),
            (game.get_player_diamonds_count(self.current_player),0,0,0,0,0,0),
            (game.get_player_hearts_count(self.current_player),0,0,0,0,0,0),
            (game.get_player_spades_count(self.current_player),0,0,0,0,0,0),
            *game.get_played_cards()
]
        
# This is necessary to normalize the shape of the states for later use with tensors
state = list(state).copy()
for i in range(54):
    if len(state) &lt; 54:
       state.append(( -1,  -1,  -1,  -1,  -1, -1, -1))
                
return np.array(state, dtype=int)

</code></pre>
<p><strong>Lastly, Predictions:</strong><br>
The final move should be a legal move, my question here it is about the <code>argmax</code> in the code below I use argmax but I reduce the predictions to the length of the legal moves.</p>
<p><strong>Question 1:</strong> does the predictions output has the shape of the states ? I mean, here I have 9 * 54 = 486, the first 13 items of them are the agent cards, does the prediction output will map the states and the first 13 item they will be the agent cards ?</p>
<p><strong>Question 2:</strong> I think I must update the prediction according to the legal move, should I put <code>float(-inf)</code> for all the predictions that are not a legal move ?</p>
<pre data-code-wrap="py"><code class="lang-py">legal_moves, is_legal_move = game._get_legal_move_bin(game.current_collection_type, self.current_player)
_state = np.array(state)
state0 = torch.tensor(_state, dtype=torch.float)
prediction = self.model(state0)
move = torch.argmax(prediction[:len(legal_moves)]).item()
final_move = legal_moves[move]
</code></pre>