# Network Performance Optimization for Low-Voltage Power Line Communications
## 3 LVPLC Network Scheme
### Improved \(Q\) Learning Approach in LVPLC Network Scheme
#### 3.2.2 Improved \(Q\)-Learning Algorithm in LVPLC Network
The receivers reply to the sender with an ACK after waiting for the response inter frame space (RIFS). The CCo selection is a success if the station successfully receives the ACK. The state machine of the station becomes UC_CCO from INIT. The state machines of the other stations become UC_STA from INIT. If the CCo selection fails, the abovementioned mechanism is repeated until the selection is successful. * CCo \(q\) first allots a beacon slot for itself, as depicted in Figure 4a. In the SLOT\({}_{0}\), [PERSON] \(q\) broadcasts a beacon frame to the other stations. The other stations delay for a short and random period of time. They then self-schedule the ASSOC_REQ frame, access the channel by CSMA, and send it to CCo \(q\). A frame collision occurs if at least two stations simultaneously access the channel to send a frame. The stations again delay for a short and random period of time and sense the channel state. The station retransmits the ASSOC_REQ frame to CCo \(q\) if the channel is idle. * CCo \(q\) directly replies to the ACK to guarantee the frame transmission success, which puts the stations in the routing table, learn the link to stations, and completes the forward path learning. Station a, which has received the ACK, establishes a connection with CCo \(q\). Similarly, stations c, e, d, and b establish a connection with CCo \(q\). When SLOT\({}_{0}\) ends, CCo \(q\) broadcasts the ASSOC_CNF frame to the stations in the convergence period 0 (CP\({}_{0}\)). Stations c, e, d, and b receive TEIs. These stations then place CCo \(q\) in the routing table, learn the link to \(q\), and complete the backward path learning. Stations a, c, e, d, and b have associated the network. The state machine of CCo \(q\) evolves C_CCO from UC_CCO. The associated station state machine evolves C_STA from UC_STA. * CCo \(q\) re-allocates the beacon slot to reduce the maintenance overhead and force as many stations as possible to associate the first layer of the network. CCo \(q\) repeats the abovementioned mechanism if new stations are available to associate; otherwise, the first layer cluster tree completes the network. * CCo \(q\) allocates a beacon slot to four stations every beacon period, as depicted in Figure 4b. If the number of stations is greater than four in the first layer, CCo \(q\) allots a beacon slot to the remaining stations in the next beacon period. Let us take stations a, c, e, d, and b as examples to explain the network process for the remaining beacon stations. If station a obtains the first beacon slot, it sends a beacon frame to the other stations by broadcasting it in SLOT\({}_{1}\). If the associated stations receive the beacon frame, they place station a in a neighboring table and establish a link. If the stations that do not associate the network receive the beacon frame, they delay for a short and random period of time, then self-schedule the ASSOC_REQ frame, access the channel by CSMA, and send it to station a. Station a replies to the ACK after waiting for the contention inter frame space (CIFS), such as that for station \(f\). The other network mechanisms are similar to \(f\). Station a sends the ASSOC_ALL_REQ frame to CCo \(q\) in the CP\({}_{1}\) when SLOT\({}_{1}\) ends. * CCo \(q\) replies to the ACK, puts the new associated stations into the routing table, learns the link to the new associated stations, and completes the forward path \(Q\)-learning to the new associated stations. CCo \(q\) broadcasts the ASSOC_ALL_CNF frame to a. Subsequently, a replies to ACK and broadcasts this frame. The new associated station \(f\) obtains the TEI, puts station \(a\) and CCo \(q\) into the routing table, and completes the backward path \(Q\)-learning. Station a becomes the proxy, and stations j, h, g, f, i, l, k, and m become the associated station. The state machine evolves C_STA from UC_STA. The second layer completes the network, as shown in Figure 4c,d.