# Network Performance Optimization for Low-Voltage Power Line Communications
## 3 LVPLC Network Scheme
### Improved \(Q\) Learning Approach in LVPLC Network Scheme
#### 3.2.2 Improved \(Q\)-Learning Algorithm in LVPLC Network  
We describe several assumptions and notations for the backbone cluster tree analysis to facilitate scheme discussion. Assumption  
The \(n\) (\(n>1\)) stations are connected with the same medium and physical links, and the following assumptions are made:* Only the MAC address is unique to the station. * Each station communicates with at least one other station when the communication environment of the channel is good. * The CCo does not repeatedly assign and then reclaim TEIs. * No more than three retransmission times are present. * The electrical signal transmission time is neglected in the copper medium. Network Information Table  
This study proposes the usage of three important tables (i.e., routing table, topology table, and neighbor table) to store the stations' information during the dynamic network process. The routing table (\(Q\)-table) is used to store the link information from the origin address to the destinations. Each station maintains a routing table. The table includes the hop count, original address, the next hop address, and destination address. According to the topology control strategy, stations put the next hop information into the network frame and send it to the destination station by broadcasting way. If the other stations receive the frame and find the address information to be different from their addresses, they discard it and avoid network loop. The topology table is used to store all the stations' connectivity relationships, and only the CCo stores this table. The neighbor table is used to store the stations' neighbor information. The information represents the stations' connectivity relationships. Each station also maintains a neighbor table. Typical CSMA and TDMA mechanisms are extensively studied. The CSMA suffers from heavy interference because of hidden terminals, and often exhibits a large delay. The TDMA has practical difficulties achieving an accurate time synchronization. Achieving such synchronization via beacon exchanges is generally expensive. Therefore, we propose an online bi-directional improved \(Q\)-learning network mechanism based on the hybrid TDMA-CSMA protocol. The structural properties of the LVPLC networks are relatively complex. This study introduces a small number of topological measurements to compute the station importance (i.e., centrality degree, closeness degree, betweenness degree, clustering coefficient, and average shortest path [25]). The value of the closeness degree is the reciprocal of the average distance between each station pair:  
\[C_{v}=1/D_{ij}=(n-1)/\sum_{j\
eq v\in V}d_{vj} \tag{15}\]  
where \(C_{v}\) is the closeness centrality of station \(v\), and \(d_{vj}\) is the distance of the shortest path connecting station \(v\) and \(j\). The relative betweenness centrality of \(B_{v}\) measures a station \(v\) that lies on the shortest path between any pair of stations passing through a specific station:  
\[B_{v}=\frac{1}{(n-1)(n-2)}\sum_{i\
eq j,i\
eq v}\frac{\left|\sigma_{ij}(v) \right|}{\left|\sigma_{ij}\right|} \tag{16}\]  
where \(\sigma_{ij}\) is the total number of shortest paths from station \(i\) to \(j\), and \(\sigma_{ivj}\) is the number of shortest paths from station \(i\) to \(j\) passing \(v\). The clustering coefficient of station \(i\) is calculated as:  
\[CC_{1}(i)=2E_{1}(i)/k_{i}(k_{i}-1) \tag{17}\]  
where \(E_{1}(i)\) is the number of links between stations \(k_{i}\). The average path \(L\) is calculated as follows:\[L=\sum_{i\
eq j}d_{ij}/n(n-1) \tag{18}\]  
where \(d_{ij}\) is the distance between \(i\) and \(j\). An improved \(Q\)-learning mechanism based on the hybrid TDMA + CSMA (control policy) mechanism is described as follows:  
* Auto selection of CCo: All stations are powered on at the same time, and their parameters are initialized. After 5 s of silence, the station with the shortest delay randomly sends the SELECT_CCO beacon frame to the other stations in the radius of communication.