# Tackling Explicit Material from Online Video Conferencing Software for Education Using Deep Attention Neural Architectures

Authors: Yongzhao Yang; Shasha Xu
Journal: Computational Intelligence and Neuroscience
 (2022)

---

## Abstract

The spread of the COVID-19 pandemic affected all areas of social life, especially education. Globally, many states have closed schools temporarily or imposed local curfews. According to UNESCO estimations, approximately 1.5 billion students have been affected by the closure of schools and the mandatory implementation of distance learning. Although rigorous policies are in place to ban harmful and dangerous content aimed at children, there are many cases where minors, mainly students, have been exposed relatively or unfairly to inappropriate, especially sexual content, during distance learning. Ensuring minors’ emotional and mental health is a priority for any education system. This paper presents a severe attention neural architecture to tackle explicit material from online education video conference applications to deal with similar incidents. This is an advanced technique that, for the first time in the literature, proposes an intelligent mechanism that, although it uses attention mechanisms, does not have a square complexity of memory and time in terms of the size of the input. Specifically, we propose the implementation of a Generative Adversarial Network (GAN) with the help of a local, sparse attention mechanism, which can accurately detect obscene and mainly sexual content in streaming online video conferencing software for education.


---

## Body

## 1. Introduction

Going through the second wave of the digital age, humanity is now called upon to manage the multilevel social effects that arise through the ever-accelerating growth of the Internet. At the international level, efforts are being made to establish an institutional framework for protecting minors using new technologies. But as children’s use of the Internet and new technologies are constantly evolving, few countries have implemented a fully operational framework in enacting regulations for illegal behaviors exclusively in the Internet environment. The harmonization of the laws of the nations is an essential precondition for the effective transnational treatment of cybercrime and the protection of minors. The prevention and response initiatives proposed as good practices by experts and stakeholders focus on children, parents, and educators, whose effectiveness is constantly being explored because Internet issues are continually evolving [1–3].Obscene and mainly sexual content, such as pornography, is not allowed in applications accessible to minors, primarily in educational environments. In general, the modern legal framework imposes strict policies on nudity and sexual content, especially when it relates to children. Implementing these policies from a technological point of view is mainly based on the development and implementation of techniques (filters) that implement these policies. Corresponding techniques are applied internationally in the educational networks of many advanced countries and prevent with significant success rate access to sites belonging to categories such as: “porn” (sites with pornographic content), “gambling” (gambling sites), “drugs” (websites promoting drugs), “aggressive” (websites promoting aggressive behavior and racism), and “violence” (websites promoting violence) [4]. Because websites are categorized in the above categories using an automated process (due to the vast number of websites on the Internet), a website can be ranked incorrectly. For this reason, every educational organization follows international practice. It enables its users to inform the competent technicians when they find any malfunction of the service, who now manually correct the database that should be excluded.In addition, social media giants enforce strict policies and established procedures for dealing with content and any harmful behavior, prohibiting content that endangers minors [5]. These include sexual harassment, abuse, and harmful and dangerous acts, uploading, streaming, commenting, engaging in activities that harm children, etc. Also, in recent years, these companies have become significant investors in the design of systems that detect sexually explicit material on the video clearly and effectively to prevent the release of material with unacceptable content [6].The huge unresolved issue now is in cases of intentional or unintentional exposure to sexual content when using real-time video conferencing software, such as online video conferencing software, used extensively during the pandemic. In these cases, where content and streaming occur in real-time, it is challenging to detect obscene or sexual content, so there is no protection for underage students [7].Obscene and primarily sexual content can be detected in streaming online video conferencing software for education with great precision. Based on the gap presented in the procedures and minors’ risks, mainly students, this paper proposes an innovative deep attention neural architecture system to tackle explicit material from online education video conference applications. It is an advanced machine learning technique, precisely computer vision, which uses an intelligent attention mechanism that does not have a square of memory and time complexity in terms of the size of its input data. Specifically, the implementation uses a GAN, with the help of a local, sparse attention mechanism, of complexity O(nn). We take advantage of the probability distributions generated within this particular attention mechanism while maintaining the 2d geometry of the multimedia content.


## 2. Related Literature

The literature concerning the field of detection mechanisms concerning specific or explicit content [1, 3] is varying due to the different approaches that the research community has:Li et al. [8] studied numerous motion classification algorithms, concentrating on video using classifiers, mostly frame-based. They divided the basic processes into three main categories: the first was frame-by-frame recognition, the second was extracting sequences, and the third was temporal-information monitoring, which used the LSTM structure or the optical flow approach to remove training data between sequences. They also divided and characterized the various types of deep learning-based cameras as follows: Convolutional Neural Networks-based methods, Restricted Boltzmann Machine-based methods [9], and Autoencoder-based techniques, all examples of unsupervised ML algorithms that could acquire the representations and produce data frames with similar attributes.Longlong et al. [10] looked at self-supervised generic image learning techniques based on deep learning from media files. They defined the key terms and examined the most prevalent self-supervised learning deep neural network topologies. They next looked at the architecture and evaluation criteria for self-supervised learning techniques, as well as the most often used samples, primarily for videos, and current self-supervised visual feature learning techniques [11]. They examined the practices of the shapes on image and video feature learning benchmark datasets. They finished their proposal by outlining several potential avenues of development for self-supervised visual feature learning.Arachchi et al. [12] introduced a state-exchanging long short-term memory (SE-LSTM) two-stream neural network approach, based on the benefits of using spatial and motion information to identify dynamic patterns. This method was used to identify movie reactions using appearance motion characteristics. It could also be used to expand the general purpose of LSTM by sharing data with past cell states in both the look and action streams. The movies could not include any other active items than the target objects to achieve better classification performance, and the contexts had to be static [13]. The trial findings showed that the technique surpassed other collections in precision, particularly when it came to static background dynamic patterns classifications. To decrease discrepancies, they proposed eliminating all mislabeled information in the next round of their study.Duboyskii et al. [14] used automated emotional state recognition and video conferencing technologies to transmit distant material in travel communication systems, surveys, and other applications. They created a peer-to-peer framework for remote communication sessions, allowing clients to share audio and visual information. At the operator end, convolutional neural networks were used for stream processing and to evaluate the customer’s emotional responses. Three mechanisms (video, audio, and text) and multimodal recognition were employed to establish the dynamic conditions. The test was carried out between persons in which one served as an operator and posed closed questions while the other answered them. The proposed technology could be used in various sectors, including service delivery and healthcare, where real-time human emotion identification is essential. The neural network produced the highest accuracy values when multimodal recognition was applied, indicating its effectiveness in video conferencing systems for classifying human emotions. Their system had the disadvantage of only supporting one-to-one user connections, which they plan to address by expanding the number of concurrent user connections.In 2016, Vondrick et al. [15] introduced a generative adversarial network for films using a Spatio-temporal convolutional architecture that untangled the scene’s images by investigating how to learn behaviors from vast volumes of unstructured camera footage. It is expected that the scene dynamics will be critical for the next phase of computer vision systems and learning from unlabeled data would be a promising option. Tests and simulations revealed that the model recognized important aspects for detecting actions with little control on the inside. Despite the fact that fully realizing the potential of unlabeled video is still a work in progress, their findings suggest that having a lot of unsupervised videos might be beneficial for both training to create films and acquire graphical images.Tulyakov et al. [16] proposed the Motion and Content deconstructed Generative Adversarial Network (MoCoGAN) framework for motion and content decomposed video production using the Generative Adversarial Network. In an unsupervised fashion, the MoCoGAN was trained to distinguish signal from content, and a movie was created by mapping a set of random vectors to a set of image sequences. They presented a unique adversarial learning method that learned motion and content decomposition unsupervised using both image and video discriminators. A Gaussian distribution was used to describe the content subspace, while a recurrent neural network to model the motion domain. The efficiency of the suggested framework was confirmed by experimental findings on datasets with qualitative and quantitative comparisons to state-of-the-art techniques [17]. They also demonstrated how their scheme could be used to produce videos with the same material but distinct motion, as well as films.To overcome the short sample issue in hyperspectral image classification, Feng et al. [18] presented a symmetric convolutional GAN based on collaborative learning and attention mechanism (CA-GAN). A combined spatial-spectral intricate attention module was used in the Generator to filter out misleading and confusing aspects of the produced samples and force the distribution of generated models to resemble the pattern of genuine hyperspectral images. To retrieve combined spatial-spectral information of images, a convolutional LSTM layer was fused in the Discriminator. In addition, by using the actual sample information retrieved by the Discriminator, a collaborative learning process was devised to aid sample production in the generator. It allowed the Generator and Discriminator to be refined alternately and collaboratively via competition. Tests on noteworthy sources of data revealed that their method outperformed the other approaches in terms of classification accuracy, particularly when the number of training samples was restricted. The studies indicated that they will look into more efficiently and automatically determining the placements and numbers of different modules, and they will experiment with different sampling methods to eliminate overlap between training and testing sets.From the literature mentioned, we see that the research community is actively focusing on finding methods and techniques to increase the performance of media classification, according to the specific needs of each individual Case [3, 19].


## 3. Methodology

The proposed implementation is based on the GANs architecture [18, 20], which uses an optimal local, sparse attention mechanism. Using a previous frame’s context, a video prediction algorithm can foretell the next frame in a video. Unlike a static image, a video allows the viewer to see the changes and motion patterns over a more extended period. For this reason, the model must take into account both time and space to accurately predict the future frames in a video. Modeling temporal dynamics is typically done using Recurrent Neural Networks. However, GANs have become the most popular method for predicting future video frames. A vital element of the structure of GANs is the existence and simultaneous training of two networks, the Generator that creates samples as close as possible to those of the training set and the Discriminator that is trained to distinguish which samples come from the training set (i.e., are they real) and which one from the Generator (i.e., are they artificial or fake). Specifically, at each training Step (i.e., inside the training loop), the Discriminator receives samples from the training data set and samples generated by the Generator and is trained to have a probability of close to 1 for the first and close to 0 for the second. In contrast, the Generator is trained so that from input noise to output images to the output realistic enough to “trick” the Discriminator.Going a little deeper into the analysis of how GANs are trained, we can say that both Generator and Discriminator are represented by (continuously) differentiable functions with trainable parameters, such as neural networks, each with its cost function. The two networks are trained through back-propagation using the Discriminator cost function, but with a different goal. The Discriminator tries to reduce the cost function for both natural and artificial samples, while Generator tries to increase the Discriminator cost function for the synthetic samples it produces. It is noteworthy that the training data set alone determines the type of samples that the Generator learns to create.The Binary Cross-Entropy cost function is used in the proposed methodology. For each predicted probability, Binary Cross Entropy compares it to the class output of 0 or 1. Once the score has been calculated, probabilities are penalized based on the distance from the expected value. This is a measure of how close or how far the calculated value is from the actual value. Specifically, for a set ofm samples per batch is as follows [17]:(1)Jm23∂⟶=−1m∑i=1myiloghxi;∂⟶+1−yilog1−hxi;∂⟶,where the initial sum and division by the number of samples approximates the mean value operator, x(i) is the i-th sample, y(i) is the label of the i-th sample, and ∼θ is the vector of the trainable model parameters. During the Discriminator training of the proposed GAN, the labels will be 1 for the actual samples and 0 for the artificial ones. In contrast, for the training of the Generator, the reverse is true, i.e., together with the synthetic samples, label 1 will be given to calculate whether it may “trick” the Discriminator.Focusing on the formation of the cost function and the values it receives for the 0/1 tags given during the training of a GAN, we see that when the tag is 1, only the first term of the sum acts. Considering the negative sign at the beginning of the equation, we see that the above Binary Cross-Entropy approach for a batch takes values from 0 to +∞ when the classification functionh(x) with parameters θ takes values from 0 to 1.Optionally, the Binary Cross Entropy cost function has two parts (one for each class) and takes values close to 0 for correct configuration (diagonal confusion matrix) while approaching the positive infinity for error (diagonal confusion matrix) - behavior graphically illustrated in Figure1 below [17]:Figure 1
The binary Cross-Entropy loss function for real (left) and fake (right) data.Thus, for the Discriminator, the Binary Cross-Entropy cost function given that during GAN training, the actual data is contractually assigned the tag 1 and the artificial data to 0, will be [17]:(2)JD∂D⟶,∂G⟶=−1m∑i=1myiloghxi;∂⟶+1−yilog1−hxi;∂⟶=−1m∑i=1mlogDxi;∂D⟶+log1−DGz⟶i;∂G⟶;∂D⟶=−1m∑i=1mlogDxi;∂D⟶−1m∑i=1mlog1−DGz⟶i;∂G⟶;∂D⟶≈−Ex∼pdatalogDx−Ez∼ppriorlog1−DGz,where D(x) is the Discriminator output (i.e., the probability of realism of the input x), G(z) is the output of the Generator network for random vector input z (i.e., an artificial image), pdata is the distribution followed by the data input (in these images it will be a very high dimensional distribution that can only be indirectly and approximately modeled by GAN), and pprior the prior distribution from which we sample to get the random vector at the Generator input. Since Discriminator predicts probability and therefore D(x) ∈ [0, 1], it follows that to minimize its cost function, Discriminator must learn to assign a high probability to samples labeled 1 (derived from the set of training data) and low on those generated by the Generator [21].The Generator network, in turn, tries to “trick” the Discriminator so that the chances it assigns to the artificial samples at its output are high. It aims to maximize the second term of the Discriminator cost function - after all, only this term can affect the Discriminator’s cost function to increase it. Therefore, the following will apply to the Generator [8]:(3)JG∂G⟶,∂D⟶=1m∑i=1m1−yilog1−hxi;∂⟶=1m∑i=1mlog1−DGz⟶i;∂G⟶;∂D⟶=1m∑i=1mlog1−DGz⟶i;∂G⟶;∂D⟶≈Ez∼ppriorlog1−DGz,where the negative sign at the beginning has now been removed as the Generator tries by minimizing its cost function to increase that of the Discriminator, while all other sizes are as before. Because the first term of the equation depends only on the training data set, the above Generator cost function is declared as negative of the Discriminator cost function [22, 23]:(4)JG∂G⟶,∂C⟶=−JD∂C⟶,∂G⟶.Focusing on the continuous 1-Lipschitz functionf, in the proposed GAN is the Discriminator network itself, which, taking an image, x, is called upon to give a real number. Therefore, the function will be(5)c:X⟶ℝ,cL≤1.To successfully approach a neural network with trainable parameters ∼θ a continuous 1-Lipschitz function, the measure of some of the network output derivatives in terms of trainable parameters must be at most 1 at each point in the domain. Thus, the Discriminator neural network must satisfy the following continuity condition to be a 1-Lipschitz continuous function [24, 25]:(6)∇∂⌢CCx;∂C⟶2≤1∀x∈X↔cL≤1.This condition enforcement ensures that the cost function is valid when measuring the allocation distance. It is continuous and differentiable and does not increase too fast. The proposed model introduces a normalization term that imposes a penalty when the norm of some of the output derivatives of Discriminator concerning its input is greater than 1 so that [21](7)regGP=∇∂⌢CCx˜;x˜2−12,and so, the cost functions that the two neural networks try to minimize will be [8, 21, 22]:(8)JC∂C⟶,∂G⟶=−1m∑i=1mCxi;∂C⟶+1m∑i=1mCGz⟶i;∂G⟶;∂C⟶+λ^GP∗regGP=−1m∑i=1mCxi;∂C⟶+1m∑i=1mCGz⟶i;∂G⟶;∂C⟶+λGP∗1m∑i=1m∇∂C⟶Cε∗x+1−ε∗x^;∂C⟶2−12≈−Ex∼pdataCx+Ez∼ppriarCGz+λ^GP∗Ex˜∼px˜∇x˜Cx˜2−12.To model the sequence of input symbols under a single framework, we propose in this work the use of optimal attention mechanisms both qualitatively and computationally. The proposed sparse attention mechanism requires much less memory, is faster, achieves better performance, and requires fewer training steps than intensive attention due to incorporating appropriate assumptions into its architectural design.In particular, the quadratic complexity of attention is due to the calculation of the table [17, 23]:(9)MQ,K=Q⋅KT,∈ℝNX×NY.Instead, we propose multidimensional attention mechanisms in this work. In each Step i, attention is limited to a set of predefined positions given by a mask:(10)Ai∈0,1NX×NY.In each Step i, we calculate(11)MQ,Kia,b=MQ,Ka,b,Aia,b=1,−∞,Aia,b=0.In addition, using information flow charts and the two-dimensional geometry conservation mechanism, we construct a sparse multistep attention layer that can model any dependencies on the input data and respects the native pixel locality in a video. An indicative representation of spherical 2-D points far away from the sphere is very unlikely to fall in the same area at all random rotations, which is reversed for very close points to the sphere, as shown in Figure2.Figure 2
Spherical 2-D case study.This process is directly related to the tendency of the softmax function to yield sparse distributions. So, by this logic, we argue that the dense models produce sparse attention maps:(12)EXk:n≤μ+σk−1n−k+1.Based on the above relation, we can prove the rarity of the probabilistic distributions obtained from softmax [21–23]:(13)eEXk:n∑i=1neEXi≤eμ+σk−1/n−k+1∑i=1neEXi=eμ+σk−1/n−k+1neμ.which with limits μ = 0 and σ2 = 1 can be calculated as(14)EeXk:n∑i=1neXi:n≤ϵ,ek−1/n−k+1n≤ϵ⇌n∈≥1k−1n−k+1≤lnnϵk≤1+n+1ln2nϵ1+ln2nϵ.The challenge in multistep attention mechanisms is the design of dual masks for each step. This paper uses an information theory tool to successfully design sparse attention patterns. Specifically, information flow graphs are used, which are guided, acyclic graphs that model the flow of network information into graphs of distributed systems. For our problem, these graphs show the flow of information between the attention steps and the corresponding transformations that follow. The most common of the proposed transformations are [8, 22, 23]:(15)Fqi=qi;12,…;12,Gki=Uki;Uki22;…;Uki22,Fqi=qi;0,Gki=ki;MK2−ki22,Fq=MKq2⋅q;0,Gk=k;MK2−k22.To smooth out the deformations resulting from the above transformations, the proposed system allows the focus on the previous and next stage, as shown in Figure3 below:Figure 3
Depiction of transformer attention.For each set of masksA1,…,Ap we make a polymer graph GV=V0,V1,…,Vp,E where the edges between Vi,Vi+1 are determined by the mask Mi. Thus, a sparse pattern has complete information if the relevant information graph has a path from each node a ∈ V0 to each node b∈ Vp. So, in addition to the computational improvement of the dense attention mechanism, the sparse attention mechanisms also achieve better results due to the integration of prior knowledge of locality into the information flow chart.Our mechanism has O(nn) memory complexity and speed, significantly reducing the square complexity of intensive attention. The probability distributions created within the attention map make a new method for reversing the proposed attention GAN. Essentially, the proposed technique provides the methodology for evaluating the boundaries of indeterminate forms so that by applying them, an indefinite form can be quickly assessed by substitution [21, 22]:(16)limh⟶0+1h∫xjxj+hfx|θdx=limh⟶0+d/dh∫xjxj+hfx|θdxdh/dh=limh⟶0+fxj+h|θ1=fxj|θ.Then,(17)argmaxθℒθ|xj=argmaxθlimh⟶0+ℒθ|x∈xj,xj+h=argmaxθlimh⟶0+1h∫xjxj+hfx|θdx=argmaxθfxj|θ.Therefore,(18)argmaxθℒθ|xj=argmaxθfxj|θ,and so, maximizing the probability density in xj equals maximizing the probability of that observation in xj, thus creating the method of the proposed attention.As a novel approach, this technique is an intelligent advanced mechanism that uses attention mechanisms but does not have a square complexity of memory and time in terms of the input size. So, it is possible to accurately detect obscene and primarily sexual content in streaming online video conferencing software.


## 4. Scenarios and Results

The research was also conducted to assess the likelihood that the user will engage in abnormal behavior related to displaying inappropriate content [7, 26]. A specialized scenario was implemented to model the proposed system to calibrate the user’s actions during the live video stream about an activity that might be considered provocative or inappropriate. This process was based on the technique of visual flow, which involves the movement of objects between successive snapshots of a video, which arises due to the action of objects. Sparse optical flow detects characteristic points, such as angles and edges of the image, and their monitoring in successive snapshots, while dense visual flow refers to the estimation of the motion vectors of the whole image, i.e., all pixels.More specifically, the scenario assumes that the optical flux is a standard estimate where the position of each point is defined using a square polynomial of the formf1x=xTA1x+bTx+c1, where Α is a symmetric array, b vector, and c graded number. An adjustment of least squares determines the coefficients. Respectively for the second scene, it applies that [27–29]:(19)f2x=f1x−d.Therefore, we have(20)f2x=f1x−d=x−dTA1x−d+b1Tx−d+c1=xTA1x+b1−2A1dTx+dTA1d−b1Td+c1=xTA2x+b2Tx+c2.If the coefficients of the square polynomials are equated, we have(21)A2A1,b2=b1−2A1d,c2=dTA1d−b1Td+c1.And since A is reversible, we have(22)d=−12A1−1b2−b1.This condition does not apply to the entire image signal, as there is no universal permutation. Thus, the universal polynomial equation is converted to local with coefficientsA1(x), b1(x), and c1(x). Even the condition A1 = A2 is practically not valid, so it is estimated as [27, 28](23)Ax=A1x+A2x2.Finally, we define(24)Δbbx=−12b2x−b1x.We have(25)Axdx=Δbx,where d(x) now has local power and is not universal. Finally, to improve the accuracy, we can apply this condition to the whole neighboring area and not to each pixel separately, minimizing the relationship [13, 23, 26]:(26)∑Δx∈IwΔxAx+Δxdx−Δbx+Δx2,where w(∆x), weight function of adjacent points. So, the field of view is ultimately(27)dx=∑wATA−1∑wATb.So, the algorithm’s operation is based on the minimization of a function that includes an information term using the L1 norm and a normalization term using the optical fluctuation. Brightness constancy assumption is initially considered as(28)ddtIxt,yt,t=0,where I(x(t), y(t), t) the video and (x(t), y(t)) the trajectory of a point in the image. Applying the chain rule results in(29)∇I⋅x˙,y˙+∂∂tI=0.It is also defined as the speed of the orbits:(30)ux,y=u1x,y,u2x,y,and the visual flow is committed to locating the reference point, which in the resulting case is the inappropriate material:(31)∇I⋅u+∂∂tI=0.For each point in the image, this equation has 2 unknown variables, the velocity componentsu. Therefore, the system does not have a unique solution. To solve this problem, we use a smoothing term to force the normalization of u.In the proposed model, the solution is performed by minimizing the energy function resulting from the sum of the variability ofu and the term L1 when the following function is applied [13, 17, 21]:(32)Eu=∫Ω∇u1+∇u2+λρu.The minimization process for findingu is performed for different image scales. The vector u is initially calculated for large scales, initial values for the more minor scales. Thus, the vector u is gradually determined more accurately.Finally, for the proposed algorithm to better render the classification coded features, the Gaussian Mixture Model (GMM) is first calculated to model the distributions of video descriptions. The vectors then encode the slope of the logarithmic probability of the features according to the GMM parameters. Let Χ = {x1, x2, xt} the n-dimensional features. The GMM parameters are estimated based on these characteristics: weights, averages, and variability. Accordingly, the logarithmic probability slopes for the GMM parameters are calculated as follows [11, 30, 31]:(33)∇αklogpX=∑i=1t∇αklogpxt,∇μklogpX=∑i=1t∇μklogpxt,∇σklogpX=∑i=1t∇σklogpxt,where from the sum of the three vectors results [31, 32]:(34)FV=∇αklogpX,∇μklogpX,∇σklogpX.The pornography database [4, 6, 19, 30], which contains nearly 80 hours of 400 pornographic and 400 non-pornographic videos, was used to locate the scenes of inappropriate material. The pornographic material comes from relevant sites that host only such material. At the same time, it should be emphasized that the set consists of various types of pornography and depicts actors of many ethnicities. Respectively, the non-pornographic content came from browsing the web with general-purpose videos.During pre-processing, all videos were initially segmented into shots. A basic (non-inappropriate) frame was used to summarize the content of the picture into a still image. Some typical static images from photos contained in this dataset are shown in Figure4 below [1, 5, 30].Figure 4
Pornography database.All the exterior shots, such as beach shots, were removed, and only indoor pictures were used. In total, 12,182 videos were used, of which 6,743 were inappropriate, and 5,439 were inappropriate.The video observations based on the density estimation were given in time-series images, where thex-axis symbolizes time. In probability and statistics, density estimation is constructing an estimate of an unobservable underlying probability density function using observed data. The unobservable density function describes the distribution of a vast population; the data are typically viewed as a random sampling from that population. Density estimation techniques such as Parzen windows and various data clustering techniques, including vector quantization, are used. The simplest method for estimating density is to use a rescaled histogram. In this paper, for uniformity and comparison of the results, along with the pictures of the model estimation, a heuristic method was used based on the images of the experts’ observations and their votes in terms of content for each scene. Models trained with batch learning in the material in question were used as specialists. This procedure was done for each video, based on the total time in seconds that each category lasted within the video [6, 30].The 10-fold cross-validation method was used for the experimental evaluation. In contrast, the Mean Average Precision (MAP) and Accuracy Rate (AcR) were used as the scoring measure, where most evaluators take the final class of the examined video. Finally, the ROC Curve and F-measure metrics displayed the results. The results of the procedure are shown in Table1 below.Table 1
Performance metrics of the classification process - 1.
MAPAcRROC curveF-measurePorn92.53291.91295.00492.301noPorn88.02489.03188.55889.065As can be seen from the table above, the results look pretty satisfactory. In some cases, the model finds it challenging to locate the noPorn category, slightly reducing its overall performance. This is because the vector representations are identical. Although experimentally, this did not reduce the performance for the problems tested, there may be other problems with a drop. Even more importantly, this limits its use to situations where the number of classes is multiple.For this problem, a simple solution was used to replace the imaging function to group vectors with short Euclidean distances or large internal products to have data located in some lower norm sphere or even data without geometric constraints. The results of the procedure are shown in Table2 below.Table 2
Performance metrics of the classification process - 2.
MAPAcRROC curveF-measurePorn95.87195.56895.99796.163noPorn92.95891.59791.86991.733As can be seen, alternative display schemes achieve much better results without imposing such strong constraints on the nature of the input data. The main problem of the proposed solution is that it requires network retraining and, therefore, cannot operate on pretrained networks. This significantly reduces its usefulness as retraining costs are vast, and the chances of mastering sparse attention mechanisms are slim.The groups are created randomly in random attention, and the attention occurs within the group. To increase the probability of success of the method, we repeat the process a few times. For this reason, we propose a comparison model, the randomization, which can be used to create sparse models that do not require retraining. As shown in Table3 below, the model in question achieves impressive results.Table 3
Performance metrics of the classification process - 3.
MAPAcRROC curveF-measurePorn99.22099.11899.67299.258noPorn98.48798.59698.60498.599It seems that this model can begin the search to find attention mechanisms that do not require retraining.


## 5. Conclusions

In this work, we proposed and studied solutions for efficient attention mechanisms. The methods presented are based on either predetermined sparse patterns or dynamic dilution. The advanced technique first introduced in the literature suggests a GAN assisted by attention mechanisms, which can speed up and even be more efficient, allowing for faster processing and fewer memory requirements. The methodology is used in a case study to deal with incidents of fair or unfair exposure to offshore content to underage students during distance learning in online education video conference applications. A significant disadvantage of the proposed method is that it requires an extensive bandwidth network.Changes that can lead to simpler variants of attention that operate without imposing restrictions on attention inputs are critical future developments in this work. Also, the search for even more efficient computing methods and, in general, the solutions that can significantly improve the performance of solving complex real-time problems like the one studied. Finally, it is crucial to investigate how an external classification scheme can be implemented that can achieve high acceleration for a sufficiently large input size.



---

*Source: 6334802-2022-05-11.xml*