Patent Description:
In order to satisfy wireless data traffic demands that tend to increase after <NUM> communication system commercialization, efforts to develop an enhanced <NUM> communication system or a pre-<NUM> communication system are being made. For this reason, the <NUM> communication system or pre-<NUM> communication system is called a beyond <NUM> network communication system or a post LTE system. In order to achieve a high data transfer rate, an implementation of the <NUM> communication system in a mmWave band (e.g., <NUM> Giga (<NUM>) band) is taken into consideration. In order to reduce a path loss of a radio wave and increase the transfer distance of a radio wave in the mmWave band, beamforming, massive MIMO, full dimensional MIMO (FD-MIMO), an array antenna, analog beamforming and large scale antenna technologies are being discussed in the <NUM> communication system. Furthermore, in order to improve the network of a system, technologies, such as an improved small cell, an advanced small cell, a cloud radio access network (cloud RAN), an ultra-dense network, device to device communication (D2D), wireless backhaul, a moving network, cooperative communication, coordinated multi-points (CoMP), and reception interference cancellation, are being developed in the <NUM> communication system. In addition, hybrid FSK and QAM modulation (FQAM) and sliding window superposition coding (SWSC) that are advanced coding modulation (ACM) schemes, and filter bank multi-carrier (FBMC), non-orthogonal multiple access (NOMA) and sparse code multiple access (SCMA), that is, advanced access technologies, are being developed in the <NUM> system.

Meanwhile, the Internet evolves from a human-centered connection network over which human generates and consumes information to Internet of things (IoT) through which information is exchanged and processed between distributed elements, such as things. An Internet of everything (IoE) technology in which a big data processing technology through a connection with a cloud server is combined with the IoT technology is emerging. In order to implement the loT, technical elements, such as the sensing technology, wired/wireless communication and network infrastructure, service interface technology and security technology, are required. Accordingly, technologies, such as a sensor network, machine to machine (M2M) and machine type communication (MTC) for a connection between things, are recently researched. In the IoT environment, an intelligent Internet technology (IT) service in which a new value is created for human life by collecting and analyzing data generated from connected things may be provided. The IoT may be applied to fields, such as a smart home, a smart building, a smart city, a smart car or a connected car, a smart grid, health care, smart home appliances, and advanced medical services, through convergence and composition between the existing information technology (IT) and various industries.

Accordingly, various attempts to apply the <NUM> communication system to the IoT are being made. For example, technologies, such as a sensor network, machine to machine (M2M) and machine type communication (MTC), are implemented by schemes, such as beamforming, MIMO, and an array antenna, that is, <NUM> communication technologies. The application of a cloud radio access network (cloud RAN) as the aforementioned big data processing technology may be said to be an example of convergence between the <NUM> technology and the IoT technology.

Meanwhile, if voice signals are transmitted and received between terminals as in a voice call in a wireless communication system, a voice inputted to a transmission terminal is converted into a wideband signal having an electrical form within the transmission terminal. The transmission terminal converts the wideband signal into a narrowband signal due to restrictions on a transmission bandwidth, and transmits the signal to a reception terminal. To this end, the transmission terminal converts the wideband signal into the narrowband signal through down-sampling processing, and transmits the narrowband signal to the reception terminal. The reception terminal that has received the narrowband signal restores the narrowband signal to the wideband signal through up-sampling (bandwidth expansion) processing, and outputs the voice based on the restored wideband signal.

In the case of the aforementioned voice signal transmission and reception technology, incomplete restoration is inevitably performed because a reception terminal has to restore the original waveform based on some sampled data in the state in which the reception terminal is unaware of a wideband signal generated by a transmission terminal. This directly boils down to the deterioration of voice quality. Accordingly, an efficient bandwidth expansion technology is required.

The disclosure proposes a technology for effectively restoring, by a reception terminal, a narrowband signal received from a transmission terminal to an original wideband signal by using an artificial neural network learnt based on the wideband signal and the narrowband signal down-sampled from the wideband signal. <CIT> discloses methods and apparatus for improving speech communication and speech interface quality using neural networks. <CIT> discloses a method for video processing. <CIT> pertains to neuroevolution based artificial bandwidth expansion of telephone band speech. <CIT> discusses deep neural networks for artificial noise robust speech bandwidth extension.

A method of a transmission terminal transmitting is provided as defined in the appended claims.

Furthermore, a method of a reception terminal is provided as defined in the appended claims.

Furthermore, a method of a server is provided as defined in the appended claims.

A bandwidth expansion method based on an artificial neural network, which enables a narrowband signal to be effectively restored to a wideband signal based on an artificial neural network, is disclosed.

Hereinafter, embodiments of the disclosure are described in detail with reference to the accompanying drawings. In describing the embodiments, a description of technology contents that are well known in the art to which the disclosure pertains and that are not directly related to the disclosure is omitted.

For the same reason, in the accompanying drawings, some components are enlarged, omitted or schematically depicted. Furthermore, the size of each component does not accurately reflect its real size. In the drawings, the same or similar components are assigned the same reference numerals.

Advantages and characteristics of the disclosure and a method for achieving the advantages and characteristics will become apparent from the embodiments described in detail in conjunction with the accompanying drawings. However, the disclosure is not limited to the disclosed embodiments, but may be implemented in various different forms. The embodiments are provided to only complete the disclosure and to fully notify a person having ordinary knowledge in the art to which the disclosure pertains of the category of the disclosure. The disclosure is defined by the category of the claims. Throughout the specification, the same reference numerals denote the same components.

In the disclosure, it will be understood that each block of the flowchart illustrations and combinations of the blocks in the flowchart illustrations can be executed by computer program instructions. These computer program instructions may be mounted on the processor of a general purpose computer, a special purpose computer, or other programmable data processing equipment, so that the instructions executed by the processor of the computer or other programmable data processing equipment create means for executing the functions specified in the flowchart block(s). These computer program instructions may also be stored in a computer-usable or computer-readable storage medium that can direct a computer or other programmable data processing equipment to implement a function in a particular manner, such that the instructions stored in the a computer-usable or computer-readable memory produce an article of manufacture including instruction means that implement the function specified in the flowchart block(s). The computer program instructions may also be loaded on a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable data processing equipment to produce a computer-executed process, so that the instructions performing the computer or other programmable data processing equipment provide steps for executing the functions described in the flowchart block(s).

Furthermore, each block of the flowchart illustrations may represent a portion of a module, a segment, or code, which includes one or more executable instructions for implementing a specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of order. For example, two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

The term "unit", as used in the present embodiment means software or a hardware component, such as an FPGA or an ASIC, and the "unit" performs specific tasks. However, the term "~unit" does not mean that it is limited to software or hardware. The "unit" may advantageously be configured to reside on an addressable storage medium and configured to operate on one or more processors. Accordingly, the "unit" may include, for example, components, such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functionalities provided in the components and "units" may be combined into fewer components and "units" or may be further separated into additional components and "units". Furthermore, the components and "units" may be implemented to operate on one or more CPUs within a device or a security multimedia card. Furthermore, in embodiments, '~ unit' may include one or more processors.

A term to denote a signal, a term to denote a channel, a term to denote control information, terms to denote network entities, a term to denote a component of an apparatus, a term for identifying an access node, terms to denote messages, a term to denote an interface between network entities, terms to denote various types of identification information, etc., which are used in the following description, have been exemplified for convenience of description. Accordingly, the disclosure is not limited to terms described later, and another term to denote a target having an equivalent technical meaning may be used.

<FIG> is a diagram illustrating an example of a process between the transmission and reception of voice signals between terminals according to an embodiment of the disclosure.

<FIG> illustrates an example of the transmission and reception of voice signals between a terminal (hereinafter a transmission terminal or a first terminal) <NUM> transmitting a voice signal and a terminal (hereinafter a reception terminal or a second terminal) <NUM> receiving a voice signal according to an embodiment of the disclosure.

With reference to <FIG>, a voice inputted to the transmission terminal <NUM> may be converted into an electrical voice signal within the transmission terminal <NUM>, and may be delivered to the reception terminal <NUM> as an analog signal or a digital signal. In this case, the voice inputted to the transmission terminal <NUM> is converted into the wideband signal having a wide bandwidth within the transmission terminal <NUM>, but the signal transmitted to the reception terminal <NUM> may be transmitted after being converted in the form of the narrowband signal due to restrictions on a transmission bandwidth. To this end, the transmission terminal <NUM> may convert, into the narrowband signal, the wideband signal generated based on the inputted voice through down-sampling processing, and may transmit the narrowband signal to the reception terminal <NUM>. The reception terminal <NUM> that has received the narrowband signal may restore the narrowband signal to the wideband signal through up-sampling (bandwidth expansion) processing, and may output a voice based on the restored wideband signal.

<FIG> is a diagram illustrating an example of down-sampling and up-sampling processing for a voice signal.

With reference to <FIG>, the transmission terminal <NUM> may generate an electrical voice signal based on an inputted voice. In this case, an original waveform of the voice signal generated by the transmission terminal <NUM> may be generated as a wideband signal including a large amount of data. The transmission terminal <NUM> may convert the wideband signal into a narrowband signal through down-sampling processing for sampling some data of the original waveform, and may transmit the converted narrowband signal to the reception terminal <NUM>. The reception terminal <NUM> may restore the received narrowband signal to the wideband signal (original waveform) through linear or cubic spline interpolation. In this case, incomplete restoration is performed because the reception terminal <NUM> has to restore the original waveform based on some sampled data in the state in which the reception terminal <NUM> is unaware of the wideband signal (original waveform) generated by the transmission terminal <NUM>. This directly boils down to the deterioration of voice quality.

The disclosure discloses a method of expanding the bandwidth of a voice signal based on an artificial neural network learnt based on a wideband signal and a narrowband signal.

Hereinafter, a method of expanding the bandwidth of a voice signal in a reception terminal through artificial neural network structure signaling according to an embodiment of the disclosure is described with reference to drawings.

<FIG> is a diagram illustrating a bandwidth expansion method based on an artificial neural network according to an embodiment of the disclosure.

With reference to <FIG>. the bandwidth expansion method based on an artificial neural network according to an embodiment of the disclosure may be performed by a system including a transmission terminal <NUM> transmitting a voice signal, a reception terminal <NUM> receiving a voice signal, and a server <NUM>. Hereinafter, in this specification, an artificial neural network learning process will be described as being performed by the separate server <NUM> for convenience of description, but this does not limit the scope of the disclosure. The artificial neural network learning process may be implemented by any apparatus capable of implementing each of steps performed by the server <NUM> to be described later, for example, a terminal.

According to an embodiment, the server <NUM> may perform the learning of an artificial neural network by using, as learning data, a wideband signal and a narrowband signal generated by down-sampling the wideband signal. The learning data for the learning may be transmitted from the transmission terminal <NUM> to the server <NUM>. According to an embodiment, the transmission terminal <NUM> may transmit, to the server <NUM>, the wideband signal and the narrowband signal generated by down-sampling the wideband signal. The server <NUM> may perform the learning of the artificial neural network based on the received wideband signal and narrowband signal. Alternatively, the transmission terminal <NUM> may transmit only the wideband signal to the server <NUM>. The server <NUM> may generate the narrowband signal by down-sampling the received wideband signal, and may perform the learning of the artificial neural network based on the received wideband signal and the generated narrowband signal. That is, the server <NUM> may perform the learning of the artificial neural network by updating coefficients included in the artificial neural network so that the wideband signal corresponding to an original waveform of the narrowband signal is restored from the down-sampled narrowband signal and outputted. According to an embodiment, the server <NUM> may perform the learning of an artificial neural network by using frequency components (amplitude, a phase) of a narrowband signal and a wideband signal as learning data in addition to a change in the waveforms of the narrowband signal and the wideband signal for the time. In this case, the learning data used may be constructed in the form of (the waveform of the narrowband signal over time, the frequency component of the narrowband signal, the waveform of the wideband signal over time, or the frequency component of the wideband signal), for example, or may be constructed in the form of 2D data based on a change in the frequency (an x axis: time, a y axis: the amount of change of a frequency) of the narrowband signal and wideband signal over time. According to an embodiment, the server <NUM> may perform the learning of an artificial neural network by using a wideband signal and a narrowband signal as learning data based on neural network structure information related to the transmission terminal <NUM>. The neural network structure information related to the transmission terminal <NUM> may be received by the server <NUM> from the transmission terminal <NUM> or may be previously stored in the server <NUM> in accordance with the transmission terminal <NUM>. Neural network structure information according to an embodiment may include at least one of the number of layers constituting an artificial neural network, the number of filters for each layer, the sizes of filters for each layer, an activation function, a stride, and padding. The detailed definition of each of parameters included in the neural network structure information is described in detail below.

According to an embodiment, the server <NUM> may generate neural network coefficient information related to a learnt artificial neural network. The neural network coefficient information is a set of coefficients that constitute the learnt artificial neural network, and may be a set including values that constitute weights of filters within each layer of the artificial neural network to be described later, for example. The server <NUM> may transmit the generated neural network coefficient information to the reception terminal <NUM>. The reception terminal <NUM> may store the received neural network coefficient information in a storage device (e.g., a memory) of the reception terminal <NUM>.

According to an embodiment, the transmission of the neural network coefficient information from the server <NUM> to the reception terminal <NUM> and the storage of the neural network coefficient information in the reception terminal <NUM> accordingly may be performed before session negotiations between the transmission terminal <NUM> and the reception terminal <NUM> are performed. That is, since the information on the coefficients that constitute the learnt artificial neural network has a huge capacity and requires security, the information may be previously delivered and stored in the reception terminal <NUM> prior to session negotiations between the transmission terminal <NUM> and the reception terminal <NUM>.

Thereafter, in the session negotiations and voice signal transmission and reception step, the transmission terminal <NUM> may transmit, to the reception terminal <NUM>, neural network structure information, including parameters related to a structure of the learnt artificial neural network, and a narrowband signal down-sampled from a wideband signal generated based on a voice inputted to the transmission terminal <NUM>.

Neural network structure information according to an embodiment of the disclosure may include at least one of the number of layers constituting an artificial neural network, the number of filters for each layer, the sizes of filters for each layer, an activation function, a stride, and padding. The number of layers L may indicate the total number of layers that constitute an artificial neural network. The number of filters (N_filters) and the sizes of filters (N_filtersizes) for each layer may indicate the number of filters included in each layer that constitutes an artificial neural network and the sizes of filters included in the each layer. The activation function may indicate the type of activation function corresponding to each layer within an artificial neural network, and may include any one of a sigmoid function, an Relu function, and a Leaky Relu function. The stride may indicate a sampling stride at which filtering for input data is applied within each layer. The padding may mean an indicator indicating whether padding will be applied to data inputted to each layer.

The reception terminal <NUM> may restore an artificial neural network learnt by the server <NUM>, based on neural network structure information received from the transmission terminal <NUM> and previously stored neural network coefficient information. That is, the reception terminal <NUM> may restore the entire artificial neural network by configuring, in an artificial neural network structure corresponding to the neural network structure information received from the transmission terminal <NUM>, coefficients included in the previously stored neural network coefficient information. As described above, the artificial neural network has been learnt to restore a wideband signal, that is, an original waveform of a narrowband signal, from the narrowband signal that has been down-sampled. Accordingly, the reception terminal <NUM> may restore, to a wideband signal, a narrowband signal received from the transmission terminal <NUM> by using an artificial neural network.

Hereinafter, the definition of parameters included in neural network structure information is described along with a structure and learning process of an artificial neural network according to an embodiment of the disclosure with reference to <FIG>.

Hereinafter, a description will be given by using a convolutional neural network as an example of an artificial neural network with reference to drawings, but this does not limit the scope of a right of the disclosure, and a bandwidth expansion method using any form of an artificial neural network, which may be implemented by a person having ordinary knowledge in the art to which the disclosure pertains, may be performed. Furthermore, hereinafter, a description will be given by taking a learning process of two-dimensional data using an artificial neural network as an example in the drawings, but this does not limit the scope of a right of the disclosure, and the learning of an artificial neural network using any form of data including one-dimensional or three-dimensional data depending on a form of data that needs to be learnt, which may be implemented by a person having ordinary knowledge in the art to which the disclosure pertains, may be performed.

<FIG> is a diagram illustrating an example of a structure of an artificial neural network according to an embodiment of the disclosure.

With reference to <FIG>, the artificial neural network according to an embodiment of the disclosure may include a plurality of layers <NUM> and <NUM>. The layers may include respective convolutional layers <NUM> and <NUM>, which learn a weight based on input data, and respective activation functions <NUM> and <NUM>. As illustrated in <FIG>, data (input) inputted to Layer <NUM> may be subjected to filtering processing through a filter to which a given weight has been assigned in the convolutional layer <NUM>, and is made to pass through the activation function. The data outputted by the convolutional layer may be converted into non-linear data and then delivered to Layer <NUM><NUM>. Layer <NUM> may process data by using the same method as that in Layer <NUM> based on the data received from Layer <NUM>, may deliver output data to a next layer (Layer <NUM>). The last layer (Layer_N) may output the final result value by repeating such a process. According to an embodiment, in the artificial neural network learning step, the artificial neural network having the structure may update weights of the filters within each layer in a way to output a proper (desired) output value in accordance with given input data. According to an embodiment of the disclosure, the artificial neural network may perform learning by using a narrowband signal and a wideband signal as learning data in a way to output the wideband signal, that is, an original waveform of the narrowband signal, from the narrowband signal that has been down-sampled.

According to an embodiment, the activation function within each layer may include a sigmoid function, an Relu function, and a Leaky Relu function or any non-linear function, for example.

According to an embodiment, the total number of layers within the artificial neural network illustrated in <FIG> may be indicated by the number of layers L included in neural network structure information. An activation function type corresponding to each layer may be indicated as an activation function included in neural network structure information.

<FIG> illustrate data filtering within a layer of an artificial neural network according to various embodiments of the disclosure. Hereinafter, the remaining parameters included in neural network structure information are described with reference to <FIG>.

<FIG> is a diagram illustrating data filtering within a layer of an artificial neural network according to an embodiment of the disclosure.

In <FIG>, input data having a <NUM> by <NUM> size and a filter having a <NUM> by <NUM> size are described as examples of input data and a filter, for convenience of description. With reference to <FIG>, in an operation within a convolutional layer, the sum may be calculated by multiplying input data and elements corresponding to each other between filters while moving the filters within the input data at a given interval, and may be outputted as an element of output data. In this case, the given interval at which the filters are moved is configured as a stride, and indicates a case where stride = <NUM> in <FIG>.

For example, an element (<NUM>,<NUM>) of output data may be calculated by multiplying elements disposed at (<NUM>,<NUM>), (<NUM>,<NUM>), (<NUM>,<NUM>), and (<NUM>,<NUM>) within input data corresponding to a filter having a <NUM> by <NUM> size and elements disposed at (<NUM>,<NUM>), (<NUM>,<NUM>), (<NUM>,<NUM>), and (<NUM>,<NUM>) of a filter, respectively, and then calculating the sum. Accordingly, the element (<NUM>,<NUM>) of the output data may be calculated as <NUM>*<NUM> + <NUM> * <NUM> + <NUM> * <NUM> + <NUM>*<NUM> = <NUM>.

By using the same method as that described above, an element (<NUM>,<NUM>) of output data may be calculated by moving the filter by <NUM> (=stride) in an x axis direction (a right direction in the drawing), then multiplying elements disposed at (<NUM>,<NUM>), (<NUM>,<NUM>), (<NUM>,<NUM>), and (<NUM>,<NUM>) of the input data corresponding to the filter and elements disposed at the (<NUM>,<NUM>), (<NUM>,<NUM>), (<NUM>,<NUM>), and (<NUM>,<NUM>) of the filter, respectively, and then calculating the sum. Accordingly, an element (<NUM>,<NUM>) of output data may be calculated as <NUM>*<NUM> + <NUM>* <NUM> + <NUM>* <NUM> + <NUM>*<NUM> = <NUM>. An element (<NUM>,<NUM>) of output data may be calculated as <NUM>*<NUM> + <NUM>* <NUM> + <NUM>* <NUM> + <NUM>*<NUM> = <NUM>.

<FIG> is a diagram illustrating data filtering within a layer of an artificial neural network according to another embodiment of the disclosure.

<FIG> illustrates data filtering in a case where the same input data and filters as those in <FIG> are constructed, but a stride is configured to be different from that in <FIG> illustrates an example if stride = <NUM>, and <FIG> illustrates an example if stride = <NUM>.

With reference to <FIG>, an element (<NUM>,<NUM>) of output data may be calculated according to the same method as that in <FIG>, and may be calculated as <NUM>*<NUM> + <NUM>*<NUM> + <NUM>*<NUM> + <NUM>*<NUM> = <NUM>. An element (<NUM>,<NUM>) of output data may be calculated by moving a filter in an x axis direction (a right direction in the drawing) by <NUM> (=stride), then multiplying elements disposed at (<NUM>,<NUM>), (<NUM>,<NUM>), (<NUM>,<NUM>), and (<NUM>,<NUM>) of input data corresponding to the filter and elements disposed at (<NUM>,<NUM>), (<NUM>,<NUM>), (<NUM>,<NUM>), and (<NUM>,<NUM>) of the filter, respectively, and then calculating the sum. Accordingly, if stride = <NUM> is indicated as in <FIG>, the element (<NUM>,<NUM>) of the output data may be calculated as <NUM>*<NUM> + <NUM>* <NUM> + <NUM>* <NUM> + <NUM>*<NUM> = <NUM>.

<FIG> illustrates data filtering in a case where the same input data and filter as those in <FIG> are constructed, the same stride (stride=<NUM>) is configured, and padding is applied.

If input data is outputted by using a filter according to the method illustrated in <FIG>, the size of output data is certainly reduced compared to the size of input data. Accordingly, the number of pieces of filtering which may be performed within a layer or the total number of layers may be limited. Furthermore, data disposed within the input data is repeatedly incorporated in a process of calculating each element of output data (e.g., data disposed in input data (<NUM>,<NUM>) in <FIG> is used in a process of calculating elements (<NUM>,<NUM>), (<NUM>,<NUM>), (<NUM>,<NUM>), and (<NUM>,<NUM>) within output data). In contrast, as data is closely disposed at an edge of the input data, the number of times that the data is incorporated in a filtering process may be reduced (e.g., data disposed at the input data (<NUM>,<NUM>) in <FIG> is incorporated only in a process of calculating the element (<NUM>,<NUM>) within output data). Accordingly, in order to adjust the size of output data and properly filter data disposed at an edge of input data, the edge of the input data may be extended by filling the surroundings of the edge with a specific value, and filtering may be then performed, which is called padding. <FIG> illustrates zero-padding in which the <NUM> by <NUM> input data described with reference to <FIG> has been extended by filling the surroundings of the edge with <NUM>. With reference to <FIG>, if padding is applied, filtering may be performed and output data may be calculated in the same manner as <FIG> and <FIG> by using, as input data, data obtained by filling the surroundings of the edge of the original input data with a specific value.

Each of the values <NUM>, <NUM>, <NUM>, and <NUM> of the elements included in the filter, which have been described with reference to <FIG>, corresponds to a weight of the artificial neural network. In the disclosure, the learning of an artificial neural network may mean that values of weights included in a filter are updated so that proper output data is outputted in accordance with input data. Each filter may be configured in each of a plurality of layers that construct an artificial neural network. A plurality of filters may be configured within the same layer.

According to an embodiment of the disclosure, the server <NUM> may perform the learning of the artificial neural network by using, as learning data, a wideband signal and a narrowband signal generated by down-sampling the wideband signal. That is, the server <NUM> may perform the learning of the artificial neural network by updating a filter or weight included in the artificial neural network so that the wideband signal, that is, an original waveform of the narrowband signal, is outputted from the down-sampled narrowband signal. The filter and weight of the learnt artificial neural network may be transmitted to the reception terminal <NUM> as neural network coefficient information.

<FIG> is a diagram illustrating neural network structure information and a neural network structure corresponding to the neural network structure information according to an embodiment of the disclosure.

As described with reference to <FIG>, a transmission terminal according to an embodiment of the disclosure may transmit, to a reception terminal, neural network structure information related to a learnt artificial neural network. The neural network structure information may include at least one of the number of layers L, the number of filters (N_filters) for each layer, the sizes of filters (N_filtersizes), an activation function, a stride, and padding.

Detailed values of the parameters corresponding to the neural network structure illustrated in <FIG> are as follows.

As illustrated in <FIG>, an artificial neural network indicated by the parameters has seven layers of Layer <NUM> to Layer <NUM>. Each of the seven layers may have <NUM> filters having a size <NUM>, <NUM> filters having a size <NUM>, <NUM> filters having a size <NUM>, <NUM> filters having a size <NUM>,. , <NUM> filters having a size <NUM>. Furthermore, an activation function may be Leakyrelu, a sampling stride may be <NUM>, and padding may be applied as the same value as a corresponding edge value. All the aforementioned values for indicating the parameters are for an example, and may be delivered as any form of information which may be recognized by a reception terminal.

Meanwhile, the learning process of the artificial neural network described with reference to <FIG> may be performed based on voice information into which a characteristic of a person corresponding to each transmission terminal has been incorporated.

<FIG> is a diagram for describing a voice signal transmission and reception method based on an artificial neural network into which a characteristic of a person has been incorporated according to an embodiment of the disclosure.

According to an embodiment, a server <NUM> may learn a first artificial neural network by using, as learning data, a wideband signal received from a first transmission terminal <NUM>-<NUM> and a narrowband signal corresponding to the wideband signal. Furthermore, the server <NUM> may learn a second artificial neural network by using, as learning data, a wideband signal received from a second transmission terminal (not illustrated) and a narrowband signal corresponding to the wideband signal. As also described with reference to <FIG>, the server <NUM> may receive, from the first transmission terminal <NUM>-<NUM> and the second transmission terminal, a wideband signal and a narrowband signal corresponding to the wideband signal and learn each artificial neural network by using the wideband signal and the narrowband signal as learning data, or may receive a wideband signal from the first transmission terminal <NUM>-<NUM> and the second transmission terminal and the server <NUM> may down-sample the wideband signal as a narrowband signal and use the narrowband signal as learning data for learning each artificial neural network.

According to an embodiment, the server <NUM> may perform the learning of the first artificial neural network and the second artificial neural network based on first neural network structure information and second neural network structure information related to the first transmission terminal <NUM>-<NUM> and the second transmission terminal. The server <NUM> may receive, from the first transmission terminal <NUM>-<NUM> or the second transmission terminal, the first neural network structure information and the second neural network structure information related to the first transmission terminal <NUM>-<NUM> and the second transmission terminal, or the first neural network structure information and the second neural network structure information related to the first transmission terminal <NUM>-<NUM> and the second transmission terminal may be previously stored in the server <NUM> in accordance with the first transmission terminal <NUM>-<NUM> or the second transmission terminal, respectively. Neural network structure information according to an embodiment of the disclosure may include at least one of the number of layers constituting an artificial neural network, the number of filters for each layer, the sizes of filters for each layer, an activation function, a stride, and padding.

According to an embodiment, the server <NUM> may generate first neural network coefficient information related to the learnt first artificial neural network and second neural network coefficient information related to the second artificial neural network. The neural network coefficient information is a set of coefficients that constitute each of the learnt artificial neural networks, and may be a set including values that constitute weights of filters within layers that constitute each artificial neural network, for example. The server <NUM> may transmit the generated first neural network coefficient information and second neural network coefficient information to a reception terminal <NUM>. The reception terminal <NUM> may store the received first and second neural network coefficient information in a storage device (e.g., a memory) of the reception terminal <NUM>. According to an embodiment, the reception terminal <NUM> may store the received first neural network coefficient information and second neural network coefficient information by making the received first neural network coefficient information and second neural network coefficient information correspond to information of the first transmission terminal <NUM>-<NUM> and the second transmission terminal, which is stored in the reception terminal <NUM>. For example, the received first neural network coefficient information and second neural network coefficient information may be stored in accordance with information of respective transmission terminals, which is stored in an address directory of the reception terminal.

According to an embodiment, the transmission of the first and second neural network coefficient information from the server <NUM> to the reception terminal <NUM> and the storage of the first and second neural network coefficient information in the reception terminal <NUM> accordingly may be performed before session negotiations between the first and second transmission terminals and the reception terminal <NUM> are performed.

Thereafter, the first transmission terminal <NUM>-<NUM> that attempts session negotiations may transmit, to the reception terminal <NUM>, a narrowband signal down-sampled from a wideband signal generated based on first neural network structure information, including parameters related to a structure of the learnt first artificial neural network, and a voice inputted to the first transmission terminal <NUM>-<NUM>. <FIG> illustrates that the first transmission terminal <NUM>-<NUM> initiates session negotiations and a call with the reception terminal <NUM>, and is merely for an example. The same may be applied to even a case where the second transmission terminal attempts session negotiations.

The reception terminal <NUM> may restore the first artificial neural network learnt by the server <NUM>, based on the first neural network structure information received from the first transmission terminal <NUM>-<NUM> and the first neural network coefficient information previously stored in accordance with the information on the first transmission terminal <NUM>-<NUM>. That is, the reception terminal <NUM> may restore the first artificial neural network by configuring coefficients included in the first neural network coefficient information that has been previously stored in the artificial neural network structure corresponding to the first neural network structure information received from the transmission terminal <NUM>-<NUM>. The first artificial neural network has been learnt to restore a wideband signal, that is, an original waveform of a narrowband signal, from the narrowband signal by incorporating a characteristic of a voice inputted to the first transmission terminal <NUM>-<NUM> (e.g., a voice characteristic of a user of the first transmission terminal <NUM>-<NUM>). Accordingly, the reception terminal <NUM> may restore, to the wideband signal, the narrowband signal received from the transmission terminal <NUM>-<NUM> by using the first artificial neural network.

<FIG> is a diagram illustrating a method of transmitting, by a transmission terminal, a voice signal according to an embodiment of the disclosure.

With reference to <FIG>, a transmission terminal according to an embodiment of the disclosure may first identify whether a reception terminal, that is, the subject of session negotiations, has stored neural network coefficient information related to the transmission terminal (<NUM>). If the reception terminal does not store the neural network coefficient information related to the transmission terminal, as in a conventional technology, the transmission terminal may generate a wideband signal based on an inputted voice, may generate a narrowband signal by down-sampling the wideband signal, and may transmit the generated narrowband signal to the reception terminal (<NUM>). In this case, the reception terminal may restore the wideband signal by up-sampling the received narrowband signal through spline interpolation, etc. by using the same method as a conventional method. If the reception terminal stores the neural network coefficient information related to the transmission terminal, the transmission terminal may transmit, to the reception terminal, the neural network structure information related to the transmission terminal so that the reception terminal can restore a learnt artificial neural network (<NUM>). Thereafter, the transmission terminal may generate a wideband signal based on an inputted voice, may generate a narrowband signal by down-sampling the wideband signal, and may transmit the generated narrowband signal to the reception terminal (<NUM>). In this case, the reception terminal may restore an artificial neural network based on the neural network structure information received from the transmission terminal and the previously stored neural network coefficient information, and may restore, to the wideband signal, the narrowband signal received from the transmission terminal by using the artificial neural network. Neural network coefficient information according to an embodiment may be a set of coefficients that constitute an artificial neural network learnt in a server. The neural network structure information may include at least one of the number of layers constituting an artificial neural network, the number of filters for each layer, the sizes of filters for each layer, an activation function, a stride, and padding. The neural network structure information has been described with reference to the previous drawing, and a detailed description thereof is omitted.

<FIG> is a diagram illustrating a method of receiving, by a reception terminal, a voice signal according to an embodiment of the disclosure.

With reference to <FIG>, a reception terminal according to an embodiment of the disclosure may identify whether the reception terminal has stored neural network coefficient information related to a transmission terminal, that is, the subject of session negotiations (<NUM>). If the reception terminal does not store the neural network coefficient information related to the transmission terminal, the reception terminal may restore a wideband signal by up-sampling a narrowband signal received from the transmission terminal through spline interpolation, etc. by using the same method as a conventional method (<NUM>). If the reception terminal stores the neural network coefficient information related to the transmission terminal, the reception terminal may receive, from the transmission terminal, neural network structure information for restoring an artificial neural network (<NUM>). In this case, the reception terminal may restore the artificial neural network based on the neural network structure information received from the transmission terminal and the previously stored neural network coefficient information (<NUM>), and may restore, to a wideband signal, a narrowband signal received from the transmission terminal (<NUM>) by using the restored artificial neural network. Neural network coefficient information according to an embodiment may be a set of coefficients that constitute an artificial neural network learnt in a server. The neural network structure information may include at least one of the number of layers constituting an artificial neural network, the number of filters for each layer, the sizes of filters for each layer, an activation function, a stride, and padding. The neural network structure information has been described with reference to the previous drawing, and a detailed description thereof is omitted.

<FIG> is a diagram illustrating a method of a server according to an embodiment of the disclosure.

With reference to <FIG>, a server according to an embodiment of the disclosure may perform the learning of an artificial neural network based on a wideband signal and a narrowband signal (<NUM>). According to an embodiment, the server may receive, from a transmission terminal, the wideband signal or the narrowband signal to be used as learning data for the artificial neural network. The server may received, from the transmission terminal, both the wideband signal and the narrowband signal corresponding to the wideband signal, or may receive the wideband signal from the transmission terminal and generate the narrowband signal by down-sampling the wideband signal. The server may perform the learning of the artificial neural network so that the wideband signal corresponding to an original waveform of the narrowband signal is restored from the narrowband signal down-sampled by using the wideband signal and the narrowband signal as learning data and is outputted. Thereafter, the server may generate neural network coefficient information related to the learnt artificial neural network (<NUM>), and may transmit the neural network coefficient information to the reception terminal (<NUM>).

According to an embodiment, a server may perform the learning of an artificial neural network by using a wideband signal and a narrowband signal as learning data based on neural network structure information related to a transmission terminal. The server may receive, from the transmission terminal, the neural network structure information related to the transmission terminal, or the neural network structure information related to the transmission terminal may be previously stored in the server in accordance with the transmission terminal. Furthermore, neural network coefficient information according to an embodiment of the disclosure may be a set of coefficients that constitute an artificial neural network learnt in a server. The neural network structure information may include at least one of the number of layers constituting an artificial neural network, the number of filters for each layer, the sizes of filters for each layer, an activation function, a stride, and padding. The neural network structure information has been described with reference to the previous drawing, and a detailed description thereof is omitted. According to an embodiment, the transmission of the neural network coefficient information from the server to the reception terminal and the storage of the neural network coefficient information in the reception terminal accordingly may be performed before session negotiations between the transmission terminal and the reception terminal are performed.

As described above, in this specification, the artificial neural network learning process has been described as being performed by a separated server, for convenience of description, but this does not restrict the scope of the disclosure. The artificial neural network learning process may be implemented by any apparatus capable of implementing the steps performed by the aforementioned server, for example, a terminal.

<FIG> is a diagram illustrating a construction of a transmission terminal <NUM> according to an embodiment of the disclosure.

With reference to <FIG>, a transmission terminal <NUM> may include a controller <NUM>, a transceiver unit <NUM>, and a memory <NUM>. However, the components of the transmission terminal <NUM> are not limited to the aforementioned example. For example, the transmission terminal <NUM> may include more components or less components than the aforementioned components. Furthermore, the controller <NUM>, the transceiver unit <NUM>, and the memory <NUM> may be implemented in one chip form.

According to an embodiment of the disclosure, the controller <NUM> may control a series of processes on which the transmission terminal <NUM> may operated according to the aforementioned embodiments of the disclosure. For example, the controller <NUM> according to an embodiment of the disclosure may generate a wideband signal based on an inputted voice, may generate a narrowband signal by down-sampling a voice signal, may perform the learning of an artificial neural network by using the generated wideband signal and narrowband signal as learning data, may generate neural network coefficient information and neural network structure information based on the learnt artificial neural network, and may control the transmission terminal <NUM> to deliver or transmit the neural network coefficient information and neural network structure information to a reception terminal <NUM>. Furthermore, the controller <NUM> according to an embodiment of the disclosure may control the transceiver unit <NUM> and the memory <NUM> so that the transceiver unit <NUM> and the memory <NUM> perform an operation necessary to perform an operation according to the aforementioned embodiments of the disclosure.

According to an embodiment of the disclosure, the transceiver unit <NUM> may transmit and receive signals to and from the reception terminal <NUM> or a server <NUM>. The aforementioned signal may include a voice signal and data.

According to an embodiment of the disclosure, the memory <NUM> may store a program and data necessary for an operation of the terminal. Furthermore, the memory <NUM> may store control information or data included in signals transmitted and received by the terminal. The memory <NUM> may be constructed as a storage medium, such as a ROM, a RAM, a hard disk, a CD-ROM, and a DVD, or a combination of storage media. Furthermore, the memory <NUM> may be constructed as a plurality of pieces of memory.

<FIG> is a diagram illustrating a construction of a reception terminal <NUM> according to an embodiment of the disclosure.

With reference to <FIG>, the reception terminal <NUM> may include a controller <NUM>, a transceiver unit <NUM>, and a memory <NUM>. However, the components of the reception terminal <NUM> are not limited to the aforementioned example. For example, the reception terminal <NUM> may include more components or less components than the aforementioned components. Furthermore, the controller <NUM>, the transceiver unit <NUM>, and the memory <NUM> may be implemented in one chip form.

According to an embodiment of the disclosure, the controller <NUM> may control a series of processes on which the reception terminal <NUM> may operate according to the aforementioned embodiments of the disclosure. For example, the controller <NUM> according to an embodiment of the disclosure may receive a narrowband signal from the transmission terminal <NUM>, may receive neural network coefficient information and neural network structure information, may restore an artificial neural network based on the received neural network coefficient information and neural network structure information, and may control the reception terminal <NUM> to convert, into a wideband signal, a narrowband signal received from the transmission terminal <NUM> by using the restored artificial neural network. Furthermore, the controller <NUM> according to an embodiment of the disclosure may control the transceiver unit <NUM> and the memory <NUM> so that the transceiver unit <NUM> and the memory <NUM> perform an operation necessary to perform an operation according to the aforementioned embodiments of the disclosure.

According to an embodiment of the disclosure, the transceiver unit <NUM> may transmit and receive signals to and from the transmission terminal <NUM> or the server <NUM>. The aforementioned signal may include a voice signal and data.

<FIG> is a diagram illustrating a construction of a server according to an embodiment of the disclosure.

With reference to <FIG>, a server <NUM> may include a controller <NUM>, a connection unit <NUM>, and a memory <NUM>. However, the components of the server are not limited to the aforementioned example. For example, the server may include more components or less components than the aforementioned components. Furthermore, the controller <NUM>, the connection unit <NUM>, and the memory <NUM> may be implemented in one chip form.

According to an embodiment of the disclosure, the controller <NUM> may control a series of processes on which the server <NUM> may operate according to the aforementioned embodiments of the disclosure. For example, the controller <NUM> according to an embodiment of the disclosure may receive a signal from the transmission terminal <NUM>, may perform the learning of an artificial neural network, may generate neural network coefficient information, and may control the server <NUM> to transmit the neural network coefficient information to the reception terminal <NUM>. Furthermore, the controller <NUM> according to an embodiment of the disclosure may control the connection unit <NUM> and the memory <NUM> so that the connection unit <NUM> and the memory <NUM> perform an operation necessary to perform an operation according to the aforementioned embodiments of the disclosure.

According to an embodiment of the disclosure, the connection unit <NUM> may be connected to a network entity that transmits and receives signals to and from the transmission terminal <NUM> or the reception terminal <NUM>. The aforementioned signal may include a voice signal and data.

Methods according to embodiments written in the claims or specification of the disclosure may be implemented in the form of hardware, software, or a combination of hardware and software.

If a method is implemented as software, a computer-readable storage medium or a computer program product which stores one or more programs (software modules) may be provided. The one or more programs stored in the computer-readable storage medium or the computer program product are configured for being executable by one or more processors within an electronic device. The one or more programs include instructions that enable the electronic device to execute methods according to embodiments, which are written in the claims or specification of the disclosure.

Such a program (software module or software) may be stored in a random access memory, a nonvolatile memory including a flash memory, a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic disc storage device, a compact disc-ROM (CD-ROM), digital versatile discs (DVDs) or other types of optical storage devices, or a magnetic cassette. Alternatively, the program (software module or software) may be stored in a memory consisting of a combination of all or some of them. Furthermore, multiple configuration memories may be included.

Furthermore, the program may be stored in an attachable storage device which may be accessed over a communication network, such as the Internet, Intranet, a local area network (LAN), a wide LAN (WLAN), or a storage area network (SAN), or a communication network consisting of a combination of them. Such a storage device may access a device that performs an embodiment of the disclosure through an external port. Furthermore, a separate storage device in a communication network may access a device that performs an embodiment of the disclosure.

In the aforementioned detailed embodiments of the disclosure, the components included in the disclosure have been expressed in the singular or plural form depending on a proposed detailed embodiment. However, the singular or plural expression has been selected suitably for a situation proposed for convenience of description, and the disclosure is not limited to singular or plural components. Although a component has been expressed in the plural form, it may be configured in the singular form. Although a component has been expressed in the singular form, it may be configured in the plural form.

Claim 1:
A method performed by a transmission terminal (<NUM>, <NUM>, <NUM>-<NUM>, <NUM>) in a communication system, the method comprising:
identifying whether neural network coefficient information related to the transmission terminal (<NUM>, <NUM>, <NUM>-<NUM>, <NUM>) has been stored in a reception terminal (<NUM>, <NUM>, <NUM>, <NUM>);
transmitting, to the reception terminal (<NUM>, <NUM>, <NUM>, <NUM>), neural network structure information related to the transmission terminal (<NUM>, <NUM>, <NUM>-<NUM>, <NUM>) based on the neural network coefficient information identified as having been stored in the reception terminal (<NUM>, <NUM>, <NUM>, <NUM>);
generating a wideband voice signal based on an inputted voice;
generating a narrowband voice signal by down-sampling the wideband voice signal; and
transmitting the narrowband voice signal to the reception terminal (<NUM>, <NUM>, <NUM>, <NUM>).