Office Action 1: Non-Final Rejection (2025-02-04)
Rejection types: 103 - Obviousness

CLAIMS:
WHAT IS CLAIMED IS:

1\. A washing machine appliance, comprising:
a wash tub positioned within a cabinet;
a wash basket rotatably mounted within the wash tub and defining a wash chamber configured for receiving a load of clothes;
a water supply configured to selectively dispense a flow of wash fluid through a discharge nozzle into the wash tub;
a bulk dispenser for selectively adding a wash additive to the flow of wash fluid;
a controller operably coupled to the water supply, the bulk dispenser, and a camera assembly, the controller being configured to:
initiate a bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive;
obtain one or more images of the flow of wash fluid using the camera assembly;
analyze the one or more images using an image recognition process to determine an additive content of the flow of wash fluid;
determine that the additive content falls below a predetermined additive threshold; and
terminate the bulk flush operation in response to determining that the additive content falls below the predetermined additive threshold.

2\. The washing machine appliance of claim 1, wherein the controller is further configured to:
determine that the additive content exceeds the predetermined additive threshold; and
continue the bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive.

3\. The washing machine appliance of claim 1, wherein the controller is further configured to: analyze the one or more images using the image recognition process to determine that a flow rate of the flow of wash fluid falls below a flow rate threshold;
and provide a user notification that there is a fault condition with the water supply.

4\. The washing machine appliance of claim 1, wherein terminating the bulk flush operation comprises stopping the flow of wash fluid using the water supply and stopping the dispensing of the wash additive using the bulk dispenser.

5\. The washing machine appliance of claim 1, wherein terminating the bulk flush operation comprises providing a user notification that a bulk dispensing tank is empty.

6\. The washing machine appliance of claim 5, further comprising:
a user interface panel, wherein the user notification is provided through the user interface panel.

7\. The washing machine appliance of claim 5, wherein the controller is in operative communication with a remote device through an external network, and
wherein the user notification is provided through the remote device.

8\. The washing machine appliance of claim 1, wherein the image recognition process is a machine learning image recognition process comprising at least one of a convolution neural network ("CNN"), a region-based convolution neural network ("R-CNN"), a deep belief network ("DBN"), or a deep neural network ("DNN") image recognition process.

9\. The washing machine appliance of claim 1, wherein the camera assembly is mounted within the cabinet in view of the flow of wash fluid.

10\. The washing machine appliance of claim 1, wherein the camera assembly is mounted above the wash chamber and faces the discharge nozzle or the wash chamber.

11\. The washing machine appliance of claim 1, wherein the controller is in operative communication with a remote device through an external network, wherein the remote device comprises the camera assembly.

12\. The washing machine appliance of claim 1, wherein the washing machine appliance is a vertical axis washing machine appliance.

13\. A method of operating a washing machine appliance, the washing machine appliance comprising a wash basket rotatably mounted within a wash tub and defining a wash chamber configured for receiving a load of clothes, a water supply configured to selectively dispense a flow of wash fluid through a discharge nozzle into the wash tub, and a bulk dispenser for selectively adding a wash additive to the flow of wash fluid, the method comprising:
initiating a bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive;
obtaining one or more images of the flow of wash fluid using a camera assembly;
analyzing the one or more images using an image recognition process to determine an additive content of the flow of wash fluid;
determining that the additive content falls below a predetermined additive threshold; and
terminating the bulk flush operation in response to determining that the additive content falls below the predetermined additive threshold.

14\. The method of claim 13, further comprising:
determining that the additive content exceeds the predetermined additive threshold; and continuing the bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive.

15\. The method of claim 13, further comprising:
analyzing the one or more images using the image recognition process to determine that a flow rate of the flow of wash fluid falls below a flow rate threshold;
and providing a user notification that there is a fault condition with the water supply.

16\. The method of claim 13, wherein terminating the bulk flush operation comprises stopping the flow of wash fluid using the water supply and stopping the dispensing of the wash additive using the bulk dispenser.

17\. The method of claim 13, wherein terminating the bulk flush operation comprises providing a user notification that a bulk dispensing tank is empty.

18\. The method of claim 13, wherein the image recognition process is a machine learning image recognition process comprising at least one of a convolution neural network ("CNN"), a region-based convolution neural network ("R-CNN"), a deep belief network ("DBN"), or a deep neural network ("DNN") image recognition process.

19\. The method of claim 13, wherein the camera assembly is mounted within a cabinet of the washing machine appliance in view of the flow of wash fluid.

20\. The method of claim 13, wherein the washing machine appliance is in operative communication with a remote device through an external network, wherein the remote device comprises the camera assembly. 24 25 26 27

OFFICE ACTION:
DETAILED ACTION
Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2, 4-14, and 16-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 2020/0071871), and in further view of Gallagher (US 2017/0037559).
Regarding claim 1, Kim figures 1-4 teach a washing machine appliance, comprising:
a wash tub (8 tub) positioned within a cabinet (2);
a wash basket (10 drum) rotatably mounted within the wash tub and defining a wash chamber configured for receiving a load of clothes;
a water supply (30 water supply part) configured to selectively dispense a flow of wash fluid into the wash tub;
a bulk dispenser (60 detergent supply part) for selectively adding a wash additive to the flow of wash fluid;
a controller (360) operably coupled to the water supply, the bulk dispenser, and a camera assembly (322 image sensor).[0037-41][0054-57]
Kim teaches a detergent bubble recognizer 110 for recognizing presence of detergent bubble from image data. The detergent bubble recognizer 110 may recognize presence of detergent bubble from the image data acquired by the image sensor 322 using an image recognition model based on machine learning as a field of artificial intelligence.  As an example of artificial intelligence technology applied to the detergent bubble recognizer 110 of the additional rinsing control module 100. The additional rinsing control module 100 may include a bubble ratio calculator 130 for calculating the ratio of detergent bubble based on the detergent bubble region extracted from the image data. The additional rinsing control module 100 may include an additional rinsing setting unit 140 for setting an additional rinsing cycle based on the ratio of detergent bubble calculated by the bubble ratio calculator 130. The additional rinsing setting unit 140 may set the number of times of performing the additional rinsing cycle and/or a time when the additional rinsing cycle is performed based on the calculated ratio of detergent bubble. The washing machine 1 according to the embodiment of the present invention may recognize presence of residual detergent (detergent bubble) from image data using an image recognition model based on artificial intelligence and perform control to perform the additional rinsing cycle based on the result of recognition, thereby more efficiently performing the rinsing cycle of the washing operation thereby reading on the controller being configured to obtain one or more images of the flow of wash fluid using the camera assembly, analyze the one or more images using an image recognition process to determine an additive content of the flow of wash fluid; determine that the additive content falls below a predetermined additive threshold; and terminate the bulk flush operation in response to determining that the additive content falls below the predetermined additive threshold.[0074, 0077, 0110, 0112-113, 0116]
Kim is silent to a discharge nozzle. Kim is also silent to the controller being configured to: initiate a bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive.
Gallagher is directed towards a washing machine wherein the mixture of water and detergent outputted by the pump 126 can be sprayed from the nozzle 66 generally directly down and along a side of the basket while the basket is being rotated thereby reading on a discharge nozzle and the controller being configured to initiate a bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive.[0141]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to provide the nozzle and operation of Gallagher in the washing machine of Kim to provide a proper dose of treating chemistry to the laundry.[0005]
Regarding claim 2, Kim teaches the washing machine 1 according to the embodiment of the present invention may recognize presence of residual detergent (detergent bubble) from image data using an image recognition model based on artificial intelligence and perform control to perform the additional rinsing cycle based on the result of recognition, thereby more efficiently performing the rinsing cycle of the washing operation thereby suggesting the controller is further configured to: determine that the additive content exceeds the predetermined additive threshold; and continue the bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive.[0016]
Regarding claim 4, Kim teaches when the rinsing cycle refers to a cycle for discharging detergent and contaminants from a tub or a drum to the outside. After the rinsing cycle is completed, it is preferable that residual detergent is not present in the tub or the drum. However, in a conventional washing machine, detergent may remain in the laundry or the drum (the internal glass of a door) even after the rinsing cycle is completed. When the washing operation is completed and the laundry is dried in a state in which the detergent remains, the laundry may deteriorate or the skin of a user may be damaged thereby reading on wherein terminating the bulk flush operation comprises stopping the flow of wash fluid using the water supply and stopping the dispensing of the wash additive using the bulk dispenser.[0006]
Regarding claim 5, Gallagher teaches a bubble detection sensor was described for determining when the bulk container 120 is empty or nearing empty thereby suggesting it to be an obvious modification wherein terminating the bulk flush operation comprises providing a user notification that a bulk dispensing tank is empty.[0185]
Regarding claim 6, Kim teaches output unit 330 may include an output portion for notifying the user of a variety of information related to operation of the washing machine 1. For example, the output unit 330 may include a speaker or a buzzer as an audio output unit, and include a display as a graphic or text output unit. For example, the output unit 330 may be disposed on the control panel 20 of FIG. 1, without being limited thereto, thereby reading on a user interface panel, wherein the user notification is provided through the user interface panel.[0059]
Regarding claim 7, Kim teaches communication unit 350 may include at least one communication module for connecting the washing machine 1 to a server or a user terminal. For example, the communication unit 350 may include a short-range communication module such as Bluetooth or near field communication (NFC), a wireless Internet module such as Wi-Fi, or a mobile communication module. The controller 360 may transmit the state information or operation information of the washing machine 1 to the server or the terminal through the communication unit 350 thereby reading on the controller is in operative communication with a remote device through an external network, and wherein the user notification is provided through the remote device.[0062]
Regarding claim 8, Kim teaches the detergent bubble recognizer 110 may recognize presence of detergent bubble from the image data acquired by the image sensor 322 using an image recognition model based on machine learning as a field of artificial intelligence. For example, the image recognition model may include a software or hardware artificial neural network (ANN). For example, the artificial neural network may include a convolutional neural network (CNN) learned through deep learning, a recurrent neural network (RNN), and a deep neural network (DNN) such as a deep belief network (DBN) thereby reading on the image recognition process is a machine learning image recognition process comprising at least one of a convolution neural network (CNN), a region-based convolution neural network (R-CNN), a deep belief network (DBN), or a deep neural network (DNN) image recognition process.[0074]
Regarding claim 9, Kim teaches mage sensor 322 may be included in at least one camera installed inside or outside the washing machine 1 thereby reading on the camera assembly is mounted within the cabinet in view of the flow of wash fluid.[0055]
Regarding claim 10, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to provide the camera assembly is mounted above the wash chamber and faces the discharge nozzle or the wash chamber as a rearrangement of parts is an obvious design choice.(MPEP 2144.04)
Regarding claim 11, Kim teaches communication unit 350 may include at least one communication module for connecting the washing machine 1 to a server or a user terminal. For example, the communication unit 350 may include a short-range communication module such as Bluetooth or near field communication (NFC), a wireless Internet module such as Wi-Fi, or a mobile communication module. The controller 360 may transmit the state information or operation information of the washing machine 1 to the server or the terminal through the communication unit 350 thereby suggesting the controller is in operative communication with a remote device through an external network, wherein the remote device comprises the camera assembly.[0062]
Regarding claim 12, Gallagher figure 21-26 teach the washing machine appliance is a vertical axis washing machine appliance.[0143]
Regarding claim 13, Gallagher teaches a method of operating a washing machine appliance, the washing machine appliance comprising a wash basket (10 drum) rotatably mounted within a wash tub (8 tub) and defining a wash chamber configured for receiving a load of clothes, a water supply (30 water supply part) configured to selectively dispense a flow of wash fluid into the wash tub, and a bulk dispenser(60 detergent supply part) for selectively adding a wash additive to the flow of wash fluid.[0037-41][0054-57]
Kim teaches a detergent bubble recognizer 110 for recognizing presence of detergent bubble from image data. The detergent bubble recognizer 110 may recognize presence of detergent bubble from the image data acquired by the image sensor 322 using an image recognition model based on machine learning as a field of artificial intelligence.  as an example of artificial intelligence technology applied to the detergent bubble recognizer 110 of the additional rinsing control module 100. The additional rinsing control module 100 may include a bubble ratio calculator 130 for calculating the ratio of detergent bubble based on the detergent bubble region extracted from the image data. The additional rinsing control module 100 may include an additional rinsing setting unit 140 for setting an additional rinsing cycle based on the ratio of detergent bubble calculated by the bubble ratio calculator 130. The additional rinsing setting unit 140 may set the number of times of performing the additional rinsing cycle and/or a time when the additional rinsing cycle is performed based on the calculated ratio of detergent bubble. The washing machine 1 according to the embodiment of the present invention may recognize presence of residual detergent (detergent bubble) from image data using an image recognition model based on artificial intelligence and perform control to perform the additional rinsing cycle based on the result of recognition, thereby more efficiently performing the rinsing cycle of the washing operation thereby reading obtaining one or more images of the flow of wash fluid using a camera assembly; analyzing the one or more images using an image recognition process to determine an additive content of the flow of wash fluid; determining that the additive content falls below a predetermined additive threshold; and terminating the bulk flush operation in response to determining that the additive content falls below the predetermined additive threshold.
Kim is silent to a discharge nozzle. Kim is also silent to initiating a bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive.
Gallagher is directed towards a washing machine wherein the mixture of water and detergent outputted by the pump 126 can be sprayed from the nozzle 66 generally directly down and along a side of the basket while the basket is being rotated thereby reading on a discharge nozzle and the controller being configured to initiate a bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive.[0141]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to provide the nozzle and operation of Gallagher in the washing machine of Kim to provide a proper dose of treating chemistry to the laundry.[0005]
Regarding claim 14, Kim teaches the washing machine 1 according to the embodiment of the present invention may recognize presence of residual detergent (detergent bubble) from image data using an image recognition model based on artificial intelligence and perform control to perform the additional rinsing cycle based on the result of recognition, thereby more efficiently performing the rinsing cycle of the washing operation thereby suggesting determining that the additive content exceeds the predetermined additive threshold; and continuing the bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive.[0016]
Regarding claim 16, Kim teaches when the rinsing cycle refers to a cycle for discharging detergent and contaminants from a tub or a drum to the outside. After the rinsing cycle is completed, it is preferable that residual detergent is not present in the tub or the drum. However, in a conventional washing machine, detergent may remain in the laundry or the drum (the internal glass of a door) even after the rinsing cycle is completed. When the washing operation is completed and the laundry is dried in a state in which the detergent remains, the laundry may deteriorate or the skin of a user may be damaged thereby reading wherein terminating the bulk flush operation comprises stopping the flow of wash fluid using the water supply and stopping the dispensing of the wash additive using the bulk dispenser.[0006]
Regarding claim 17, Gallagher teaches a bubble detection sensor was described for determining when the bulk container 120 is empty or nearing empty thereby suggesting it to be an obvious modification wherein terminating the bulk flush operation comprises providing a user notification that a bulk dispensing tank is empty.[0185]
Regarding claim 18, Kim teaches the detergent bubble recognizer 110 may recognize presence of detergent bubble from the image data acquired by the image sensor 322 using an image recognition model based on machine learning as a field of artificial intelligence. For example, the image recognition model may include a software or hardware artificial neural network (ANN). For example, the artificial neural network may include a convolutional neural network (CNN) learned through deep learning, a recurrent neural network (RNN), and a deep neural network (DNN) such as a deep belief network (DBN) thereby reading on wherein the image recognition process is a machine learning image recognition process comprising at least one of a convolution neural network (CNN), a region-based convolution neural network (R-CNN), a deep belief network (DBN), or a deep neural network (DNN) image recognition process.[0074]
Regarding claim 19, Kim teaches mage sensor 322 may be included in at least one camera installed inside or outside the washing machine 1 thereby reading the camera assembly is mounted within a cabinet of the washing machine appliance in view of the flow of wash fluid.[0055]
Regarding claim 20, Kim teaches communication unit 350 may include at least one communication module for connecting the washing machine 1 to a server or a user terminal. For example, the communication unit 350 may include a short-range communication module such as Bluetooth or near field communication (NFC), a wireless Internet module such as Wi-Fi, or a mobile communication module. The controller 360 may transmit the state information or operation information of the washing machine 1 to the server or the terminal through the communication unit 350 thereby suggesting the washing machine appliance is in operative communication with a remote device through an external network, wherein the remote device comprises the camera assembly.[0062]
Claim(s) 3 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 2020/0071871) and Gallagher (US 2017/0037559), as applied to claim 1, and in further view of Ha (US 2021/0025104).
Regarding claim 3, Kim in view of Gallagher is silent to the controller is further configured to: analyze the one or more images using the image recognition process to determine that a flow rate of the flow of wash fluid falls below a flow rate threshold; and provide a user notification that there is a fault condition with the water supply.
Ha is directed towards an apparatus and method for detecting water supply in a household appliance wherein FIGS. 6B to 6E are exemplary views illustrating a state in which water supply detection image data designates the water supplier in the tub as the water supply region of interest based on data having 3 to 5 second images (30 frames per second) and uses the optical flow algorithm as the vision algorithm. Referring to FIGS. 6B to 6E, the optical flow is detected, and a magnitude of the optical vector has a minimum threshold value of 208 and is detected from 208 or more. The optical vector is represented by a set of dots at the part at the upper end region of the tub where the water is supplied, the optical vector means that the darker and wider the optical vector, the more washing water is supplied. In addition, when the water supply is stably performed, the optical vector has a wide distribution and the vector values are arranged in the region of interest.[0087-93]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to provide the apparatus and method of Ha in the washing machine of Kim and Gallagher to monitor the blockage in the water supply path.[0002]
Regarding claim 15, Ha is directed towards an apparatus and method for detecting water supply in a household appliance used for water supply equipment that needs to monitor the blockage in the water supply path. FIGS. 6B to 6E are exemplary views illustrating a state in which water supply detection image data designates the water supplier in the tub as the water supply region of interest based on data having 3 to 5 second images (30 frames per second) and uses the optical flow algorithm as the vision algorithm. Referring to FIGS. 6B to 6E, the optical flow is detected, and a magnitude of the optical vector has a minimum threshold value of 208 and is detected from 208 or more. The optical vector is represented by a set of dots at the part at the upper end region of the tub where the water is supplied, the optical vector means that the darker and wider the optical vector, the more washing water is supplied. In addition, when the water supply is stably performed, the optical vector has a wide distribution and the vector values are arranged in the region of interest thereby reading on analyzing the one or more images using the image recognition process to determine that a flow rate of the flow of wash fluid falls below a flow rate threshold; and providing a user notification that there is a fault condition with the water supply.[0002][0087-93]
Conclusion


NEXT OFFICE ACTION: DETAILED ACTION
Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see pages 6-8, filed March 18, 2024, with respect to the rejection(s) of claim(s) 1-20 under 35 U.S.C. 103 have been fully considered and are persuasive.  Therefore, the rejection has been withdrawn.  However, upon further consideration, a new ground(s) of rejection is made in view of Yoon (US 2023/0067550).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2, 4-8, 11-14, 16-18, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 2020/0071871), and in further view of Yoon (US 2023/0067550) and Gallagher (US 2017/0037559).
Regarding claim 1, Kim figures 1-4 teach a washing machine appliance, comprising:
a wash tub (8 tub) positioned within a cabinet (2);
a wash basket (10 drum) rotatably mounted within the wash tub and defining a wash chamber configured for receiving a load of clothes;
a water supply (30 water supply part) configured to selectively dispense a flow of wash fluid into the wash tub;
a bulk dispenser (60 detergent supply part) for selectively adding a wash additive to the flow of wash fluid;
a controller (360) operably coupled to the water supply, the bulk dispenser, and a camera assembly (322 image sensor).[0037-41][0054-57]
Kim teaches a detergent bubble recognizer 110 for recognizing presence of detergent bubble from image data. The detergent bubble recognizer 110 may recognize presence of detergent bubble from the image data acquired by the image sensor 322 using an image recognition model based on machine learning as a field of artificial intelligence.  As an example of artificial intelligence technology applied to the detergent bubble recognizer 110 of the additional rinsing control module 100. The additional rinsing control module 100 may include a bubble ratio calculator 130 for calculating the ratio of detergent bubble based on the detergent bubble region extracted from the image data. The additional rinsing control module 100 may include an additional rinsing setting unit 140 for setting an additional rinsing cycle based on the ratio of detergent bubble calculated by the bubble ratio calculator 130. The additional rinsing setting unit 140 may set the number of times of performing the additional rinsing cycle and/or a time when the additional rinsing cycle is performed based on the calculated ratio of detergent bubble. The washing machine 1 according to the embodiment of the present invention may recognize presence of residual detergent (detergent bubble) from image data using an image recognition model based on artificial intelligence and perform control to perform the additional rinsing cycle based on the result of recognition, thereby more efficiently performing the rinsing cycle of the washing operation thereby reading on the controller being configured to obtain one or more images of the flow of wash fluid using the camera assembly, analyze the one or more images using an image recognition process to determine an additive content of the flow of wash fluid; determine that the additive content falls below a predetermined additive threshold; and terminate the bulk flush operation in response to determining that the additive content falls below the predetermined additive threshold.[0074, 0077, 0110, 0112-113, 0116]
Kim is silent to a discharge nozzle, a camera assembly mounted within the cabinet above the wash chamber and faces the discharge nozzle to monitor the flow of wash fluid, and the controller configured to obtain one or more images of the flow of wash fluid using the camera assembly before the wash fluid reaches the load of clothes.
Yoon is directed towards a washing machine where figure 3 teaches camera 182 of camera assembly 180 may be oriented such that it has a field of view that encompasses the flow of wash fluid 178 as dispensed from discharge nozzle 176. The images may be used to determine whether the flow of wash fluid 178 contains detergent or other additives.[0056]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to provide a camera and nozzle configuration as taught in Yoon to accurately and efficiently identify issues with the detergent dispenser.[0004]
Kim in view of Yoon is also silent to the controller being configured to: initiate a bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive.
Gallagher is directed towards a washing machine having a special flushing cycle for the pump 126 would need to be executed prior to switching chemistries to avoid contamination  thereby reading on the controller being configured to initiate a bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive.[0184]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to provide the operation of Gallagher in the washing machine of Kim and Yoon to provide a avoid contamination when switching chemistries.[0184]
Regarding claim 2, Kim teaches the washing machine 1 according to the embodiment of the present invention may recognize presence of residual detergent (detergent bubble) from image data using an image recognition model based on artificial intelligence and perform control to perform the additional rinsing cycle based on the result of recognition, thereby more efficiently performing the rinsing cycle of the washing operation thereby suggesting the controller is further configured to: determine that the additive content exceeds the predetermined additive threshold; and continue the bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive.[0016]
Regarding claim 4, Kim teaches when the rinsing cycle refers to a cycle for discharging detergent and contaminants from a tub or a drum to the outside. After the rinsing cycle is completed, it is preferable that residual detergent is not present in the tub or the drum. However, in a conventional washing machine, detergent may remain in the laundry or the drum (the internal glass of a door) even after the rinsing cycle is completed. When the washing operation is completed and the laundry is dried in a state in which the detergent remains, the laundry may deteriorate or the skin of a user may be damaged thereby reading on wherein terminating the bulk flush operation comprises stopping the flow of wash fluid using the water supply and stopping the dispensing of the wash additive using the bulk dispenser.[0006]
Regarding claim 5, Gallagher teaches a bubble detection sensor was described for determining when the bulk container 120 is empty or nearing empty thereby suggesting it to be an obvious modification wherein terminating the bulk flush operation comprises providing a user notification that a bulk dispensing tank is empty.[0185]
Regarding claim 6, Kim teaches output unit 330 may include an output portion for notifying the user of a variety of information related to operation of the washing machine 1. For example, the output unit 330 may include a speaker or a buzzer as an audio output unit, and include a display as a graphic or text output unit. For example, the output unit 330 may be disposed on the control panel 20 of FIG. 1, without being limited thereto, thereby reading on a user interface panel, wherein the user notification is provided through the user interface panel.[0059]
Regarding claim 7, Kim teaches communication unit 350 may include at least one communication module for connecting the washing machine 1 to a server or a user terminal. For example, the communication unit 350 may include a short-range communication module such as Bluetooth or near field communication (NFC), a wireless Internet module such as Wi-Fi, or a mobile communication module. The controller 360 may transmit the state information or operation information of the washing machine 1 to the server or the terminal through the communication unit 350 thereby reading on the controller is in operative communication with a remote device through an external network, and wherein the user notification is provided through the remote device.[0062]
Regarding claim 8, Kim teaches the detergent bubble recognizer 110 may recognize presence of detergent bubble from the image data acquired by the image sensor 322 using an image recognition model based on machine learning as a field of artificial intelligence. For example, the image recognition model may include a software or hardware artificial neural network (ANN). For example, the artificial neural network may include a convolutional neural network (CNN) learned through deep learning, a recurrent neural network (RNN), and a deep neural network (DNN) such as a deep belief network (DBN) thereby reading on the image recognition process is a machine learning image recognition process comprising at least one of a convolution neural network (CNN), a region-based convolution neural network (R-CNN), a deep belief network (DBN), or a deep neural network (DNN) image recognition process.[0074]
Regarding claim 11, Kim teaches communication unit 350 may include at least one communication module for connecting the washing machine 1 to a server or a user terminal. For example, the communication unit 350 may include a short-range communication module such as Bluetooth or near field communication (NFC), a wireless Internet module such as Wi-Fi, or a mobile communication module. The controller 360 may transmit the state information or operation information of the washing machine 1 to the server or the terminal through the communication unit 350 thereby suggesting the controller is in operative communication with a remote device through an external network, wherein the remote device comprises the camera assembly.[0062]
Regarding claim 12, Gallagher figure 21-26 teach the washing machine appliance is a vertical axis washing machine appliance.[0143]
Regarding claim 13, Gallagher teaches a method of operating a washing machine appliance, the washing machine appliance comprising a wash basket (10 drum) rotatably mounted within a wash tub (8 tub) and defining a wash chamber configured for receiving a load of clothes, a water supply (30 water supply part) configured to selectively dispense a flow of wash fluid into the wash tub, and a bulk dispenser(60 detergent supply part) for selectively adding a wash additive to the flow of wash fluid.[0037-41][0054-57]
Kim teaches a detergent bubble recognizer 110 for recognizing presence of detergent bubble from image data. The detergent bubble recognizer 110 may recognize presence of detergent bubble from the image data acquired by the image sensor 322 using an image recognition model based on machine learning as a field of artificial intelligence.  as an example of artificial intelligence technology applied to the detergent bubble recognizer 110 of the additional rinsing control module 100. The additional rinsing control module 100 may include a bubble ratio calculator 130 for calculating the ratio of detergent bubble based on the detergent bubble region extracted from the image data. The additional rinsing control module 100 may include an additional rinsing setting unit 140 for setting an additional rinsing cycle based on the ratio of detergent bubble calculated by the bubble ratio calculator 130. The additional rinsing setting unit 140 may set the number of times of performing the additional rinsing cycle and/or a time when the additional rinsing cycle is performed based on the calculated ratio of detergent bubble. The washing machine 1 according to the embodiment of the present invention may recognize presence of residual detergent (detergent bubble) from image data using an image recognition model based on artificial intelligence and perform control to perform the additional rinsing cycle based on the result of recognition, thereby more efficiently performing the rinsing cycle of the washing operation thereby reading obtaining one or more images of the flow of wash fluid using a camera assembly; analyzing the one or more images using an image recognition process to determine an additive content of the flow of wash fluid; determining that the additive content falls below a predetermined additive threshold; and terminating the bulk flush operation in response to determining that the additive content falls below the predetermined additive threshold.
Kim is silent to a discharge nozzle and obtaining one or more images of the flow of wash fluid using a camera assembly being mounted to face the discharge nozzle to monitor the flow of wash fluid before the wash fluid reaches the load of clothes.
Yoon is directed towards a washing machine where figure 3 teaches camera 182 of camera assembly 180 may be oriented such that it has a field of view that encompasses the flow of wash fluid 178 as dispensed from discharge nozzle 176. The images may be used to determine whether the flow of wash fluid 178 contains detergent or other additives.[0056]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to provide a camera and nozzle configuration as taught in Yoon to accurately and efficiently identify issues with the detergent dispenser.[0004]
Kim in view of Yoon  is also silent to initiating a bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive.
Gallagher is directed towards a washing machine having a special flushing cycle for the pump 126 would need to be executed prior to switching chemistries to avoid contamination  thereby reading on the controller being configured to initiate a bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive.[0184]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to provide the operation of Gallagher in the washing machine of Kim and Yoon to provide a avoid contamination when switching chemistries.[0184]
Regarding claim 14, Kim teaches the washing machine 1 according to the embodiment of the present invention may recognize presence of residual detergent (detergent bubble) from image data using an image recognition model based on artificial intelligence and perform control to perform the additional rinsing cycle based on the result of recognition, thereby more efficiently performing the rinsing cycle of the washing operation thereby suggesting determining that the additive content exceeds the predetermined additive threshold; and continuing the bulk flush operation by operating the water supply to provide the flow of wash fluid and the bulk dispenser to dispense the wash additive.[0016]
Regarding claim 16, Kim teaches when the rinsing cycle refers to a cycle for discharging detergent and contaminants from a tub or a drum to the outside. After the rinsing cycle is completed, it is preferable that residual detergent is not present in the tub or the drum. However, in a conventional washing machine, detergent may remain in the laundry or the drum (the internal glass of a door) even after the rinsing cycle is completed. When the washing operation is completed and the laundry is dried in a state in which the detergent remains, the laundry may deteriorate or the skin of a user may be damaged thereby reading wherein terminating the bulk flush operation comprises stopping the flow of wash fluid using the water supply and stopping the dispensing of the wash additive using the bulk dispenser.[0006]
Regarding claim 17, Gallagher teaches a bubble detection sensor was described for determining when the bulk container 120 is empty or nearing empty thereby suggesting it to be an obvious modification wherein terminating the bulk flush operation comprises providing a user notification that a bulk dispensing tank is empty.[0185]
Regarding claim 18, Kim teaches the detergent bubble recognizer 110 may recognize presence of detergent bubble from the image data acquired by the image sensor 322 using an image recognition model based on machine learning as a field of artificial intelligence. For example, the image recognition model may include a software or hardware artificial neural network (ANN). For example, the artificial neural network may include a convolutional neural network (CNN) learned through deep learning, a recurrent neural network (RNN), and a deep neural network (DNN) such as a deep belief network (DBN) thereby reading on wherein the image recognition process is a machine learning image recognition process comprising at least one of a convolution neural network (CNN), a region-based convolution neural network (R-CNN), a deep belief network (DBN), or a deep neural network (DNN) image recognition process.[0074]
Regarding claim 20, Kim teaches communication unit 350 may include at least one communication module for connecting the washing machine 1 to a server or a user terminal. For example, the communication unit 350 may include a short-range communication module such as Bluetooth or near field communication (NFC), a wireless Internet module such as Wi-Fi, or a mobile communication module. The controller 360 may transmit the state information or operation information of the washing machine 1 to the server or the terminal through the communication unit 350 thereby suggesting the washing machine appliance is in operative communication with a remote device through an external network, wherein the remote device comprises the camera assembly.[0062]
Claim(s) 3 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 2020/0071871), Yoon (US 2023/0067550), and Gallagher (US 2017/0037559), as applied to claim 1, and in further view of Ha (US 2021/0025104).
Regarding claim 3, Kim in view of Yoon and Gallagher is silent to the controller is further configured to: analyze the one or more images using the image recognition process to determine that a flow rate of the flow of wash fluid falls below a flow rate threshold; and provide a user notification that there is a fault condition with the water supply.
Ha is directed towards an apparatus and method for detecting water supply in a household appliance wherein FIGS. 6B to 6E are exemplary views illustrating a state in which water supply detection image data designates the water supplier in the tub as the water supply region of interest based on data having 3 to 5 second images (30 frames per second) and uses the optical flow algorithm as the vision algorithm. Referring to FIGS. 6B to 6E, the optical flow is detected, and a magnitude of the optical vector has a minimum threshold value of 208 and is detected from 208 or more. The optical vector is represented by a set of dots at the part at the upper end region of the tub where the water is supplied, the optical vector means that the darker and wider the optical vector, the more washing water is supplied. In addition, when the water supply is stably performed, the optical vector has a wide distribution and the vector values are arranged in the region of interest.[0087-93]
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to provide the apparatus and method of Ha in the washing machine of Kim, Yoon, and Gallagher to monitor the blockage in the water supply path.[0002]
Regarding claim 15, Ha is directed towards an apparatus and method for detecting water supply in a household appliance used for water supply equipment that needs to monitor the blockage in the water supply path. FIGS. 6B to 6E are exemplary views illustrating a state in which water supply detection image data designates the water supplier in the tub as the water supply region of interest based on data having 3 to 5 second images (30 frames per second) and uses the optical flow algorithm as the vision algorithm. Referring to FIGS. 6B to 6E, the optical flow is detected, and a magnitude of the optical vector has a minimum threshold value of 208 and is detected from 208 or more. The optical vector is represented by a set of dots at the part at the upper end region of the tub where the water is supplied, the optical vector means that the darker and wider the optical vector, the more washing water is supplied. In addition, when the water supply is stably performed, the optical vector has a wide distribution and the vector values are arranged in the region of interest thereby reading on analyzing the one or more images using the image recognition process to determine that a flow rate of the flow of wash fluid falls below a flow rate threshold; and providing a user notification that there is a fault condition with the water supply.[0002][0087-93]
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.

CITED REFERENCE SUMMARIES: Ref US20200071871A1: [content unavailable]
Ref US20170037559A1: Claims:
A laundry treating appliance for treating laundry according to a cycle of operation, the laundry treating appliance comprising: a cabinet defining an interior and defining an access opening; a treating chamber located within the interior and accessible through the access opening; and a treating chemistry station comprising at least one actuator and a treating chemistry conduit, the at least one actuator located on the cabinet, and the treating chemistry conduit directed toward the treating chamber, wherein actuation of the at least one actuator causes a discharge of treating chemistry from the treating chemistry conduit toward the treating chamber.
a cabinet defining an interior and defining an access opening;
a treating chamber located within the interior and accessible through the access opening; and
a treating chemistry station comprising at least one actuator and a treating chemistry conduit, the at least one actuator located on the cabinet, and the treating chemistry conduit directed toward the treating chamber, wherein actuation of the at least one actuator causes a discharge of treating chemistry from the treating chemistry conduit toward the treating chamber.
The laundry treating appliance of claim 1 further comprising at least one nozzle disposed on a terminal end of the treating chemistry conduit.
The laundry treating appliance of claim 2 wherein the at least one actuator comprises multiple actuators, and the at least one nozzle comprises multiple nozzles, with the multiple actuators and nozzles arranged in pairs.
The laundry treating appliance of claim 1 further comprising at least one sensor providing a sensor output indicative of an amount of treating chemistry dispensed through the treating chemistry conduit.
The laundry treating appliance of claim 4 further comprising a controller operably coupled to the at least one sensor to receive the sensor output and reduce a volume of treating chemistry supplied to the treating chamber for a selected cycle of operation based on the sensor output.
The laundry treating appliance of claim 5 wherein the volume of treating chemistry is reduced by the amount of treating chemistry indicated by the sensor output.
The laundry treating appliance of claim 1 wherein the cabinet further comprises a fascia at least partially defining the access opening, having the actuator disposed on the fascia.
The laundry treating appliance of claim 7 wherein the treating chemistry conduit is adjacent the actuator.
The laundry treating appliance of claim 8 wherein the treating chemistry conduit passes through the fascia.
The laundry treating appliance of claim 9 wherein the treating chemistry conduit is directed toward the treating chamber.
The laundry treating appliance of claim 1 wherein the treating chemistry conduit is fluidly coupled to a bulk treating chemistry reservoir.
The laundry treating appliance of claim 11 further comprising a pump fluidly coupling the bulk treating chemistry reservoir and the treating chemistry conduit.
The laundry treating appliance of claim 1 wherein the treating chemistry station further comprises a water conduit, providing for a mixed discharge of water and treating chemistry form the treating chemistry station.
A stain station for a laundry treating appliance having a treating chamber for treating laundry according to a cycle of operation, the stain station comprising: one or more nozzles for dispensing a volume of treating chemistry to the treating chamber; and at least one actuator disposed on the laundry treating appliance for selectively dispensing treating chemistry from one or more of the nozzles.
one or more nozzles for dispensing a volume of treating chemistry to the treating chamber; and
at least one actuator disposed on the laundry treating appliance for selectively dispensing treating chemistry from one or more of the nozzles.
The stain station of claim 14 further comprising at least one sensor providing a sensor output indicative of an amount of treating chemistry dispensed through one or more nozzles.
The stain station of claim 15 further comprising a controller operably coupled to the at least one sensor to receive the sensor output and reduce the treating chemistry supplied to the treating chamber for a selected cycle of operation based on the sensor output.
The stain station of claim 14 wherein the stain station is fluidly coupled to a bulk treating chemistry reservoir.
The stain station of claim 17 further comprising a pump fluidly coupling the bulk treating chemistry reservoir.
A method of treating laundry in a laundry treating appliance according to a selected cycle of operation, the method comprising: determining an amount of treating chemistry dispensed during a pre-treating operation to define a determined amount of pre-treating chemistry; reducing a predetermined amount of treating chemistry for the selected cycle of operation based on the determined amount of pre-treating chemistry to define a reduced treating chemistry amount; and dispensing the reduced treating chemistry amount during an execution of the selected cycle of operation.
determining an amount of treating chemistry dispensed during a pre-treating operation to define a determined amount of pre-treating chemistry;
reducing a predetermined amount of treating chemistry for the selected cycle of operation based on the determined amount of pre-treating chemistry to define a reduced treating chemistry amount; and
dispensing the reduced treating chemistry amount during an execution of the selected cycle of operation.
The method of claim 19 wherein the reduced treating chemistry amount is equal to a difference between the predetermined amount and the determined amount.
Ref US20210025104A1: Claims:
An apparatus for detecting a fluid supply level, the apparatus comprising: a fluid supply container configured to receive a fluid supplied therein; a vision sensor mounted in a vicinity of the fluid supply container or on the fluid supply container, the vision sensor being configured to acquire an image including a portion in which the fluid is supplied into the fluid supply container; and a fluid supply level determiner configured to determine the fluid supply level in accordance with an amount of the fluid being supplied into the fluid supply container through a predetermined vision algorithm based on image information acquired by the vision sensor.
a fluid supply container configured to receive a fluid supplied therein;
a vision sensor mounted in a vicinity of the fluid supply container or on the fluid supply container, the vision sensor being configured to acquire an image including a portion in which the fluid is supplied into the fluid supply container; and
a fluid supply level determiner configured to determine the fluid supply level in accordance with an amount of the fluid being supplied into the fluid supply container through a predetermined vision algorithm based on image information acquired by the vision sensor.
The apparatus of claim 1 , wherein the fluid supply level determiner is configured to: set at least one fluid supply region of interest with respect to the image information; detect a feature of the at least one fluid supply region of interest through the predetermined vision algorithm; and determine the fluid supply level based on the detected feature of the fluid supply region of interest.
set at least one fluid supply region of interest with respect to the image information;
detect a feature of the at least one fluid supply region of interest through the predetermined vision algorithm; and
determine the fluid supply level based on the detected feature of the fluid supply region of interest.
The apparatus of claim 2 , wherein the fluid supply level determiner is configured to set a portion of an upper end of the fluid supply container as the at least one fluid supply region of interest with respect to the image information.
The apparatus of claim 2 , wherein, based on the amount of supplied fluid, the fluid supply level determiner is configured to: classify the fluid supply level into three classifications including a no fluid supply, a cautious fluid supply, and a normal fluid supply; or classify the fluid supply level into four classifications including a no fluid supply, a cautious fluid supply, a quasi-normal fluid supply, and a normal fluid supply.
classify the fluid supply level into three classifications including a no fluid supply, a cautious fluid supply, and a normal fluid supply; or
classify the fluid supply level into four classifications including a no fluid supply, a cautious fluid supply, a quasi-normal fluid supply, and a normal fluid supply.
The apparatus of claim 4 , wherein, based on the amount of fluid being supplied for each fluid supply level, the fluid supply level determiner is configured to predict a fluid supply time to a point in time at which the fluid supply will be completed.
The apparatus of claim 5 , wherein, based on label values which are the amount of fluid being supplied for each fluid supply level and the fluid supply time to a point in time at which the fluid supply will be completed for each fluid supply level, the fluid supply level determiner is configured to: create a training data set; perform machine learning; and predict, based on a trained model, the fluid supply time to the point in time at which the fluid supply will be completed for each fluid supply level.
create a training data set;
perform machine learning; and
predict, based on a trained model, the fluid supply time to the point in time at which the fluid supply will be completed for each fluid supply level.
The apparatus of claim 4 , wherein the fluid supply level determiner is configured to determine the fluid supply level as an erroneous fluid supply when a result of determining the fluid supply level is no fluid supply.
The apparatus of claim 2 , wherein the vision algorithm is an optical flow algorithm, and the feature is an optical vector.
The apparatus of claim 8 , wherein the optical flow algorithm is an algorithm that determines the fluid supply level based on a distribution of the optical vectors and a magnitude of the optical vectors with respect to the fluid supply region of interest.
The apparatus of claim 8 , wherein the optical flow algorithm is configured to use an average value of vector values calculated by applying a frame moving average to determine the fluid supply level.
The apparatus of claim 1 , further comprising a lighting device configured to illuminate an inside of the fluid supply container.
A washing machine comprising an apparatus for detecting a water supply level, the apparatus comprising: a washing machine tub configured to receive a washing water therein; a vision sensor mounted in a vicinity of the washing machine tub or on the washing machine tub, the vision sensor being configured to acquire an image including a portion in which the washing water is supplied into the washing machine tub; and a water supply level determiner configured to determine the water supply level in accordance with an amount of fluid being supplied into the washing machine tub through a predetermined vision algorithm based on image information acquired by the vision sensor.
a washing machine tub configured to receive a washing water therein;
a vision sensor mounted in a vicinity of the washing machine tub or on the washing machine tub, the vision sensor being configured to acquire an image including a portion in which the washing water is supplied into the washing machine tub; and
a water supply level determiner configured to determine the water supply level in accordance with an amount of fluid being supplied into the washing machine tub through a predetermined vision algorithm based on image information acquired by the vision sensor.
A method of detecting a water supply level in a washing machine tub, the method comprising: acquiring, by a vision sensor mounted in a vicinity of the washing machine tub, a water supply image; setting at least one water supply region of interest with respect to image information acquired by the vision sensor; detecting a feature of the at least one water supply region of interest through a predetermined vision algorithm; and determining a water supply level in accordance with an amount of water being supplied into the washing machine tub based on the detected feature of the at least one water supply region of interest.
acquiring, by a vision sensor mounted in a vicinity of the washing machine tub, a water supply image;
setting at least one water supply region of interest with respect to image information acquired by the vision sensor;
detecting a feature of the at least one water supply region of interest through a predetermined vision algorithm; and
determining a water supply level in accordance with an amount of water being supplied into the washing machine tub based on the detected feature of the at least one water supply region of interest.
The method of claim 13 , wherein, in the detecting of the feature of the at least one water supply region of interest, the vision algorithm is an optical flow algorithm, and the feature is an optical vector.
The method of claim 14 , wherein the optical flow algorithm is an algorithm that determines the water supply level based on a distribution of optical vectors and a magnitude of the optical vectors with respect to the at least one water supply region of interest.
The method of claim 14 , wherein the optical flow algorithm uses an average value of optical vector values calculated by applying a frame moving average to determine the water supply level.
The method of claim 13 , wherein, based on the water supply amount, the determining of the water supply level classifies the water supply level into three classifications including a no water supply, a cautious water supply, and a normal water supply, or wherein, based on the water supply amount, the determining of the water supply level classifies the water supply level into four classifications including a no water supply, a cautious water supply, a quasi-normal water supply, and a normal water supply.
wherein, based on the water supply amount, the determining of the water supply level classifies the water supply level into four classifications including a no water supply, a cautious water supply, a quasi-normal water supply, and a normal water supply.
The method of claim 17 , further comprising: determining the water supply level as an erroneous water supply when the water supply level is classified as the no water supply; and informing a user of the erroneous water supply through a user terminal or an output device.
determining the water supply level as an erroneous water supply when the water supply level is classified as the no water supply; and
informing a user of the erroneous water supply through a user terminal or an output device.
The method of claim 17 , further comprising predicting a water supply time to a point in time at which the water supply will be completed based on the water supply amount for each water supply level.
The method of claim 19 , wherein, based on label values which are the water supply amount for each water supply level and the water supply time to a point in time at which the water supply will be completed for each water supply level, the predicting of the water supply time to the point in time at which the water supply will be completed comprises: creating a training data set; performing machine learning; and predicting, based on a trained model, the water supply time to the point in time at which the water supply will be completed for each water supply level.
creating a training data set;
performing machine learning; and
predicting, based on a trained model, the water supply time to the point in time at which the water supply will be completed for each water supply level.
Ref US20200071871: Claims:
1. A washing machine comprising:
a door comprising an external cover and an internal glass configured to open and close a laundry entrance of the washing machine; at least one image sensor disposed to face the internal glass and configured to acquire image data; a detergent bubble recognizer configured to recognize detergent bubbles from the image data through a machine learning based image recognition model; and a controller configured to control the washing machine to perform an additional rinsing cycle based on a result of recognition.
2. The washing machine of claim 1, further comprising a bubble ratio calculator configured to process the image data and to calculate a ratio of an area of the detergent bubbles to an area of the internal glass, based on the detergent bubbles being recognized by the detergent bubble recognizer,
wherein the controller is configured to control the washing machine to perform the additional rinsing cycle based on the ratio calculated by the bubble ratio calculator.
3. The washing machine of claim 2, wherein the controller is further configured to update the machine learning based image recognition model of the detergent bubble recognizer using the acquired image data, and at least one of (i) information regarding an amount of laundry, (ii) information regarding an amount of detergent, or (iii) washing mode information.
4. The washing machine of claim 2, wherein the controller is further configured to set a number of times for performing the additional rinsing cycle based on the ratio calculated by the bubble ratio calculator.
5. The washing machine of claim 4, wherein the controller is further configured to:
set the number of times for performing the additional rinsing cycle to a first number based on the calculated ratio being a first ratio, and set the number of times for performing the additional rinsing cycle to a second number based on the calculated ratio being a second ratio greater than the first ratio.
6. The washing machine of claim 2, wherein the controller is further configured to increase a duration of time for performing the additional rinsing cycle, as the calculated ratio increases.
7. The washing machine of claim 1, wherein the controller is further configured to control the at least one image sensor to acquire the image data based on completion of a first rinsing cycle prior to the additional rinsing cycle.
8. The washing machine of claim 7, wherein the first rinsing cycle is one of a plurality of cycles in a washing operation of the washing machine, and
wherein the controller is further configured to perform a next cycle that follows the first rinsing cycle in the washing operation, based on the detergent bubbles not being recognized from the image data by the detergent bubble recognizer.
9. The washing machine of claim 1, wherein the at least one image sensor is disposed between the external cover and the internal glass of the door.
10. The washing machine of claim 9, wherein the at least one image sensor is fastened to an internal wall of the external cover of the door.
11. The washing machine of claim 1, further comprising at least one light source provided on an inner circumferential surface of the laundry entrance,
wherein the controller is configured to activate the at least one light source during acquisition of the image data by the at least one image sensor.
12. The washing machine of claim 1, wherein the image recognition model comprises a deep neural network that is configured to implement machine-based deep learning.
13. The washing machine of claim 2, further comprising a communication unit configured to transmit, to a server or a terminal, the acquired image data or at least one of (i) a number of times for performing the additional rinsing cycle, or (ii) duration of time for performing the additional rinsing cycle based on the ratio calculated by the bubble ratio calculator.
14. The washing machine of claim 13, wherein the controller is further configured to:
[Truncated to input limit]