Parallel planar weight sensing device

A parallel planar weight sensing device includes an accessory mounting bracket with an upper portion and a lower portion. Part of the accessory mounting bracket extends around a crossmember that also has an upper portion and a lower portion. Affixed to the upper portion of the crossmember is a first load cell while a second load cell is affixed to the lower portion of the crossmember. The upper portion of the accessory mounting bracket is affixed to the first load cell while the lower portion is affixed to the second load cell. An accessory, such as a shelf or hook, may be attached to the accessory mounting bracket and used to stow items. As the load on the accessory changes, such as due to a pick or place of items, a total weight change may be determined by summing the output from the load cells.

BACKGROUND

Retailers, wholesalers, and other product distributors typically maintain an inventory of various items that may be ordered, purchased, leased, borrowed, rented, viewed, and so forth, by clients or customers. For example, inventory may be stored and distributed from a fulfillment center. When a customer orders an item, the item is picked from inventory, routed to a packing station, packed, and shipped to the customer. Likewise, retail stores maintain inventory in customer accessible areas, such as in a shopping area, and customers can pick items from inventory and take for purchase, rental, and so forth. Other examples of entities that maintain facilities holding inventory include libraries, museums, rental centers, and so forth. In each instance, for an item to be moved from one location to another, it is picked from its current location and transitioned to a new location. It is often desirable to monitor quantity and movement of inventory within the facility.

DETAILED DESCRIPTION

A material handling facility (facility) or other setting may utilize fixtures that provide inventory locations used to stow or otherwise hold items. For example, the fixtures may include a crossmember that is used to support one or more accessories such as shelves, hooks, bins, baskets, and so forth that are then used to store items. The accessories are cantilevered, extending away from the crossmember. In some implementations, each accessory may be used to store a single type of item. Sensors in the environment provide sensor data. For example, the fixtures may include weight sensors that measure the weight of the accessory and any items that it supports and provide weight sensor data. In another example, cameras may provide image data.

The facility may include, or have access to, an inventory management system. The inventory management system may be configured to maintain information about items, users, condition of the facility, and so forth based at least in part on the sensor data. The inventory management system may use the sensor data to determine interaction data. The interaction data may include information about the type of item involved, quantity involved, whether the interaction was a pick or place, and so forth. Interactions may include the user picking an item from an inventory location, placing an item at the inventory location, touching an item at the inventory location, rummaging through items at the inventory location, and so forth. For example, the inventory management system may generate interaction data that indicates what item the user picked from a particular lane on a shelf, and then use this interaction data to adjust the count of inventory stowed at that lane. The interaction data may then be used to bill an account associated with the user for the item that was picked or credit the account for the item that was placed.

In a traditional device, a cantilevered arm supporting a load produces a relatively large mechanical movement on a load cell. A change in loading of a cantilevered arm in a traditional device will result in twisting and this device behaves like a torsion spring. This results in undesirable deflection of the cantilevered arm as well as ongoing oscillation and overall susceptibility to vibration. For example, even if the load on the cantilevered arm does not change, ambient vibration to the overall structure (such as from a passing truck) may result in undesirable movement of the cantilevered arm. In traditional devices, these undesirable movements produce noisy output signals from weight sensors that may be unusable or require significant signal processing to provide usable data. These undesirable movements may also impair use of the device. For example, trying to add or remove an object to a moving cantilevered arm is more demanding than the same action on a stationary cantilevered arm. Undesirable movements may introduce mechanical interactions with material supported by the cantilevered arms. For example, if the load comprises liquids in containers, movement of the cantilevered arm may introduce undesired movement of the liquids in the containers.

Traditional devices that attempt to minimize some of these issues are limited to relatively light loads, are prohibitively expensive to implement, and do not scale well to larger form factors.

Described in this disclosure is a parallel planar weight sensing device (device) that provides support for a cantilevered accessory to support a load. Compared to traditional devices, the device described herein provides significantly improved resistance to torsion, reducing displacement of the cantilevered arm and oscillation of the device. The output from weight sensors in the parallel planar weight sensing device may experience less noise as a result of reduced undesirable movement.

The device may be implemented in different size form factors, allowing use with loads ranging from grams to thousands of kilograms or more. Implementations of the device described herein for larger size form factors, such as devices supporting a load of tens to hundreds of kilograms, are also significantly less expensive than traditional devices. This reduction in cost facilitates increased deployment, allowing the device to be used more extensively.

The inventory locations may use one or more parallel planar weight sensing devices. For example, an inventory location may comprise one or more crossmembers. Each crossmember may be part of one or more parallel planar weight sensing devices.

The parallel planar weight sensing device comprises a crossmember. An accessory mounting bracket (bracket) is attached to the crossmember. One or more accessories may then be attached to, or are integral with, the bracket. The accessory may include a hook, shelf, bin, basket, dispenser, and so forth that stows an item. An accessory may be supported by more than one bracket. A crossmember may support more than one bracket. For example, a single crossmember may support sixteen brackets. Three brackets may support accessories with hooks from which bagged items may be suspended, seven brackets may support shelves upon which boxed items may sit, and six brackets may support bins to contain loose items. As the needs of the facility change, the accessories may be easily added to or removed from particular brackets.

The crossmember and the bracket are rigid and may comprise unitary structures or may be made up of several pieces. For example, the crossmember may comprise several pieces that are joined together while the bracket also comprises several pieces that are joined together. As a result, the device may be easily constructed in larger sizes that would otherwise require prohibitively expensive fabrication of single piece elements.

The crossmember and the bracket are joined to one another at two locations, each location comprising a weight sensor. The locations may be located opposite one another with respect to the crossmember. The weight sensor at each of the two locations may comprise a planar beam load cell having a live end and a fixed end. In one implementation the live end of the planar beam load cell is affixed to the bracket while the fixed end is affixed to the crossmember. The planar beam load cells may be arranged coplanar to one another. For example, the body of each planar beam load cell describes a plane that extends through a length and width of the body. In this implementation, an axis extending through the live ends of the two planar beam load cells is perpendicular to the plane of each of the planar beam load cells. A load is applied to the accessory that cantilevers from the bracket, the bracket exerts a vertical force and a twisting force. The twisting force results in a lateral force on each load cell. The vertical force on each of the load cells is summed while the lateral forces as applied in opposing directions are opposed by the rigidity of the crossmember and cancel one another out. The vertical force is measured by each of the weight sensors and used to produce weight sensor data. The weight sensor data for the two load cells may be summed to calculate a total weight value.

The device described in this disclosure provides a relatively low cost, physically robust device for determining weight of a cantilevered load. Information about the weight may be used by the inventory management system to facilitate and improve operations by providing information as to the current quantity and movement of items in the facility.

Illustrative Device

FIG. 1illustrates a parallel planar weight sensing device100with an accessory to support items, according to one implementation. An accessory102is depicted. The accessory102may comprise a hook, a shelf, a bin, a basket, or other structure that may be used to stow items. The accessory102may also include one or more engagement features that mechanically engage one or more engagement features on an accessory support bracket (bracket)104. The bracket104is a rigid structure. For example, the bracket104may comprise one or more pieces of metal, plastic, composite material, and so forth. The bracket104may exhibit a “C” shape as shown here. The bracket104has an upper portion106and a lower portion108.

The device includes a crossmember110. The crossmember110is a rigid structure. For example, the crossmember110may comprise one or more pieces of metal, plastic, composite material, and so forth. The crossmember110may exhibit a “C” shape as shown here. The crossmember110has an upper portion112and a lower portion114.

The crossmember110may be arranged within at least a portion of the bracket104. For example, the upper portion106of the bracket104may be proximate to the upper portion112of the crossmember110while the lower portion108of the bracket104may be proximate to the lower portion114of the crossmember110.

The device uses one or more load cells116. The load cell116may comprise a body or structure and one or more transducers that provide information indicative of a force applied to that structure. The structure of the load cell116may include a live end and a fixed end. The live end is affixed to a load while the fixed end is affixed to a supporting structure.

In one implementation the load cells116may comprise planar beam load cells116. The structure of each planar beam load cell116may comprise a flat piece of material, such as metal, having strain concentration features such as slots, grooves, and so forth. For example, the flat piece of material may have a “U” shaped cut. The tab formed by the “U” may be the live end while the remaining structure surrounding the “U” is the fixed end. A strain gauge is affixed to a portion of the structure, such as at the point where the tab joins the remainder of the structure. A change in load on the live end results in a deformation of the structure of the load cell which is measured by the strain gauge. Strain gauges may be electrical, optical, acoustic, or combinations thereof. Output from the strain gauge may be used to determine weight sensor data indicative of the force that produced the deformation.

A fixed end of a first load cell116is affixed to the upper portion112of the crossmember110. For example, the load cell116may be affixed to the crossmember110using mechanical fasteners such as screws, bolts, rivets, clips, and so forth. The upper portion106of the bracket104is arranged proximate to the live end of the first load cell116. A first fastener118may be used to join the live end of the first load cell116to the upper portion106of the bracket104. For example, as shown inFIG. 1the first fastener118may comprise a bolt that is secured with a first nut120. A first standoff122may be used to maintain spacing between the outer surface of the load cell116and the inner surface of the bracket104.

A fixed end of a second load cell116is affixed to the lower portion114of the crossmember110. The lower portion108of the bracket104is arranged proximate to the live end of the second load cell116. A second fastener118may be used to join the live end of the second load cell116to the lower portion108of the bracket104. For example, as shown here the second fastener118may comprise a bolt that is secured with a second nut120. A second standoff122may be used to maintain spacing between the outer surface of the load cell116and the inner surface of the lower portion108of the bracket104.

In other implementations other arrangements may be used. For example, the fixed end of the load cells116may be affixed to the bracket104while the live ends are affixed to the crossmember110.

In another implementation a portion of the bracket104may be arranged at least partially within the crossmember110. For example, an inner surface of the upper portion112of the crossmember110may be proximate to an outer surface of the upper portion106of the bracket104while an inner surface of the lower portion114of the crossmember110may be proximate to an outer surface of the lower portion108of the bracket104.

While the bracket104and the crossmember110are depicted as having “C” shaped profiles, in other implementations other profiles may be used. For example, the crossmember110may be semi-circular or circular, trapezoidal, and so forth. The profiles of the bracket104and the crossmember110may also differ from one another.

FIG. 2illustrates a side view of the device ofFIG. 1, according to one implementation. In this view an accessory mounting face202of the bracket104is visible. The accessory mounting face202may include one or more engagement features to engage the accessory102. For example, the engagement features may include slots, tabs, holes, ridges, grooves, and so forth.

As illustrated here, the first load cell116and the second load cell116may be aligned. For example, an axis204is shown extending through the live ends of the first load cell116and the second load cell116. In other implementations the axis204may extend through another portion of the first load cell116and the second load cell116.

In this illustration, the axis204extends along a long axis of the fasteners118. The axis204in this implementation is perpendicular to the accessory102. In other implementations, the axis204may be oriented to be vertical while the accessory102may extend away from the bracket104at an angle. For example, the accessory102may slope downward from the bracket104.

The axis204is also perpendicular to the plane of the first load cell116and the plane of the second load cell116. As shown in this implementation, the plane of the first load cell116and the plane of the second load cell116are parallel to one another.

In other implementations the first load cell116and the second load cell116may be placed in different alignments. A line extending through the live end of the first load cell116and the live end of the second load cell116may not be aligned with vertical. For example, the first load cell116may be displaced along a Z axis (left and right along the crossmember110) with respect to the second load cell116. In another example, the first load cell116may be displaced along the X axis (front to back with respect to the crossmember110) with respect to the second load cell116.

A distance X is shown extending between the axis204and a location206on the accessory102of a load that exerts a downward force FXL on the accessory102. A distance Y is indicative of the distance between the load cells116. Distances X and Y are perpendicular to one another in this illustration. An origin208is depicted at a midpoint of distance Y, within the crossmember110.

The downward force FXL produces a torque or twist on the bracket104with respect to the origin208. This torque produces a pair of horizontal forces on the bracket104with corresponding reaction forces FY by the rigid system formed by the load cells116affixed to the crossmember110. A corresponding counter reaction force FXC provided by the crossmember110supports the device. For example, the FXC may be provided by a vertical upright of a rack, which in turn is supported by the ground.

The use of the load cells116at the distance Y reduces the reaction force needed to counteract the movement about the origin208as generated by the load FXL at the distance X. A vertical shift of FXL is enforced by the arrangement of the device. As a result, accurate measurement at the first and second load cells116is possible regardless of X or FXL. For example, as the location206moves towards or away from the origin208, the force along the axis204(measured as the weight) remains the same while the reaction forces FY change. Likewise, regardless the distance X at which the load applies the force FXL, as the force FXL changes the force along the axis204(measured as weight) remains the same while the reaction forces FY may change.

The reaction force FY may be expressed with this equation:

The sum of applied moments M with respect to the origin208may be expressed with this equation:
ΣM=FXL*X−FY*Y=0  Equation 2

As the distance Y between the load cells116is increased, the device exhibits improved resistance to changes in X as well as FXL. The device may thus be scaled in size, increasing Y to allow for increased distances X to provide for the use of longer accessories102without incurring undesired movements.

FIG. 3illustrates an exploded view of the device ofFIG. 1, according to one implementation. In this view, additional components are visible. For example, a washer302may be used in conjunction with the fastener118. Individual screws304used to attach the fixed end of the load cells116to the crossmember110are also shown.

In this illustration features of the load cells116are visible. For example, the load cells116depicted here comprise planar beam load cells with a tab306. The tab306is also known as the live end of the planar beam load cell116.

The load cells116may be asymmetrical in construction and may be aligned in a particular direction. The first load cell116is arranged in a first direction308while the second load cell116is arranged in a second direction310. In the implementation illustrated here, the first direction308is opposite the second direction310.

The load cells116may exhibit improved performance when installed with a particular alignment with respect to the bracket104. The load cell116has a structure. For example, the load cells116depicted here are planar beam load cells with an overall flat structure with a plane that is described by the length and width of the structure. Relative to the plane, the forces applied to the load cell116may be perpendicular, parallel, or a combination thereof. For example, the force FXL may introduce a transverse or lateral force imposed on the load cell116as well as the weight which is perpendicular.

In some implementations one or more of the load cells116may be aligned in a particular direction. For example, as illustrated here the first load cell116that is affixed to the upper portion112of the crossmember110is affixed in the first direction308. In this first direction308, the tab306of the structure of the load cell is placed under tension when the force FXL is applied to the bracket104. For example, the weight of the load on the accessory102applies a force tending to rotate the bracket104with respect to the origin, tipping the upper portion106of the bracket104towards the front of the device. However, the rigid nature of the components prevents substantial rotation. In particular, the rigid structure of the first load cell116prevents the rotation, with the reaction force FY manifesting as tension on the tab306within the plane of the load cell116. Similarly, the second load cell116is oriented in the second direction310, opposite the first direction308. The second tab306of the second load cell116is also under tension when the force FXL is applied to the bracket104. In other implementations other arrangements may be used. For example, the load cells116may be arranged such that the force FXL places a portion of the individual load cells116in compression.

In other implementations, the load cells116may be arranged in other directions. For example, the first direction308of the first load cell116may be parallel to a long axis of the crossmember110.

As indicated here in dashed lines, the crossmember110may extend left and right. The crossmember110may support a plurality of other brackets104. For example, the crossmember110may support a total of sixteen brackets104, each with the corresponding components described herein.

FIG. 4illustrates a perspective view of the device ofFIG. 1, according to one implementation. In this view, the first load cell116that is beneath the upper portion106of the bracket104has been depicted using dashed lines. A force402on the first load cell116resulting from the force FXL is shown. A force404on the second load cell116resulting from the force FXL is also shown. The direction of force402is opposite the force404. The forces402and404are countered by reaction force FY as described above.

As illustrated here, the direction308of the first load cell116and the direction310of the second load cell116may be arranged such that the individual tabs306of each load cell116are under tension. For example, force402is the force provided by the tab306of the first load cell116when the force FXL is applied.

FIG. 5illustrates a side view500of the device, according to another implementation. In this illustration the accessory102comprises a shelf support having a plurality of engagement features504. The engagement features504of the accessory102may engage corresponding engagement features on the accessory mounting face202. For example, the engagement features504may comprise hooks that fit within holes in the accessory mounting face202. In other implementations the accessory102may be supported by engagement features on other portions of the bracket104. For example, the bracket104may include engagement features on one or more of the upper portion106or the lower portion108. The accessory102may also include an engagement feature502, such as a tab, notch, lip, and so forth.

FIG. 6is a block diagram600illustrating a material handling facility (facility)602using the device100, according to some implementations. A facility602comprises one or more physical structures or areas within which one or more items604(1),604(2), . . . ,604(Q) may be held. The items604may comprise physical goods, such as books, pharmaceuticals, repair parts, electronic gear, and so forth.

The facility602may include one or more areas designated for different functions with regard to inventory handling. In this illustration, the facility602includes a receiving area606, a storage area608, and a transition area610.

The receiving area606may be configured to accept items604, such as from suppliers, for intake into the facility602. For example, the receiving area606may include a loading dock at which trucks or other freight conveyances unload the items604. In some implementations, the items604may be processed, at the receiving area606, to generate at least a portion of item data as discussed below. For example, an item604may be weighed, imaged or otherwise scanned to develop reference images or representations of the item604at the receiving area606.

The storage area608is configured to store the items604. The storage area608may be arranged in various physical configurations. In one implementation, the storage area608may include one or more aisles612. The aisle612may be configured with, or defined by, inventory locations614on one or both sides of the aisle612. The inventory locations614may include one or more of a platform, a rack, a case, a cabinet, a bin, a floor location, or other suitable storage mechanisms for holding, supporting, or storing the items604. For example, the inventory locations614may comprise racks that support one or more devices100with accessories102to hold items604. The inventory locations614may be affixed to the floor or another portion of the structure of the facility602. The inventory locations614may also be movable such that the arrangements of aisles612may be reconfigurable. In some implementations, the inventory locations614may be configured to move independently of an outside operator. For example, the inventory locations614may comprise a platform with a power source and a motor, operable by a computing device to allow the platform to move from one location within the facility602to another.

One or more users616and carts618or other material handling apparatus may move within the facility602. For example, the user616may move about within the facility602to pick or place the items604in various inventory locations614, placing them in the cart618for ease of transport. The cart618is configured to carry or otherwise transport one or more items604. For example, the cart618may include a basket, bag, bin, and so forth. In other implementations, other material handling apparatuses such as robots, forklifts, cranes, aerial drones, and so forth, may move about the facility602picking, placing, or otherwise moving the items604. For example, a robot may pick an item604from a first inventory location614(1) and move the item604to a second inventory location614(2).

The facility602may include one or more sensors620. For example, the sensors620may provide sensor data such as weight sensor data from the device(s)100, image data from cameras, and so forth.

While the storage area608is depicted as having one or more aisles612, sensors620, inventory locations614storing the items604, and so forth, it is understood that the receiving area606, the transition area610, or other areas of the facility602may be similarly equipped. Furthermore, the arrangement of the various areas within the facility602is depicted functionally rather than schematically. For example, in some implementations, multiple different receiving areas606, storage areas608, and transition areas610may be interspersed rather than segregated in the facility602.

The facility602may include, or be coupled to, an inventory management system622. The inventory management system622is configured to interact with the user616or devices such as the sensors620, robots, material handling equipment, computing devices, and so forth, in one or more of the receiving area606, the storage area608, or the transition area610.

During operation of the facility602, the sensor data based on output from the devices100or other sensors620may be provided to the inventory management system622. The sensors620may include, but are not limited to, cameras620(1), touch sensors620(6), and so forth. The sensors620may be stationary or mobile, relative to the facility602. For example, the facility602may include cameras620(1) to obtain images of the user616or other objects in the facility602. In another example, the inventory locations614may contain cameras620(1) to acquire images of picking or placement of items604on shelves, and so forth. The sensors620are described in more detail below with regard toFIG. 7.

The inventory management system622or other systems may use the sensor data to determine the location of objects within the facility602, movement of the objects, or provide other functionality. Objects may include, but are not limited to, items604, users616, carts618, and so forth. For example, a series of images acquired by the cameras620(1) may indicate removal by the user616of an item604from a particular location at the inventory location614and placement of the item604on or at least partially within the cart618.

The facility602may be configured to receive different kinds of items604from various suppliers and to store them until a customer orders or retrieves one or more of the items604. A general flow of items604through the facility602is indicated by the arrows ofFIG. 6. Specifically, as illustrated in this example, items604may be received from one or more suppliers, such as manufacturers, distributors, wholesalers, and so forth, at the receiving area606. In various implementations, the items604may include merchandise, commodities, perishables, or any suitable type of item604, depending on the nature of the enterprise that operates the facility602.

Upon being received from a supplier at the receiving area606, the items604may be prepared for storage in the storage area608. For example, in some implementations, items604may be unpacked or otherwise rearranged. The inventory management system622may include one or more software applications executing on a computer system to provide inventory management functions. These inventory management functions may include maintaining information indicative of the type, quantity, condition, cost, location, weight, or any other suitable parameters with respect to the items604. The items604may be stocked, managed, or dispensed in terms of countable units, individual units, or multiple units, such as packages, cartons, crates, pallets, or other suitable aggregations. Alternatively, some items604, such as bulk products, commodities, and so forth, may be stored in continuous or arbitrarily divisible amounts that may not be inherently organized into countable units. Such items604may be managed in terms of a measurable quantity such as units of length, area, volume, weight, time, duration, or other dimensional properties characterized by units of measurement. Generally speaking, a quantity of an item604may refer to either a countable number of individual or aggregate units of an item604or a measurable amount of an item604, as appropriate.

After arriving through the receiving area606, items604may be stored within the storage area608. In some implementations, like items604may be stored or displayed together in the inventory locations614such as in accessories102. In this implementation, items604of a single type are stored in a particular inventory location614. In other implementations, different types of items604may be stored in different inventory locations614. For example, to optimize retrieval of certain items604having frequent turnover within a large physical facility602, those items604may be stored in several different inventory locations614to reduce congestion during picking that might occur at a single inventory location614.

When a customer order specifying one or more items604is received, or as a user616progresses through the facility602, the corresponding items604may be selected or “picked” from the inventory locations614containing those items604. In various implementations, item picking may range from manual to completely automated picking. For example, in one implementation, a user616may have a list of items604they desire and may progress through the facility602picking items604from inventory locations614within the storage area608and placing those items604into a cart618. In other implementations, employees of the facility602may pick items604using written or electronic pick lists derived from customer orders. These picked items604may be placed into the cart618as the employee progresses through the facility602.

After items604have been picked, the items604may be processed at a transition area610. The transition area610may be any designated area within the facility602where items604are transitioned from one location to another or from one entity to another. For example, the transition area610may be a packing station within the facility602. When the item604arrives at the transition area610, the items604may be transitioned from the storage area608to the packing station. Information about the transition may be maintained by the inventory management system622.

In another example, if the items604are departing the facility602, a list of the items604may be obtained and used by the inventory management system622to transition responsibility for, or custody of, the items604from the facility602to another entity. For example, a carrier may accept the items604for transport with that carrier accepting responsibility for the items604indicated in the list. In another example, a user616may purchase or rent the items604and remove the items604from the facility602. During use of the facility602, the user616may move about the facility602to perform various tasks, such as picking or placing the items604in the inventory locations614.

To facilitate operation of the facility602, the inventory management system622is configured to use the sensor data such as the item data, physical layout data, and so forth, to generate interaction data.

The interaction data may provide information about an interaction, such as a pick of an item604from the inventory location614, a place of an item604to the inventory location614, a touch made to an item604at the inventory location614, a gesture associated with an item604at the inventory location614, and so forth. The interaction data may include one or more of the type of interaction, interaction location identifier indicative of where from the inventory location614the interaction took place, item identifier, quantity change to the item604, user identifier, and so forth. The interaction data may then be used to further update the item data. For example, the quantity of items604on hand at a particular lane on the platform may be changed based on an interaction that picks or places one or more items604.

In one implementation, the sensor data may be received by the inventory management module. The sensor data may be processed to determine a weight change of +230 g occurred at a device at position2at inventory location614(73). The item data may indicate that items604of the type “12 oz can of Brand X dogfood” are stowed at this position and have an expected weight of 234 grams. Based on the sensor data a hypothesis may be generated that indicates “Qty+1 (return) of 12 oz can of Brand X dogfood to position2at inventory location614(73)”. If the hypothesis has a confidence value that exceeds a threshold value, the hypothesis may be deemed to represent the interaction which took place. As a result, interaction data may be generated that is representative of this interaction.

As described above, the inventory management system622may perform other operations, determining inventory to restock, determining user billing data, and so forth.

FIG. 7is a block diagram700illustrating additional details of the facility602, according to some implementations. The facility602may be connected to one or more networks702, which in turn connect to one or more computing devices, such as servers704. The network702may include private networks such as an institutional or personal intranet, public networks such as the Internet, or a combination thereof. The network702may utilize wired technologies (e.g., wires, fiber optic cables, and so forth), wireless technologies (e.g., radio frequency, infrared, acoustic, optical, and so forth), or other connection technologies. The network702is representative of any type of communication network, including one or more of data networks or voice networks. The network702may be implemented using wired infrastructure (e.g., copper cable, fiber optic cable, and so forth), a wireless infrastructure (e.g., cellular, microwave, satellite, and so forth), or other connection technologies.

The servers704may be configured to execute one or more modules or software applications associated with the inventory management system622or other systems. While the servers704are illustrated as being in a location outside of the facility602, in other implementations, at least a portion of the servers704may be located at the facility602. The servers704are discussed in more detail below with regard toFIG. 8.

The user616, the carts618, items604, or other objects in the facility602may be equipped with one or more tags706. The tags706may be configured to emit a signal708. In one implementation, the tag706may be a radio frequency identification (RFID) tag706configured to emit a RF signal708upon activation by an external signal. For example, the external signal may comprise a radio frequency signal or a magnetic field configured to energize or activate the RFID tag706. In another implementation, the tag706may comprise a transmitter and a power source configured to power the transmitter. For example, the tag706may comprise a Bluetooth Low Energy (BLE) transmitter and battery. In other implementations, the tag706may use other techniques to indicate presence of the tag706. For example, an acoustic tag706may be configured to generate an ultrasonic signal708, which is detected by corresponding acoustic receivers. In yet another implementation, the tag706may be configured to emit an optical signal708.

The inventory management system622may be configured to use the tags706for one or more of identification of the object, determining a location of the object, and so forth. For example, the user616may wear tags706, the carts618may have tags706affixed, items604may have tags706affixed to their packaging, and so forth, which may be read and, based at least in part on signal strength, used to determine one or more of identity or location.

Generally, the inventory management system622or other systems associated with the facility602may include any number and combination of input components, output components, and servers704.

The one or more sensors620may be arranged at one or more locations within the facility602. For example, the sensors620may be mounted on or within a floor, wall, at a ceiling, at an inventory location614, on a cart618, may be carried or worn by a user616, and so forth.

The sensors620may include one or more cameras620(1) or other imaging sensors. The one or more cameras620(1) may include imaging sensors configured to acquire images of a scene. The cameras620(1) are configured to detect light in one or more wavelengths including, but not limited to, terahertz, infrared, visible, ultraviolet, and so forth. The cameras620(1) may comprise charge coupled devices (CCD), complementary metal oxide semiconductor (CMOS) devices, microbolometers, and so forth. The inventory management system622may use image data acquired by the cameras620(1) during operation of the facility602. For example, the inventory management system622may identify items604, users616, carts618, and so forth, based at least in part on their appearance within the image data acquired by the cameras620(1). The cameras620(1) may be mounted in various locations within the facility602. For example, cameras620(1) may be mounted overhead, on inventory locations614, may be worn or carried by users616, may be affixed to carts618, and so forth.

The sensors620may include a smart floor620(2). The smart floor620(2) is able to provide information about the location of objects, such as users616, carts618, and so forth. This information may include identifying the object, determining a location of the object, tracking the object, and so forth. The smart floor620(2) may utilize smart floor devices that comprise one or more of transmitters or receivers that radiate or receive electromagnetic signals from antennas located at or underneath the floor. Based on information about what antenna radiated a signal and what antenna acquired the signal, information about an object on or above the floor may be determined. For example, the smart floor620(2) may comprise sensing elements, or segments. Each segment may include an antenna that is coupled to one or more of a transmitter or a receiver. During operation, the segment may transmit an electromagnetic signal708that is radiated by the antenna, receive an electromagnetic signal708that is acquired by the antenna, or both. In some implementations the smart floor620(2) may operate as a physically large touch sensor that is deployed at floor level. The electromagnetic signals provide information about the presence of an object thereon. For example, the segments may electromagnetically couple to objects that are close by, allowing for the detection of objects that are either in contact with the floor or above the floor. In some implementations, instead of or in addition to the visual tracking of the object, the smart floor620(2) may be used to provide object representation movement data. For example, the output from the segments obtained during a particular window of time may be processed in a fashion similar to the image data.

One or more radio receivers620(3) may also be included as sensors620. In some implementations, the radio receivers620(3) may be part of transceiver assemblies. The radio receivers620(3) may be configured to acquire RF signals708associated with RFID, Wi-Fi, Bluetooth, ZigBee, 2G, 3G, 4G, LTE, or other wireless data transmission technologies. The radio receivers620(3) may provide information associated with data transmitted via radio frequencies, signal strength of RF signals708, and so forth. For example, information from the radio receivers620(3) may be used by the inventory management system622to determine a location of an RF source, such as a transmitter carried by the user616, a transmitter on the cart618, a tag706on the item604, and so forth.

One or more depth sensors620(4) may also be included in the sensors620. The depth sensors620(4) are configured to acquire spatial or three-dimensional (3D) data, such as depth information, about objects within a field-of-view (FOV). The depth sensors620(4) may include range cameras, lidar systems, sonar systems, radar systems, structured light systems, stereo vision systems, optical interferometry systems, and so forth. The inventory management system622may use the 3D data acquired by the depth sensors620(4) to identify objects, determine a location of an object in 3D real space, identify a user616, and so forth.

One or more buttons620(5) may be configured to accept input from the user616. The buttons620(5) may comprise mechanical, capacitive, optical, or other mechanisms. For example, the buttons620(5) may comprise mechanical switches configured to accept an applied force from a touch of the user616to generate an input signal. The inventory management system622may use data from the buttons620(5) to receive information from the user616. For example, the cart618may be configured with a button620(5) to accept input from the user616and send information indicative of the input to the inventory management system622.

The sensors620may include one or more touch sensors620(6). The touch sensors620(6) may use resistive, capacitive, surface capacitance, projected capacitance, mutual capacitance, optical, Interpolating Force-Sensitive Resistance (IFSR), or other mechanisms to determine the position of a touch or near-touch. For example, the IFSR may comprise a material configured to change electrical resistance responsive to an applied force. The location within the material of that change in electrical resistance may indicate the position of the touch. The inventory management system622may use data from the touch sensors620(6) to receive information from the user616. For example, the touch sensor620(6) may be integrated with the cart618to provide a touchscreen with which the user616may select from a menu one or more particular items604for picking, enter a manual count of items604at an inventory location614, and so forth.

One or more microphones620(7) may be configured to acquire information indicative of sound present in the environment. In some implementations, arrays of microphones620(7) may be used. These arrays may implement beamforming techniques to provide for directionality of gain. The inventory management system622may use the one or more microphones620(7) to acquire information from acoustic tags706, accept voice input from the user616, determine ambient noise level, and so forth.

The sensors620may include instrumented auto facing units (IAFUs)620(8). The IAFU620(8) may comprise a position sensor configured to provide data indicative of displacement of a pusher. As an item604is removed from the IAFU620(8), the pusher moves, such as under the influence of a spring, and pushes the remaining items604in the IAFU620(8) to the front of the inventory location614. By using data from the position sensor, and given item data such as a depth of an individual item604, a count may be determined, based on a change in position data. For example, if each item604is 1 inch deep, and the position data indicates a change of 7 inches, the quantity held by the IAFU620(8) may have changed by 7 items604. This count information may be used to confirm or provide a cross check for a count obtained by other means, such as analysis of the sensor data from the weight sensors620(16).

The sensors620may include one or more optical sensors620(9). The optical sensors620(9) may be configured to provide data indicative of one or more of color or intensity of light impinging thereupon. For example, the optical sensor620(9) may comprise a photodiode and associated circuitry configured to generate a signal708or data indicative of an incident flux of photons. As described below, the optical sensor array620(14) may comprise a plurality of the optical sensors620(9). The optical sensors620(9) may include photodiodes, photoresistors, photovoltaic cells, quantum dot photoconductors, bolometers, pyroelectric infrared detectors, and so forth. For example, the optical sensor620(9) may use germanium photodiodes to detect infrared light.

One or more radio frequency identification (RFID) readers620(10), near field communication (NFC) systems, and so forth, may be included as sensors620. For example, the RFID readers620(10) may be configured to read the RF tags706. Information acquired by the RFID reader620(10) may be used by the inventory management system622to identify an object associated with the RF tag706such as the item604, the user616, the cart618, and so forth. For example, based on information from the RFID readers620(10) detecting the RF tag706at a particular inventory location, an item604being placed or picked may be determined.

The sensors620may include one or more accelerometers620(11), which may be worn or carried by the user616, mounted to the cart618, and so forth. The accelerometers620(11) may provide information such as the direction and magnitude of an imposed acceleration. Data such as rate of acceleration, determination of changes in direction, speed, and so forth, may be determined using the accelerometers620(11).

A gyroscope620(12) may provide information indicative of rotation of an object affixed thereto. For example, the cart618or other objects may be equipped with a gyroscope620(12) to provide data indicative of a change in orientation of the object.

A magnetometer620(13) may be used to determine an orientation by measuring ambient magnetic fields, such as the terrestrial magnetic field. The magnetometer620(13) may be worn or carried by the user616, mounted to the cart618, and so forth. For example, the magnetometer620(13) mounted to the cart618may act as a compass and provide information indicative of which direction the cart618is oriented.

An optical sensor array620(14) may comprise one or optical sensors620(9). The optical sensors620(9) may be arranged in a regular, repeating, or periodic two-dimensional arrangement such as a grid. The optical sensor array620(14) may generate image data. For example, the optical sensor array620(14) may be arranged within or below an inventory location614and obtain information about shadows of items604, hand of the user616, and so forth.

The sensors620may include proximity sensors620(15) used to determine presence of an object, such as the user616, the cart618, and so forth. The proximity sensors620(15) may use optical, electrical, ultrasonic, electromagnetic, or other techniques to determine a presence of an object. In some implementations, the proximity sensors620(15) may use an optical emitter and an optical detector to determine proximity. For example, an optical emitter may emit light, a portion of which may then be reflected by the object back to the optical detector to provide an indication that the object is proximate to the proximity sensor620(15). In other implementations, the proximity sensors620(15) may comprise a capacitive proximity sensor620(15) configured to provide an electrical field and determine a change in electrical capacitance due to presence or absence of an object within the electrical field.

The proximity sensors620(15) may be configured to provide sensor data indicative of one or more of a presence or absence of an object, a distance to the object, or characteristics of the object. An optical proximity sensor620(15) may use time-of-flight (ToF), structured light, interferometry, or other techniques to generate the distance data. For example, ToF determines a propagation time (or “round-trip” time) of a pulse of emitted light from an optical emitter or illuminator that is reflected or otherwise returned to an optical detector. By dividing the propagation time in half and multiplying the result by the speed of light in air, the distance to an object may be determined. In another implementation, a structured light pattern may be provided by the optical emitter. A portion of the structured light pattern may then be detected on the object using a sensor620such as a camera620(1). Based on an apparent distance between the features of the structured light pattern, the distance to the object may be calculated. Other techniques may also be used to determine distance to the object. In another example, the color of the reflected light may be used to characterize the object, such as skin, clothing, cart618, and so forth.

The one or more weight sensors620(16) are configured to measure the weight of a load, such as the item604, the cart618, or other objects. The weight sensors620(16) may be configured to measure the weight of the load at one or more of the inventory locations614, the cart618, on the floor of the facility602, and so forth. For example, the device100may include a plurality of weight sensors620(16). The weight sensors620(16) may include one or more sensing mechanisms to determine the weight of a load. These sensing mechanisms may include piezoresistive devices, piezoelectric devices, electrically resistive device, capacitive devices, electromagnetic devices, optical devices, potentiometric devices, microelectromechanical devices, and so forth. The sensing mechanisms of weight sensors620(16) may operate as transducers that generate one or more signals based on an applied force, such as that of the load due to gravity. For example, the weight sensor620(16) may comprise a strain gauge and a structural member that deforms slightly when weight is applied. The strain gauge may be a resistive strain gauge, optical strain gauge, capacitive strain gauge, and so forth. By measuring a change in the characteristic of the strain gauge, such as a change in electrical capacitance or resistance, a change in polarization of light in an optical fiber, and so forth, the weight may be determined. Instead of a strain gauge, the weight sensor620(16) may use a piezoelectric element that generates a voltage when a load is applied or removed. In another example, the weight sensor620(16) may comprise a force sensing resistor (FSR). The FSR may comprise a resilient material that changes one or more electrical characteristics when compressed. For example, the electrical resistance of a particular portion of the FSR may decrease as the particular portion is compressed. In another example, the weight sensor620(16) may comprise an optical fiber. As strain is applied to the optical fiber, changes to light passing through the fiber may be detected and used to determine an applied force, such as weight. In yet another example, the weight sensor620(16) may comprise an ultrasonic transducer that is used to detect changes in acoustic transmission in a material in response to an applied force on the material.

The inventory management system622may use the data acquired by the weight sensors620(16) to identify an object, determine a change in the quantity of objects, determine a location of an object, maintain shipping records, and so forth.

The sensors620may include other sensors620(S) as well. For example, the other sensors620(S) may include light curtains, ultrasonic rangefinders, thermometers, barometric sensors, hygrometers, and so forth.

In some implementations, the sensors620may include hardware processors, memory, and other elements configured to perform various functions. For example, the cameras620(1) may be configured to generate image data, send the image data to another device such as the server704, and so forth.

The facility602may include one or more access points710configured to establish one or more wireless networks. The access points710may use Wi-Fi, NFC, Bluetooth, or other technologies to establish wireless communications between a device and the network702. The wireless networks allow the devices to communicate with one or more of the sensors620, the inventory management system622, the tags706, a communication device of the cart618, or other devices.

Output devices712may also be provided in the facility602. The output devices712are configured to generate signals708, which may be perceived by the user616or detected by the sensors620. In some implementations, the output devices712may be used to provide illumination of the optical sensor array620(14).

Haptic output devices712(1) are configured to provide a signal708that results in a tactile sensation to the user616. The haptic output devices712(1) may use one or more mechanisms such as electrical stimulation or mechanical displacement to provide the signal708. For example, the haptic output devices712(1) may be configured to generate a modulated electrical signal, which produces an apparent tactile sensation in one or more fingers of the user616. In another example, the haptic output devices712(1) may comprise piezoelectric or rotary motor devices configured to provide a vibration, which may be felt by the user616.

One or more audio output devices712(2) may be configured to provide acoustic output. The acoustic output includes one or more of infrasonic sound, audible sound, or ultrasonic sound. The audio output devices712(2) may use one or more mechanisms to generate the acoustic output. These mechanisms may include, but are not limited to, the following: voice coils, piezoelectric elements, magnetostrictive elements, electrostatic elements, and so forth. For example, a piezoelectric buzzer or a speaker may be used to provide acoustic output.

The display devices712(3) may be configured to provide output, which may be seen by the user616or detected by a light-sensitive sensor such as a camera620(1) or an optical sensor620(9). In some implementations, the display devices712(3) may be configured to produce output in one or more of infrared, visible, or ultraviolet light. The output may be monochrome or in color. The display devices712(3) may be one or more of emissive, reflective, microelectromechanical, and so forth. An emissive display device712(3), such as using LEDs, is configured to emit light during operation. In comparison, a reflective display device712(3), such as using an electrophoretic element, relies on ambient light to present an image. Backlights or front lights may be used to illuminate non-emissive display devices712(3) to provide visibility of the output in conditions where the ambient light levels are low.

The display devices712(3) may be located at various points within the facility602. For example, the addressable displays may be located on inventory locations614, carts618, on the floor of the facility602, and so forth.

Other output devices712(P) may also be present. For example, the other output devices712(P) may include scent/odor dispensers, document printers, 3D printers or fabrication equipment, and so forth.

FIG. 8illustrates a block diagram800of a server704configured to support operation of the facility602, according to some implementations. The server704may be physically present at the facility602, may be accessible by the network702, or a combination of both. The server704does not require end-user knowledge of the physical location and configuration of the system that delivers the services. Common expressions associated with the server704may include “on-demand computing”, “software as a service (SaaS)”, “platform computing”, “network-accessible platform”, “cloud services”, “data centers”, and so forth. Services provided by the server704may be distributed across one or more physical or virtual devices.

One or more power supplies802may be configured to provide electrical power suitable for operating the components in the server704. The one or more power supplies802may comprise batteries, capacitors, fuel cells, photovoltaic cells, wireless power receivers, conductive couplings suitable for attachment to an external power source such as provided by an electric utility, and so forth. The server704may include one or more hardware processors804(processors) configured to execute one or more stored instructions. The processors804may comprise one or more cores. One or more clocks806may provide information indicative of date, time, ticks, and so forth. For example, the processor804may use data from the clock806to associate a particular interaction with a particular point in time.

The server704may include one or more communication interfaces808such as input/output (I/O) interfaces810, network interfaces812, and so forth. The communication interfaces808enable the server704, or components thereof, to communicate with other devices or components. The communication interfaces808may include one or more I/O interfaces810. The I/O interfaces810may comprise Inter-Integrated Circuit (I2C), Serial Peripheral Interface bus (SPI), Universal Serial Bus (USB) as promulgated by the USB Implementers Forum, RS-232, and so forth.

The I/O interface(s)810may couple to one or more I/O devices814. The I/O devices814may include input devices such as one or more of a sensor620, keyboard, mouse, scanner, and so forth. The I/O devices814may also include output devices712such as one or more of a display device712(3), printer, audio speakers, and so forth. In some embodiments, the I/O devices814may be physically incorporated with the server704or may be externally placed.

The network interfaces812may be configured to provide communications between the server704and other devices, such as the carts618, routers, access points710, and so forth. The network interfaces812may include devices configured to couple to personal area networks (PANS), local area networks (LANs), wireless local area networks (WLANS), wide area networks (WANs), and so forth. For example, the network interfaces812may include devices compatible with Ethernet, Wi-Fi, Bluetooth, ZigBee, and so forth.

The server704may also include one or more buses or other internal communications hardware or software that allow for the transfer of data between the various modules and components of the server704.

As shown inFIG. 8, the server704includes one or more memories816. The memory816may comprise one or more non-transitory computer-readable storage media (CRSM). The CRSM may be any one or more of an electronic storage medium, a magnetic storage medium, an optical storage medium, a quantum storage medium, a mechanical computer storage medium, and so forth. The memory816provides storage of computer-readable instructions, data structures, program modules, and other data for the operation of the server704. A few example functional modules are shown stored in the memory816, although the same functionality may alternatively be implemented in hardware, firmware, or as a system on a chip (SoC).

The memory816may include at least one operating system (OS) module818. The OS module818is configured to manage hardware resource devices such as the I/O interfaces810, the I/O devices814, the communication interfaces808, and provide various services to applications or modules executing on the processors804. The OS module818may implement a variant of the FreeBSD operating system as promulgated by the FreeBSD Project; other UNIX or UNIX-like variants; a variation of the Linux operating system as promulgated by Linus Torvalds; the Windows operating system from Microsoft Corporation of Redmond, Wash., USA; and so forth.

Also stored in the memory816may be a data store820and one or more of the following modules. These modules may be executed as foreground applications, background tasks, daemons, and so forth. The data store820may use a flat file, database, linked list, tree, executable code, script, or other data structure to store information. In some implementations, the data store820or a portion of the data store820may be distributed across one or more other devices including the servers704, network attached storage devices, and so forth.

A communication module822may be configured to establish communications with one or more of the carts618, sensors620, display devices712(3), other servers704, or other devices. The communications may be authenticated, encrypted, and so forth.

The memory816may store an inventory management module824. The inventory management module824is configured to provide the inventory functions as described herein with regard to the inventory management system622. For example, the inventory management module824may track items604between different inventory locations614, to and from the carts618, generate restocking orders, direct operation of robots within the facility602, and so forth. During operation the inventory management module824may access sensor data826or information based on the sensor data826.

Information used by the inventory management module824may be stored in the data store820. For example, the data store820may be used to store sensor data826such as weight sensor data828, cart data830, user location data832, item data834, physical layout data836, threshold data838, output data840, interaction data842, user billing data844, and so forth.

Output from the devices100may be used to determine the weight sensor data828. For example, the weight sensor data828may comprise information indicative of a weight at a particular time for a particular bracket104.

The cart data830comprises information about the items604that are determined to be in the custody of the user616. For example, the cart data830may indicate the items604that are within the cart618that is associated with the account of the user616, currently being carried by the user616, and so forth. The cart data830may be based at least in part on the interaction data842. For example, the interaction data842may be associated with a particular user616, changing the contents of the cart data830based on the interaction.

The inventory management module824may generate the user location data832that is indicative of the location of the user616within the facility602. For example, the inventory management module824may use image data obtained by the cameras620(1) to determine a location of the user616. In other implementations, other techniques may be used for determining the user location data832. For example, data from the smart floor620(2) may be used to determine the location of the user616. Based on the user location data832and the interaction data842, a particular interaction may be associated with an account of a particular user616. For example, if the user location data832indicates that the user616is present in front of inventory location614(492) at time17:47:20and the interaction data842indicates a pick of a quantity of one item604(27) from the device associated with accessory102at inventory location614(492) at17:47:27, the user616may be billed for that pick.

The data store820may store item data834that comprises information about a particular type of item604. The item data834may include information indicative of a weight of a single item604, or a package, kit, or other grouping considered to be a single item604. The item data834may include other characteristics of that type of item604such as physical dimensions, characteristics about how the item604appears, and so forth. The item data834may indicate the types and quantities of items604that are expected to be stored at a particular area at a particular inventory location614. The item data834may include other data. For example, the other data may comprise weight distribution of the item604, and so forth. The item data834may also be associated with the physical layout data836. For example, the item data834may indicate that a particular type of item604is stowed at a particular device.

The data store820may store physical layout data836that provides information indicative of where inventory locations614are in the facility, location of sensors620, information about sensor orientation and field of view (where applicable), and so forth. For example, the physical layout data836may comprise information representative of a map or floor plan of the facility602with relative positions of inventory locations614, accessories102within those inventory locations614, and so forth. In some implementations the physical layout data836may include planogram data indicative of how types of items604are to be arranged at the inventory locations614, location of the areas, and so forth.

The physical layout data836may also comprise information about the type of inventory location614in use, information indicative of the physical configuration or placement of parallel planar weight sensing devices, accessories102, weight sensors620(16), and so forth. For example, the physical layout data836may indicate that a particular model parallel planar weight sensing device that is in use at a particular inventory location614. Using this information, data about the device may be retrieved and used to generate output data840. The inventory management module824may utilize the physical layout data836during operation.

The threshold data838comprises the various thresholds used by the system. For example, the threshold data838may specify values for confidence thresholds that are used to determine if a hypothesis is deemed suitable to be used to generate the interaction data842.

During operation, electronics associated with the inventory location614or the inventory management module824may determine output data840. The output data840may be indicative of a detection of a weight change at a particular parallel planar weight sensing device.

As described above, the interaction data842may provide information about an interaction, such as a pick of an item604from the inventory location614, a place of an item604to the inventory location614, a touch made to an item604at the inventory location614, a gesture associated with an item604at the inventory location614, and so forth. The interaction data842may include one or more of the type of interaction, interaction location identifier indicative of where from the inventory location614the interaction took place, item identifier, quantity change to the item604, user identifier, and so forth.

The inventory management module824may generate user billing data844based on the cart data830. For example, when a user616leaves the facility602, the items in the cart data830and their respective quantities may be used to generate user billing data844. In some implementations, the user billing data844may be used to bill a payment account associated with the user616.

The inventory management module824may process the sensor data826and generate other data. For example, based on the interaction data842, a quantity of a type of item604at a particular inventory location614may drop below a threshold restocking level. The inventory management system622may generate data comprising a restocking order indicative of the inventory location614and a quantity needed to replenish stock to a predetermined level. The restocking order may then be used to direct a robot to restock that inventory location614.

Other modules846may also be present in the memory816as well as other data848in the data store820.