Electromagnetic user tracking system

A facility is equipped with floor tiles, each tile having several segments, each segment with an antenna. Each segment on a tile is associated with a particular timeslot. A transmitter at the tile transmits on a specific frequency. During the particular timeslot for that segment, a signal at the specific frequency is transmitted and radiated from the antenna for that segment. A user electromagnetically couples to one or more antennas of the tile, acting as a signal path for the signal. A receiver in a second tile detects the signal, and information about the frequency detected, timeslot for the signal, and relative signal strength is generated. Where the user has stepped from is determined using the information. By concatenating these steps, a path of the user may be determined. Receivers in shelves facilitate disambiguation of one user from another when interacting with items.

BACKGROUND

Retailers, wholesalers, and other product distributors typically maintain an inventory of various items that may be ordered, purchased, leased, borrowed, rented, viewed, and so forth, by clients or customers. For example, an e-commerce website may maintain inventory in a fulfillment center. When a customer orders an item, the item is picked from inventory, routed to a packing station, packed, and shipped to the customer. Likewise, physical stores maintain inventory in customer accessible areas, such as in a shopping area, and customers can pick items from inventory and take them to a cashier for purchase, rental, and so forth.

Many physical stores also maintain inventory in a storage area, fulfillment center, or other facility that can be used to replenish inventory located in the shopping areas or to satisfy orders for items that are placed through other channels (e.g., e-commerce). Other examples of entities that maintain facilities holding inventory include libraries, museums, rental centers, and so forth. In each instance, for an item to be moved from one location to another, it is picked from its current location and transitioned to a new location. It is often desirable to monitor quantity or movement of users, inventory, or other objects within the facility.

Other types of facilities may also benefit from tracking of users or other objects. For example, hospitals may wish to track patients, airports may wish to track passengers, and so forth.

DETAILED DESCRIPTION

Described in this disclosure are systems and techniques for generating data in a materials handling facility (facility). The facility may include, or have access to, an inventory management system. The inventory management system may be configured to maintain information about items, users, condition of the facility, and so forth. For example, the inventory management system may maintain data indicative of a number of items at a particular fixture, what items a particular user is ordered to pick, how many items have been picked or placed at the fixture, requests for assistance, environmental status of the facility, and so forth.

Operation of the facility may be facilitated by using one or more sensors to acquire information about interactions in the facility. The inventory management system may process sensor data from the one or more sensors to determine tracking data, interaction data, and so forth. The tracking data provides information about the location of a user within the facility, their path through the facility, and so forth. The interaction data is indicative of an action such as picking or placing an item at a particular location on the fixture, touching an item at a particular location on the fixture, presence of the user at the fixture without touching the item, and so forth. For example, the inventory management system may use the sensor data to generate tracking data and interaction data that determines a type of item a user picked from a particular fixture.

A fixture may include one or more item stowage areas such as shelves, hangers, and so forth, that hold or otherwise support a type of item. The fixture may be arranged into sections, such as lanes on a shelf. For example, a shelf may have three lanes, with each lane holding a different type of item. Items may be added to (placed) or removed (picked) from the fixture, moved from one fixture to another, and so forth.

The floor of the facility may comprise a plurality of smart floor tiles. The smart floor tiles may include transmitters that generate low frequency radio signals and a receiver that detects the low frequency radio signals. For example, the carrier of these signals may be less than or equal to 30 MHz. The smart floor tiles may also include sensors such as touch or pressure sensors that provide object data indicative of an object such as a foot or wheel that is in contact with the smart floor tile.

The floor of the facility is composed of clusters of smart floor files. Each cluster includes a plurality of smart floor tiles. Each smart floor tile, in turn, has a plurality of segments. One or more transmitters of the smart floor tile are configured to transmit several signals at a particular frequency that is associated with that smart floor tile. Within the cluster, each smart floor tile transmits on a different frequency. The same tile frequencies may be reused by other smart floor tiles in another cluster.

In one implementation that utilizes time division multiple access (TDMA), transmissions radiated from a particular segment are scheduled to occur during a particular timeslot. The segment signal is transmitted on the particular frequency that is associated with the smart floor tile. Each segment signal within a given smart floor tile is transmitted during a different timeslot, or interval of time. Each segment includes at least one antenna that radiates the segment signal assigned to that segment during the timeslot. In some implementations, an initial signal may be transmitted using all of the antennas in the smart floor tile prior to the transmission of the segment signals. The initial signal may operate as a preamble, providing a reference against which timing may be synchronized to determine what timeslot a particular segment signal is associated with. In some implementations, the initial signal may also be used to minimize a receiver sampling noise. For example, the initial signal may break a squelch.

In another implementation that uses code division multiple access (CDMA), each segment may transmit a segment signal that is modulated to include a CDMA code. The CDMA code is thus used to identify a particular segment in the smart floor tile, while the frequency of the signal is used to determine the particular smart floor tile within a cluster. In this implementation, the segment signals may be transmitted at the same time from their respective segments and the segment antennas therein.

An object may electromagnetically couple to a proximate antenna in the smart floor tile. For example, when a user is standing with their left foot on a first segment in a first smart floor tile, their left foot electromagnetically couples to the antenna in that segment. As a result of this coupling, a first set of the signals transmitted by the first segment are transferred along the body of the user by way of this electromagnetic coupling. Continuing the example, the signals are propagated along the body of the user standing on that segment to the other extremities such as the right foot and both hands.

As the user walks, their right foot comes to rest on a second smart floor tile. The body of the user now acts as a bridge, providing a signal path along which signals may travel between the first and second smart floor tiles. A receiver in the second smart floor tile detects the first set of signals that originated by the first smart floor tile under the left foot. Meanwhile, the reverse happens with the first smart floor tile detecting a second set of signals that originate from the second smart floor tile and are passed from the right foot through the user's body to the left foot.

The smart floor tiles may generate tile output data that includes received characteristic data. The received characteristic data provides information about the signals received and may include the received signal strength of those signals. The tile output data from the first and second smart floor tiles may be used to determine that the user is in contact with both tiles. For example, a server may receive the tile output data and determine that these two smart floor tiles and their respective segments are reporting received characteristic data indicative of the other smart floor tile. Given this correspondence, the two locations of the received characteristic data may be associated with the feet of a single user, and a location of the user may be determined. The server may also analyze the received characteristic data obtained from several segments and estimate a shape of a user's foot.

By determining a successive series of locations of the user over time, tracking data may be generated. The tracking data comprises information indicative of the user's path through the facility.

The signals provided by the transmitters may be used to determine the relative position of the user's hand(s) with respect to a fixture, to determine an item interacting with a location, and so forth. For example, a smart floor tile may transmit the signals that are then conducted through the user and detected using antennas arranged along a shelf that are connected to one or more receivers. By using the relative signal strength at the different antennas and the known position of the antennas, a position of the user's hand may be determined with respect to the shelf. When the user touches an item stored on the shelf, the signals transfer from the user to the item and from there transfer to the shelf. For example, the amplitude of the electromagnetic signal received at an antenna that is located beneath the item that is being touched may increase significantly relative to the level obtained when there is no contact. As a result of this increase, the user may be deemed to have had contact with the item stored at that location on the shelf.

Information about which user is interacting with the fixture, touching an item, and so forth, may be determined by analyzing the particular signals that are received. For example, the receiver of the shelf may generate characteristic data for the signal received at the shelf. This characteristic data may be compared with the tile output data described above to determine which user is in contact with the particular smart floor tiles and segments. This contact produces a characteristic pattern of signals that corresponds to the received characteristic data for the signal received at the shelf. In some implementations, the characteristic data may be obtained using signals received from a subset of the antennas at the shelf. For example, the antennas corresponding to peak received signal strength values that are used to determine the relative position may be used to produce the characteristic data. The spatial diversity between different antennas on the shelf may be used to separate out different hands, and the different characteristics may be used to distinguish one user from another.

By using the techniques described herein, operation of the facility may be improved. Details about movement of the users in the facility, the interactions between users and items in the facility, and so forth, may be quickly and accurately determined. For example, as items are picked, placed, and so forth, information such as inventory levels based on changes in the count of items at the fixtures may be readily and more accurately determined. As a result, the inventory management system may be able to quickly track what item a user has interacted with, maintain up-to-date inventory information, and so forth. Tracking of users may be facilitated, allowing for enhanced services to the users of the facility, such as making the facility respond to the presence of a user. For example, as an authorized user approaches a fixture holding items that is locked, the fixture may unlock to provide access.

The smart floor tiles provide various technical advantages including, but not limited to, reductions in bandwidth compared other sensor methodologies, improved tracking of individual users in congested environments, detection of potential hazards, detection of user incapacity, and so forth. The smart floor tiles are mechanically robust and provide high resolution tracking data for users as well as providing the ability to identify who is interacting with a particular fixture, item, and so forth. The system described herein allows for reduced capital expenditures, as well as reduced operating expenditures relative to other sensor methodologies. For example, compared to vision tracking systems, installation of smart floor tiles is less expensive and, during operation, requires fewer computational resources, is less prone to failure or environmental interference, and so forth. The smart floor tiles and the information obtained thereby may be used in conjunction with other systems, such as vision tracking systems, tag tracking systems, and so forth.

The system described herein may be used in other types of facilities, both commercial and non-commercial. For example, the smart floor tiles may be installed within a home or care facility and provide information such as user tracking, if the user is standing, lying on the floor, and so forth. The system may be used to improve user safety by determining the whereabouts of the user, determining if the user has fallen, and so forth. The system may also provide enhanced functionality, such as operating in conjunction with building operation. For example, by tracking the user in the facility, lighting, environmental controls, and so forth, may be controlled based on the location of the user.

Illustrative System

FIG. 1illustrates a system100using a variety of sensors to generate tracking data and other information within a facility, according to some implementations. The facility includes a floor102. The floor102may comprise a plurality of smart floor tiles (SFTs)104. A group of the SFTs104is a cluster. The floor102may include a plurality of clusters.

Each of the SFTs104may include various components such as antennas, transmitters, receivers, hardware processors, sensors, and so forth. The SFT104may itself be subdivided into segments. For example, each segment may comprise a different antenna. The SFT104may be configured to transmit and receive electromagnetic signals (EMS)106. Each SFT104may transmit at a particular frequency that is different from the other SFTs104in a cluster. As a result, within the cluster, a particular SFT104may be distinguished by the EMS106that it transmits at a particular frequency. The EMS106may be transmitted at a low power. For example, the EMS106may have a power level of less than 500 microwatts.

These EMS106may be propagated by the body of a user. For example, the EMS106may be propagated along the skin or clothing of the user, travelling from one SFT104to another, or from one SFT104to another device such as the fixtures108. Each SFT104may transmit several signals. The transmissions may be continuous or may be made at particular times, using one or more of the antennas of the SFT104. The different types of signals that may be transmitted are discussed in more detail below with regard toFIG. 2. The SFT104is discussed in more detail below with regard toFIG. 3.

Within the facility may be one or more fixtures108. The fixture108may include shelves, hangers, and so forth, that hold or otherwise support a type of item. The fixture108may be arranged into sections, such as lanes on a shelf. For example, a shelf may have three lanes, with each lane holding a different type of item. Items may be added to (placed) or removed (picked) from the fixture108, moved from one fixture108to another, and so forth. In some implementations, the SFTs104may be installed, and the fixtures108and other objects may then be installed on the SFTs104. In other implementations, the fixtures108may be installed and then the SFTs104may be installed around the fixtures108. Some portions of the floor102may omit SFTs104. For example, SFTs104may be omitted from around the perimeter of a room, immediately adjacent to a wall, underneath a fixture108, and so forth.

An entry110provides access for a user112to the facility. For example, the entry110may comprise a foyer, door, gated entry area, and so forth. In some implementations, an identity of the user112may be asserted at the entry. For example, the user112may provide identification credentials such as swiping a card, carrying a device that transmits or displays authentication credentials, and so forth. The user112may move throughout the facility, with movement depicted in this illustration as a user path114across the floor102. The user112may use various tools while in the facility, such as a tote116, pallet jack, and so forth. The tote116may include a basket, cart, bin, bag, and so forth. During operation of the facility, users112thus move around, picking, placing, or otherwise interacting with items at the fixtures108.

The SFTs104may obtain electrical power from a power supply118. For example, the power supply118may provide 24 volts direct current (VDC) to one or more of the SFTs104. The power supply118may be configured to obtain power from building mains and then provide conditioned power for use. The SFTs104are connected to a network120. The network120allows for communication between SFTs104and other devices, such as described below.

A clock122may provide a clock signal124or other clock data that is transmitted to the SFTs104using the network120. In some implementations, the clock signal124may be distributed via another mechanism, such as by the power supply118by way of a power distribution network. For example, the clock signal124may be overlaid as an alternating current signal along one or more of the electrical conductors used to supply direct current power to the SFTs104. In some implementations, the clock signal124may be omitted, with each SFT104operating with independent clocks122or “free running”.

One or more processors of the SFTs104may generate tile output data126. The tile output data126may include characteristic data128. The characteristic data128is indicative of a plurality of EMS106, each at different frequencies, and the received signal strength of the signals at each of the different frequencies. In some implementations, the characteristic data128may include information such as a timestamp associated with the EMS106. The characteristic data128is indicative of a particular SFT104and one or more segments of the SFT104. The tile output data126may include information about the SFT104itself and the segments thereon that received the signals that are represented by the characteristic data128. For example, the tile output data126may comprise characteristic data128for the EMS106received at each segment.

During operation, a first foot of the user112is in contact with a first SFT104(1). The particular EMS106(1) transmitted by the first SFT104(1) is electromagnetically coupled to the body of the user112and transferred along a signal path that includes the body of the user112from the first foot to the second foot of the user112. Meanwhile, a first receiver in the first SFT104(1) is listening for EMS106. As the second foot comes into contact with a second SFT104(2), a bidirectional exchange of EMS106may take place. The first SFT104(1) transmits a first set of EMS106(1) (at a first tile frequency and with segment signals sent during respective timeslots), which is received by a receiver of the second SFT104(2). Meanwhile, the second SFT104(2) transmits a second set of EMS106(2) (at a second tile frequency and with one or more segment signals sent during respective timeslots), which is received by a receiver of the first SFT104(1).

As the user112walks across the floor102, they act as a bridge between successive SFTs104, resulting in a trail of pairs of SFTs104(or the segments therein) that have been trod upon. Tile output data126may be generated that is indicative of the identity of the receiving SFT104and the characteristic data128indicative of the EMS106that were received. The tile output data126may be transferred from the SFT104in the floor102to an inventory management system130via the network120. Other information, such as the fixture data132, may also be provided to the inventory management system130.

The inventory management system130may include a tracking module134. The tracking module134may use one or more of the tile output data126or the fixture data132to generate tracking data136. The tracking data136may include one or more of information indicative of the user path114within the facility, current location, location at a particular time, and so forth. In some implementations, the tracking module134may be executed as a tracking system, such as provided by one or more computing devices. In some implementations, the tracking module134may use the characteristic data128to further distinguish between users112or other objects. For example, the user112, tote116, or other object may include a transmitter that emits a discrete EMS106or a receiver that receives the EMS106and provides characteristic data128. In some implementations, the distribution of received EMS106signal amplitude with respect to feet (such as greater signal strength at the toe than at the heel) may be used to determine an approximate shape of the foot that is indicative of a particular user112or other object to be tracked. This data may be used instead of, or in conjunction with, the characteristic data128to generate the tracking data136.

An analysis module138may use the tracking data136to generate group data140. The group data140may comprise information that associates a plurality of users112as belonging to a common group or having a common affiliation. For example, members of a family within the facility may be deemed to be a group, members of the same picking crew may be members of a group, and so forth. In some implementations, the tile output data126may be processed to determine the group data140. For example, several users112may be holding hands or otherwise in physical contact with one another. As a result of this contact, the EMS106from a first SFT104(1) may be transferred through those users112to the receivers of the SFTs104beneath each of the other members of the group. By determining the presence of a plurality of users112, such as by multiple footprints detected by the sensors within the SFTs104that share a common EMS106that encode the same characteristic data128, group data140may be determined.

The analysis module138may also generate interaction data142. The interaction data142is indicative of an action such as picking or placing an item at a particular fixture108, approaching but not touching an item stowed at the fixture108, presence of the user112at the fixture108, and so forth. For example, the analysis module138may use tracking data136to determine that a particular user112was in front of a particular fixture108at a time when that fixture108experienced a change in quantity of items stowed therein. Based on this correspondence, a particular user112may be associated with that change in quantity, and interaction data142indicative of this may be generated.

The analysis module138may also use the fixture data132or other data obtained from one or more sensors or other devices located at or near the fixture108to generate the interaction data142. In one implementation, the fixture108may include one or more receivers that are able to receive the EMS106. As the user112comes into contact with the item stowed at the fixture108, their body and the item itself provide a pathway for the EMS106to be transferred to an antenna located at the fixture108. As a result, use of the SFT104and the EMS106provides the additional benefit of unambiguously identifying an item that the particular user112interacted with. The analysis module138is configured to generate the interaction data142based on inputs including, but not limited to, the tile output data126, the fixture108, and so forth.

WhileFIG. 1depicts the floor102as being completely covered with SFTs104, in some implementations, only a portion of the floor102may include SFTs104. For example, SFTs104may be placed within an aisle and not underneath the fixtures108. In another example, the SFTs104may be deployed in front of the fixtures108.

The inventory management system130may access data from other sensors within the facility. For example, image data may be obtained from a plurality of cameras located within the facility. Various image processing techniques may be used, such as object recognition, blob tracking, and so forth, to generate information from this image data. In some implementations, the image data may be processed by human operators. For example, a human operator may be presented with images as well as tracking data136to resolve an ambiguity or loss of tracking.

FIG. 2illustrates an arrangement200of SFTs104and their respective segments, according to some implementations.

A portion of the floor102is depicted which is made up of several clusters202. A cluster202is a grouping of SFTs104. For example, the portion of the floor102depicted here includes 25 clusters202, each cluster202including 9 SFTs104. Each SFT104in turn may include one or more segments204. Continuing the example depicted here, each SFT104includes 16 segments204. In other implementations, the cluster202may include different numbers of SFTs104, each SFT104may include different numbers of segments204, and so forth.

In some implementations, segments204may comprise portions of a SFT104or may be discrete devices that are joined together to form a SFT104. For example, the segments204may be connected to one another, a backplane, wiring harness, and so forth, to form a SFT104.

The physical size of a cluster202may be determined in some implementations based on a maximum expected stride length for a user112. For example, a user112may be expected to have a stride length that is less than 3 feet while walking. If the SFTs104are 1 foot on each side, then the cluster202depicted here is 3 feet by 3 feet. Likewise, each segment204is 3 inches by 3 inches. In other implementations, other sizes of segments204, SFTs104, and clusters202may be used. Also, other shapes of segments204, SFTs104, and clusters202may be used. For example, the segments204may be triangular shaped, SFTs104may be rectangular, and so forth.

The SFT104transmits one or more EMS106. For example, an initial signal206and one or more segment signals208may be transmitted. Together, these signals comprise the EMS106emitted by the SFT104. The segment signal208may be unmodulated, or may contain null data. The EMS106transmitted by the SFT104may be at a first frequency. The first frequency is representative of that particular SFT104within a particular cluster202. The segment signal208may also be transmitted at the first frequency, and includes a signal that is transmitted at a particular time within a timeslot210. The timeslot210, in turn, is associated with the particular segment204within a particular SFT104. For example, a segment signal208received during a particular timeslot210may be deemed associated with the segment204assigned to that timeslot. In some implementations, the occurrence of the segment signal208at a particular time within a timeslot210is thus representative of the particular timeslot210, and the corresponding segment204associated with that timeslot210. In some implementations, the timeslot210may be 1 millisecond (ms) or less in duration. The characteristic data128may include timestamp data associated with receipt of one or more of the initial signal206or the segment signal208. In some implementations, the timestamp data may be used to determine the timeslot210.

In this figure, the particular frequency of the signals transmitted by the SFT104is represented by a letter, such as “A”, “B”, “C”, and so forth, while each segment204with a corresponding timeslot210is represented by a number “1”, “2”, “3”, and so forth. For example, the frequencies used by the SFTs104may begin at 40 kHz with 1 kHz spacing, resulting in “A” representing 40 kHz, “B” representing 41 kHz, “C” representing 42 kHz, and so forth. Likewise, the segment signals208may be configured such that the first timeslot210(1) is associated with segment204(1), a second timeslot210(2) is associated with a segment204(2), and so forth. In one implementation, each timeslot210may be less than or equal to 1 ms in duration. The notation, frequencies, and timeslot size are provided by way of illustration and not necessarily as limitations. In some implementations, the duration of timeslots210may differ. For example, timeslot210(1) may be 5 ms in duration while timeslot210(2) is 1 ms in duration.

The EMS106as emitted may exhibit sinusoidal waveforms. In other implementations, other waveforms such as square, triangle, sawtooth, and so forth, may be used. Use of sinusoidal waveforms may allow for reduced channel spacing and minimize adjacent channel interference. The EMS106may be transmitted at fixed carrier frequencies of between 20 kilohertz and 15 megahertz. In other implementations, other frequencies may be used. In some implementations, the waveforms of the initial signal206may differ from the waveforms of the segment signal208.

In some implementations, as depicted here, each SFT104may utilize the same spatial arrangement of segments204, segment number scheme, and corresponding timeslot210associated with that segment204. For example, the SFTs104in the floor102may have the same arrangement of segments204, such as beginning at the top left of the SFT104with segment1and increasing from left to right and into subsequent rows, such as in SFT104(A).

In other implementations, the SFTs104may be arranged such that adjacent segments204of adjacent SFTs104use the same timeslots210. For example, the SFTs104may be arranged such that a first physical arrangement of segments204and their respective segment signal208timeslots210for a first SFT104(A) are mirrored in a second SFT104(B) that is adjacent to the first SFT104(A). In this configuration, immediately adjacent segments204utilize the same segment signal208timeslots210. Use of this mirrored arrangement may improve performance of the system by producing an increase in the total amplitude of the segment signals208in situations where a user112(or other object) has a foot (or other portion) in contact with two different SFTs104. This arrangement may also provide additional benefits with regard to computing the location of an object, such as the position of the feet of the users112.

In this illustration, three users112are depicted. The right foot of user112(1) is above the following SFTs104and their respective segments204:104(6)(5),104(6)(9), and104(6)(13). The left foot of user112(1) is above the following SFTs104and their respective segments204:104(D)(6),104(D)(7),104(D)(10),104(D)(11),104(D)(12),104(D)(15), and104(D)(16). Also shown are the feet of users112(2) and112(3) at other locations within the cluster202. A representation of the characteristic data128associated with the second user112(2) is depicted below with regard toFIG. 6.

The SFTs104may be configurable, such that they may be installed and then configured to transmit at a particular frequency after physical installation of the SFT104. For example, SFT104(A) may be electronically switched to generate EMS106at a specified frequency.

The SFTs104may be installed inside or outside of a building. For example, the floor102of an uncovered area, yard, exterior shed, and so forth may be equipped with the SFTs104.

While the hierarchy of floor102, cluster202, SFT104, and segments204is discussed and used herein, it is understood that other hierarchies or arrangements may utilize the techniques described herein. For example, instead of segments204being part of an SFT104, the floor102may comprise a plurality of single-segment SFTs104. The segment signals208may then be transmitted, in the appropriate timeslot210, to indicate a particular timeslot210. In another example, instead of arranging SFTs104into clusters202, each SFT104may have a unique combination of one or more of frequency, timeslot, or other characteristics of EMS106that are used to identify a particular SFT104on the floor102.

FIG. 3illustrates the arrangement300of components included in a SFT104, according to some implementations. A side view of a portion of the SFT104depicts a top layer comprising a protective material, such as flooring material302. The flooring material302is electrically non-conductive under ordinary conditions. For example, the flooring material302may include plastic, ceramic, wood, textile, or other material. Beneath a layer of flooring material302may be one or more antennas304and one or more sensors306. The antennas304may comprise structures designed to accept or emit EMS106. In some implementations, the antennas304may also serve as the flooring material302. For example, the antennas304may comprise aluminum or steel sheets upon which the users112walk. The active portion of the antenna304comprises that portion of the antenna304that is used to radiate or receive an EMS106.

The SFT104may include a plurality of antennas304. For example, the antennas304may be arranged to form an array. In some implementations, the active portion of the antennas304may have a surface area that occupies at least 1 square inch. Each segment204includes at least one segment antenna304. The segment antenna304of the segment204may be the same size as the segment204or may be smaller. For example, the segment204may be 3 inches by 3 inches square, but the segment antenna304in that segment204may only be 2 inches by 2 inches square. In another example, the segment204may be 3 inches by 3 inches square and the segment antenna304in that segment204may be 3 inches by 3 inches square. Each segment antenna304may have a maximum size of sixteen square inches, in some implementations. The size of the segment antennas304may be determined at least in part based on the expected size of the objects in contact with the floor102, such as the size of the foot of the user112. In one implementation, antennas304may be shared, with a single antenna304being used to both transmit and receive simultaneously or at different times. In another implementation, separate antennas304may be used to transmit and receive.

The SFT104may also include a plurality of sensors306that may be arranged to form one or more arrays. For example, the sensors306may include weight sensors that measure the weight applied to a particular segment204. The sensors306provide sensor output data. The arrangement of an array of one type of sensor may differ from another type of sensor. In some implementations, the sensors306may include a magnetometer that provides information about local magnetic fields.

As illustrated here, the antennas304may be located within a common plane. In other implementations, the antennas304may be arranged within a layer that is above the sensors306, below the sensors306, and so forth. A load bearing support structure308may be beneath the sensors306and the antennas304and provides mechanical and physical separation between the underlying subfloor310upon which the SFT104rests and the flooring material302. The support structure308may comprise a series of pillars, posts, ribs, or other vertical elements. The support structure308may comprise a composite material, plastic, ceramic, metal, or other material. In some implementations, the support structure308may be omitted, and electronics312or structures associated with the electronics312may be used to support a load on the flooring material302. For example, the electronics312may comprise a glass fiber circuit board that provides mechanical support while also providing a surface for mounting the electronics312. The subfloor310may comprise concrete, plywood, or existing flooring materials over which the SFT104is installed. In some implementations, the SFT104may be affixed to the subfloor310, or may be unaffixed or “floating”. For example, the SFT104may be adhered to the subfloor310using a pressure sensitive adhesive.

The SFT104includes the electronics312. The electronics312may include the elements described elsewhere in more detail. In the implementation depicted here, the electronics312are arranged within the support structure308. In some implementations, one or more of the antennas304or the sensors306may be located within the support structure308. The support structure308may operate as a heat sink to dissipate heat generated by operation of the electronics312.

The SFT104may incorporate a wiring recess314on an underside of the SFT104. For example, the support structure308and the electronics312may be formed or arranged to provide a pathway for a wiring harness316to pass beneath at least a portion of the SFT104. The wiring recess314may extend from one edge of the SFT104to another, may extend in different directions, and so forth. For example, the wiring recess314may be arranged in a “+” or cross shape, allowing for wiring harnesses316to pass along the X or Y axes as depicted here.

The wiring harness316may provide a coupling to one or more of the power supply118, the network120, and so forth. For example, the wiring harness316may include conductors that allow for the SFT104to receive electrical power from an electrical distribution network, allow for connection to a Controller Area Network (CAN) bus network that services a cluster202of SFTs104, and so forth. The wiring harness316may include electrical conductors, electromagnetic waveguides, fiber optics, and so forth. In some implementations, a plurality of wiring harnesses316may be used. For example, a first wiring harness316(1) may provide electrical power while a second wiring harness316(2) provides network connectivity. In some implementations, the wiring harness316may be used to provide information that is then processed to determine a relative arrangement of SFTs104.

The electronics312of the SFT104may include a power supply318. The power supply318may include an electric power interface that allows for coupling to the power supply118. For example, the electrical power interface may comprise connectors, voltage converters, frequency converters, and so forth. The power supply318may include circuitry that is configured to provide monitoring or other information with regard to the consumption of electrical power by the other electrical power components of the SFT104. For example, the power supply318may include power conditioning circuitry, DC to DC converters, current limiting devices, current measurement devices, voltage measurement devices, and so forth. In some implementations, the SFT104may be configured to connect to redundant power buses. For example, a first electrical distribution network such as an “A” bus and a second electrical distribution network such as a “B” bus may be provided, each of which can provide sufficient electrical power for operation. In some implementations, the SFT104may incorporate redundant power supplies318.

The SFT104may include one or more hardware processors320. Hardware processors320may include microprocessors, microcontrollers, systems on a chip (SoC), field programmable gate arrays (FPGAs), and so forth. The SFT104may also include one or more memories322. The memory322may comprise one or more non-transitory computer-readable storage media (CRSM). The CRSM may be any one or more of an electronic storage medium, a magnetic storage medium, an optical storage medium, a quantum storage medium, a mechanical computer storage medium, and so forth. The memory322provides storage of computer-readable instructions, data structures, program modules, and other data for the operation of the SFT104.

The SFT104may include electronics312. The electronics312may be configured to acquire information from sensors306of the SFT104. In one implementation, the sensors306may comprise electrodes or other electrically conductive elements that are used as part of a capacitive sensor array. In one implementation, the electrodes may be arranged in an array. Each electrode may be rectangular with a first side and a second side, with the length of the first side and the second side being between 10 millimeters and 50 millimeters. In other implementations, other shapes and sizes of electrodes may be used.

The electronics312may include capacitance measurement circuitry that generates capacitance data. The capacitance measurement circuitry may use various techniques to determine capacitance. For example, the capacitance measurement circuitry may include a source that provides a predetermined voltage, a timer, and circuitry to measure voltage of the conductive element relative to the ground. By determining an amount of time that it takes to charge the conductive element to a particular voltage, the capacitance may be calculated. The capacitance measurement circuitry may use one or more of analog or digital circuits to determine capacitance. During operation, the capacitive sensor uses a conductive element located beneath the flooring material302to produce capacitance data indicating capacitance values at particular times. Based on the capacitance data, information such as a presence of an object, shape of an object, and so forth, may be generated to produce sensor output data324. The sensor electronics312may be configured to scan the sensors306and generate sensor output data324at least 30 times per second. The sensor output data324may include information about proximity of an object with respect to a particular electrode. The sensor output data324may be further processed to generate the other data.

In other implementations, the sensors306may comprise optical touch sensors comprising one or more illuminators and one or more photodetector elements, resistive touch sensors comprising electrically resistive material, acoustic touch sensors comprising one or more transducers, and so forth. The sensors306may include other sensors, such as weight sensors, moisture detectors, microphones, and so forth.

The SFT104may include a receiver326. The receiver326is configured to detect the EMS106. The receiver326may be implemented as discrete circuitry, as a software defined radio (SDR), and so forth. The receiver326is coupled to one or more of the antennas304. In some implementations, a single receiver326may be coupled to a single antenna304. In other implementations, a single receiver326may be coupled to a plurality of antennas304by way of switching circuitry, matching network, and so forth. The switching circuitry may allow the selective connection of a particular antenna304to the receiver326. This selective connection may include the disconnection of one antenna304and connection of another antenna304at a particular time. The receiver326may be configured to detect the EMS106at a particular frequency and generate information indicative of a received signal strength.

In some implementations, elements of the sensors306may be combined or used in conjunction with the antennas304. For example, electrically conductive elements may be used for both capacitive sensing by the sensor306and as antennas304. This dual use may occur at the same time or may be multiplexed over time. For example, switching circuitry may, at a first time, selectively connect the sensor electronics312to the electrically conductive element for use as a capacitive sensor pad. The switching circuitry may then selectively connect, at a second time, the receiver326to the same electrically conductive element for use as an antenna304.

The EMS106is acquired by the antenna304and then provided to the receiver326. For example, the receiver326may comprise a superheterodyne receiver, with an incoming radio signal being converted to an intermediate frequency by a mixer. At the intermediate frequency stage, the downconverted signal is amplified and filtered before being fed to a demodulator. One or more antennas304may be dedicated for use by the receiver326, while one or more other antennas304may be dedicated for use by the transmitter(s)328. The use of separate antennas to transmit and receive may improve isolation between the receiver326and the transmitter328. The receiver326or the hardware processor320processes the EMS106to determine the characteristic data128, such as a received frequency and the signal strength received at that frequency. In another implementation, the receiver326may comprise a SDR.

In some implementations, the EMS106may encode data. The receiver326or the hardware processor320may decode, decrypt, or otherwise demodulate and process the demodulated signal to determine the characteristic data128. For example, the receiver326may provide as output the digital representation of a signal that incorporates binary phase shift keying (BPSK) or other techniques. The hardware processor320may process this digital representation to recover a serial data stream that includes framing, error control data, payload, and other information. The payload may then be processed to produce output. The error control data may include error detection data such as parity check data, parity bits, hash values, and so forth. For example, a hash function may be applied to the characteristic data128to generate hash output. A comparison of the hash output may be made to determine if an error is present.

The SFT104includes one or more transmitters328. For example, the transmitter328may comprise a voltage-controlled oscillator that generates an output signal that is fed directly to a power amplifier. The transmitter328couples to an antenna304, which then radiates the EMS106. The transmitter328may be implemented as discrete circuitry, SDR, or a combination thereof.

The transmitter328may accept multiple signals to generate the EMS106that is emitted from an antenna304connected to the output of the transmitter328. In some implementations, each segment204may utilize a single transmitter328that produces an EMS106that includes at least the segment signal208. In other implementations, a single transmitter328may be used to generate all of the EMS106from a given SFT104. For example, the transmitter328may generate the initial signal206and all the respective segment signals208for that SFT104. Filters may be used on the output such that the antenna304at a particular segment204emits only the desired frequency associated with that particular segment204.

The transmitter328may be configured to produce an output signal that is amplitude modulated, frequency modulated, phase modulated, and so forth. The transmitters328for the SFTs104in a given floor102may operate on a single frequency, or may be frequency agile and operate on a plurality of different frequencies. For example, at a first time, a single transmitter328may generate the segment signals208at a first frequency and then transition to transmitting at a second frequency. In some implementations, the receiver326and the transmitter328may be combined or share one or more components. For example, the receiver326and the transmitter328may share a common oscillator or frequency synthesizer.

In some implementations, a single antenna304may be used to both transmit and receive. For example, the receiver326may include notch filters to attenuate the frequencies of the transmitted EMS106. A single antenna304may also be used to transmit different signals. For example, a single antenna304may be used to transmit the initial signal206and a segment signal208. In some implementations, a diplexer may be used that accepts input from two or more transmitters328and provides output of the EMS106to an antenna304or group of antennas304. In other implementations, the diplexer or other filtering may be omitted, and one or more transmitters328may be coupled to a single antenna304or group of antennas304.

The hardware processor320may acquire data from one or more of the sensors306, the receiver326, the transmitter328, and so forth, to generate other data330. The other data330comprises information about an object that is resting on or proximate to the flooring material302. The information may be indicative of a shape of the object. In some implementations, the other data330may comprise information that is representative of the contours of an object. For example, the other data330may comprise a bitmap representative of the output from a plurality of sensors306and indicative of their relative arrangement. In another example, the other data330may comprise a vector value that is indicative of polygons used to represent an outline of an object. In some implementations, the other data330may be indicative of an area of the object. For example, the other data330may indicate that the total area of an object is 48 square centimeters. The other data330may include other information such as information about amplitude of a received EMS106with respect to different portions of the object. For example, other data330may be generated that indicates the shape of the object with information about amplitude, frequency, or other details about the EMS106at particular points or areas within that shape.

In some implementations, one or more of the receiver326or the transmitter328may be used to generate the sensor output data324. For example, sensors306may communicate with the power supply318to determine the amount of electrical current that is being drawn at a particular time by the transmitter328. As the electrical coupling between an object above the SFT104and one or more of the antennas304changes, one or more operating characteristics of the devices in the SFT104may change. For example, the impedance of the antenna304may experience change. Changes in the impedance may result in a change in the power output of the transmitter328during operation. For example, the transmitter328may exhibit an impedance mismatch with the antenna304in the presence of an object, such as a foot. This impedance mismatch may result in reduced power consumption by the radio frequency amplifier of the transmitter328. Information about changes in the operational characteristics, such as a change in current draw by the transmitter328, may be processed to determine the presence or absence of an object with respect to the antenna304. The operating characteristics may include, but are not limited to: received signal strength at the receiver326, power consumption of the transmitter328, radio frequency power output of the transmitter328, impedance presented at an antenna304, standing wave ratio (SWR), and so forth. For example, the impedance of the antenna304may be measured at a radio frequency input to the receiver326, a radio frequency output of the transmitter328, and so forth. In another example, the SWR presented by one or more of the antennas304may be similarly measured. In other implementations, other operating characteristics may be used. For example, a change in the noise detected by the receiver326may be used to determine presence or absence of an object. In yet another implementation, the transmitter328of the SFT104may generate a signal that is then received by the receiver326of the same SFT104. A change in the received signal at a particular antenna304may be used to determine the presence of an object. In still another implementation, the EMS106received from the other SFT104may be measured, and the received signal strength at particular segments204may be used to generate information indicative of the presence of an object.

By combining information from a plurality of antennas304, other data330may be generated. In other implementations, other characteristics of the receiver326or the transmitter328may be assessed to generate the other data330or other information indicative of proximity of an object to the antenna304. For example, the change in impedance may be measured, a change in background noise level may be measured, and so forth. In some implementations, radio ranging may be utilized in which the transmitter328emits a pulse and the receiver326listens for a return or echo of that pulse. Data indicative of proximity from several antennas304may then be processed to generate the other data330. In another implementation, distance between the object and the antenna304may be determined using the amplitude of the received EMS106. For example, a lookup table may be used that associates a particular received signal strength with a particular distance from the antenna304.

The communication interface332connects the SFT104to the network120. For example, the communication interface332may be able to connect to one or more of a Controller Area Network (CAN bus), Inter-Integrated Circuit (I2C), Serial Peripheral Interface bus (SPI), 1-Wire bus, Universal Serial Bus (USB) as promulgated by the USB Implementers Forum, RS-232, Ethernet, Wi-Fi, Bluetooth, and so forth. The communication may be facilitated by data connectors, such as optical connectors, electrical connectors, and so forth. The data connectors provide a pathway for signals to be exchanged between the communication interface332and the network120.

The SFT104may include non-transitory computer readable media that is used to store instructions, data, and so forth. Tile identifier data334comprises information indicative of a particular SFT104. The tile identifier data334may be unique within the particular network120, the facility, unique across the production of all SFTs104manufactured, and so forth. In some implementations, a media access control (MAC) address, network address, bus address, and so forth, that is associated with the communication interface332may be used as tile identifier data334.

During operation, the hardware processor320may generate tile output data126. As described above, the tile output data126may include the characteristic data128. In some implementations, the tile output data126may indicate the characteristic data128that was received by the SFT104, the particular antennas304or segments204associated with that reception, information about the frequencies of EMS106that are being transmitted, and so forth. The tile output data126may also include the tile identifier data334, timestamp data, and so forth. For example, the timestamp data included in the tile output data126may indicate when the characteristic data128was received by the receiver326.

The SFT104may include multiple hardware processors320with different capabilities. For example, individual elements of the sensors306may utilize dedicated state machines to perform simple processing functions. These dedicated state machines may then send output data to a microcontroller that provides additional processing to generate sensor output data324. In one implementation, the dedicated state machine may comprise a complex programmable logic device (CPLD). Continuing the example, a dedicated state machine may provide a 4 bit value indicative of the capacitance measured by a capacitive sensor306at a particular location on the SFT104. The microcontroller may have information that describes a relative arrangement of the sensors306, and may use this information in conjunction with the dedicated state machine output to generate a bitmap that may be included in the other data330.

Various techniques may be used to increase the overall uptime of an individual SFT104, and functionality of the floor102as a whole. In one implementation, the SFT104may include additional components to provide for failover redundancy. For example, the SFT104may include at least two hardware processors320, each of which is able to generate other data330, generate tile output data126, and so forth. In another example, the SFT104may include two power supplies318, each connected to a different bus or power supply118.

To provide additional redundancy, adjacent SFTs104may be connected to different networks120. For example, an SFT104may be connected to a first network120(1) while the SFT104immediately to the right may be connected to a second network120(2).

The SFT104may be configured to perform diagnostics of onboard components, adjacent SFTs104, and so forth. For example, the SFT104may be configured to test the receiver326and the transmitter328by transmitting a signal from the first antenna304(1) and listening with the receiver326with a second antenna304(2) that is adjacent to the first antenna304(1). In some implementations, the SFT104may be configured to send diagnostic data using the network120. For example, diagnostic data may be sent to the inventory management system130indicating that a particular SFT104has a fault and requires repair or replacement. The SFT104may be designed in a modular fashion to allow for repair or replacement without affecting adjacent SFTs104.

In some implementations, operation of the SFT104or the segments204therein may be responsive to presence or absence of an object. For example, segments204that are proximate to or underneath the object forming a shape may be deemed active segments. Antennas304associated with these active segments may be used transmit or receive the EMS106. Inactive segments comprise segments204that are not underneath or proximate to the shape. The determination of whether a segment204is active or not may be based at least in part on output from the sensor elements, antennas304, or other sensors. For example, a segment204may be deemed to be an active segment when the associated sensor element exhibits a capacitance value that exceeds a threshold level.

During operation, the determination of which segments204are active may be used to determine which antennas304are used to one or more of transmit or receive the EMS106. For example, the antennas304beneath inactive segments may be disconnected from receivers326, or the receivers326associated with those antennas304may be placed in a low power mode or turned off. As an object is detected by the sensor element as driven using the sensor electronics312, a particular segment204may be designated as an active segment. In this illustration, the active segments are represented with a crosshatch pattern. The antenna304and associated radio frequency elements such as the receiver326and the transmitter328associated with that antenna304may be transitioned to an operational mode. Continuing the example, the receiver326may begin listening for an EMS106.

The SFT104, or portions thereof such as segments204, may transition from a receive mode to a transmit mode or vice versa. This transition may be responsive to the detection of an object by the sensor306. For example, the presence of an object followed by the absence of the object may result in the SFT104transitioning from the transmit mode to the receive mode.

By selectively transmitting the EMS106using antennas304that are within a threshold distance of the shape as determined by the sensors306, performance of the system may be improved. For example, power consumption may be reduced by transmitting using only those antennas304that are proximate to the object producing the shape. In other implementations, the transmitters328may be activated on a particular schedule, such as transmitting for 50 milliseconds duration with a gap waiting time of 100 ms before the next transmission. This reduction in duty cycle decreases power consumption.

In some implementations, segments204may be in transmit mode while the receiver326is still active. For example, the transmitters328may transmit while the receiver326is listening.

The sensors306in the SFT104may be used to determine the presence of hazardous conditions at the SFT104. For example, the sensors306may be able to detect a liquid that is present on the flooring material302that may comprise a slipping hazard. Continuing the example, a puddle of water on the flooring material302may be detected. Information indicative of the puddle may be provided to the inventory management system130for mitigation, such as clean up. In another example, the sensors306may be able to detect a user112lying on the flooring material302. Upon such detection, an attendant of the facility may be alerted to provide assistance to the user112. With this example, the floor102provides information to the operators of the facility that may be used to improve the safety of the facility for the users112.

FIG. 4illustrates at400the mixing of EMS106transmitted simultaneously by the SFTs104, according to some implementations. The body of the user112, or another object proximate to the antenna304, may electromagnetically couple to the antenna304. This electromagnetic coupling may include, but is not limited to, capacitive coupling, electrostatic coupling, inductive coupling, and so forth. In other implementations, other types of coupling may take place. Once coupled, a signal path402is provided that incorporates the body of the user112, their clothing, other users112they are in contact with, and so forth.

As described above and illustrated here, each SFT104and the respective segments204thereof are transmitting signals at a particular frequency and at particular timeslots. Each segment204emits from its respective one or more antennas304both the initial signal206and the respective segment signal208. In this simplified example, each SFT104includes two segments204. The user112is standing with a left foot on SFT104(K) at segment204(6), and thus the body of the user112acts as the signal path402of the EMS106to a right foot on the SFT104(L) at segment204(6), and vice versa. Each of the SFTs104produces tile output data126that is indicative of the tile identifier data334of the receiving SFT104.

The SFT104(K) produces tile output data126(21) that is indicative of the frequency of the various signals received by the SFT104(K) at the different timeslots210and their received signal strength, while the tile output data126(22) is indicative of the various signals received by the SFT104(L). With the characteristic data128, the frequency of the signal, the combination of the various timeslots210presented and their received signal strength provides information as to the placement of the foot with respect to a SFT104. As the foot of the user112rests across different segments204, and possibly different SFTs104, it electromagnetically couples to the antennas304therein.

The tile output data126thus provides information about the location of a foot. Given the exchange of EMS106from one SFT104to another, a pair of SFTs104(or locations therein) may be determined. Given the reciprocity of the exchanges of EMS106and the resulting characteristic data128, the two feet may be associated with a single user112. In some implementations, a location of the user112may be determined to be between the feet locations. For example, the tracking module134may generate tracking data136that indicates the location of the user112(1) is at a midpoint between their left and right footprints.

The tracking module134may utilize certain assumptions or rules in the determination of a location of the user112. For example, the user112may be assumed to have two feet, the feet may be assumed to have a minimum length of 4 inches but less than 20 inches, and so forth. The tracking module134may also utilize data about the physical layout of the facility. For example, the physical arrangement of the SFTs104with respect to one another, the arrangement of the segments204therein, and so forth, may be used.

FIG. 5illustrates a chart500of the antennas304of the SFT104that are used to emit EMS106in their respective timeslots210, according to some implementations. As described above, each SFT104emits an EMS106. The EMS106may be at a particular frequency that is associated with that SFT104. During specified timeslots210, the segment signal208is radiated from a particular antenna304in a particular segment204.

In this illustration, the horizontal axis indicates time502increasing left to right. Various timeslots210ranging from timeslot210(0) to210(16) are depicted along the horizontal axis as well. The number of timeslots210may be associated with the number of segments204for an SFT104. For example, an SFT104with 4 segments may have five timeslots210, while an SFT104with 20 segments may have 21 timeslots210. During the first timeslot210(0), the initial signal206may be transmitted using one or more of the antennas304. For example, as depicted here, all of the antennas304of the SFT104may be used to radiate the initial signal206. In other implementations, other arrangements of antennas304may be used. For example, every other antenna304may be used, or a dedicated antenna304that extends across the SFT104may be used. As described above, the initial signal206may be used by a receiver326for synchronization. In some implementations, the initial signal206and corresponding timeslot210(0) may be omitted. For example, instead of the initial signal206being transmitted, the clock signal124may be used to synchronize the SFTs104. The synchronization of the SFTs104does not have to be stringent. Instead, adjacent SFTs104may be synchronized to one another within a threshold limit. For example, SFT104(1) and SFT104(2) which are next to one another may be synchronized to within 1 ms, while SFT104(1) and SFT104(127) at opposite ends of the facility may be out of sync due to propagation delays of the clock signal124, with the system still operating normally. In one implementation, the clock signal124may be received by the SFT104. A clock onboard the SFT104may be set based on the timing signal. Once set, the clock may be used to determine timing for when timeslots210begin, end, and so forth.

The initial signal206may also provide other operational benefits. For example, the initial signal206may be used to minimize the sampling of noise by the receiver326by acting as a signal to break squelch on the receiver.

The vertical axis depicts the various elements that the EMS is being radiated from504. For example, the initial signal206is radiated using all antennas304. During timeslot210(1), the segment signal208(1) is radiated from the one or more antennas304associated with the first segment204(1). Likewise, during timeslot210(2), the segment signal208(2) is radiated from the one or more antennas304associated with the second segment204(2), and so forth. During the particular timeslot210, each segment204thus radiates an EMS106.

A sequence506comprises the particular pattern of emission of the EMS106from the respective antennas304as described. The sequence506may be repeated. For example, the sequence506may continuously loop, with the SFT104transmitting at a designated frequency the initial signal206and segment signal208in the respective timeslots210.

In some implementations, the SFT104may use different frequencies to transmit the segment signals208. For example, in addition to transmitting during particular timeslots210and using particular antennas304for a particular segment204, each segment signal208may be transmitted at a different frequency.

In some implementations, one or more of the initial signal206, the segment signal208, or other EMS106may be modulated to convey information. For example, the segment signal208may be modulated to include a 4 bit value. The information conveyed may be indicative of the SFT104identifier, a segment identifier, power output level of the transmitted signal, and so forth. In another example, the initial signal206may be modulated to include a predetermined preamble value, such as a predetermined series of all binary 0's, all binary 1's, alternating binary 0's and 1's, and so forth.

As described above, in some implementations, this technique may be used to identify particular SFTs104. For example, each SFT104may comprise a single segment204, with that segment204associated with a particular timeslot210. As a result, each single-segment SFT104would transmit within a particular timeslot210to identify itself.

FIG. 6illustrates graphs600of combined received signal characteristic data128for the second user112(2) shown inFIG. 2. The characteristic data128may be generated using data obtained by a receiver326in a SFT104or a fixture108, according to some implementations. In this illustration, a portion of the characteristic data128(2) of the EMS106that are propagated through the signal path402of the user112(2) is shown. For example, the graphs600may result from the combination of characteristic data128obtained from the SFTs104indicated.

A first graph depicts a horizontal axis indicative of tile frequencies602that represent individual SFTs104and a vertical axis that indicates a received signal strength604for each tile frequency602. The received signal strength604by tile may be indicative of a maximum value for all signals received for that frequency, a cumulative signal strength that comprises a sum of the received signal strengths received for that frequency, an average signal strength of the values of received signal strengths for that frequency, and so forth. In some implementations, the received signal strengths604by tile as described above may be calculated for the received signal strength of a signal received during a particular timeslot210for one or more sequences506.

A second graph depicts indicated segments606for frequency “C”. For example, the tile frequencies602depict nine tile frequencies602“A” through “I”. Each of these tile frequencies602may be associated with a particular SFT104in the cluster202. In the second graph, an enlargement is shown for the sixteen indicated segments606“1” through “16” that include information about the signal strength of the segment signals208for those respective timeslots210as transmitted on the tile frequency602of “C”.

In the second graph, received signal strength608by segment is depicted. The received signal strength608by segment may be indicative of a maximum value for all signals received during the timeslot210for the indicated segment606, a cumulative signal strength that comprises a sum of the received signal strengths received for that timeslot210, an average signal strength of the values of received signal strengths for that timeslot210, and so forth. In some implementations, the received signal strengths608by segment may be calculated for the received signal strength of a signal received during one or more sequences506.

The characteristic data128is depicted here in graphical format, but it is understood that the characteristic data128may be represented using various data structures including, but not limited to tables, linked lists, delimiter separated values, serialized data, and so forth.

Because the users112are standing on different portions of their respective SFTs104, each user112exhibits different arrangements of received signal strengths at the different tile frequencies602and indicated segments606, and so forth. For example, as user112(1) is standing on SFTs104(B) and104(D), the characteristic data128(1) will show peaks corresponding to SFTs104(B) and104(D).

By processing the characteristic data128, a location for each foot may be obtained. For example, based on the characteristic data128(2), the user112(2) is determined to be standing on SFTs104(B), (C), and (F). Given the distribution of the tile frequencies602and the indicated segments606and the known arrangement of the SFTs104and segments204with respect to one another, the relative positions of the left and right feet of a user112may be reconstructed.

In one implementation, another frequency allocation scheme may be used. For example, all of the SFTs104in a cluster202may use the same frequency to transmit, but each segment204in the cluster202may radiate a segment signal208at a particular timeslot210within the cluster202. Continuing the example, given the nine SFTs104with their respective sixteen segments, 144 different timeslots210may be used in a cluster202.

FIG. 7illustrates tracking700of a user112as they move across the SFTs104, according to some implementations. At702, the user112is shown at a first time=0. As described above, based on the tile output data126(11), a first location of the user112is determined as being between SFT104(M) and SFT104(J) based on the location of the left and right feet of the user112. At704, the user112is shown at a second time t=1. A second location of the user112is determined as being between SFT104(1) and SFT104(E). A time series of these user locations may be used to describe the user path114. As described above, if the entry110involves identification, authentication, or other functions, this identity may be asserted with the user112as they move throughout the facility102along the user path114.

FIG. 8illustrates the use 800 of a portable receiver802to detect the signals transmitted by the SFTs104, according to some implementations. The user112, a tote116, or other object may be equipped with the portable receiver802. The portable receiver802may be electromagnetically coupled to the user112and is configured to receives the EMS106and generate data such as the characteristic data128, an identifier of the portable receiver802, a timestamp, and so forth. For example, the portable receiver802may include a communication interface such as Wi-Fi or Bluetooth compliant network interface that allows for wireless exchange of data with another computing device. The portable receiver802may acquire the characteristic data128and send this characteristic data128to a server or other computing device. The portable receiver802may be associated with a particular user account, such as that of an associate or affiliate of the facility. The portable receiver802may obtain the characteristic data128such as shown inFIG. 5. The characteristic data128may be sent via the communication interface to a server that determines the user112(1) is located at a position centered on SFT104(F), segment (6).

In another implementation, the user112or other object may utilize a portable transmitter804. The portable transmitter804transmits an EMS106at a particular frequency, timeslot210, or coding that will result in a receiver generating characteristic data128associated with that particular object. Similar to that described above, the system may use the characteristic data128to specifically identify one or more of a particular category or specific identity of a particular user112, tote116, or other object. For example, all totes116may be issued a portable transmitter804that emits a signal at 76 kHz, each transmitting a tote signal (similar to the segment signal208) in a particular timeslot210. In another example, each tote116may have a different assigned frequency, such that tote116(1) has a portable transmitter804that emits at 78 kHz while another tote116(2) transmits at 81 kHz. As a result, the receivers(s)326of the SFTs104proximate to the wheels of the tote116detect the signal and produce tile output data126with characteristic data128showing the signal(s) emitted by the portable transmitter804.

The portable transmitter804may be provided in a variety of different form factors. For example, the portable transmitter804may comprise a device that may be mounted on the belt, worn as a wristband, a necklace, or a headband, attached to safety equipment worn by the user112, and so forth. In some implementations, the portable transmitter804may be incorporated into another device, such as a smartphone, point-of-sale terminal, and so forth.

Other information may be gathered with this configuration, or in the earlier configurations, without the portable transmitter804. For example, it may be determined which user112is in contact with a particular tote116based on the characteristic data128reported by their portable receiver802. In this example, the characteristic data128may include the EMS106emitted from the SFTs104under the tote116that are propagated via the tote116into the user112, the EMS106emitted by the portable transmitter804, and so forth.

In some implementations, the portable receiver802, portable transmitter804, and so forth, may be in communication with the inventory management system130. For example, these devices may communicate using Wi-Fi with an access point. In another example, data may be transferred using the SFTs104. Continuing this example, a signal may be transferred that encodes data which is then received by the receiver326in the floor102. Likewise, the transmitter328in the SFT104may send data to a receiver onboard the tote116or other device.

In some implementations, the functions of the portable receiver802and the portable transmitter804may be combined in a single device. For example, a portable transceiver may be configured to transmit EMS106and receive EMS106.

Fixed installations may also use these devices. For example, the components and functions of the portable receiver802may be incorporated into or associated with a fixed device, such as a door handle. When the user112touches the door handle, the EMS106propagated through their body from the SFT104to the door handle may provide characteristic data128that may be used to identify that user112. In another example, the components and functions of the portable transmitter804may be incorporated into or associated with a fixed device, such as a handrail. The handrail may emit the EMS106and a receiver, such as the portable receiver802or a receiver in the SFT104, may be used to provide characteristic data128that may be used to identify that user112.

FIG. 9is an illustration900of the use of EMS106to determine a particular user112is interacting with a particular portion of a fixture108, according to some implementations.

As described above, the fixtures108may be used to store one or more items902. As illustrated here, the fixture108includes items902stowed on four shelves904(1),904(2),904(3), and904(4). In other implementations, the fixture108may comprise racks, bins, hangers, and so forth.

As depicted here, a first SFT104(1) transmits a first combination of EMS106(1) along a signal path402of the body of the first user112(1), while a fourth SFT104(4) transmits a second combination of EMS106(2) along a signal path402of the body of the second user112(2). The first EMS106(1) conveys first characteristic data128(1), while the second EMS106(2) conveys second characteristic data128(2). As respective users112pick or place items902on the one or more of the shelves904, their respective combinations of EMSs106are propagated along their respective bodies. The shelves904are equipped with one or more antennas304and one or more receivers326(not shown). In some implementations, shields or other arrangements of antennas304may be present to provide directionality to the patterns of the antennas304. The electronics of the shelves904generate the fixture data132. The fixture data132may comprise the characteristic data128of the EMS106that has been received by the shelf904. In some implementations, the fixture data132may include fixture identifier data indicative of a particular fixture108or portion thereof, a timestamp, and so forth.

The shelves904may include an array of antennas304, allowing for a determination of gesture data indicative of where the hand of the user112is relative to the fixture108, motion of the hand, and so forth. For example, each shelf904may include two antennas304, one on the left side and one on the right side. By analyzing the relative signal strength of the EMS106as conveyed by a signal path402from the foot of the user112to their hand as it is near or in contact with the shelf904, a position of the hand at a particular time may be determined.

By utilizing data from the antennas304and receivers326on different shelves904, information about the position of the hand in three-dimensional space may be determined. For example, antennas304on shelf904(1) and904(2) may be used to determine the position of the hand of the user112relative to those shelves904.

In some implementations, an antenna304may be located beneath the item902. As a result of the user112coming into contact with the item902, an increase in the amplitude of the EMS106as measured by the receiver326connected to the antenna304may be determined. Given predetermined information specifying that a particular type of item902is stowed on the shelf904proximate to the antenna304(1), based on the fixture data132, the inventory management system130is able to generate interaction data142. For example, item902of the type “pet food” is assigned for stowage on shelf904(1) in a lane that is above antenna304(16). The fixture data132may indicate that the signal strength of one or more frequencies of the second EMS106(2) that conveyed the second characteristic data128(2) exceeded a threshold value. The amplitude of the signals as indicated in the second characteristic data128(2) is thus indicative of the user112coming into contact with the item902. Based on the particular characteristic data128, a particular user112may thus be associated with a particular user account, and the particular user112may be assessed a charge for the pick of the can of pet food.

Other sensors306, such as weight sensors, capacitive sensors, and so forth, may also be used. Data from these other sensors306may then be used in conjunction with the characteristic data128and information obtained from the receivers326about the EMS106to generate the interaction data142. The characteristic data128transferred by way of the EMS106to the antenna304in the shelf904may be used to determine who is picking what item902. A change in weight of the shelf904as measured by one or more weight sensors306may be used to determine the quantity of the items902that are either picked or placed. For example, the change in weight may be divided by a known weight of a sample of the item902. By using these techniques, the inventory management system130is able to quickly and inexpensively determine which user112interacted with a particular item902, the fixtures108, or portion thereof.

In some implementations, information about how the EMS106is propagated may be used to distinguish between one type of item902and another type of item902that the user112may be interacting with. For example, the same antenna304may service two lanes on the shelf904. In a first lane are stowed boxes of dried pasta, while the second lane stows metal cans of tomato sauce. The metal can provides a better signal pathway306for the EMS106compared to the box of dried pasta. By analyzing the received signal strength of the EMS106, the user112coming into contact with the metal can may be distinguished from the user112coming into contact with the box of dried pasta. For example, if the received signal strength of the EMS106exceeds a threshold value, the contact may be determined to be with the metal can in the second lane. Similarly, if the received signal strength of the EMS106is below a threshold value, the contact may be determined to be with the box of dried pasta in the first lane.

As described above with regard toFIG. 8, in some implementations, the EMS106may be transmitted by portable transmitter804. In other implementations, the shelf904may emit one or more EMS106that are detected using the receiver326in the SFT104or the portable receiver802.

In other implementations, the same techniques may be used to determine if the user112is touching other objects in the environment. For example, the placement of a user's112hand with respect to a table or countertop may be determined. In another example, the techniques may be used to determine that a user112is touching a door handle, sitting in a chair, sitting on a bench, and so forth.

The fixture108may include a transmitter in some implementations. For example, the shelf904may include a transmitter to generate EMS106, that is then propagated via the signal path402to the SFT104, a portable receiver802, and so forth.

FIG. 10illustrates an enlarged side view1000of the use of an EMS106to generate gesture data and other information indicative of which item902a user112interacted with at the fixture108, according to some implementations.

As described above, the shelves904or other fixtures108may incorporate one or more antennas304that couple to one or more receivers326. As a hand1002of the user112approaches the fixture108, antennas304may receive the EMS106as transmitted by an SFT104, portable transmitter804, and so forth. In some implementations, the shelves904or other fixtures108may incorporate one or more transmitters328that are coupled to one or more antennas304. The transmitters328may be used to generate an EMS106.

Electronics1004associated with the shelf904recover the characteristic data128conveyed by the EMS106. The electronics1004may be similar to the electronics312described above with regard to the SFT104. For example, the electronics1004may include a power supply318, a receiver326, the hardware processor320, a communication interface332, one or more antennas304, and so forth. In some implementations, the electronics1004may include one or more transmitters328.

The hardware processor320may be configured to generate the fixture data132. The fixture data132may include one or more of characteristic data128, fixture identifier data1006, gesture data1008, and so forth. As described above, the characteristic data128comprises information that is conveyed by an EMS106. Fixture identifier data1006is used to identify a particular fixture108or portion thereof, such as a shelf904, lane upon the shelf904, and so forth. The gesture data1008may comprise information indicative of a location of the hand1002of the user112with respect to the fixture108or portion thereof, duration of contact by the hand1002, direction of movement of the hand1002, and so forth. The gesture data1008may be generated based on information about the EMS106obtained by one or more antennas304. For example, based on the changes over time of an amplitude or received signal strength of the EMS106at a given antenna304, a position of the hand1002or portion thereof may be determined.

The gesture data1008may include information such as a time series of position. In some implementations, the gesture data1008may be used to generate trajectory data indicative of a trajectory of the hand1002. This trajectory may then be used to help determine which lane the user112is interacting with, disambiguate the user112from among several users112if the characteristic data128is unavailable, and so forth.

The gesture data1008may include information indicative of contact duration between the user112and the item902. For example, a contact threshold time may indicate a minimum amount of time that the user112has to be in contact with the item902before a contact is deemed to occur. The comparison of the contact duration and the contact threshold time may be used to reduce false positives, minimize the impact of noise, and so forth. In some implementations, the contact may also be determined at least in part by the received signal strength of the EMS106during contact. For example, contact may be determined when the received signal strength is above a threshold strength value. Contact may be determined when the contact duration exceeds the contact threshold time and the received signal strength is above the threshold strength value.

The gesture data1008may comprise a time series of coordinates, each set of coordinates indicating a position of the hand1002at successive times. The gesture data1008may provide coordinates in one, two, or three-dimensional space. For example, coordinates in a one-dimensional space for the gesture data1008may indicate where along the shelf904from left to right the hand1002is determined to be. In another example, coordinates in three-dimensional space for the gesture data1008may indicate where the hand1002is in terms of left to right, front to back and height above the shelf904.

To generate gesture data1008, the hand of the user1002does not necessarily need to be in contact with the portion of the fixture108. For example, proximity of the hand1002may be sufficient to allow for coupling between the hand1002and the antenna304that is sufficient to transfer the EMS106.

As described below in more detail with regard toFIG. 11, the fixture108may incorporate other sensors as well.

WhileFIGS. 9 and 10depict the EMS106as originating in the SFT104, in other implementations, the signal pathway may be reversed. For example, a transmitter328may be located at the shelf904that generates an EMS106associated with a particular type of item902. As the user112approaches and then grasps the item902, a signal path402may be provided that conveys the EMS106from the shelf904to a receiver in the SFT104. In other implementations, the EMS106may be produced by the portable transmitter804.

FIG. 11depicts a block diagram1100of a fixture108such as a shelf904that is configured to generate gesture data1008, characteristic data128, and so forth, according to some implementations. A top view1102of a shelf904and side view1104of an enlarged portion of the shelf904are depicted.

As shown in the top view1102, a plurality of conductive elements1106are distributed in rows and columns across the shelf904to form an array. The conductive elements1106may be planar and formed into shapes such as rectangles (as shown here). Arranged proximate to each of the four corners of the shelf904are weight sensors1108. The conductive elements1106may be configured for dual use as antennas304and as elements of a capacitive sensor array. In other implementations, other shapes and arrangements of the conductive elements1106may be used.

As shown in the side view1104, the conductive elements1106may be connected by wire or other electrical conductor. The wire transfers a capacitive signal1110between the conductive element1106and other circuitry, such as a switch module1112. The switch module1112may in turn connect to a capacitance measurement/receiver module1114. For example, the capacitive signal1110may be used to supply a charge to the conductive element1106. The capacitance measurement/receiver module1114determines a change in this charge over time and generates capacitance data1116. The capacitance measurement/receiver module1114may also generate the characteristic data128in some implementations.

The switch module1112may comprise switching circuitry that allows for the capacitance measurement/receiver module1114to be selectively connected to a particular conductive element1106. In some implementations, a plurality of switch modules1112may be used to allow for different switching configurations. For example, a first switch module1112(1) may have 4 outputs, each connecting to additional switch modules1112(2),1112(3),1112(4),1112(5). Each of those switch modules1112(2)-(5) may have 4 outputs in which each output is connected to additional switch modules1112, and so forth. The switching circuitry may comprise microelectromechanical switches, relays, transistors, diodes, and so forth. Other configurations or networks of switch modules1112may be implemented as well.

The capacitance measurement/receiver module1114may be used to generate the capacitance data1116. The capacitance data1116may include information such as a capacitance value, information indicative of a particular conductive element1106, timestamp, and so forth. In some implementations, circuitry or functionality of the switch module1112and the capacitance measurement/receiver module1114may be combined. The capacitance measurement/receiver module1114may also include a receiver326to allow for the reception of the EMS106.

A bottom plate1118may provide mechanical support for one or more of the conductive elements1106. In some implementations, the bottom plate1118may comprise an electrical conductor that acts as a shield for an electric field present at the conductive element1106.

A shelf top1120may be arranged atop one or more of the conductive elements1106and the bottom plate1118. One or more items902may rest on or above the shelf top1120. For example, the shelf top1120may comprise a non-conductive material such as a plastic or ceramic.

The conductive element1106may comprise one or more electrically conductive materials. The electrically conductive elements1106may be formed as one or more of a coating, thin-film, paint, deposited material, foil, mesh, and so forth. For example, the conductive element1106may comprise an electrically conductive paint, silver paste, aluminum film, a copper sheet, and so forth. The conductive element1106may be deposited upon, embedded within, laminated to, or otherwise supported by the bottom plate1118, the shelf top1120, and so forth. These conductive elements1106may then be connected to the capacitance measurement circuitry in the capacitance measurement/receiver module1114.

One or more shields1122may be provided. A shield1122may be adjacent to one or more of the conductive elements1106. The shield1122comprises an electrically conductive material and is separated by an electrical insulator, such as air, plastic, ceramic, and so forth, from the conductive element1106. A single shield1122may be used to provide shielding for one or more conductive elements1106. During operation, the shield1122may be driven at the same voltage potential of the input of the capacitive signal1110. In this configuration, there is no difference in electrical potential between the shield1122and the conductive element1106. External interference may then couple to the shield1122producing little interaction with the conductive element1106. The shield1122may also be used to direct the electric field produced by the conductive element1106during operation. For example, the electric field is directed generally away from the shield1122. Using this technique, the capacitive sensor may detect objects on the side opposite that of the shield1122, with the shield1122preventing the capacitive sensor from “seeing” or being affected by an object behind the shield1122.

The shelf904may include other layers or structures. For example, an electrical insulator1124such as polyethylene terephthalate may be arranged between the bottom plate1118and the shield1122(if present) or the conductive element1106. Wires, circuit traces, or other electrically conductive pathways may conduct the capacitive signal1110between the capacitance measurement/receiver module1114and the conductive element1106.

The bottom plate1118may be supported by one or more of the weight sensors1108. In some implementations, the bottom plate1118may comprise an electrically conductive material and act as a ground plane, such as if connected to an earth ground. The weight sensor1108may in turn be supported by a shelf support1126.

The one or more of the weight sensors1108may be connected to a weight sensor module1128. The weight sensor module1128may comprise circuitry that is used to generate the weight data1130. The weight data1130may include information such as a weight value, information indicative of a particular weight sensor1108, timestamp, and so forth. In some implementations, circuitry or functionality of the weight sensor module1128and the weight sensor1108may be combined.

One or more image sensors (not shown) may be used to acquire image data at or near the shelf904or other fixture108. The image data may comprise one or more still images, video, or a combination thereof. The image sensor may have a field of view (FOV) that includes at least a portion of the shelf904or other type of fixture108. For example, a camera may be mounted within the shelf904to acquire image data of one or more lanes of items902in the shelf904.

FIG. 12depicts a scenario1200showing the signal strengths as received using different antennas304at the fixture108, according to some implementations. In this scenario, a graph is depicted along with a corresponding schematic of the antennas304laid out on a shelf904. With regard to the graph, along a horizontal axis are bins indicative of shelf position1202. Along the vertical axis of the graph is the received signal strength604received at the particular shelf position1202.

Below the graph are the array of antennas304that may be positioned along the shelf904. In this scenario, the shelf904includes eighteen antennas304arranged side by side. In one implementation, each lane of the shelf904may be associated with a particular antenna304. A right hand1002of the first user112(1) is shown reaching towards the antenna304(6). A left hand1002of the second user112(2) is shown reaching towards the antenna304(16). As depicted by the graph above, the received signal strength604for the respective hands1002exhibit spikes at a first location1204and a second location1206. As described above, the received signal strength604may be for a particular frequency, group of frequencies, and so forth.

To distinguish between the hands1002detected at the first location1204and the second location1206, the characteristic data128may be assessed. For example, given where the respective users112are standing, they will exhibit a particular set of characteristics or spectra that is the combination of the EMS106.

FIG. 13is a block diagram1300illustrating a materials handling facility (facility)1302using the system100, according to some implementations. A facility1302comprises one or more physical structures or areas within which one or more items902(1),902(2), . . . ,902(Q) may be held. The items902may comprise physical goods, such as books, pharmaceuticals, repair parts, electronic gear, and so forth.

The facility1302may include one or more areas designated for different functions with regard to inventory handling. In this illustration, the facility1302includes a receiving area1304, a storage area1306, and a transition area1308. Throughout the facility1302, the plurality of SFTs104may be deployed as described above.

The receiving area1304may be configured to accept items902, such as from suppliers, for intake into the facility1302. For example, the receiving area1304may include a loading dock at which trucks or other freight conveyances unload the items902. In some implementations, the items902may be processed, such as at the receiving area1304, to generate at least a portion of item data as described below. For example, an item902may be tested at the receiving area1304to determine the attenuation of an EMS106passing through it, and this information stored as item data.

The storage area1306is configured to store the items902. The storage area1306may be arranged in various physical configurations. In one implementation, the storage area1306may include one or more aisles1310. The aisle1310may be configured with, or defined by, the fixtures108on one or both sides of the aisle1310. The fixtures108may include one or more of a shelf904, a rack, a case, a cabinet, a bin, a floor location, or other suitable storage mechanisms for holding, supporting, or storing the items902. For example, the fixtures108may comprise shelves904with lanes designated therein. The fixtures108may be affixed to the floor102or another portion of the structure of the facility1302. The fixtures108may also be movable such that the arrangements of aisles1310may be reconfigurable. In some implementations, the fixtures108may be configured to move independently of an outside operator. For example, the fixtures108may comprise a rack with a power source and a motor, operable by a computing device to allow the rack to move from one location within the facility1302to another.

One or more users112and totes116or other material handling apparatus may move within the facility1302. For example, the user112may move about within the facility1302to pick or place the items902in various fixtures108, placing them on the tote116for ease of transport. The tote116is configured to carry or otherwise transport one or more items902. For example, the tote116may include a basket, cart, bag, bin, and so forth. In other implementations, other material handling apparatuses such as robots, forklifts, cranes, aerial drones, and so forth, may move about the facility1302picking, placing, or otherwise moving the items902. For example, a robot may pick an item902from a first fixture108(1) and move the item902to a second fixture108(2).

One or more sensors1312may be configured to acquire information in the facility1302. The sensors1312may include, but are not limited to, weight sensors1312(1), capacitive sensors1312(2), image sensors1312(3), depth sensors1312(4), and so forth. The weight sensors1312(1) may comprise the same or different hardware as the weight sensors1108described above. The sensors1312may be stationary or mobile, relative to the facility1302. For example, the fixtures108may contain weight sensors1312(1) to acquire weight sensor data of items902stowed therein, image sensors1312(3) to acquire images of picking or placement of items902on shelves904, optical sensor arrays1312(14) to detect shadows of the user's112hands1002at the fixtures108, and so forth. In another example, the facility1302may include image sensors1312(3) to obtain images of the user112or other objects in the facility1302. The sensors1312are discussed in more detail below with regard toFIG. 14.

While the storage area1306is depicted as having one or more aisles1310, fixtures108storing the items902, sensors1312, and so forth, it is understood that the receiving area1304, the transition area1308, or other areas of the facility1302may be similarly equipped. Furthermore, the arrangement of the various areas within the facility1302is depicted functionally rather than schematically. For example, in some implementations, multiple different receiving areas1304, storage areas1306, and transition areas1308may be interspersed rather than segregated in the facility1302.

The facility1302may include, or be coupled to, the inventory management system130. The inventory management system130is configured to interact with one or more of the users112or devices such as sensors1312, robots, material handling equipment, computing devices, and so forth, in one or more of the receiving area1304, the storage area1306, or the transition area1308.

During operation of the facility1302, the sensors1312may be configured to provide sensor data, or information based on the sensor data, to the inventory management system130. The sensor data may include the weight data1130, the capacitance data1116, the image data, and so forth. The sensors1312are described in more detail below with regard toFIG. 14.

The inventory management system130or other systems may use the sensor data to track the location of objects within the facility1302, movement of the objects, or provide other functionality. Objects may include, but are not limited to, items902, users112, totes116, and so forth. For example, a series of images acquired by the image sensor1312(3) may indicate removal by the user112of an item902from a particular location on the fixture108and placement of the item902on or at least partially within the tote116.

The facility1302may be configured to receive different kinds of items902from various suppliers and to store them until a customer orders or retrieves one or more of the items902. A general flow of items902through the facility1302is indicated by the arrows ofFIG. 13. Specifically, as illustrated in this example, items902may be received from one or more suppliers, such as manufacturers, distributors, wholesalers, and so forth, at the receiving area1304. In various implementations, the items902may include merchandise, commodities, perishables, or any suitable type of item902, depending on the nature of the enterprise that operates the facility1302.

Upon being received from a supplier at the receiving area1304, the items902may be prepared for storage in the storage area1306. For example, in some implementations, items902may be unpacked or otherwise rearranged. The inventory management system130may include one or more software applications executing on a computer system to provide inventory management functions. These inventory management functions may include maintaining information indicative of the type, quantity, condition, cost, location, weight, or any other suitable parameters with respect to the items902. The items902may be stocked, managed, or dispensed in terms of countable units, individual units, or multiple units, such as packages, cartons, crates, pallets, or other suitable aggregations. Alternatively, some items902, such as bulk products, commodities, and so forth, may be stored in continuous or arbitrarily divisible amounts that may not be inherently organized into countable units. Such items902may be managed in terms of a measurable quantity such as units of length, area, volume, weight, time, duration, or other dimensional properties characterized by units of measurement. Generally speaking, a quantity of an item902may refer to either a countable number of individual or aggregate units of an item902or a measurable amount of an item902, as appropriate.

After arriving through the receiving area1304, items902may be stored within the storage area1306. In some implementations, like items902may be stored or displayed together in the fixtures108such as in bins, on shelves904, hanging from pegboards, and so forth. For example, all items902of a given kind are stored in one fixture108. In other implementations, like items902may be stored in different fixtures108. For example, to optimize retrieval of certain items902having frequent turnover within a large physical facility1302, those items902may be stored in several different fixtures108to reduce congestion that might occur at a single fixture108.

When a customer order specifying one or more items902is received, or as a user112progresses through the facility1302, the corresponding items902may be selected or “picked” from the fixtures108containing those items902. In various implementations, item picking may range from manual to completely automated picking. For example, in one implementation, a user112may have a list of items902they desire and may progress through the facility1302picking items902from the fixtures108within the storage area1306and placing those items902into a tote116. In other implementations, employees of the facility1302may pick items902using written or electronic pick lists derived from customer orders. These picked items902may be placed into the tote116as the employee progresses through the facility1302.

After items902have been picked, the items902may be processed at a transition area1308. The transition area1308may be any designated area within the facility1302where items902are transitioned from one location to another or from one entity to another. For example, the transition area1308may be a packing station within the facility1302. When the items902arrive at the transition area1308, the items902may be transitioned from the storage area1306to the packing station. Information about the transition may be maintained by the inventory management system130.

In another example, if the items902are departing the facility1302, a list of the items902may be obtained and used by the inventory management system130to transition responsibility for, or custody of, the items902from the facility1302to another entity. For example, a carrier may accept the items902for transport with that carrier accepting responsibility for the items902indicated in the list. In another example, a user112may purchase or rent the items902and remove the items902from the facility1302. During use of the facility1302, the user112may move about the facility1302to perform various tasks, such as picking or placing the items902in the fixtures108.

The inventory management system130may generate the interaction data142. The interaction data142may be based at least in part on one or more of the tile output data126, the fixture data132, and so forth. The interaction data142may provide information about an interaction, such as a pick of an item902from the fixture108, a place of an item902to the fixture108, a touch made to an item902at the fixture108, a gesture associated with an item902at the fixture108, and so forth. The interaction data142may include one or more of the type of interaction, duration of interaction, interaction location identifier indicative of where from the fixture108the interaction took place, item identifier, quantity change to the item902, user identifier, and so forth. The interaction data142may then be used to further update the item data. For example, the quantity of items902on hand at a particular lane on the shelf904may be changed based on an interaction that picks or places one or more items902.

The inventory management system130may combine or otherwise utilize data from different sensors1312of different types. For example, weight data1130obtained from weight sensors1312(1) at the fixture108may be used instead of, or in conjunction with, one or more of the capacitance data1116or image data to determine the interaction data142.

FIG. 14is a block diagram1400illustrating additional details of the facility1302, according to some implementations. The facility1302may be connected to one or more networks1402, which in turn connect to one or more servers1404. The network1402may include private networks such as an institutional or personal intranet, public networks such as the Internet, or a combination thereof. The network1402may utilize wired technologies (e.g., wires, fiber optic cables, and so forth), wireless technologies (e.g., radio frequency, infrared, acoustic, optical, and so forth), or other connection technologies. The network1402is representative of any type of communication network, including one or more of data networks or voice networks. The network1402may be implemented using wired infrastructure (e.g., copper cable, fiber optic cable, and so forth), a wireless infrastructure (e.g., cellular, microwave, satellite, and so forth), or other connection technologies.

The servers1404may be configured to execute one or more modules or software applications associated with the inventory management system130or other systems. While the servers1404are illustrated as being in a location outside of the facility1302, in other implementations, at least a portion of the servers1404may be located at the facility1302. The servers1404are discussed in more detail below with regard toFIG. 15.

The users112, the totes116, or other objects in the facility1302may be equipped with one or more tags1406. The tags1406may be configured to emit a signal1408. In one implementation, the tag1406may be an RFID tag1406configured to emit an RF signal1408upon activation by an external signal. For example, the external signal may comprise a radio frequency signal or a magnetic field configured to energize or activate the RFID tag1406. In another implementation, the tag1406may comprise a transmitter and a power source configured to power the transmitter. For example, the tag1406may comprise a Bluetooth Low Energy (BLE) transmitter and battery. In other implementations, the tag1406may use other techniques to indicate presence of the tag1406. For example, an acoustic tag1406may be configured to generate an ultrasonic signal1408, which is detected by corresponding acoustic receivers. In yet another implementation, the tag1406may be configured to emit an optical signal1408.

The inventory management system130may be configured to use the tags1406for one or more of identification of the object, determining a location of the object, and so forth. For example, the users112may wear tags1406, the totes116may have tags1406affixed, and so forth, which may be read and, based at least in part on signal strength, used to determine identity and location. In other implementations, such as described above, the users112may wear portable transmitters804, the totes116may be equipped with a portable receiver802, portable transmitter804, and so forth. In some implementations, the two may be combined, such as tags1406and the use of a portable transmitter804.

Generally, the inventory management system130or other systems associated with the facility1302may include any number and combination of input components, output components, and servers1404.

The one or more sensors1312may be arranged at one or more locations within the facility1302. For example, the sensors1312may be mounted on or within a floor102, wall, at a ceiling, at fixture108, on a tote116, may be carried or worn by a user112, and so forth.

The sensors1312may include one or more weight sensors1312(1) that are configured to measure the weight of a load, such as the item902, the tote116, or other objects. The weight sensors1312(1) may be configured to measure the weight of the load at one or more of the fixtures108, the tote116, on the floor102of the facility1302, and so forth. For example, the shelf904may include a plurality of lanes or platforms, with one or more weight sensors1312(1) beneath each one to provide weight sensor data about an individual lane or platform. The weight sensors1312(1) may include one or more sensing mechanisms to determine the weight of a load. These sensing mechanisms may include piezoresistive devices, piezoelectric devices, capacitive devices, electromagnetic devices, optical devices, potentiometric devices, microelectromechanical devices, and so forth. The sensing mechanisms of weight sensors1312(1) may operate as transducers that generate one or more signals based on an applied force, such as that of the load due to gravity. For example, the weight sensor1312(1) may comprise a load cell having a strain gauge and a structural member that deforms slightly when weight is applied. By measuring a change in the electrical characteristic of the strain gauge, such as capacitance or resistance, the weight may be determined. In another example, the weight sensor1312(1) may comprise a force sensing resistor (FSR). The FSR may comprise a resilient material that changes one or more electrical characteristics when compressed. For example, the electrical resistance of a particular portion of the FSR may decrease as the particular portion is compressed. The inventory management system130may use the data acquired by the weight sensors1312(1) to identify an object, determine a change in the quantity of objects, determine a location of an object, maintain shipping records, and so forth.

The sensors1312may include capacitive sensors1312(2). As described above with regard toFIG. 11, the capacitive sensor1312(2) may comprise one or more conductive elements1106and the capacitance measurement/receiver module. In some implementations, the capacitive sensor1312(2) may include or utilize a switch module1112. The capacitive sensor1312(2) may be configured to use a far-field capacitance effect that may comprise measuring the self-capacitance of the conductive elements1106, rather than a mutual capacitance. In one implementation, a fixed charge may be provided to the conductive element1106, and the resultant voltage may be measured between the conductive element1106and the ground.

In other implementations, the capacitive sensor1312(2) may be configured to operate in a mutual capacitance mode, surface capacitance mode, and so forth. In mutual capacitance mode, at least two conductive layers are arranged in a stack with a dielectric material between the layers. The dielectric may be a solid, such as a plastic, a gas such as air, a vacuum, and so forth. The mutual capacitance at points between these layers is measured. When another object touches the outermost conductive layer, the mutual capacitance between the two layers changes, allowing for detection. In surface capacitance mode, voltages are applied to different points of a conductive element1106to produce an electrostatic field. By measuring the changes in current draw (or another electrical characteristic) from the different points at which voltage is applied, a location of an object may be determined.

The sensors1312may include one or more image sensors1312(3). The one or more image sensors1312(3) may include imaging sensors configured to acquire images of a scene. The image sensors1312(3) are configured to detect light in one or more wavelengths including, but not limited to, terahertz, infrared, visible, ultraviolet, and so forth. The image sensors1312(3) may comprise charge coupled devices (CCD), complementary metal oxide semiconductor (CMOS) devices, microbolometers, and so forth. The inventory management system130may use image data acquired by the image sensors1312(3) during operation of the facility1302. For example, the inventory management system130may identify items902, users112, totes116, and so forth, based at least in part on their appearance within the image data acquired by the image sensors1312(3). The image sensors1312(3) may be mounted in various locations within the facility1302. For example, image sensors1312(3) may be mounted overhead, on the fixtures108, may be worn or carried by users112, may be affixed to totes116, and so forth.

One or more depth sensors1312(4) may also be included in the sensors1312. The depth sensors1312(4) are configured to acquire spatial or three-dimensional (3D) data, such as depth information, about objects within a FOV. The depth sensors1312(4) may include range cameras, lidar systems, sonar systems, radar systems, structured light systems, stereo vision systems, optical interferometry systems, and so forth. The inventory management system130may use the 3D data acquired by the depth sensors1312(4) to identify objects, determine a location of an object in 3D real space, and so forth.

One or more buttons1312(5) may be configured to accept input from the user112. The buttons1312(5) may comprise mechanical, capacitive, optical, or other mechanisms. For example, the buttons1312(5) may comprise mechanical switches configured to accept an applied force from a touch of the user112to generate an input signal. The inventory management system130may use data from the buttons1312(5) to receive information from the user112. For example, the tote116may be configured with a button1312(5) to accept input from the user112and send information indicative of the input to the inventory management system130.

The sensors1312may include one or more touch sensors1312(6). The touch sensors1312(6) may use resistive, capacitive, surface capacitance, projected capacitance, mutual capacitance, optical, Interpolating Force-Sensitive Resistance (IFSR), or other mechanisms to determine the position of a touch or near-touch. For example, the IFSR may comprise a material configured to change electrical resistance responsive to an applied force. The location within the material of that change in electrical resistance may indicate the position of the touch. The inventory management system130may use data from the touch sensors1312(6) to receive information from the user112. For example, the touch sensor1312(6) may be integrated with the tote116to provide a touchscreen with which the user112may select from a menu one or more particular items902for picking, enter a manual count of items902at fixture108, and so forth.

One or more microphones1312(7) may be configured to acquire information indicative of sound present in the environment. In some implementations, arrays of microphones1312(7) may be used. These arrays may implement beamforming techniques to provide for directionality of gain. The inventory management system130may use the one or more microphones1312(7) to acquire information from acoustic tags1406, accept voice input from the users112, determine ambient noise level, and so forth.

The sensors1312may include one or more optical sensors1312(8). The optical sensors1312(8) may be configured to provide data indicative of one or more of color or intensity of light impinging thereupon. For example, the optical sensor1312(8) may comprise a photodiode and associated circuitry configured to generate a signal or data indicative of an incident flux of photons. As described below, the optical sensor array1312(14) may comprise a plurality of the optical sensors1312(8). For example, the optical sensor array1312(14) may comprise an array of ambient light sensors such as the ISL76683 as provided by Intersil Corporation of Milpitas, Calif., USA, or the MAX44009 as provided by Maxim Integrated of San Jose, Calif., USA. In other implementations, other optical sensors1312(8) may be used. The optical sensors1312(8) may be sensitive to one or more of infrared light, visible light, or ultraviolet light. For example, the optical sensors1312(8) may be sensitive to infrared light, and infrared light sources such as light emitting diodes (LEDs) may provide illumination.

The optical sensors1312(8) may include photodiodes, photoresistors, photovoltaic cells, quantum dot photoconductors, bolometers, pyroelectric infrared detectors, and so forth. For example, the optical sensor1312(8) may use germanium photodiodes to detect infrared light.

One or more radio frequency identification (RFID) readers1312(9), near field communication (NFC) systems, and so forth, may be included as sensors1312. For example, the RFID readers1312(9) may be configured to read the RF tags1406. Information acquired by the RFID reader1312(9) may be used by the inventory management system130to identify an object associated with the RF tag1406such as the item902, the user112, the tote116, and so forth. For example, based on information from the RFID readers1312(9) detecting the RF tag1406at different times and RFID readers1312(9) having different locations in the facility1302, a velocity of the RF tag1406may be determined.

One or more RF receivers1312(10) may also be included as sensors1312. In some implementations, the RF receivers1312(10) may be part of transceiver assemblies. The RF receivers1312(10) may be configured to acquire RF signals1408associated with Wi-Fi, Bluetooth, ZigBee, 4G, 3G, LTE, or other wireless data transmission technologies. The RF receivers1312(10) may provide information associated with data transmitted via radio frequencies, signal strength of RF signals1408, and so forth. For example, information from the RF receivers1312(10) may be used by the inventory management system130to determine a location of an RF source, such as a communication interface onboard the tote116.

The sensors1312may include one or more accelerometers1312(11), which may be worn or carried by the user112, mounted to the tote116, and so forth. The accelerometers1312(11) may provide information such as the direction and magnitude of an imposed acceleration. Data such as rate of acceleration and determination of changes in direction, speed, and so forth, may be determined using the accelerometers1312(11).

A gyroscope1312(12) may provide information indicative of rotation of an object affixed thereto. For example, the tote116or other objects may be equipped with a gyroscope1312(12) to provide data indicative of a change in orientation of the object.

A magnetometer1312(13) may be used to determine an orientation by measuring ambient magnetic fields, such as the terrestrial magnetic field. The magnetometer1312(13) may be worn or carried by the user112, mounted to the tote116, and so forth. For example, the magnetometer1312(13) mounted to the tote116may act as a compass and provide information indicative of which direction the tote116is oriented.

An optical sensor array1312(14) may comprise one or more optical sensors1312(8). The optical sensors1312(8) may be arranged in a regular, repeating, or periodic two-dimensional arrangement such as a grid. The optical sensor array1312(14) may generate image data. For example, the optical sensor array1312(14) may be arranged within or below fixture108and obtain information about shadows of items902, hand1002of the user112, and so forth.

The sensors1312may include proximity sensors1312(15) used to determine presence of an object, such as the user112, the tote116, and so forth. The proximity sensors1312(15) may use optical, electrical, ultrasonic, electromagnetic, or other techniques to determine a presence of an object. In some implementations, the proximity sensors1312(15) may use an optical emitter and an optical detector to determine proximity. For example, an optical emitter may emit light, a portion of which may then be reflected by the object back to the optical detector to provide an indication that the object is proximate to the proximity sensor1312(15). In other implementations, the proximity sensors1312(15) may comprise a capacitive proximity sensor1312(15) configured to provide an electrical field and determine a change in electrical capacitance due to presence or absence of an object within the electrical field.

The proximity sensors1312(15) may be configured to provide sensor data indicative of one or more of a presence or absence of an object, a distance to the object, or characteristics of the object. An optical proximity sensor1312(15) may use time-of-flight (ToF), structured light, interferometry, or other techniques to generate the distance data. For example, ToF determines a propagation time (or “round-trip” time) of a pulse of emitted light from an optical emitter or illuminator that is reflected or otherwise returned to an optical detector. By dividing the propagation time in half and multiplying the result by the speed of light in air, the distance to an object may be determined. In another implementation, a structured light pattern may be provided by the optical emitter. A portion of the structured light pattern may then be detected on the object using a sensor1312such as an image sensor1312(3). Based on an apparent distance between the features of the structured light pattern, the distance to the object may be calculated. Other techniques may also be used to determine distance to the object. In another example, the color of the reflected light may be used to characterize the object, such as skin, clothing, tote116, and so forth.

The sensors1312may also include an instrumented auto-facing unit (IAFU)1312(16). The IAFU1312(16) may comprise a position sensor configured to provide data indicative of displacement of a pusher. As an item902is removed from the IAFU1312(16), the pusher moves, such as under the influence of a spring, and pushes the remaining items902in the IAFU1312(16) to the front of the fixture108. By using data from the position sensor, and given item data such as a depth of an individual item902, a count may be determined, based on a change in position data. For example, if each item902is 1 inch deep, and the position data indicates a change of 17 inches, the quantity held by the IAFU1312(16) may have changed by 17 items902. This count information may be used to confirm or provide a cross check for a count obtained by other means, such as analysis of the weight data1130, the capacitance data1116, the image data1532, and so forth.

The sensors1312may include other sensors1312(S) as well. For example, the other sensors1312(S) may include light curtains, ultrasonic rangefinders, thermometers, barometric sensors, air pressure sensors, hygrometers, and so forth. For example, the inventory management system130may use information acquired from thermometers and hygrometers in the facility1302to direct the user112to check on delicate items902stored in a particular fixture108, which is overheating, too dry, too damp, and so forth.

In one implementation, a light curtain may utilize a linear array of light emitters and a corresponding linear array of light detectors. For example, the light emitters may comprise a line of infrared LEDs or vertical cavity surface emitting lasers (VCSELs) that are arranged in front of the fixture108, while the light detectors comprise a line of photodiodes sensitive to infrared light arranged below the light emitters. The light emitters produce a “lightplane” or sheet of infrared light that is then detected by the light detectors. An object passing through the lightplane may decrease the amount of light falling upon the light detectors. For example, the user's112hand1002would prevent at least some of the light from light emitters from reaching a corresponding light detector. As a result, a position along the linear array of the object may be determined that is indicative of a touchpoint. This position may be expressed as touchpoint data, with the touchpoint being indicative of the intersection between the hand1002of the user112and the sheet of infrared light. In some implementations, a pair of light curtains may be arranged at right angles relative to one another to provide two-dimensional touchpoint data indicative of a position of touch in a plane. Input from the light curtain, such as indicating occlusion from a hand1002of a user112may be used to generate interaction data142.

In some implementations, the image sensor1312(3) or other sensors1312(S) may include hardware processors, memory, and other elements configured to perform various functions. For example, the image sensors1312(3) may be configured to generate image data1532, send the image data1532to another device such as the server1404, and so forth.

The facility1302may include one or more access points1410configured to establish one or more wireless networks. The access points1410may use Wi-Fi, NFC, Bluetooth, or other technologies to establish wireless communications between a device and the network1402. The wireless networks allow devices to communicate with one or more of the sensors1312, the inventory management system130, the optical sensor arrays1312(14), the tags1406, a communication device of the tote116, or other devices.

Output devices1412may also be provided in the facility1302. The output devices1412are configured to generate signals, which may be perceived by the user112or detected by the sensors1312. In some implementations, the output devices1412may be used to provide illumination of the optical sensor array1312(14).

Haptic output devices1412(1) are configured to provide a signal that results in a tactile sensation to the user112. The haptic output devices1412(1) may use one or more mechanisms such as electrical stimulation or mechanical displacement to provide the signal. For example, the haptic output devices1412(1) may be configured to generate a modulated electrical signal, which produces an apparent tactile sensation in one or more fingers of the user112. In another example, the haptic output devices1412(1) may comprise piezoelectric or rotary motor devices configured to provide a vibration, which may be felt by the user112.

One or more audio output devices1412(2) may be configured to provide acoustic output. The acoustic output includes one or more of infrasonic sound, audible sound, or ultrasonic sound. The audio output devices1412(2) may use one or more mechanisms to generate the acoustic output. These mechanisms may include, but are not limited to, the following: voice coils, piezoelectric elements, magnetostrictive elements, electrostatic elements, and so forth. For example, a piezoelectric buzzer or a speaker may be used to provide acoustic output.

The display devices1412(3) may be configured to provide output, which may be seen by the user112or detected by a light-sensitive sensor such as an image sensor1312(3) or an optical sensor1312(8). In some implementations, the display devices1412(3) may be configured to produce output in one or more of infrared, visible, or ultraviolet light. The output may be monochrome or in color. The display devices1412(3) may be one or more of emissive, reflective, microelectromechanical, and so forth. An emissive display device1412(3), such as using LEDs, is configured to emit light during operation. In comparison, a reflective display device1412(3), such as using an electrophoretic element, relies on ambient light to present an image. Backlights or front lights may be used to illuminate non-emissive display devices1412(3) to provide visibility of the output in conditions where the ambient light levels are low.

The display devices1412(3) may be located at various points within the facility1302. For example, the addressable displays may be located on the fixtures108, totes116, on the floor of the facility1302, and so forth.

Other output devices1412(P) may also be present. For example, the other output devices1412(P) may include scent/odor dispensers, document printers, 3D printers or fabrication equipment, and so forth.

FIG. 15illustrates a block diagram1500of a server1404configured to support operation of the facility1302, according to some implementations. The server1404may be physically present at the facility1302, may be accessible by the network1402, or a combination of both. The server1404does not require end-user knowledge of the physical location and configuration of the system that delivers the services. Common expressions associated with the server1404may include “on-demand computing”, “software as a service (SaaS)”, “platform computing”, “network-accessible platform”, “cloud services”, “data centers”, and so forth. Services provided by the server1404may be distributed across one or more physical or virtual devices.

One or more power supplies1502may be configured to provide electrical power suitable for operating the components in the server1404. The one or more power supplies1502may comprise batteries, capacitors, fuel cells, photovoltaic cells, wireless power receivers, conductive couplings suitable for attachment to an external power source such as provided by an electric utility, and so forth. The server1404may include one or more hardware processors1504(processors) configured to execute one or more stored instructions. The processors1504may comprise one or more cores. One or more clocks1506may provide information indicative of date, time, ticks, and so forth. For example, the processor1504may use data from the clock1506to associate a particular interaction with a particular point in time.

The server1404may include one or more communication interfaces1508such as input/output (I/O) interfaces1510, network interfaces1512, and so forth. The communication interfaces1508enable the server1404, or components thereof, to communicate with other devices or components. The communication interfaces1508may include one or more I/O interfaces1510. The I/O interfaces1510may comprise Inter-Integrated Circuit (I2C), Serial Peripheral Interface bus (SPI), Universal Serial Bus (USB) as promulgated by the USB Implementers Forum, RS-232, and so forth.

The I/O interface(s)1510may couple to one or more I/O devices1514. The I/O devices1514may include input devices such as one or more of a sensor1312, keyboard, mouse, scanner, and so forth. The I/O devices1514may also include output devices1412such as one or more of a display device1412(3), printer, audio speakers, and so forth. In some embodiments, the I/O devices1514may be physically incorporated with the server1404or may be externally placed.

The network interfaces1512may be configured to provide communications between the server1404and other devices, such as the SFTs104, totes116, routers, access points1410, and so forth. The network interfaces1512may include devices configured to couple to personal area networks (PANs), local area networks (LANs), wireless local area networks (WLANS), wide area networks (WANs), and so forth. For example, the network interfaces1512may include devices compatible with Ethernet, Wi-Fi, Bluetooth, ZigBee, and so forth.

The server1404may also include one or more busses or other internal communications hardware or software that allow for the transfer of data between the various modules and components of the server1404.

As shown inFIG. 15, the server1404includes one or more memories1516. The memory1516may comprise one or more non-transitory computer-readable storage media (CRSM). The CRSM may be any one or more of an electronic storage medium, a magnetic storage medium, an optical storage medium, a quantum storage medium, a mechanical computer storage medium, and so forth. The memory1516provides storage of computer-readable instructions, data structures, program modules, and other data for the operation of the server1404. A few example functional modules are shown stored in the memory1516, although the same functionality may alternatively be implemented in hardware, firmware, or as a system on a chip (SoC).

The memory1516may include at least one operating system (OS) module1518. The OS module1518is configured to manage hardware resource devices such as the I/O interfaces1510, the I/O devices1514, the communication interfaces1508, and provide various services to applications or modules executing on the processors1504. The OS module1518may implement a variant of the FreeBSD operating system as promulgated by the FreeBSD Project; other UNIX or UNIX-like variants; a variation of the Linux operating system as promulgated by Linus Torvalds; the Windows operating system from Microsoft Corporation of Redmond, Wash., USA; and so forth.

Also stored in the memory1516may be a data store1520and one or more of the following modules. These modules may be executed as foreground applications, background tasks, daemons, and so forth. The data store1520may use a flat file, database, linked list, tree, executable code, script, or other data structure to store information. In some implementations, the data store1520or a portion of the data store1520may be distributed across one or more other devices including the servers1404, network attached storage devices, and so forth.

A communication module1522may be configured to establish communications with one or more of the totes116, sensors1312, display devices1412(3), other servers1404, or other devices. The communications may be authenticated, encrypted, and so forth.

The memory1516may store an inventory management module1524. The inventory management module1524is configured to provide the inventory functions as described herein with regard to the inventory management system130. For example, the inventory management module1524may track items902between different fixtures108, to and from the totes116, and so forth.

The inventory management module1524may include one or more of a data acquisition module1526, the tracking module134, the analysis module138, an action module1528, and so forth. The data acquisition module1526may be configured to acquire and access information associated with operation of the facility1302. For example, the data acquisition module1526may be configured to acquire tile output data126from the SFTs104, fixture data132, sensor data1530such as the weight data1130, capacitance data1116, image data1532, other sensor data1534, and so forth. The sensor data1530may be accessed by the other modules for use.

The data store1520may also store item data1536. The item data1536provides information about a particular type of item902, including characteristics of that type of item902such as physical dimensions, where that type of item902is located in the facility1302, characteristics about how the item902appears, capacitance values associated with the type of item902, attenuation characteristics of an EMS106, and so forth. For example, the item data1536may indicate that the type of item902is “Bob's Low Fat Baked Beans, 10 oz can” with a stock keeping unit number of “24076513”. The item data1536may indicate the types and quantities of items902that are expected to be stored at that particular fixture108such as in a particular lane on a shelf904, width and depth of that type of item902, weight of the item902individually or in aggregate, sample images of the type of item902, and so forth.

The item data1536may include an item identifier. The item identifier may be used to distinguish one type of item902from another. For example, the item identifier may include a stock keeping unit (SKU) string, Universal Product Code (UPC) number, radio frequency identification (RFID) tag data, and so forth. The items902that are of the same type may be referred to by the same item identifier. For example, cans of beef flavor Brand X dog food may be represented by the item identifier value of “9811901181”. In other implementations, non-fungible items902may each be provided with a unique item identifier, allowing each to be distinguished from one another.

The item data1536may include one or more of geometry data, item weight data, sample image data, sample capacitance data, or other data. The geometry data may include information indicative of size and shape of the item902in one-, two-, or three-dimensions. For example, the geometry data may include the overall shape of an item902, such as a cuboid, sphere, cylinder, and so forth. The geometry data may also include information such as length, width, depth, and so forth, of the item902. Dimensional information in the geometry data may be measured in pixels, centimeters, inches, arbitrary units, and so forth. The geometry data may be for a single item902, or a package, kit, or other grouping considered to be a single item902.

The item weight data comprises information indicative of a weight of a single item902, or a package, kit, or other grouping considered to be a single item902. The item data1536may include other data. For example, the other data may comprise weight distribution of the item902, point cloud data for the item902, and so forth.

The sample capacitance data may comprise data indicative of a previously measured or calculated change in capacitance obtained by a representative capacitive sensor1312(2) based on the presence or absence of a sample of the type of item902. For example, during processing or intake of the item902at the facility1302, a sample of the type of item902may be placed on a capacitive sensor1312(2) to generate the sample capacitance data. Similar data may be obtained for the attenuation or propagation of the EMS106across the item902.

The sample image data may comprise one or more images of one or more of that type of item902. For example, sample image data may be obtained during processing or intake of the item902to be used by the facility1302.

The item data1536may include one or more fixture identifiers (IDs). The fixture ID is indicative of a particular area or volume of fixture108such as a shelf904that is designated for stowage of the type of item902. For example, a single shelf904may have several lanes, each with a different fixture ID. Each of the different fixture IDs may be associated with a lane having a particular area on the shelf904designated for storage of a particular type of item902. A single type of item902may be associated with a particular fixture ID, a plurality of fixture IDs may be associated with the single type of item902, more than one type of item902may be associated with the particular fixture ID, and so forth.

The item data1536may also include quantity data. The quantity data may comprise a count or value indicative of a number of items902. The count may be a measured or an estimated value. The quantity data may be associated with a particular fixture ID, for an entire facility1302, and so forth. For example, the same type of item902may be stored at different shelves904within the facility1302. The quantity data may indicate the quantity on hand for each of the different fixtures108.

The analysis module138may utilize the tile output data126, fixture data132, weight data1130, capacitance data1116, item data1536, and other information to generate interaction data142. The interaction data142is indicative of action such as picking or placing an item902for a particular fixture108, presence of the user112at the fixture108, and so forth.

In some implementations, the analysis module138may generate output data1544about the user112. The analysis module138may determine if the user112is standing, moving, lying on the floor102, and so forth. For example, the analysis module138may determine an area of contact with the floor102based on the tile output data126. If the area of contact exceeds a threshold value, the user112may be determined to be lying on the floor102. Based on this determination, other actions may be taken. For example, alarm data may be generated to summon assistance if a user112is deemed to be lying on the floor102.

The analysis module138, or other modules, may be configured to determine portions of the SFTs104which are to be deactivated or from which information is to be disregarded. In one implementation, during setup of the system, the antennas304of a SFT104that are located underneath a fixture108may be deactivated. In another implementation, the analysis module138may determine SFTs104or portions thereof that report presence of an object that is unchanging over long periods of time, such as hours or days. These objects, such as a fixture108above the SFT104, may then be subsequently disregarded and information about these positions may be removed from further processing. If a change is detected, such as when the fixture108above the SFT104is moved, information about that change may be used to re-enable consideration of data from that SFT104or portion thereof.

The inventory management module1524may utilize the physical layout data1538. The physical layout data1538may provide information indicative of location of the SFTs104, where sensors1312and the fixtures108are in the facility1302with respect to one another, FOV of sensors1312relative to the fixture108, and so forth. For example, the physical layout data1538may comprise information representative of a map or floor plan of the facility1302with relative positions of the fixtures108, location of individual SFTs104therein, arrangements of the segments204, planogram data indicative of how items902are to be arranged at the fixtures108, and so forth. Continuing the example, the physical layout data1538may be based on using the relative arrangement of the SFTs104in conjunction with their physical dimensions to specify where the SFTs104are placed within the facility1302.

The physical layout data1538may associate a particular fixture ID with other information such as physical location data, sensor position data, sensor direction data, sensor identifiers, and so forth. The physical layout data1538provides information about where in the facility1302objects are, such as the fixture108, the sensors1312, and so forth. In some implementations, the physical layout data1538may be relative to another object. For example, the physical layout data1538may indicate that a particular weight sensor1312(1), capacitive sensor1312(2), or image sensor1312(3) is associated with the shelf904or portion thereof.

The inventory management module1524may utilize the physical layout data1538and other information during operation. For example, the tracking module134may utilize physical layout data1538to determine what capacitance data1116acquired from particular capacitive sensors1312(2) corresponds to a particular shelf904, lane, or other fixture108.

The tracking module134may access information from sensors1312within the facility1302, such as those at the shelf904or other fixture108, onboard the tote116, carried by or worn by the user112, and so forth. For example, the tracking module134may receive the fixture data132and use the characteristic data128to associate a particular user112with a pick or place of an item902at the associated fixture108.

The account item data1540may also be included in the data store1520and comprises information indicative of one or more items902that are within the custody of a particular user112, within a particular tote116, and so forth. For example, the account item data1540may comprise a list of the contents of the tote116. Continuing the example, the list may be further associated with the user account representative of the user112. In another example, the account item data1540may comprise a list of items902that the user112is carrying. The tracking module134may use the account item data1540to determine subsets of possible items902with which the user112may have interacted.

The inventory management module1524, and modules associated therewith, may access sensor data1530, threshold data1542, and so forth. The threshold data1542may comprise one or more thresholds, ranges, percentages, and so forth, that may be used by the various modules in operation.

The inventory management module1524may generate output data1544. For example, the output data1544may include the interaction data142, inventory levels for individual types of items902, overall inventory, and so forth.

The action module1528may be configured to initiate or coordinate one or more actions responsive to output data1544. For example, the action module1528may access output data1544that indicates a particular fixture108is empty and in need of restocking. An action such as a dispatch of a work order or transmitting instructions to a robot may be performed to facilitate restocking of the fixture108.

Processing sensor data1530, such as the image data1532, may be performed by a module implementing, at least in part, one or more of the following tools or techniques. In one implementation, processing of the image data1532may be performed, at least in part, using one or more tools available in the OpenCV library as developed by Intel Corporation of Santa Clara, Calif., USA; Willow Garage of Menlo Park, Calif., USA; and Itseez of Nizhny Novgorod, Russia, with information available at www.opencv.org. In another implementation, functions available in the OKAO machine vision library as promulgated by Omron Corporation of Kyoto, Japan, may be used to process the sensor data1530. In still another implementation, functions such as those in the Machine Vision Toolbox for Matlab (MVTB) available using MATLAB as developed by Math Works, Inc. of Natick, Mass., USA, may be utilized.

Techniques such as artificial neural networks (ANNs), active appearance models (AAMs), active shape models (ASMs), principal component analysis (PCA), cascade classifiers, and so forth, may also be used to process the sensor data1530or other data. For example, the ANN may be trained using a supervised learning algorithm such that object identifiers are associated with images of particular objects within training images provided to the ANN. Once trained, the ANN may be provided with the sensor data1530and the item data1536to allow for a determination of similarity between two or more images.

The sensor data1530obtained from different sensors1312may be used to compare or validate output data1544. For example, the image data1532may indicate the presence of a person based on a coat or jacket that is arranged across the back of a chair. However, the tile output data126provides information that no user112is currently present at that location in the facility1302. This difference may be used to generate an alarm, notify an associate in the facility1302, and so forth.

Other data1546may be stored in the data store1520as well as other modules1548in the memory1516. For example, the other modules1548may include a billing module while the other data1546may include billing data.

FIG. 16depicts a flow diagram1600of a process of using SFTs104that emit a sequence of segment signals208in respective timeslots210to generate tracking data136, according to some implementations.

At1602, an SFT104transmits an initial signal206at a first frequency. The first frequency is representative of the SFT104within a cluster202or other grouping of SFTs104. In one implementation, the initial signal206may be radiated using antennas304of the SFT104such that the initial signal206is radiated across at least half of the surface area of the SFT104. For example, an antenna switch may couple the transmitter328to all of the segment antennas304.

At1604, one or more segment signals208are transmitted in a time sequence506from a plurality of segments204of the SFT104. Each segment signal208is transmitted during a timeslot210associated with a particular segment204and using the antennas304associated with the particular segment204. For example, the antenna switch may connect particular antennas304at different segments204to the transmitter328during the respective timeslot210associated with that segment204. Continuing the example, during the first timeslot210(1), the first segment antenna304(1) is connected to the transmitter328, while at the second time slot210(2), the second segment antenna304(2) is connected to the transmitter328.

One or more of the initial signal206, segment signals208, or other EMS106may be modulated using amplitude modulation, frequency modulation, phase modulation, or other modulation techniques. The EMS106may have a fixed carrier frequency of between 20 kilohertz and 15 megahertz and with a sinusoidal waveform. In some implementations, an unmodulated carrier may be used. The overall envelope of the waveform of the EMS106may be configured to ramp up in amplitude over a period of time to avoid spurious emissions at other frequencies.

At1606, the SFT104receives a plurality of EMS106. One or more of the receiver(s)326, the transmitter(s)328, or the antennas304may operate or be used simultaneously in some implementations. For example, an antenna304may be used to receive and radiate EMS106simultaneously. The plurality of EMS106may comprise a single signal that is transmitted and has the initial signal206, segment signals208, and so forth modulated therein. For example, a continuous carrier may be transmitted with the initial signal206and segment signals208produced by modulating the carrier at particular times.

At1608, received characteristic data128is determined for at least a portion of the received plurality of EMS106. The characteristic data128may be indicative of one or more of: a frequency of the received EMS106, timeslot210within which the EMS106was received, signal strength of the EMS106, and so forth. In one implementation, the determination of the timeslot210may include one or more of the following steps. A reception time elapsed between the reception time of the initial signal206and the particular segment signal208may be determined.

Time differences between signals may be done from a leading edge of the signals. Time may be measured based on a start time of the respective signals, end time of the respective signals, and so forth. For example, the reception time elapsed may be measured from the time at which the initial signal206first exceeds a threshold amplitude and the time at which the segment signal208exceeds a threshold amplitude. In other implementations, differences in timing may be done using different portions of the signals.

The timeslot210value associated with the reception time may be determined. For example, the reception time elapsed between the initial signal206and the segment signal208(3) is 10 milliseconds. A lookup table or other data structure may be used to associate the reception time elapsed value with a particular timeslot210value. In this example, the 10 ms is associated with the third timeslot210(3). As a result, receiving a segment signal208(3) starting at 10 ms results in determination of a timeslot value of “3” that indicates an EMS106that was transmitted in the third timeslot210(3). Given the timeslot value of “3”, the segment signal208(3) is associated with the third segment204(3).

As described, the characteristic data128may indicate the received signal strength of the EMS106. By using data about the received signal strength, the distance to a particular antenna304that is radiating the EMS106may be estimated.

The initial signal206may be used to synchronize a local clock or start a timer, to determine the reception time elapsed between the initial signal206and the segment signal208.

At1610, tile output data126is generated. The tile output data126is indicative of the SFT104and the received characteristics data128. In some implementations, the tile output data126may be sent using a CAN bus interface or other communication interface to another computing device.

At1612, tracking data136is generated using the tile output data126. For example, tile output data126from a plurality of smart floor tiles104may be analyzed to determine which SFTs104and segments204thereon the user112is in contact with or bridging.

FIG. 17depicts a flow diagram1700of a process of generating tracking data136using SFTs104with segments204that emit signals using code division multiple access (CDMA), according to some implementations.

At1702, a first segment signal208(1) modulated with a first CDMA code and a second segment signal208(2) modulated with a second CDMA code are generated at a first frequency.

At1704, the first segment signal208(1) is transmitted using a first segment antenna304(1) to radiate the signal.

At1706, the second segment signal208(2) is transmitted using a second segment antenna304(2) to radiate the signal. In some implementations, one or more of the segment signals208for a particular SFT104may be transmitted contemporaneously.

At1708, a plurality of EMS106are received.

At1710, received characteristic data128of the plurality of EMS106is determined. The received characteristic data128may include one or more of frequency of the received signal, CDMA code, signal strength, and so forth. In some implementations, the tile output data126may be sent using a CAN bus interface or other communication interface to another computing device.

At1712, tile output data126indicative of the SFT104and the received characteristic data128is generated.

At1714, tracking data136is generated using the tile output data126.

The system described above may be utilized in a variety of different settings including, but not limited to, commercial, non-commercial, medical, and so forth. For example, the SFTs104may be deployed in a home, hospital, care facility, correctional facility, transportation facility, office, and so forth. The tracking module134may provide tracking data136, such as the location of users112within a facility. In some implementations, the tracking module136may provide tracking data136that is indicative of the identity of a particular user112. The analysis module138may be used to generate output data1544that is indicative of a status of the user112, such as whether the user112is standing, sitting, lying on the floor, and so forth.