Method and system for order-free spoken term detection

A method for spoken term detection, comprising generating a time-marked word list, wherein the time-marked word list is an output of an automatic speech recognition system, generating an index from the time-marked word list, wherein generating the index comprises creating a word loop weighted finite state transducer for each utterance, i, receiving a plurality of keyword queries, and searching the index for a plurality of keyword hits.

TECHNICAL FIELD

The field generally relates to systems and methods for spoken term detection and, in particular, systems and methods for order-free spoken term detection

BACKGROUND

Finding a target term in an audio corpus is one of the fundamental problems in automatic speech processing. Given the vast amount of existing spoken information, there is an increasing need for small indices and fast search. Typically, known spoken term detection (STD) systems search for terms in an index built from the output of an automatic speech recognition (ASR) system. The ASR output representation is the 1-best hypothesis, and using it for indexing results in good STD performance if the ASR system has low word error rate. However, many known STD systems, which may have to deal with degraded inputs, can benefit from using a richer ASR output representation. Lattices and confusion networks (CNs) are two used representations of multiple hypotheses from an ASR system, and have been used for building STD indices. The lattice approach requires large disk space to store an index. Although CNs require less disk space, CN computation can be prohibitive for large lattices.

SUMMARY

In general, exemplary embodiments of the invention include systems and methods for spoken term detection and, in particular, systems and methods for order-free spoken term detection.

Embodiments of the present invention use Time-Marked Word (TMW) lists as a replacement for lattices and CNs used as indexing vehicles for STD. In a TMW list, candidates are tagged with posterior probabilities and time information, and stored as a large list of words. The TMW list does not use the additional word ordering present in a lattice or CN. TMW lists compactly summarize a large ASR search space. Representing a large search space can be critical for STD metrics such as actual term-weighted value (ATWV) that heavily penalize misses of rare keywords. As set forth below in experimental examples, comparisons on the OpenKWS 2014 Tamil limited language pack task show that the TMW-based indexing results in better performance than conventional methods, while being faster and having a smaller footprint.

According to an exemplary embodiment of the present invention, a method for spoken term detection, comprises generating a time-marked word list, wherein the time-marked word list is an output of an automatic speech recognition system, generating an index from the time-marked word list, wherein generating the index comprises creating a word loop weighted finite state transducer for each utterance, i, receiving a plurality of keyword queries, and searching the index for a plurality of keyword hits.

According to an exemplary embodiment of the present invention, a computer program product for spoken term detection, comprises a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform the above method.

According to an exemplary embodiment of the present invention, an apparatus, for spoken term detection comprises a memory, and a processing device operatively coupled to the memory and configured to generate a time-marked word list, wherein the time-marked word list is an output of an automatic speech recognition system, generate an index from the time-marked word list, wherein generating the index comprises creating a word loop weighted finite state transducer for each utterance, i, receive a plurality of keyword queries, and search the index for a plurality of keyword hits.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Exemplary embodiments of the invention will now be discussed in further detail with regard to systems and methods for spoken term detection and, in particular, systems and methods for order-free spoken term detection. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein.

As used herein, “word label” can refer to the word identity of an ASR hypothesis.

As used herein, “start and end times” can refer to the beginning and end times of hypothesized words.

As used herein, “posterior probabilities/scores” can refer to the probability of a hypothesized word for a given start and end time, given an entire observed acoustic sequence. For certain Weighted Finite State Transducer (WFST) operations, the probabilities are converted to the log domain.

As used herein, “zero-cost epsilon-arcs” can refer to links in a finite state transducer with <epsilon> input and output labels, and with cost=0.

As used herein, “full connectivity” can refer to all the nodes in a finite state transducer being connected.

As noted above, embodiments of the present invention use TMW lists as a replacement for lattices and CNs. TMW lists include a set of words with start and end times, and posterior scores. Unlike lattices and CNs, which explicitly represent word ordering in their topologies, TMW lists lack such structure, encoding word ordering implicitly in the time marks. The structural relationship between lattices or CNs and TMW lists can be explained as being like the structural relationship between a sentence and its bag-of-words representation. To accommodate the lack of explicit word-order information in the TMW lists, embodiments of the present invention utilize a Weighted Finite State Transducer (WFST) architecture for STD.

ASR Output Representations

Many speech recognition systems produce lattices or CNs to be used for STD indexing. Lattices are partially ordered networks of word hypotheses, with links in the networks carrying word identity, time information, language model (LM) and acoustic model (AM) scores. Posterior probabilities for the links in a lattice can be computed from the LM and AM scores using, for example, the Forward-Backward algorithm. CNs have a linear structure, representing the competing word hypotheses and their posterior probabilities in consecutive time intervals (referred to as confusion bins). CNs are produced from lattices through a 2-step process: (1) Intra-word clustering, in which the lattice arcs which have the same word label, and start and end time, are merged, and their posteriors summed up, and (2) Inter-word clustering, in which all the lattice arcs are clustered until the partial order becomes a total order, leading to the linear structure. CNs are orders of magnitude smaller than lattices, but they take extra time to compute. The inter-word clustering step can account for almost all, for example, 99%, of the computation time. To avoid this time-consuming step, embodiments of the present invention use a TMW list, which comprises the output of the intra-word clustering step, including an enumeration of word labels, start and end times, and posterior probabilities, (w,s,e,p).

According to an embodiment, silence, hesitations and other filler words are not written into this list. A lattice is computed in memory, but only the TMW list is produced on disk. In order to reduce the size of the TMW lists further, an embodiment of the present invention relaxes an exact time match constraint to allow for arcs with large overlap to merge as well. This disclosure reports results for exact match. An exact match time constraint refers to having two links in a TMW list having identical start and end times in order to merge the two links. In the relaxed constraint scenario, links can be merged that have less than 100% overlap, for example, links are merged which have 95% overlap.

Data and ASR System Description

The following includes a description of the task, metric, and ASR system used for indexing. In connection with the non-limiting illustrative experimental examples discussed below, experiments were conducted in the context of the IARPA Babel program, which focuses on spoken term detection for low-resource languages. In the non-limiting illustrative experimental examples, the STD task is defined by the National Institute of Standards and Technology (NIST®) in the OpenKWS14 Evaluation Plan. The limited language pack track (LP) of the program was chosen, in which only 20 hours of audio, (10 hours of transcribed data) is used for building ASR models and lexicons, making it arguably more interesting for out-of-vocabulary (OOV) keyword searches. The non-limiting illustrative experimental examples focus on the Tamil language, which was the OpenKWS 2014 evaluation task. The limited language pack includes a 20-hour development set (DEV). For these experiments two keyword sets were used: IBM-1, containing 1721 in-vocabulary (IV) queries and 654 OOV queries, and IBM-2, containing 1978 IV queries and 617 OOV queries, generated by International Business Machines (IBM®) and supplied to all OpenKWS participants.

The metric used for the Babel program is Term-Weighted Value (TWV), which was first used in the NIST® 2006 STD Evaluation. As shown in Tables 1, 3 and 4 set forth herein, keyword search performance is reported in terms of maximum Term-Weighted Value (MTWV), which is the best TWV for all values of a decision threshold, Optimal TWV (OTWV), which gives an upper-bound of the performance under perfect keyword-specific thresholding, and Supremum TWV (STWV), which gives an upper bound of the performance assuming perfect detection scores and thresholding.

The acoustic model used in the experimental examples is a collection of three deep neural networks (DNNs) which differ in the number of output states (1000, 2000, 3000). The DNNs take 9 consecutive frames as input where each frame is a concatenation of a 40-dimensional feature space maximum likelihood linear regression (fMLLR) vector and a 7-dimensional fundamental frequency variation (FFV) vector. Each DNN has 5 hidden layers with 1024 sigmoid units. During decoding, the output scores of the DNNs are combined at the frame level with equal weights. The training of the nets comprises (1) layer-wise discriminative pre-training using the cross-entropy criterion, (2) stochastic gradient training using back-propagation and the cross-entropy criterion, and (3) sequence discriminative training using stochastic gradient and the state-level minimum Bayes risk criterion. The dictionary has 14.1K words and 21.3K pronunciations. The language model (LM) is a trigram LM with modified Kneser-Ney smoothing trained only on the acoustic transcripts.

According to an embodiment, the lattices, CNs and TMW lists are produced using a dynamic decoder. The word error rates for the 1-best hypotheses from the lattices and confusion networks are 73.9% and 73.1%, respectively. For simplicity, the results are presented for this acoustic model only, which is the IBM® model with the best ATWV performance in the OpenKWS 14 evaluation. Similar improvements can be obtained for other acoustic models.

Indexing

In accordance with an embodiment of the present invention, the order-free method proposed for indexing TMW lists is described. An index containing necessary information for keyword searching (e.g., audio file identity, start time, end time, and word label) is constructed from a TMW list using the following steps.1. For each utterance, i, a word loop WFST is created, which has Sias the start node, Eias the end node, and arcs from Sito Eifor each item (w,s,e,p) in the TMW list. These arcs have w as the input label, (s,e) as the output label and −log(p) as the cost. Eiis connected to Siby a zero-cost epsilon arc, thus creating a word loop.2. The final single index is obtained by creating a new start node, S, that is connected to each Siby zero-cost arcs with input label epsilon and output label i (or audio file id), and a new end node, E, that is connected to each Eiby zero-cost epsilon-arcs.

FIG. 1shows a TMW-based index, in accordance with an embodiment of the present invention. The set of keywords that can be retrieved by this index is larger than the set of keywords that can be retrieved by a lattice index due to the full connectivity of the word components of the TMW-based index. A multi-word keyword might not be found in a lattice index if there is no path connecting the word components in the lattice. This can be a problem especially for large keywords. In the case of a CN-based index, which is already a much more connected structure than the lattice index, the TMW-based index allows for new sequences of words which might be missed in a CN due to an inter-word alignment error.

Although embodiments of the present invention provide an ASR system that outputs TMW lists instead of lattices and CNs, in the case the lattice and CN outputs already exist, alternative embodiments of the present invention convert the lattice and CN outputs to TMW lists, which are indexed in a similar fashion to TMW lists created instead of the lattices and CNs. According to an embodiment of the present invention, in the case of converting a CN output to a TMW list, epsilon arcs in a CN are ignored when creating the index, and only the words in a CN are used to obtain the word-loop index. The lattice-TMW list and CN-TMW list indexing is discussed further herein.

Search

According to embodiments of the present invention, each query is converted into a word automaton to search the index described in connection withFIG. 1. More specifically, a multi-word query containing N words is converted into an automaton with N links, each link having as a label the corresponding word. In-vocabulary (IV) query automata are directly composed with the word index transducer. For OOV searches, (1) queries can be converted to IV queries (proxies) using a phone confusability (P2P) transducer (see U.S. patent application Ser. No. 14/230,790, filed on Mar. 31, 2014, which is assigned to International Business Machines Corporation of Armonk, N.Y., USA, and titled “Method And System For Efficient Spoken Term Detection Using Confusion Networks,” the complete disclosure of which is expressly incorporated herein by reference in its entirety for all purposes), and then composed with the word index, or (2) the index is converted to phone level by replacing all words with their pronunciations and is then searched via composition with phone automata. A phone automaton is generated by (1) converting an OOV word automaton to a phone automaton P using the lexicon, (2) composing P with P2P, and (3) extracting N-best paths. Both methods produce identical results, with the choice for which method to use depending on, for example, memory and computational constraints, as well as on the size of the vocabulary. The proxy method can result in a smaller index size and faster search. However, for large vocabulary sizes, the conversion of OOV queries to IV proxies is computationally and memory intensive, in which case the phonetic method may be preferred. For many tasks, the IV search can also benefit from expansion using a P2P transducer, in which case the indexing and search pipeline for all the queries will be the same, and only the degree of phonetic expansion (N-best) will differ (less expansion for IV queries).

Regardless of the type of composition, word-based or phone-based, the result of the composition, after projecting on the output label, is a list of hits for each query and the corresponding score. A hit contains the audio file id, as well as a sequence of start and end time pairs (si,ei) corresponding to the word components of a multi-word query “audio file id” (s1,e1) (s2,e2) . . . (sn,en). In contrast to the previous lattice and CN-based WFST approaches, in which the start and end time pairs are ordered due to the structure of the index, when employing TMW lists, all the hits containing consecutive time pairs that are not ordered are eliminated. Two time pairs (si,ei) and (si+1,ei+1) are ordered if si<si+1and si+1−ei<thresh, where thresh is empirically determined. In other words, the start times have to be sorted in time, and the putative locations of the word components should not be far from each other. Note that si+1−eicould be negative if the two time pairs overlap. The final posting list includes the surviving hits, which have start time s1and end time en. In case there are two overlapping hits for a keyword, only the hit with the maximum score is kept. For each keyword, the scores below a threshold are normalized (e.g., using the methods described in U.S. patent application Ser. No. 14/230,790, referenced above), while high scores are kept intact.

Experimental Examples and Results

In non-limiting illustrative experimental examples, the OpenFST Library was used for both indexing and search. It is to be understood that there are many methods for creating the phone confusability transducer. For the OpenKWS evaluation a method was used that compares the Viterbi alignment of the training data transcripts to the decoded output to accumulate state-level confusions which are then converted to phone-level confusions.

As a baseline for the TMW based STD, known lattice and CN WFST STD architectures that were successfully deployed in both the DARPA RATS and IARPA Babel evaluations were used. In the lattice architecture, a word index built from lattices was used for IV search and a phone index was used for OOV search, after the OOV queries were expanded using the P2P transducer. In the CN approach, a word index built from CNs was used for both IV and OOV searching. All queries were mapped to IV proxies after expansion with the P2P transducer. The same confusability transducer was used for all approaches, and the same degree of expansion for IV (N-best=2000) and OOV queries (N-best=20000) was used. Table 1 and Table 2 set forth below show the performance, index size and computational time for TMW lists, CNs, and lattices produced by the acoustic model described above.

TABLE 2Comparison of size and computational timesSystemIndex SizeTime to produceLattice21G82 hoursCN110M124 hoursTMW295M80 hours

It can be seen that TMW STD has the best MTWV, OTWV, and STWV, requires the least amount of time for index generation, and produces a smaller index than lattice STD. While CN STD has an even smaller index size, if decoding beams for CN STD are increased to match the TMW STD index size, the CN STD performance is still worse (MTWV=0.1525) and the time to produce the CN STD index increased by 20%.

The difference between order-free indexing and structured indexing for a given ASR output type was also investigated. Order-free indexing based on lattices (lattice-TMW) is a matter of converting lattices to TMW lists and then applying TMW indexing and search. This can be identical to TMW STD, except that the lattices have been written to disk. For order-free indexing based on CNs (CN-TMW), TMW lists are created by extracting the words with time information and their posterior probabilities from CNs, and then applying TMW indexing and search. The comparison between lattice STD and lattice-TMW STD is made in Table 1, while the comparison between CN STD and CN-TMW STD is made in Table 3 set forth below. Even if CNs are used as an intermediate representation, order-free indexing improves STD performance.

The STD results above are obtained using the same ASR decoding parameters, namely the ASR decoding parameters used in the evaluation. For the ATWV metric it is very important that rare words are not missed; therefore, better performance can be achieved if the index is rich enough to contain instances of those words, even if the scores are low. If the only hit for a word has a very low score, after normalization this score becomes 1, and will survive any thresholding. Given that TMW lists are much smaller than lattices and faster to produce than CNs, increasing the decoding beams and thus pruning fewer hypotheses can be afforded. As seen in Table 4 set forth below, with an index that is 150 times smaller, better performance is obtained.

In all the above experiments, indexing is based on word ASR decoding. However, the embodiments of the present invention are not limited thereto. For example, another evaluation system, in accordance with an embodiment of the present invention, can use three indexes: (1) word-based, (2) word-based but with no language model scores, and (3) morph-based. For each query, searching in the three indexes is performed simultaneously and the results are merged. According to an embodiment of the present invention,FIG. 2shows the architecture of an index that can be used for this parallel search. The labels T1,T2,T3identify the sub-index that produces a given hit in the resulting posting list. These identifiers are needed due to the different merging strategies used in case of overlapping hits. For hits coming from the same sub-index, only the maximum scoring hit is kept, while for hits coming from different sub-indexes, the scores are totaled. As seen in Table 5 set forth below, parallel indexing and search results in 40% relative improvement in ATWV, and this improvement holds when TMW STD is used instead of CN STD.

TABLE 5Comparison of parallel STD architecturefor CN STD and TMW STDSystemMTWVCN STD0.2194TMW STD0.2210

Comparisons only against CN STD are shown because this was the system that was submitted in the OpenKWS evaluation. TMW STD can be especially beneficial for parallel indexing and search. Given the complex structure of a parallel index, it is important to have small sub-indexes which can also be produced quickly.

As noted herein, embodiments of the present invention provide (TMW) lists as input for STD indexing, and as a replacement for lattices and CNs. TMW lists are much smaller than lattices, and faster to compute than CNs. To accomodate for a lack of explicit word-order information in the TMW lists, embodiments of the present invention provide a new word-loop FST architecture for STD. The burden of insuring that the words in a multi-word query are correctly ordered in an STD hit is transferred from the indexing step to the search step. While previously the index encoded this information, causing the index to be large (lattices) or slower to produce (CNs), the current approach simply imposes an efficient time order test during search. According to an embodiment of the present invention, the proposed STD architecture can be applied to lattices and CNs by converting the lattices and CNs to TMW lists. For example, TMW lists are created after creating a lattice in memory, which for computation of word posterior probabilities.

FIG. 3shows the proposed system architecture, in accordance with an embodiment of the present invention. As shown inFIG. 3by lines and/or arrows, the components of the system300are operatively coupled to each other via, for example, physical connections, such as wired and/or direct electrical contact connections, and wireless connections, such as, for example, WiFi, BLUETOOTH®, IEEE 802.11, and/or networks, including but not limited to, a local area network (LAN), wide area network (WAN), cellular network, satellite network or the Internet.

The system300for spoken term detection, comprises a query module310capable of receiving keyword queries301, for example, phone level OOV keyword queries and IV keyword queries. In accordance with an embodiment of the present invention, the system300includes a list generation module320comprising, for example, an ASR system including an ASR decoder, which generates a TMW list. The TMW list comprises the output of intra-word clustering, including an enumeration of word labels, start and end times, and posterior probabilities, (w,s,e,p). As noted above, according to an embodiment, silence, hesitations and other filler words are not written into this list. A lattice is computed in memory, but only the TMW list is produced on disk, and to reduce the size of the TMW lists, an embodiment of the present invention relaxes an exact time match constraint to allow for arcs with large overlap to merge as well.

The TMW list is sent to an indexing module330. The indexing module330generates an index like that shown inFIG. 1from a TMW list. The index includes necessary information for keyword searching (e.g., audio file identity, start time, end time, and word label) and is constructed from the TMW list using the steps for indexing described above. For each utterance, the indexing module330creates a word loop WFST as explained hereinabove, which has Sias the start node, Eias the end node, and arcs from Sito Eifor each item (w,s,e,p) in the TMW list. The indexing module330generates a final single index by creating a new start node, S, that is connected to each Siby zero-cost arcs with input label epsilon and output label i (or audio file id), and a new end node, E, that is connected to each Eiby zero-cost epsilon-arcs.

A search module340receives the queries from the query module310, and converts each query into a word automaton to search the index described in connection withFIG. 1. As noted above, IV query automata are directly composed with the word index transducer, and for OOV searches, queries can be converted to IV queries (proxies) using a phone confusability (P2P) transducer, and then composed with the word index, or (2) the index is converted to phone level by replacing all words with their pronunciations and is then searched via composition with phone automata. The search module340generates a phone automaton by (1) converting an OOV word automaton to a phone automaton P using the lexicon, (2) composing P with P2P, and (3) extracting N-best paths.

Regardless of the type of composition, word-based or phone-based, the search module outputs to an output module350, a list of hits for each query and a corresponding score. As noted above, a hit contains the audio file id, as well as a sequence of start and end time pairs (si,ei) corresponding to the word components of a multi-word query “audio file id” (s1,e1) (s2,e2) . . . (sn,en). The output module350eliminates all the hits containing consecutive time pairs that are not ordered, and orders two time pairs (si,ei) and (si+1,ei+1) if si<si+1and si+1−ei<thresh, where thresh is empirically determined. The final posting list provided by the output module350includes the surviving hits, which have start time s1and end time en. In case there are two overlapping hits for a keyword, the output module350keeps only the hit with the maximum score. The output module350includes a normalization component360, which normalizes the scores below a threshold for each keyword, while keeping the high scores intact.

FIG. 4is a flow diagram illustrating a method for spoken term detection, in accordance with an exemplary embodiment of the present invention. The method for spoken term detection400comprises generating a TMW list, wherein the time-marked list is an output of an ASR system (block402). The TMW list includes an enumeration of word labels, start and end times, and posterior probabilities. In accordance with an embodiment, generating the TMW list may comprise converting a lattice output or a confusion network output to the TMW list.

The method400further comprises generating an index from the TMW list, wherein generating the index comprises creating a word loop WFST for each utterance, i (block404). In accordance with an embodiment of the present invention, the word loop WFST includes Sias a start node, Eias an end node, and arcs from Sito Eifor each word label, start and end time, and posterior probability (w,s,e,p) in the TMW list. Each arc has w as an input label, (s,e) as an output label and −log(p) as a cost. Eiis connected to Siby a zero-cost epsilon arc. Generating the index may further comprise creating a new start node, S, that is connected to each Siby zero-cost arcs with input label epsilon and output label i, and creating a new end node, E, that is connected to each Eiby zero-cost epsilon-arcs.

According an embodiment, the index may comprise a plurality of indexes that are simulataneously searched. The plurality of indexes can comprise at least two of a word-based index, a word-based index with no language model scores, and a morph-based index.

The method400further comprises receiving a plurality of keyword queries (block406), and searching the index for a plurality of keyword hits (block408). Searching can comprise converting each query of the plurality of keyword queries into a word automaton to search the index. According to an embodiment, a keyword hit may include an audio file id, a sequence of start and end time pairs (si,ei) corresponding to word components of a multi-word query audio file id (s1,e1) (s2,e2) . . . (sn,en).

The method may further comprise eliminating all hits containing consecutive time pairs that are not ordered, wherein two time pairs (si,ei) and (si+1,ei+1) are ordered if si<si+1and si+1−ei<thresh, where thresh is empirically determined.

As shown inFIG. 5, computer system/server512in computing node510is shown in the form of a general-purpose computing device. The components of computer system/server512may include, but are not limited to, one or more processors or processing units516, a system memory528, and a bus518that couples various system components including system memory528to processor516.

The computer system/server512typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer system/server512, and it includes both volatile and non-volatile media, removable and non-removable media.

The system memory528can include computer system readable media in the form of volatile memory, such as random access memory (RAM)530and/or cache memory532. The computer system/server512may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system534can be provided for reading from and writing to a non-removable, non-volatile magnetic media (not shown and typically called a “hard drive”). Although not shown, a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk (e.g., a “floppy disk”), and an optical disk drive for reading from or writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical media can be provided. In such instances, each can be connected to the bus518by one or more data media interfaces. As depicted and described herein, the memory528may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention. A program/utility540, having a set (at least one) of program modules542, may be stored in memory528by way of example, and not limitation, as well as an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data or some combination thereof, may include an implementation of a networking environment. Program modules542generally carry out the functions and/or methodologies of embodiments of the invention as described herein.

Computer system/server512may also communicate with one or more external devices514such as a keyboard, a pointing device, a display524, etc., one or more devices that enable a user to interact with computer system/server512, and/or any devices (e.g., network card, modem, etc.) that enable computer system/server512to communicate with one or more other computing devices. Such communication can occur via Input/Output (I/O) interfaces522. Still yet, computer system/server512can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and/or a public network (e.g., the Internet) via network adapter520. As depicted, network adapter520communicates with the other components of computer system/server512via bus518. It should be understood that although not shown, other hardware and/or software components could be used in conjunction with computer system/server512. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.