HICO-DET-SG and V-COCO-SG: New Data Splits to Evaluate Systematic Generalization in Human-Object Interaction Detection
Abstract
Human-Object Interaction (HOI) detection is a task to predict interactions between humans and objects in an image.
In real-world scenarios, HOI detection models are required systematic generalization, i.e., generalization to novel combinations of objects and interactions, because it is highly probable that the train data only cover a limited portion of all possible combinations.
However, to our knowledge, no open benchmark or existing work evaluates the systematic generalization in
HOI detection. To address this issue, we created two new sets of HOI detection data splits named HICO-DET-SG and V-COCO-SG based on HICO-DET and V-COCO datasets.
We evaluated representative HOI detection models on the new data splits and observed large degradation in the test performances compared to those on the original datasets.
This result shows that systematic generalization is a challenging goal in HOI detection.
We hope our new data splits encourage more research toward this goal.
Introduction
Human-Object Interaction (HOI) detection has been attracting large interest in computer vision, as it is useful for various applications such as self-driving cars, anomaly detection, analysis of surveillance video, and so on.
The task is to detect humans and objects as well as interactions between them and the output is typically represented as <human, interaction , object> triplet.
After some datasets [1–6] were published, a large number of studies has tackled this problem [7–20].
HOI detection is an advanced computer vision task as it requires a model not only to detect humans and objects but also to predict interactions between them.
Moreover, humans can have different interactions with the same object (e.g., wash a horse and walk a horse) and the same interaction occurs for different objects (e.g., wash a horse and wash a car).
In real-world scenarios, it is highly probable that the train data only cover a limited portion of all possible combinations of objects and interactions.
Thus, it is important for HOI detection models to generalize to novel combinations of known objects and interactions. [Figure]
[Figure] [Figure] This type of generalization to novel combinations of known concepts is called systematic generalization and attracts increasing interest in recent years as it is a highly desirable property for AI systems.
Systematic generalization performance is evaluated in various tasks such as sequence-to-sequence parsing [21], language understanding [22, 23], visual properties extraction [24] and visual question answering [25–29].
However, to our knowledge, there is no open benchmark or existing work to evaluate systematic generalization in the HOI detection.
The existing HOI detection datasets cannot evaluate the systematic generalization for the novel combinations of known objects and interactions since their train and test data provide the same object-interaction combinations.
This property of the datasets may lead the model to predict the interactions depending just on the paired object classes or to predict object classes by just interactions rather than by capturing the visual cues such as human posture and positional relationships.
In this paper, we introduce two new sets of HOI detection data splits named HICO-DET-SG and
V-COCO-SG, which we created based on HICO-DET [4] and V-COCO [3] datasets to evaluate the systematic generalization (SG) capabilities of HOI detection models.
An example of such data split to evaluate systematic generalization in the HOI detection is shown in Figure 1.
In order to make sure that the test performance is not an artifact of a specific selection of combinations in the train and test data, we prepared three distinct train-test splits for both HICO-DET-SG and VCOCO-SG.
We evaluated recent representative HOI detection models on our data splits and observed large degradation in the test performances compared to those on the original datasets.
In summary, our contributions are the following:
• We created two new sets of HOI detection data splits whose train and test data have no overlapping object-interaction combinations, which serve for studying systematic generalization in HOI detection.
• We evaluated the systematic generalization performance of representative HOI detection models with our new data splits and revealed that there are large decreases in the test performance compared to the original datasets, i
e., systematic generalization is a challenging goal in HOI detection.
[Table]
HICO-DET-SG, V-COCO-SG, and the source code to create them is publicly available at https: //github.com/FujitsuResearch/hoi_sg .
HICO-DET-SG and V-COCO-SG 2.1 Creation of SG splits
The train and test data of the Systematic Generalization (SG) splits are designed not to have overlapping object-interaction combination classes, thus the HOI detection models are required to generalize to novel combinations.
For example, in Figure 1, the train data consists of combinations such as <human, wash , car>, <human, wash , elephant>, <human, walk , horse> and <human, straddle , horse>.
The HOI detection models are required to generalize to novel combinations in the test data such as <human, wash , horse>.
We make sure that, in the train data, every object class is paired with multiple interaction classes, and every interaction class is paired with multiple object classes.
This design of the splits enables the model to learn the concept of object/interaction, not depending on the specific paired interaction/object class.
To ensure that the test performance is not an artifact of a specific selection of combinations in the train and test data, we prepared three distinct train-test splits.
We eliminate triplets in the test data whose combination of classes is contained in the train data.
As a result, the SG splits contain less HOIs in total than the original.
See Appendix B for further details of the creation process of SG splits.
2.2 Statistics of HICO-DET-SG and V-COCO-SG
Based on HICO-DET dataset [4] we created HICO-DET-SG, new data splits to evaluate the systematic generalization performance in the way explained above.
The statistics of the original HICO-DET and HICO-DET-SG data splits are shown in Table 1 (upper half).
The original HICO-DET dataset contains 80 classes of objects, 117 classes of interactions, and 600 classes of object-interaction combinations.
In HICO-DET-SG splits, 540 object-interaction classes out of 600 are contained only in the train data, and the test data consists of 60 remaining classes.
Based on V-COCO dataset [3] we created V-COCO-SG, new data splits to evaluate the systematic generalization performance in the same manner.
The statistics of the original V-COCO and V-COCOSG are shown in Table 1 (lower half).
The original V-COCO dataset contains 91 classes of objects, 29 classes of interactions, and 228 classes of object-interaction combinations.
In the V-COCO-SG splits, 160 object-interaction classes out of 228 are contained only in the train data, and the test data consists of 68 remaining classes. [Table] [Table] [Table]
original paper)
[Table]
Evaluation results
In this section, we report systematic generalization performance of four representative HOI detection models, HOTR [14], QPIC [16], QAHOI [15] and STIP [20], on HICO-DET-SG and V-COCO-SG.
We trained and tested each model once on each split.
See Appendix C for the details of these HOI detection models and other experimental setup.
Further results and discussions are available in
Appendix D.
3.1 Degradation in the systematic generalization performance
We show the results on the HICO-DET-SG in Table 2 and the results on the V-COCO-SG in Table 3.
As the evaluation metrics, we use mean average precision (mAP), which is higher the better.
In order to evaluate all the models on the equal condition in terms of pre-training, we report the results of
HOTR, QPIC and STIP for both pre-trained (with object detection task on MS COCO dataset [30]) and not pre-trained encoder and decoder.
The mAPs of all models significantly decrease on all systematic generalization splits compared to the ones on the original splits.
This shows the difficulty of systematic generalization in HOI detection task, i.e., recognizing novel combinations of known objects and interactions.
Note that this degradation occurs at any selection of object-interaction combinations in the train and test data: the differences in the test mAPs are less than 3% among three systematic 3.2 Qualitative analysis
In order to further reveal the difficulty of systematic generalization in HOI detection, we analyze the failure cases here.
Figure 2 (a), (b) and (c) show the outputs of pre-trained STIP trained and tested on
HICO-DET-SG split3, which has the highest mAP among all models and all SG splits.
Figure 2 (a) shows an example of predicting wrong interaction class, which is the most frequently observed type of errors.
In this example the model predicts the interaction as straddle , even though the correct class is wash . The <human, straddle , horse> triplet appears in the train data but the <human, wash , horse> triplet appears only in the test data ( wash interaction appears with other objects in the train data). The model predicts the interaction depending just on the object class (horse), i.e., the model cannot generalize to the novel combination <human, wash , horse>. Figure 2 (b) shows an example of detecting wrong object.
The model predicts an irrelevant region as a wrong class, bench, even though it should detect a bed under the person in the image.
The <human, lie on , bench> triplet appears in the train data but the <human, lie on , bed> triplet appears only in the test data (bed appears with other interactions in the train data).
This result shows not only that the model cannot generalize to the novel combination but also that the interaction decoder predicts the lie on interaction depending mainly on the visual cue of the human posture, not on the visual cue of the object nor on the positional relationships.
Figure 2 (c) shows an example of wrong class prediction for both objects and interaction.
The model predicts the tennis racket as a baseball bat and predicts swing as hit .
The <human, hit , baseball bat> triplet appears in the train data but the <human, swing , tennis racket> triplet appears only in the test data (moreover, <human, swing , baseball bat> triplet appears in the train data but <human, hit , tennis racket> triplet does not appear in the train data).
The model detects the object as a baseball bat at the first stage, and then at the second stage it predicts the interaction as hit based on the detection results because baseball bat frequently appeared with the hit interaction in the train data.
[Figure]  Conclusion We created new splits of two HOI detection datasets, HICO-DET-SG and V-COCO-SG, which are designed not to contain overlapping combinations of object-interaction classes in the train and test data for evaluating systematic generalization.
We observed large degradation in the test performances on our SG splits compared to those on the original datasets for all the representative HOI detection models we evaluated.
This shows that systematic generalization is a challenging goal in HOI detection.
We hope our data splits encourage more research toward this goal.
A Related works
In Section A.1, we introduce some types of HOI detection datasets and models.
In Section A.2, we explain systematic generalization.
A.1 Human-Object Interaction (HOI) detection
There are two popular datasets for HOI detection: HICO-DET and V-COCO datasets.
HICO dataset was originally created for classifying objects and interactions in the image (no bounding-box) [32].
Later HICO-DET dataset was created based on HICO dataset by putting bounding-boxes of humans and objects in the image [4].
Also, at this point, one image in the dataset has become to contain multiple humans, objects and interactions between them.
V-COCO dataset [3] was created based on Microsoft COCO (MS COCO) dataset [30] by adding annotations of interactions (verbs).
Statistics of HICO-DET and V-COCO dataset are shown in Table 1.
HOI detection consists of two tasks.
The first task is to localize human and object instances in the given image.
The second task is to identify the interactions between them.
Due to this property, there are two types of model architectures to solve HOI detection: two-stage model and one-stage model.
Two-stage models [7–11, 20] detect instances at the first stage and classify the interaction for all of their combinations in the second stage.
In order to improve both instance and interaction detectors via multi-task learning and to reduce inference time, one-stage models [12, 13] have been proposed recently and becoming more popular these days.
In response to the great success of Transformer [33] in both natural language processing and computer vision [34], some of the recent one-stage works [14–19] are based on Transformer as they are designed to capture wide-range information from an image.
A.2 Systematic generalization in HOI detection
Systematic generalization [21–23, 26–29] (also called as compositional generalization [25] and combinatorial generalization [24]) can be regarded as one of the Out-of Distribution (OoD) problem settings, i.e., the model needs to generalize to different data distributions from the train data.
Scene Graph Generation (SGG) [35] is a task closely related to HOI detection. In SGG, there are some works to evaluate [36] and improve [37, 38] systematic generalization for new combinations of subject, relation and object classes under the name of zero-shot generalization. They all showed that there are large performance degradation in the systematic generalization compared to the original settings (in-distribution generalization) unless some techniques are intentionally used. There are two main differences between SGG and HOI detection.
First, the subjects in SGG can be of any type (humans, cars, etc.), while in HOI detection they are fixed as humans, which results in more various subject-object combinations in SGG.
Second, the predicates in SGG can be both positional relations (e.g., next to) and semantic actions (e.g., play with), while in HOI detection they only consist of the latter.
Considering these differences, HOI detection can be regarded as a subset of SGG with less various subject-object combinations. On the other hand, HOI detection can be regarded as a task focusing on measuring a model’s capability for complex scene understanding, because recognition of semantic actions needs more information in addition to the locations of humans and objects. Thus we believe that it is essential to work on systematic generalization in HOI detection as an early step towards the development of better models also in SGG and other visual understanding tasks. In the HOI detection task, HICO-DET dataset [4] provides rare-triplets evaluation to measure the few-shot generalization ability of the models (rare triplets are defined to appear less than 10 times in the train data).
Generalization to the rare-triplets is a type of OoD generalization and some works [39, 40] aim at improving this performance.
However, to our knowledge, there have been no benchmarks or previous works to tackle systematic generalization, i.e., zero-shot generalization in HOI detection, and our work is the first one providing the data splits to evaluate the systematic generalization performance and benchmarking the representative HOI models. train_data = [] test_data = [] for scene in Dataset do sum =
test_hois =
[] for hoi in scene.hois do match = COUNT(test_combinations, [hoi.object_class, hoi.interaction_class]) sum
= sum + match if match > 0 then test_hois.append(hoi) end if end for if sum == length(scene.hois) then test_data.append(scene) else if sum = = 0 then train_data.append(scene) else test_data.append([scene.image, test_hois]) end if end for B Further details of creating systematic generalization splits
In this section, we explain how we created systematic generalization (SG) splits for HICO-DET and
V-COCO.
First, we set the number of object-interaction combination classes to be contained in the train and test data.
We tried to match the ratio of them to that of the number of HOI triplets in the original train and test data at the beginning, but eventually more combination classes (540 in HICO-DET-SG andin V-COCO-SG) are contained in the train data in order to ensure that every object class is paired with multiple interaction classes and every interaction class is paired with multiple object classes.
This enables the model to learn the concept of object/interaction itself, not depending on the specific paired interaction/object.
Then, we created SG splits as described in Algorithm 1.
We eliminated triplets in the test data whose object-interaction combinations are contained in the train data.
Because of that, SG splits contain less
HOI triplets in total than the original.
Finally, we verified that all object and interaction classes are actually contained in the train data, even though some combinations are only contained in the test data and unseen in the train data for evaluating systematic generalization.
To ensure that the test performance is not an artifact of a specific selection of combinations in the train and test data, we prepared three distinct train-test splits.
The actual source code to create SG splits can be seen at the following repository: https://github. com/FujitsuResearch/hoi_sg .
C Experimental setup for evaluating representative HOI detection models
In this section, we describe the experiments to evaluate the systematic generalization ability of representative HOI detection models.
In Section C.1, we first explain the evaluated models and the reasons for selecting them.
Then we explain experimental setup in Section C.2.
[Table]
C.1 HOI detection models
We tested systematic generalization performance for four HOI detection models, HOTR [14],
QPIC [16], QAHOI [15] and STIP [20].
The comparison of the four models is shown in Table 4.
The selected models except STIP adopt one-stage architecture as it is popular recently.
The backbone (feature extraction) network of each model is all pre-trained with object detection task on MS COCO dataset [30].
We used the source code of these models taken from the official publicly-available repositories that we show URLs in the reference section [Table]
HOTR.
“Human-Object interaction detection TRansformer” (HOTR) [14] is one of the first
Transformer-based models for the HOI detection.
The model is one-stage parallel architecture and it has a backbone, shared encoder, instance (human + object) decoder, and interaction decoder.
The backbone network is CNN-based to extract features from images.
In order to get the matching between instance and interaction decoder outputs, three independent feed-forward networks, which are so-called HO-pointers, are trained to predict correct <human, interaction , object> combination matching.
Most of the network except HO-pointers is based on “DEtection TRansformer” (DETR) [31], a Transformer-based object detector.
Therefore we can pre-train backbone, shared encoder, instance decoder, and interaction decoder with the DETR’s weights trained on the MS
COCO dataset.
QPIC.
“Query-based Pairwise human-object interaction detection with Image-wide contextual information” (QPIC) [16] is another Transformer-based HOI detection model that is proposed around the same time as HOTR.
It is also based on DETR for most of the network.
The main difference from HOTR is that QPIC is a one-stage end-to-end architecture that has only one decoder without
HO-pointers.
We can use pre-trained weights of DETR on MS COCO datasets for most of the network, including backbone, encoder, and decoder.
QAHOI.
“Query-based Anchors for Human-Object Interaction detection” (QAHOI) [15] exhibits the best performance on HICO-DET dataset as of the submission time on Papers with Code 1 .
The model is based on deformable DETR [41] (a modified DETR), which computes self-attention from a limited range of feature maps for computational efficiency.
This enables to extract multi-scale feature maps from the image.
There is another technique to reduce computational costs in the encoder and decoder.
The encoder is trained to generate query-based anchors which represent the points with high objectness score in the image.
The decoder predicts objects and interactions only on those anchors for efficient computation.
Although the backbone of the network can be pre-trained because it is based on Swin Transformer [42], it is impossible to pre-train all the encoder and decoder because there are some modifications to deformable DETR (object detection model).
This is the reason why we do not report accuracies of pre-trained QAHOI in Tables 2 and 3.
STIP.
“Structure-aware Transformer over Interaction Proposals” (STIP) [20] exhibits the best performance on V-COCO dataset as of the submission time on Papers with Code 2 .
The model has two-stage architecture to perform HOI set prediction from non-parametric interaction queries detected by the independent instance detector.
This enables the model to explore inter-interaction and intra-interaction structure from early epochs of the training by fixing the correspondence between interaction query and each target HOI.
We can use pre-trained weights of DETR on MS COCO datasets for the backbone and object detector because the first stage of the network is exactly the same as DETR.
C.2 Pre-training, hyperparameters, and other conditions
In order to evaluate all the models on the equal condition in terms of pre-training, we report the results of HOTR, QPIC and STIP for both pre-trained and not pre-trained encoder and decoder, despite that the original papers encourage to use pre-trained weights for the best performance in the original HOI detection task.
We note that for all of our experiments, the backbone of each model, i.e., feature extraction part, is pre-trained with the same weights as the original experiments.
For all of the models, we used almost the same hyperparameters as reported in the original papers and repositories.
One change is the batch size of QAHOI: we reduced it from 2 to 1, because we got a memory error while training with the default setting.
In addition, we trained QAHOI on V-COCO and V-COCO-SG with the same hyperparameters as HICO-DET and HICO-DET-SG, because the original paper does not report hyperparameters for V-COCO dataset.
We trained and tested seven types of model once on each split, and one training took about 1 todays with 4 NVIDIA V100 GPUs.
D Further results and discussions
D.1 Difference between HICO-DET-SG and V-COCO-SG
Comparing two datasets, there are larger gaps between the original and the systematic generalization splits of V-COCO compared to HICO-DET.
We speculate that the cause of this gap is the difference in the number of images and HOIs in the dataset.
As shown in Table 1, HICO-DET-SG train data contains about 5.6 times as many images and about 12.6 times as many HOIs as the train data of
V-COCO-SG.
This means that there are more examples for one object-interaction combination in
HICO-DET-SG train data, although the data has only 3.4 times as many variety of object-interaction combination classes.
Therefore the models tend to achieve higher systematic generalization on
HICO-DET-SG compared to V-COCO-SG.
D.2 Comparison across models
Each model showed different performance on systematic generalization splits.
HOTR could not generalize at all as they achieved almost 0% mAP when its encoder and decoder are not pre-trained with object detection task, and achieved less than 3% mAP even with its encoder and decoder pretrained.
QAHOI also achieved low mAPs around 5% to show its inability to systematic generalization.
However, QPIC generalizes to novel combinations to some extent especially when using pre-trained
DETR weights: they achieved around 20% mAPs on HICO-DET-SG splits.
STIP showed the best SG performance among all the models on both HICO-DET-SG and V-COCO-SG with pre-training.
This may be because STIP has two-stage architecture.
Instance and interaction detector is less affected by each other as they are independent.
In other computer vision tasks, this type of modularity is proved to improve systematic generalization ability [29].
D.3 Importance of pre-trained encoder and decoder
For three models except QAHOI, pre-training its encoder and decoder with the weights of DETR trained on object detection task plays a key role in improving systematic generalization performance.
The accuracies on HICO-DET changed from around 0.4% to around 2.7% for HOTR, from around 12.1% to around 20.8% for QPIC and from 0.0% to around 23.2% for STIP.
Also the accuracies on
V-COCO changed from around 0.2% to around 2.1% for HOTR, from around 0.6% to around 3.8% for QPIC and from around 0.0% to 6.3% for STIP.
Generally vision Transformers require a lot of training to achieve high performance when trained from scratch [34].
So it is natural that without pre-training, the Transformer-based HOI detectors cannot solve merely HOI detection task nor systematic generalization.
Especially STIP performed quite badly without pre-training because the number of training epochs is much smaller (30 epochs) than the others (100 epochs).
For the three models, pre-trained initial weights of encoder and decoder contributes to improving systematic generalization accuracy as well as HOI detection task itself.
In this paper we could not evaluate pre-trained QAHOI because the network is designed only for HOI detection task and it is hard to obtain pre-trained weights from other tasks such as object detection.
If we modify the final part of QAHOI so that it can be trained with other tasks, we may be able to get better systematic generalization score with the pre-trained weights.
D.4 Evaluation on the train data
In order to make sure that the models are well-trained, we evaluated accuracies on the train data.
In Table 5 and Table 6, we show mAP accuracies on the train data as well as the test data for
HICO-DET-SG and V-COCO-SG, respectively (the right half of the tables are equivalent to Tableand Table 3).
For both datasets and for all the models, mAPs on the train data of SG splits are coequal to or slightly better than the original splits.
This is probably because the train data of SG splits contain less variety of triplets: in HICO-DET-SG there are only 540 combinations of objects and interactions out of 600 combination contained in the original dataset, and in V-COCO-SG there are only 160 out of 228.
T a b l e 5 : C
o m p a r
i s on o f s y s t
e m a ti c g e n e r a li za t i on p e rf
o r m a n ce s on
H
I C
O - D
ET - S
G d a t a s p lit s .
W e s ho w m A P s ( % ) f
o r bo t h t r a
i n a nd t e s t d a t a , w h
i c h a r e h
i gh e r t
h e b e tt e r ( t h e r
i g h t
h a l f o f t h
i s t a b l e i s e qu i v a l e n t t
o
T a b l e 2 ) .
T h e v a l u e s r e po r t e d i n t
h e
o r i g i n a l p a p e r s a r e w r it t e n i n b r ac k e t s .
T h e t e r m “ p r e - t r a i n ” m ea n s t h a t t h e i n iti a l w e i gh t s o f t
h
e m od e l ’ s e n c od e r a nd d ec o d e r a r e c op i e d fr o m
D ET
R [ 31 ] t r a i n e d on ob j ec t d e t ec ti on .
T e s t a cc u r ac
i e s s i
gn
i fi ca n tl y d ec r ea s e d on
H
I C
O - D
ET - S
G c o m p a r e d t o o r
i g i n a l
H
I C
O - D
ET .
E v a l u a ti on on t r a i n d a t a ( r e f e r e n ce )
E va l u a t
i o n o n t e s t d a t a ( m a i n )
HO T
R
Q P
I
C
QAHO
I
S T I P
HO T
R
Q P
I
C
QAHO
I
S T I P p r e - t r a i n ✗ ✓ ✗ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✗ ✓
O r i g i n a l
H
I C
O - D
ET 30 .44 .
[Table] [Table]
[Table] [Table]
( m
A P i n t
h
e
o r i g i n a l p a p e r) ( 25 .
[Table]
[Table] [Table]
[Table] [Table]
H
I C
O - D
ET - S
G s p lit33 .46 .
[Table]
[Table] [Table]
H
I C
O - D
ET - S
G s p lit31 .42 .
[Table]
[Table] [Table]
H
I C
O - D
ET - S
G s p lit32 .44 .
A v e r a g e o v e r
S G s p lit s 32 .44 .
T a b l e 6 :
C o m p a r i s on o f s y s t
e m a ti c g e n e r a li z a ti on p e rf o r m a n ce s on
V - C
O C
O - S
G d a t a s p lit s .
W e s ho w m A
P s ( % ) f
o r bo t h t r a
i n a nd t e s t d a t a , w h
i c h a r e h
i gh e r t
h e b e tt e r ( t
h e r i gh t
h a l f o f t h
i s t a b l e i s e qu i v a l e n t t
o
T a b l e 3 ) .
T h e v a l u e s r e po r t e d i n t
h e
o r i g i n a l p a p e r s a r e w r itt e n
i n b r ac k e t s .
“ p r e - t r a i n ” m ea n s t h a t t h e i n iti a l w e i gh t s o f t
h e m od e l ’ s e n c od e r a nd d ec od e r a r e c op i e d f r o m
D ET
R [ 31 ] t r a i n e d on ob j ec t d e t ec ti on .
T e s t acc
u r a c
y s i
gn
i fi ca n tl y d ec r ea s e d on
V - C
O C
O - S
G d a t a s p lit s c o m p a r e d t o o r
i g i n a l
V - C
O C
O f o r a ll t
h e m od e l s .
E v a l u a ti on on t r a i n d a t a ( r e f e r e n ce )
E va l u a t
i o n o n t e s t d a t a ( m a i n )
HO T
R
Q P
I
C
QAHO
I
S T I P
HO T
R
Q P
I
C
QAHO
I
S T I P p r e - t r a i n ✗ ✓ ✗ ✓ ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✗ ✗ ✓
O r i g i n a l
V - C
O C
O 28 .64 .
( m
A P i n t
h
e
o r i g i n a l p a p e r) ( 63 .
V - C
O C
O - S
G s p lit30 .65 .31 .67 .45 .23 .71 .0 .2 .
V - C
O C
O - S
G s p lit31 .67 .
V - C
O C
O - S
G s p lit28 .
A v e r a g e ov e r
S G s p lit s