SlowGuess/NeurIPS_2023_OCR huggingface dataset:
CVP Complements Other Test-Time Adaptation Methods Since our method modifies the input space and aligns the representation of adapted samples to the original manifold, it also complements established test-time adaptation approaches which adjusts the model weights. Thus, we combine the CVP with several existing methods, including MEMO [71], BN [55], and TENT [62]. For a fair comparison, we set up the batch size as 16 for all experiments. for the parameter settings, we set the kernel size for CIFAR-10-C as  $3 \times 3$  and ImageNet-C, R, S, and A as  $5 \times 5$ . The adaptation iterations
CIFAR-10-C
Avg. Error (%)	ImageNet-C
mCE ↓	ImageNet-R
Error (%)	ImageNet-S
Error (%)
CLIP(ViT/32)	58.39	77.93	32.09	60.55
SVP (patch)	58.43 (+0.04)	77.81 (-0.12)	32.12 (+0.03)	60.53 (-0.02)
CVP-F3	57.94 (-0.45)	77.43 (-0.50)	31.16 (-0.93)	59.47 (-1.08)
CVP-R3	57.91 (-0.48)	77.71 (-0.22)	31.31 (-0.78)	59.62 (-0.93)
CVP-F5	57.98(-0.41)	77.25 (-0.68)	31.14 (-0.95)	59.83 (-0.72)
CVP-R5	57.79 (-0.60)	76.67 (-1.26)	30.43 (-1.66)	59.43 (-1.12)
Table 3: Evaluation on the CLIP model. We compare CVP with other prompting baselines on CIFAR-10-C, ImageNet-C, ImageNet-R, and ImageNet-S. Overall, the  $CVP - R5^{\dagger}$  achieves the best performance, which reduces the error rate by  $1.16\%$  on average.
CIFAR-10-C
Avg. Error (%)	ImageNet-C
mCE ↓	ImageNet-R
Error (%)	ImageNet-S
Error (%)	ImageNet-A
Error (%)
Standard	58.24	76.87	63.83	75.90	100.0
Mao et al. [40]	57.94 (-0.3)	76.74 (-0.13)	68.86 (+5.03)	75.93 (+0.03)	99.94 (-0.06)
CVP (ours)	52.37 (-5.87)	75.43 (-1.49)	63.06 (-0.77)	75.30 (-0.6)	98.4 (-1.6)
MEMO [71]	56.14	73.45	60.73	73.43	99.1
MEMO + CVP	54.84 (-1.3)	72.02 (-1.43)	60.23 (-0.5)	72.67 (-0.76)	98.64 (-0.46)
BN [55]	38.51	76.20	67.29	77.98	99.8
BN + CVP	37.39 (-1.12)	76.16 (-0.04)	67.21 (-0.08)	77.92 (-0.06)	98.67 (-1.13)
TENT [62]	38.52	70.45	58.45	73.88	99.7
TENT + CVP	36.69 (-1.83)	70.34 (-0.11)	58.42 (-0.03)	73.83 (-0.05)	98.54 (-1.16)
are all set as 5. In Table 4, we show the results on five benchmarks. For CIFAR-10-C, CVP improves 1.83 points on top of TENT and reduces the error rate by  $21.55\%$  compared with the standard one. For the other datasets, CVP achieves the lowest error rate on top of the TENT method. However, the BN method degrades the performance under the small batch setting. Due to the page limit, in Appendix 7.5, we show more evaluations on other benchmark, such as the Cutout-and-Paste data [61] and other baseline Test Time Training (TTT) [59].
Low-Rank Structure to Avoid Shortcut in SSL Objective. Since SSL objective is a proxy for our visual recognition task, sole minimizing SSL objective can produce a overfitted prompt without much improvement in visual recognition. A typical way to avoid this suboptimal solution is by adding the right inductive bias. For the highly structured visual data, we study the best inductive bias we can add to prevent the shortcut during adaptation.
In addition to convolution, we also investigated the popular low-rank structure [24, 67]. We create visual prompts with low-rank structure (LVP) via singular value decomposition (SVD), and optimize the low-rank matrices using our algorithm (shown in Appendix 7.2). We compare the effectiveness of low-dimensional prompts and convolution on their ability to reverse natural corruptions. In Table 6 and Figure 3, for both LVP and CVP, increasing the prompt's rank leads to degraded performance in reverse the corruption to produce clean images, where the  $\ell_2$  norm distance between real shifting and the approximated shifting increase.
Table 4: Our prompt method complements other test-time adaptation approaches that update model weight, including MEMO, TENT, and BN. We show complements gain on every baseline when combined with CVP. Here, the Standard for CIFAR-10-C is WideResNet18 and other dataset is ResNet50. For CIFAR-10-C, on top of the TENT [62] method, we achieve the best gain on the performance, which reduces  $1.83\%$  error rate.
CIFAR-10-C
Avg. Error	ImageNet-C
mCE	ImageNet-R
Error	ImageNet-S
Error
Standard	58.24	76.87	63.83	75.90
SSL-VP	57.95 (-0.29)	76.74 (-0.13)	68.86 (+5.03)	75.93 (+0.03)
SSL-CVP	52.37 (-5.87)	75.34 (-1.53)	63.06 (-0.77)	75.30 (-0.60)
Sup-VP	42.02 (-16.22)	73.54 (-3.33)	60.17 (-3.66)	73.30 (-2.60)
Sup-CVP	49.59 (-8.65)	74.10 (-2.77)	62.60 (-1.23)	74.14(-1.76)
Table 5: Analysis on the shortcut of SSL. We compare the performance of SSL and supervised learning on VP and CVP for four benchmarks
Figure 3: The effect of structured prompt for reversing corruption. For the y-axis, we show the  $\ell_2$  distance between the ground-truth additive corruptions vector  $\Delta$  and our prompt reversing vectors  $\delta$ , where a small distance indicates our reversal is successful. For the x-axis, we show the rank for the prompt matrices and the convolutional kernel. For a convolutional kernel with size  $k \times k$ , its rank is  $k$ . We find a small rank help reverse the noise better. For the same rank, convolution is better than the additive matrix. This demonstrates the effectiveness of our convolution prompt over other low-rank structures in reversing natural corruptions.
(b)
Figure 4: (a.) Analysis of the effect of batch size for different baselines on CIFar-10-C. We show the average performance on severity 1 for every corruption type. When batch size is small, CVP has a better performance on all corruption types, compared to fine-tune and two VP methods. (b.) We show the analysis for self-supervised loss v.s. performance on VP and CVP for gaussian corruption type in CIFAR-10-C. We setup the norm-bound for VP as 8/255 or unbounded. The loss curve demonstrates that when increasing the iter. number from 1 to 30, adapting with VP without norm bound hugely over-fits the self-supervised loss. The loss of VP largely reduces, yet the performance does not improve. On the other hand, our CVP has a lower risk of over-fitting as the loss curve can reduce smoothly with accuracy gradually increasing.
CVP in Supervised Learning. CVP is necessary for our test-time adaptation because SSL is a proxy for our final task. There are solutions where SSL is minimized, but the final task performs suboptimal. We investigate the necessity of using CVP when optimized for objectives other than SSL, such as the supervised loss, where the training and evaluation objective matches. Hence, in Table 5, we show the VP and CVP under the supervised learning setting (Sup-VP and Sup-CVP) and SSL setting (SSL-VP and SSL-CVP). For all experiments, we use the same optimization setting and parameter settings as Table 4. We apply the prompts on every batch for the supervised setting and update them with ground truth. Due to training on the ground-truth label, both VP and CVP achieve higher accuracy than SSL, VP performs higher than CVP. The results demonstrate that convolution inductive bias is not necessary when ground truth is available. However, without the label, the structured convolutional prompt is necessary for improving performance.
Generalize CVP to other SSL tasks. We further show the CVP can be generalized to other self-supervision tasks. In Table 7, we compare three SSL tasks, including contrastive learning, rotation prediction and masked autoencoder (MAE) [15]. We show our CVP is effective under all SSL tasks. On ImageNet-C, CVP improves robust accuracy for every severity level (from s1 to s5) when optimized for all three SSL tasks. We find contrastive learning can perform better than rotation prediction, and MAE has the best performance.
The Effect of Batch Size and Number of Iteration for Adaptation. In Figure 4a, we empirically demonstrate that the batch size affects the performance of different prompt methods. We set up the batch sizes from 2,4,8,16 to 32 and compared the accuracy of CVP with Finetune, VP (padding), and VP (patch). When batch size is set as 2, the performance of fine-tune is only  $36.95\%$ , which is worse than CVP  $(71.53\%)$ . When increasing the batch size to 4 and 8, the performance of fine-tune, VP slightly improves, yet still worse than CVP. Overall, CVP has an advantage in adaptation under the
Standard	LVP_R3	S1	S2	S3	S4	S5
CIFAR-10-C	58.24	54.86 (-3.38)	Standard	58.82	48.25	38.65	27.15	17.57
ImageNet-C	76.87	76.42 (-0.45)	Contrast	59.53	48.84	39.82	28.55	19.53
ImageNet-R	63.83	63.57 (-0.26)	Rotation	58.97	48.30	38.88	27.50	17.99
ImageNet-S	75.90	75.69 (-0.21)	MAE	60.94	51.10	41.79	30.11	19.92
Figure 5: Visualization. From left to right we show three kinds of corruption types, including contrast, fog, and frost, on ImageNet examples. By applying our convolutional prompt on the corrupted images, our method can partially remove the corruptions and make the image easier to recognize. In addition, the saliency map calculated from Grad-Cam also shows that our approach instructs the model to look at a similar region as the original one. This highlights why our convolutional can adapt the input for robustness.
small batch setting. In Figure 4b, we show the performance on different numbers of iterations for adaption. We empirically found that when increasing the iterations, CVP has a lower risk to overfit on the self-supervised objective. More ablation studies, such as different prompt designs can be found in Appendix 7.7
Visualization of Saliency Map To better understand how CVP adapts to the corrupted inputs, we visualize the saliency map of different types of corruption. As Figure 5 shows, from left to right, the first row is the original, corrupted, and adapted samples; the second row shows their corresponding Grad-CAM with respect to the predicted labels. The red region in Grad-CAM highlights where the model focuses on target input. We empirically discover the heap map defocuses on the target object for corrupted samples. After CVP, the red region of the adapted sample's heap map is re-targeted on a similar part as the original image, demonstrating that the self-supervised visual prompts indeed improve the input adaptation and make the model refocus back on the correct areas. We provide more visualization in Appendix 7.10.
Training Cost vs. Different Kernel Size In Table 8, we evaluate different kernel sizes for CVP and empirically discover that increasing the kernel to a proper size can improve the performance slightly. We choose one corruption-type impulse noise and show the results. When increasing the kernel size, the optimization cost increases. For impulse noise, kernel size  $7^{*}7$  achieves the best robust accuracy, yet the optimization cost is much higher.
Table 6: Performance of low-rank visual prompt. We show the average error rate on four benchmarks. For CIFAR-10-C, we use WideResnet as the standard model. For ImageNet-C, R, and S, we use the ResNet50.
Table 7: Performance of CVP on three SSL tasks for ImageNet-C, including contrastive learning, rotation prediction, and MAE. We compare them with the Standard ResNet50 and show the averaged accuracy for 15 corruption types on every severity.
Kernel Size	Accuracy (%)	Training Cost/Batch
3*3	16.22	0.67s
5*5	16.3	0.68s
7*7	16.62	1.24s
13*13	16.61	1.28s
21*21	16.52	1.29s
25*25	15.4	1.32s
Table 8: Training Cost v.s. Different Kernel Size
Training Time v.s. Number of Adapt Iteration In Figure 4(b), we have shown the CVP trained under different adapt iterations v.s. their performance. When increasing the number of adapt iterations, the training time increases. The following Table 9 shows the result of CIFAR-10-C on gaussian noise type with severity 1. We compare the accuracy and per batch training time on several numbers of
adapt iters (from 0 to 20). We empirically found that CVP has a larger performance gain than VP while adapting with a few epochs (epoch number 1).
# of Adapt Iters	0	1	5	10	15	20
Cost/Batch	0.00s	0.17s	0.67s	1.29s	1,.92s	2.57s
CVP Acc.(%)	39.51	51.09	56.1	58.76	59.30	59.58
Does CVP Reverse Corrupted Images Back to Normal One? We do the quantitative measurement on the distribution distance via Sliced Wasserstein Distance (SWD) and structural similarity index measure (SSIM). We measure the distance between two input distributions: source domain distribution and target domain distribution (before/after CVP adaptation). To calculate the distance between two input distributions via the Sliced Wasserstein Distance, we first obtain a group of marginal distributions from a high dimensional probability distribution via the linear projection, then calculate the  $p$ -Wasserstein Distance for those marginal distributions. Table 10 and Figure 6 shows the result of SWD on CIFAR-10-C with severity 1. On average, CVP achieves lower SWD after adaptation, which means the target distribution is closer to the source one after adaptation. The average SWD reduce by  $0.7\%$  after prompting. In Appendix Table 18 and Figure 9, we show the more detailed analysis.
Table 9: Training Time v.s. Number of Adapt Iteration
SWD (scale: 102) ↓	SSIM ↑
before	after	before	after
Avg. Mean	7.19	6.49	0.7539	0.7884
Avg. Std	4.05	2.79	0.1294	0.7260
Table 10: Results of SWD and SSIM on CIFAR-10-C (Severity 1).
(a) Noise Group
(b) Blur Group
Figure 6: Violin Plot of SWD for different corruption groups on CIFAR-10-C (severity 1). The left figure of each subplot shows the SWD before adapting, and the right shows the SWD after CVP adaptation. CVP brings the corrupted image distribution back to the clean image distribution.
Self-supervised convolutional visual prompt (CVP) is a novel method for test-time adaptation of OOD samples. In contrast to prior works training the visual prompts with the label, CVP is label-free and lightweight. It reduces the trainable parameters to less than  $1\%$  parameters of previous visual prompts and avoids the risk of overfitting when adapting for self-supervised objectives at test time. Results on five state-of-the-art benchmarks show that CVP improves model robustness by  $5.87\%$  and complements existing weight-adaptation methods. Extensive ablation studies suggest that distribution shifts are actually structured; therefore, CVP can capture the structures better than VP during the adaptation, which provides new insight into the frontier of visual prompting techniques and test-time adaptation. Future work includes interpreting convolutional prompts and prompting with multi-modality in large-scale foundation models.
This research is supported in part by a GE/DARPA grant, a CAIT grant, and gifts from Google and Accenture. We thank suggestions from Yi Zhang, Yow-Kuan Lin, and Raphael Jedidiah Sofaer.
7.1 Flow of CVP
We illustrate the whole flow of CVP in Figure 7. There are two phases in our proposed method: (1.) Offline one-time training on SSL model: The SSL is a simple MLP model, where it takes the logit features from pre-trained backbone as input and trains the model with contrastive learning task. (2.) CVP adaptation for any OOD benchmark during test time: We leverage the SSL loss from well-trained SSL model to optimize the convolutional kernel prompts for every upcoming OOD sample.
7.2 Algorithm
7.3 Baseline Details
Here, we show the detail of the baselines that we compare with.
- Standard: The baseline uses pre-trained model without adaptation. For CIFAR-10-C, the standard is trained with 50000 clean CIFAR-10 train dataset on WideResNet18 and ResNet. For ImageNet1K-C, the standard is trained with  $\sim 1.2\mathrm{M}$  clean ImageNet train dataset on ResNet50.
- SVP: The self-supervised visual prompts attempt to reverse the adversarial attacks by modifying the input pixels with  $\ell_p$ -norm perturbations, where the perturbations are optimized via contrastive loss [40]. For the patch setting, we setup the shape of VP as  $32*32*3$  for CIFAR-C and  $224*224*3$  for all ImageNet OOD datasets. For the padding setting, we set the padding size as 1 for CIFAR-10-C and 15 for ImageNet OOD dataset. Take CIFAR data as example, we first initialize a mask with all zeros value with the shape  $30*30*3$  and set the pad value as 1 with padding size 1 so that the mask after padding is as the same shape of CIFAR data ( $32*32*3$ ). Then, we multiply the mask with the VP to preserve only the VP located at the position we just pad with 1 value. We can further optimize the VP with mask by adding it with the corrupted samples.
- BN[55]: The model adaptation method aims to adjust the BN statistics for every input batch during the test-time. It requires to adapt with single corruption type in every batch.
- TTT [59]: The test-time training trains the model with an auxiliary SSL rotation task and leverages the rotation loss for model adaptation during the testing time. In TTT method, instead of adapting the whole model, they only adapt the last few layers of the model and freeze the parameters in the front layers.
- MEMO: The model adaptation method proposed in [71] alters a single data point with different augmentations (ie., rotation, cropping, and color jitter,...etc), and the model parameters are adapted by minimizing the entropy of the model's marginal output distribution across those augmented samples.
Algorithm 1: Convolutional Visual Prompt
Input: Pretrained classifier  $\mathcal{F}(\cdot)$  , OOD images  $x$  , Self-supervised objective function  $\mathcal{L}_s(\cdot)$  Convolutional operator  $\operatorname {Conv}(\cdot)$  , Convolutional kernel  $k$  , Learning rate  $\eta$  , Number of iteration T Output: Class prediction  $\hat{y}$  for adapted sample of  $x$    
1 Inference   
2 # Initialize the kernel parameters   
3  $k^0\sim \mathcal{U}\{(\alpha ,\beta)$    
4 # Calculate initial SSL loss   
5  $loss^{0} = \mathcal{L}_{s}(x)$    
6 for  $t\in \{1,\dots,T\}$  do   
7 # Generate adapted samples   
8  $x^{t} = x + \lambda *Conv(x,k^{t})$    
9 # Calculate SSL loss with adapted samples   
10  $loss^{t} = \mathcal{L}_{s}(x^{t})$    
11 # Update kernel parameters   
12  $k^{t + 1} = k^t +\eta \frac{\partial loss^t}{\partial k^t}$    
13 # Get optimal kernel parameters   
14  $k^{\star}\gets k^{T}$    
15 if  $loss^{T} > loss^{0}$  then   
16 # Use the initial kernel parameters   
17  $k^{\star}\gets k^{0}$  .   
18 # Get final adapted samples   
19  $x^{\star} = x + \lambda *Conv(x,k^{\star})$    
20 return  $\hat{y}\gets \mathcal{F}(x^{\star})$
Algorithm 2: Low-Rank Visual Prompt
Input: Pretrained classifier  $\mathcal{F}(\cdot)$  , OOD images  $X$  , Self-supervised objective function  $\mathcal{L}_s(\cdot)$  Rank Number  $r$  , Learning rate  $\alpha$  , Number of iteration  $\kappa$  Output: Class prediction  $\hat{y}$  for adapted sample of  $x$    
1 Inference   
2 # Initialize the matrices parameters   
3 SVD on input  $X(N*C*H*W)\rightarrow$  get n pairs of initial  $U\Sigma V^T$  (3 channel)   
4 for  $t\in \{1,\dots,K\}$  do   
5 # Get N inverse matrices   
6  $M_{t}\gets U_{t}\times \Sigma_{t}\times V_{t}^{T}$    
7 # Apply low-rank SVD on  $M_t$  with rank  $r$    
8  $U_{t}^{\prime},\Sigma_{t}^{\prime},V_{t}^{T^{\prime}} = SVD_{low - rank}(M_{t})$    
9 # Generate adapted samples with low-rank inverse matrices   
10  $M_t' = U_t'\times \Sigma_t'\times V_t^{T'}$    
11  $X_{t}^{\prime} = X_{t} + M_{t}^{\prime}$    
12 # Calculate SSL loss with adapted samples.   
13  $loss^{t} = \mathcal{L}_{s}(X_{t}^{\prime})$    
14 # Update the three matrices   
15  $U_{t + 1} = U_t + \alpha \frac{\partial loss^t}{\partial U_t},\quad \Sigma_{t + 1} = \Sigma_t + \alpha \frac{\partial loss^t}{\partial\Sigma_t},\quad V_{t + 1}^T = V_t^T +\alpha \frac{\partial loss^t}{\partial V_t^T}$    
16 # Get optimal low rank matrix   
17  $M^{\star}\gets U_K\times \Sigma_K\times V_K^T$    
18 # Get final adapted samples   
19  $X^{\star} = X + M^{\star}$    
20 return  $\hat{y}\gets \mathcal{F}(X^{\star})$
- TENT [63]: The method adapts the model by minimizing the conditional entropy on batches. In our experiment, we evaluate TENT in episodic mode, which means the model parameter is reset to the initial state after every batch adaptation.
7.4 Implementation details
For the training part of SSL model, we set the training parameters with batch size as 64, training epoch as 200, and the learning rate  $(lr)$  as 0.001. The  $lr$  is decayed with a cosine annealing for each batch [37]. The transformations for contrastive learning are predefined. We augment the inputs with random resize crop, random flip, and random rotation in degree [-90, 90] for positive/negative pairs generation in every batch. The number of transformations for one sample is set as 3.
For the test-time adaptation part, we set the range of parameter  $\delta$  for VP. For the  $\ell_2$ -norm perturbations, the  $\epsilon$  is  $[-8/255, 8/255]$  and the step size is  $2/255$ . We set the iteration number  $i$  either as 1 or 5, which means each component has 1 or 5 steps during PGD. For the update iterations, as the Table 17. show a larger number of iterations has better performance. However, the training cost becomes higher.
- The choice of hyper-parameter setting on  $\lambda$  ranges and kernel
For the  $\lambda$  parameter, it controls the magnitude of convolved output when combined with the residual input. We set the range to be [0.5, 3] and run test-time optimization to automatically find the optimal solution, which does not require a validation set.
We use 3x3 for CIFar-10 and 5x5 for ImageNet. In general, small kernel size is used to avoid overfitting. We increase the kernel size for large images, such as ImageNet.
When adapting, the kernel should be initialized with fix/random initialization. We use a sharpness kernel as an initial point for the fixed initialization setting, which is a  $n^*n$  matrix. It starts from specific values, which can control the edge enhancement effect by extracting the frequency information of inputs with a high pass filter. When the kernel size is 3, we set up the sharpness kernel as  $[0,-1,0], [-1,5,-1], [0,-1,0]$ . Similar to the convolutional kernel prompt, sharpness transforms an image using a convolutional 2D filter. It simultaneously enlarges the high-frequency part of images and then filters the low frequencies part.
CIFAR-10-C	ImageNet-C,R,S,A
Kernel Size	3*3	3*3 / 5*5
λ	[0.5, 1]	[0.5, 3]
Update iters.	1, 5 (default), and 10
Initialization	fixed / random
Table 11: parameter setting
- Number of Trainable Parameters: We compare the trainable parameters v.s. accuracy for different prompting methods. As Figure 8 shows, CVP contains less than  $0.2\%$  number of trainable parameters, compared to VP(patch).
7.5 More Evaluation
- We show more detailed results for CIFAR-10-C in Table 12 and 13.
Standard	BN	Finetune	VP	CVP
patch	append	fixed (3*3) w/o update	fixed (3*3) w/ update	rand (3*3) w/o update	rand (3*3) w/ update
Gaussian Noise	19.90	24.51	19.26	20.07	19.92	23.11	23.50	24.59	26.27
Shot Noise	20.37	24.25	19.43	20.56	20.40	22.95	23.32	24.29	25.26
Impulse Noise	27.44	26.14	19.19	27.54	27.44	30.76	30.98	31.13	31.08
Defocus Blur	12.90	14.54	13.39	13.59	13.18	16.81	17.41	18.85	20.03
Motion Blur	23.26	19.95	27.67	30.50	23.29	29.21	30.03	30.37	31.89
Glass Blur	25.97	33.30	18.61	19.70	26.01	36.14	37.54	37.42	40.51
Zoom Blur	71.08	50.46	42.42	71.07	71.06	86.76	87.65	85.76	88.19
Brightness	89.38	71.47	61.03	89.39	89.40	89.37	89.39	89.25	89.31
Snow	71.21	49.73	39.95	71.52	71.22	71.23	71.44	71.17	71.52
Frost	74.83	58.40	46.35	74.93	74.84	74.79	74.81	74.69	74.90
Fog	45.69	42.45	32.96	46.52	45.76	50.08	51.42	49.64	51.65
Contrast	58.36	49.87	39.22	57.95	58.47	67.74	68.99	69.05	70.21
Elastic Transform	17.54	24.01	19.95	17.72	17.56	17.39	17.62	18.07	19.66
Pixelate	23.45	39.80	34.26	24.10	23.50	25.91	26.47	28.39	30.58
Jpeg Compression	45.06	31.37	26.42	45.65	45.17	43.99	44.44	40.74	43.43
Avg. Acc.	41.76	37.35	30.67	42.05	41.82	45.75	46.33	46.23	47.63
Avg. Error	58.24	62.65	69.33	57.95	58.18	54.25	53.67	53.77	52.37
Avg Diff.	-	4.41	11.09	-0.29	-0.06	-3.99	-4.57	-4.46	-5.87