SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Xingyu Chen $^{1,*}$ , Weiyao Wang $^{1,*}$ , Hao Tang $^{1}$ , Matt Feiszli $^{1}$
FAIR, Meta<sup>1</sup>, Equal Technical Contribution*
{xingyuchen, weiyaowang, haotang, mdf}@meta.com
3D spatial understanding is highly valuable in the context of semantic modeling of environments, agents, and their relationships. Semantic modeling approaches employed on monocular video often ingest outputs from off-the-shelf SLAM/SfM pipelines, which are anecdotally observed to perform poorly or fail completely on some fraction of the videos of interest. These target videos may vary widely in complexity of scenes, activities, camera trajectory, etc. Unfortunately, such semantically-rich video data often comes with no ground-truth 3D information, and in practice it is prohibitively costly or impossible to obtain ground truth reconstructions or camera pose post-hoc.
This paper proposes a novel evaluation protocol, Object Reprojection Error (ORE) to benchmark camera trajectories; ORE computes reprojection error for static objects within the video and requires only lightweight object tracklet annotations. These annotations are easy to gather on new or existing video, enabling ORE to be calculated on essentially arbitrary datasets. We show that ORE maintains high rank correlation with standard metrics based on ground truth. Leveraging ORE, we source videos and annotations from Ego4D-EgoTracks, resulting in EgoStatic, a large-scale diverse dataset for evaluating camera trajectories in-the-wild.
Spatial understanding is a fundamental tool in human perception [49]. In computer vision, spatial understanding tasks often come in the form of geometric reconstruction and localization [62, 76, 74], typically seeking precise camera localization and 3D scene maps. The focus is on metric structure, not relations and objects. On the other hand, neurophysiology suggests human memory is anchored in object-centric or allocentric coordinates [10, 88, 26, 32]. This interpretation suggests a different granularity of spatial modeling and is relevant to applications in human-in-the-loop settings, like augmented reality. For example, equipped with moderately-accurate object-level spatial understanding, an AR assistant can give guidance to a human performing a task, describe object locations and spatial relations, and infer user intention.
Indeed, localizing an agent relative to objects and environment has become foundational in many recent approaches to semantic understanding, e.g. spatial semantic mapping [14, 34, 64, 77, 59, 79], place categorization [83], 3D scene graph construction [4, 69, 70], spatial episodic memory [5, 45, 37, 6, 48, 65, 101], visual query [42, 43, 57, 98, 56], planning and guiding visual navigation (e.g. for AR) [17, 2, 45, 15, 16], representation learning in egocentric vision [28, 50, 90, 60, 85], action anticipation [44, 68, 66] and active object segmentation [89]. These methods use spatial information at a higher semantic level; exact locations of pixels and cameras are less critical. They focus on the spatial semantics and relations of objects and cameras. In practice however, these approaches typically rely on careful 3D geometry, in the form of depth, camera trajectories and volumes, but often discard much of the accuracy in favor of looser notions of proximity (e.g. [4, 14, 34]).
Figure 1: EgoStatic Benchmark Visualization Top row: COLMAP, bottom row: DROID-SLAM. Colored bounding boxes are ground truth tracklet annotations; dots are reprojected points tracked by estimated camera trajectory. Ideally, the points should fall in the bounding boxes.
Unfortunately, many common datasets for semantic tasks have no 3D ground truth of any kind, and it is expensive or impossible to obtain it post-hoc. How well can the underlying systems localize agents, objects and their relations in these domains? Studies on real-world ego-centric videos have often been limited to synthetic data [96] or on scan-style/walking videos [21, 72] vs. real-world videos with activities (e.g. cooking, DIY, sports) with rich semantics. While prior work [43, 57, 89, 64, 104] notes SLAM/ SfM failures, e.g. on head-mounted videos, rarely is this quantified (nor is it typically possible). In other words, we have high-value problems whose solutions currently depend on systems which cannot be benchmarked in the domains of interest.
Contributions. This work proposes a novel, object-centric metric for camera trajectory quality on essentially arbitrary video (e.g. videos in Fig. 1). As a measure of environmental understanding, the resulting benchmark uses a type of reprojection error which is a generalization of geometric reprojection error; in this way it is similar to traditional SfM / SLAM benchmarks. However, using a sparse set of semantic landmarks, specifically static objects whose identities are unknown to the method under evaluation, keeps the benchmark focused on high-level percepts in the vicinity of the camera, vs. global maps and reconstruction accuracy.
Object Reprojection Error (ORE) relaxes the need for accurate groundtruth camera trajectories. Given an arbitrary video, one identifies a few "suitable" object candidates and annotates 2D bounding boxes across the frames where the object remains unmoved (Fig. 1). Despite no groundtruth camera trajectory, ORE's rank statistics agree well with standard GT-based metrics, such as Absolute Trajectory Error (ATE) and Relative Pose Error (RPE), when properly-selected tracklets are used.
Equipped with this schema, we source a wide variety of ego-centric videos from Ego4D, a large-scale egocentric video collected in-the-wild. We leveraged the long-term tracking annotations from EgoTracks [86] on Ego4D. Our contributions include:
1. We carefully design a new evaluation protocol for camera trajectory estimation which only requires static object tracklet annotation and no camera trajectory groundtruth: ORE.
2. We benchmark 7 SLAM, Visual Odometry (VO) and Structure-from-Motion (SfM) methods on Scannet test set [21] to compare ORE with standard metrics. Rank correlation shows high agreement between ORE and standard metrics.
3. We extend and adapt Ego4D-EgoTracks [86] benchmark into a first large-scale egocentric camera trajectory benchmark. The resulting benchmark is shown to be quite challenging.
4. Finally, ORE is a useful tool: it is sensitive enough to inform hyperparameter selection and method design. The experiments reveal potential directions for future work.
2.1 SLAM, SfM and Visual Re-localization
SLAM (Simultaneous Localization And Mapping) and SfM (Structure-from-Motion) aim to jointly infer a map and localize an agent within it. SLAM systems often use multiple sensors including RGB [62], depth [63] or inertial sensor [67, 92]. They can be categorized as direct [31, 30, 106] or indirect methods [23, 62, 63, 12, 53]. Direct methods utilize pixel intensities directly and estimate 3D geometry by minimizing the photometric error; indirect methods extract intermediate representations like point features [62], then optimize camera poses and 3D point clouds via reprojection error. COLMAP [74] is a widely-adopted, general-purpose SfM pipeline, often the default method in higher-level tasks [89, 43, 57, 56]. Deep learning has recently been integrated into SLAM system, producing reliable and view-invariant features [82, 71, 39] and direct pairwise pose predictions in visual odometry [94, 99, 55, 54, 40]. The end-to-end SLAM system, DROID-SLAM [87] proposes an iterative update operator for camera poses and dense depth, using a Dense Bundle Adjustment layer. ParticleSfM [104] leverages deep learning to classify static vs dynamic point trajectories and combines with classical SfM [84]. We evaluate a representative set of these methods in what follows. Visual Re-localization [13, 78, 73] aims to recover the 6DoF camera poses of query RGB images from a known environment represented by feature embeddings [51, 3], 3D point cloud [8, 58], scene coordinates [97, 105, 9] or Neural Radiance Field (NeRF) [19, 18]. However, these relocalization methods require pose ground truth to train for unknown scenario [19, 51] and mostly focus on single-image pose estimation [51, 105], thus are not the best evaluation candidates for EgoStatic benchmark.
2.2 Datasets and Benchmarks
Most existing camera trajectory benchmarks [36, 21, 11] require accurate ground truth camera trajectories to compare with. This often relies on additional sensors other than RGB cameras: rotating 3D laser scanner and combined GPS/IMU inertial navigation system (KITTI [36]), RGB-D and IMU data (ScanNet [21]), LiDAR depth/points, IMU and radio data (LaMAR [72]). Bulky hardware can limit such datasets' domains to driving [36, 20], drones [11, 81], indoor and outdoor scans [72, 21] and synthetic data [96]. Other works [27, 80] rely on SfM to extract pose on crowd-sourced images of the same scene. We compare these benchmarks with our proposed Egostatic in Table 1: the diversity and scale of EgoStatic serve as a strong complement to existing benchmarks.
Table 1: EgoStatic provides the largest-scale evaluation, with diverse scenes and complex actions.
Seq.	Total Size	Scenario
KITTI [36]	22	13k frames	Driving
EuRoC MAV [11]	11	23 minutes	Indoor drone
TUM RGB-D [81]	15	27 minutes	Indoor drone
DISCOMAN [52]	200	5k frames	Indoor
TartanAir [96]	1037	1M frames	Synthetic
LaMAR [72]	300	100 hours	Walking in 3 sites
ScanNet [21]	1500	2.5M frames	Indoor room scan
Ours (EgoStatic)	5708	600 hours / 9M frames	137 Everyday Life scenarios
2.3 SLAM Evaluation Metrics
Most SLAM [62] or SfM [33] systems produce per-frame camera pose. Various metrics have been proposed: Absolute Trajectory Error (ATE) [103] performs a global positional alignment [91] between estimated and ground truth poses, then calculates RMSE (root mean square error). Relative Pose Error (RPE) [103] computes local errors across sub-trajectories in a temporal sliding window. Recently, alternative SLAM metrics have also been proposed. ODE(Overlap Displacement Error) [61] moves away from pose error, instead evaluating map inconsistency introduced by the pose errors. Bodin [7] evaluates reconstruction error after ICP alignment. Our proposed metric ORE is related to the classic metric reprojection error [46], but we focus specifically on object-centric reprojection.
3 EgoStatic Benchmark
In this section, we introduce EgoStatic, a benchmark for evaluating camera trajectory relative to objects. We first introduce an evaluation pipeline leveraging static objects' tracklets to compute Object Reprojection Error (ORE). This pipeline does not depend on groundtruth camera poses or sensors beyond RGB camera. The pipeline can be applied to any video, even after collection, enabling evaluation on existing large-scale egocentric data. We next provide geometric intuitions and key factors in dataset construction; these tighten the connection between ORE and classic metrics. Empirical studies (sec 4) validate the correlation. Finally, we detail the additional annotations and adopt this pipeline to Ego4D-EgoTracks video. This results in EgoStatic, with 5708 video sequences, over 9 million frames and 22k static object tracklets. To our knowledge, this is by far the largest real-world evaluation benchmark of its kind.
3.1 Existing camera trajectory metrics
We revisit two frequently used metrics: Absolute Trajectory Error (ATE), and Relative Pose Error (RPE) [103]. Their goal is to measure the quality of a series of  $N$  estimated camera poses  $\hat{\mathbf{X}} = \{\hat{\mathbf{R}}_i, \hat{\mathbf{t}}_i\}_{i=0}^{N-1}$  (rotation  $\hat{R}_i \in SO(3)$ , translation  $\hat{t}_i \in \mathbb{R}^3$ ) vs. ground truth  $\mathbf{X}^{gt} = \{\mathbf{R}_i^{gt}, \mathbf{t}_i^{gt}\}_{i=0}^{N-1}$ .
ATE is a global measure of squared error between estimated trajectory  $\hat{\mathbf{X}}$  and ground truth  $\mathbf{X}^{gt}$ . Due to coordinate system ambiguity[91], an optimal alignment  $g^{*}(\cdot) = \{s,R,t\}$  is first applied:
$$
g ^ {*} (\cdot) = \underset {g = \{s, R, t \}} {\arg \min } \sum_ {i = 0} ^ {N - 1} \left| \left| \mathbf {t} _ {i} ^ {g t} - s \mathbf {R} \hat {\mathbf {t}} _ {i} - t \right| \right| \tag {1}
$$
ATE for rotation and translation can be calculated as the root mean square error (RMSE):
$$
A T E _ {t r a n s} = \left(\frac {1}{n} \sum_ {i = 0} ^ {N - 1} \left\| \mathbf {t} _ {i} - g ^ {*} \left(\hat {\mathbf {t}} _ {\mathbf {i}}\right) \right\| ^ {2}\right) ^ {\frac {1}{2}}; A T E _ {r o t} = \left(\frac {1}{n} \sum_ {i = 0} ^ {N - 1} \left\| \angle \left(\mathbf {R} _ {i} \left(g ^ {*} \left(\hat {\mathbf {R}} _ {\mathbf {i}}\right)\right)\right) ^ {T} \right\| ^ {2}\right) ^ {\frac {1}{2}} \tag {2}
$$
where  $\angle (\cdot)$  means the angle difference between two rotation matrices.
RPE is a local consistency metric calculated by the trajectory over a fixed interval  $\Delta$ . Given a series of ground truth camera poses  $P_{i} \in SE(3)$  and estimated poses  $Q_{i} \in SE(3)$ , both within the local window, the relative pose error matrix can be defined as:
$$
E _ {i} ^ {\Delta} = \left(Q _ {i} ^ {- 1} Q _ {i + \Delta} ^ {- 1}\right) \left(P _ {i} ^ {- 1} P _ {i + \Delta}\right) \tag {3}
$$
With  $E_{i}^{\Delta} = [R_{i}^{\Delta}|t_{i}^{\Delta}]$ , the rotation and translation component of RPE can be computed as:
$$
R P E _ {r o t} = \left(\frac {1}{m} \sum_ {i = 0} ^ {m - 1} \arccos  \left(\frac {t r \left(R _ {i} ^ {\Delta}\right) - 1}{2}\right)\right); R P E _ {t r a n s} = \left(\frac {1}{m} \sum_ {i = 0} ^ {m - 1} | | t _ {i} ^ {\Delta} | | ^ {2}\right) ^ {\frac {1}{2}} \tag {4}
$$
3.2 Evaluation methodology
Given  $N$  estimated camera poses  $\hat{X}$  and  $M$  ground truth static object tracklets (e.g. bounding boxes)  $\{B_i\}_{i=0}^{M-1}$ , we aim to quantify the quality of  $\hat{X}$ , without ground truth pose  $X^{gt}$ . We summarize our entire procedure in Algorithm 1 and Figure 2. The pipeline contains two steps (Fig. 2): (a) Lift Object to 3D and (b) Re-Projecting Object down to 2D.
(a) Lift Object to 3D For each tracklet  $B_{i}$ , we select an arbitrary occurrence of object  $i$  at frame  $j$ , resulting a segmentation mask or a bounding box  $b_{i,j}$ . We now need to identify a point on the object. If masks are available we pick a central point in the mask (ScanNet), otherwise we use point  $\hat{p}$  at the box center (EgoTracks). If the depth  $\hat{d}$  of the point in this frame were known, then given camera intrinsics  $K$ , we can unproject  $\hat{p}$  to its world location
$$
\hat {p} _ {3 d} = [ R, T ] ^ {- 1} K ^ {- 1} \hat {p}, \text {w h e r e} \hat {p} = (x _ {0}, y _ {0}, 1, 1 / \hat {d})
$$
(b) Project to 2D With 3D location  $\hat{p}_{3d}$  of a point on the object, we can project to 2D for other frames to generate a point tracklet  $\{\hat{p}_k\}_{k = 0}^{N - 1}$  by applying the camera projection formula on estimated
Figure 2: Point Tracklet Generation Point tracklets are obtained by lifting and reprojecting a sampled object point  $\hat{p}$ , given camera trajectory  $\hat{X} = \{\hat{R}_i, \hat{t}_i\}_{i=0}^{N-1}$ , and point depth  $\hat{d}$
camara poses  $\hat{X}_k = \{\hat{R}_k, \hat{t}_k\}$ ,  $\hat{p}_k = K\hat{X}_k\hat{p}_{3d}$ . Since the object is static, the reprojected point will fall inside the bounding box or the segmentation mask if the estimated pose is good. We define Object Reprojection Error (ORE) as the distance between the reprojected point location  $\{\hat{p}_k\}_{k=0}^{N-1}$  and the annotated object 2D location  $\{b_{i,k}\}_{k=0}^{N-1}$ .
$$
\mathbf {O R E} \left(\hat {p} _ {k}, b _ {i, k}\right) = \left\{ \begin{array}{l l} 0, & \text {i f} \hat {p} _ {k} \in b _ {i, k} \\ | | \hat {p} _ {k} - b _ {i, k} | |, & \text {i f} \hat {p} _ {k} \notin b _ {i, k} \end{array} \right. \tag {5}
$$
We use  $l1$ -norm and normalize by the image size. ORE is averaged for all tracklets in a sequence.
Depth Optimization. In step (a), we assumed depth is known at frame  $j$  for the selected point. In practice, ground truth depth is not generally available, and any attempt to infer it could introduce model bias or systematic error, skewing results. Instead, we propose an optimization: we select the depth value  $d^{*}$  that minimizes ORE for each camera trajectory. This is similar to the global alignment in the ATE metric which minimizes RMSE, yet with fewer degrees of freedom to optimize. We tried multiple optimizers [93, 29], yet found simple grid-search sufficient to find a good value.
Algorithm 1 Calculate Object Re-Projection (ORE) Error
function ORE-D(\{B_i\}_{i=0}^{M-1},  $\hat{X} = \{\hat{R}_i, \hat{t}_i\}_{i=0}^{N-1}$ , d)  
for tracklet  $B_i \in \{B_i\}_{i=0}^{M-1}$  do  
Sample point  $\hat{p}_j = (x_0, y_0)$  from  $b_{i,j} \in B_i$  in arbitrary frame  $j$ $\hat{p}_{3d} = \hat{X}_j^{-1}K^{-1}(x_0, y_0, 1, 1/d)^T$   
ORE = {}  
for frame  $k$  in video do  
 $\hat{p}_k = K\hat{X}_k\hat{p}_{3d}$   
ORE append ORE( $\hat{p}_i, b_{i,k}$ )  
end for  
end for  
return  $\frac{1}{M}\sum_{i}\frac{1}{K}\sum_{k}\mathbf{ORE}(\hat{p}_i, b_{i,k})$   
end function  
function ORE(\{B_i\}_{i=0}^{M-1},  $\hat{X} = \{\hat{R}_i, \hat{t}_i\}_{i=0}^{N-1}$ $\mathbf{ORE} = \underset{d}{\mathrm{minimize}}\mathbf{ORE - D}(\{B_i\}_{i=0}^{M-1}, \hat{X} = \{\hat{R}_i, \hat{t}_i\}_{i=0}^{N-1}, d)$   
end function
3.3 Constructing a benchmark
When constructing an ORE benchmark, the key consideration is the choice of tracklets and camera trajectories. Here we discuss criteria for selecting good videos.
What configurations of objects will be most effective at capturing errors in camera trajectory? Ideally, a good configuration would be maximally sensitive to errors, i.e. it will reliably produce signal which grows with error magnitude. One way to identify such configurations would be, for example, to compute a probability distribution on possible errors (e.g. a distribution on SE(3) centered about the identity), and identify configurations of points which maximize the expected reprojection error under this distribution. However, we are ultimately limited by the tracklets we have available, so this feels like overkill; relatively simple intuition is sufficient to guide our choice of videos.
First, observe that a configuration of points which lies on a single camera ray carries no information about either translation error in the ray's direction or roll error about the ray; such degenerate configurations should be avoided. Further, we see that sensitivity to roll error about the optical axis, as well as pitch/yaw error, both increase as points move away from the optical center. Pitch/yaw error sensitivity is maximized when points are aligned with the rotational vector field. Putting this all together suggests a set of objects, well-distributed in angle and located relatively far from the optical center, will be sensitive to the widest possible set of rotations and will maximize overall response to error. Choosing objects with smaller bounding boxes will further reduce the nullspace of ORE and should also improve sensitivity. In what follows we employ heuristics to choose appropriate videos and study these factors empirically.
3.4 EgoStatic benchmark
To evaluate camera trajectories in real-world environment with complex activities, we source videos from Ego4D dataset [43] and object tracking annotations from EgoTracks dataset [86]. Since EgoTracks covers both active and static objects, we annotate additional attribute labels on each object occurrence of EgoTracks, identifying the static object tracklets. This results in around 22,000 static object tracklets from 5708 6-minutes video sequences ( $\sim$ 600 hours).
Here, we benchmark a representative set of methods using ORE. We divide our experiments into two sets. First, on ScanNet (which contains GT camera pose) we (i) show ORE is strongly correlated to well-established metrics and (ii) examine our design choices, motivated by geometric intuitions in section 3.3. Next, we turn to our proposed EgoStatic benchmark. We show that EgoStatic is a challenging benchmark, and second that ORE is a strong tool: it is sensitive enough to enable hyper-parameter selection and model design choices.
4.1 Baseline methods and implementations
We select 7 methods from SLAM, VO and SfM as baselines. Unless otherwise specified, we use the default setup from the original work. For SLAM, we use both geometry-based approaches such as ORB-SLAM2 [63] and ORB-SLAM3 [12] and SOTA learning-based approach DROID-SLAM [87]. For visual odometry, we include self-supervised MonoDepth $^{2}$  [41] and synthetic-data-supervised TartanVO [95]. For SfM, we include both geometry-based approach COLMAP [74] and hybrid method ParticleSfM [104] that combines learned correspondence and geometry mapping.
Tuning COLMAP. As observed in prior works [104, 89, 57], default COLMAP may fail entirely on ScanNet or egocentric video, or converge slowly with poor quality output. Following [89], we replace the exhaustive matching procedure with vocabulary tree matching procedure.
4.2 Experiments on ScanNet
ScanNet [21] is an RGB-D video dataset containing more than 1500 3D scans, each with 3D camera pose annotations and point-clouds. We pick high confidence instance mask from Mask-RCNN [47], and construct object tracklets by back-projecting masks to the point cloud. We benchmark methods on 20 test sequences on ATE, RPE and ORE (Tab. 2; see suppl. for per-scene results).
<sup>1</sup>No part of Meta's research involved any use of MonoDepth2; The data pertaining to MonoDepth2 is included solely for comparison purposes.
Table 2: ORE ranks methods consistently with  $\mathrm{ATE}_{trans}$  and  $\mathrm{ATE}_{rot}$ . Average ORE,  $\mathrm{ATE}_{trans}$  and  $\mathrm{ATE}_{rot}$  performance of 7 methods on ScanNet dataset,  $\mathrm{ATE}_{trans}$ ,  $\mathrm{ATE}_{rot}$ , and ORE are measured in meters, radians and normalized unit distance between 0 and 1 respectively.
DROID[87]	COLMAP[33]	ORB2 [63]	ORB3 [12]	Particle [104]	TartanVO [95]	Monodepth2 [41]
ORE	0.029	0.035	0.102	0.109	0.130	0.251	0.294
ATEtrans (m)	0.147	0.331	0.462	0.533	0.484	0.451	0.790
ATErot (rad)	0.135	0.448	0.896	1.065	0.629	2.096	1.861
Table 3: ORE has strong rank correlation with  $\mathbf{ATE}_{trans}$  and  $\mathbf{ATE}_{rot}$  on ScanNet. Spearman Coef. (upper triangle) and Kendall Coef. (lower triangle) between ORE and standard metrics.
Kendall\Spearman	ORE	ATEtrans	ATErot
ORE	-	0.716	0.800
ATEtrans	0.579	-	0.780
ATErot	0.650	0.681	-
Ranking statistics. To verify the effectiveness of ORE, we compare rank statistics with standard metrics  $\mathrm{ATE}_{rot}$  and  $\mathrm{ATE}_{trans}$ . We use two standard rank statistics: Spearman's rank correlation coefficient (Spearman Coef.) [100] and Kendall's  $\tau$  coefficient (Kendall Coef.) [1]. Spearman measures Pearson correlation between rankings of two variables; values above 0.6 indicate strong correlation and 0.8 is considered very strong. Kendall measures ordinal association in a pairwise manner; values above 0.35 indicate strong association. We refer to supplementary for details.
4.2.1 ORE strongly correlates to other metrics
We compute ranking statistics between ORE,  $\mathrm{ATE}_{trans}$  and  $\mathrm{ATE}_{rot}$  (3) and averaged across scenes. ORE has strong correlation with both  $\mathrm{ATE}_{trans}$  and  $\mathrm{ATE}_{rot}$ . Hence ORE leads to conclusions consistent with metrics computed from groundtruth. It is worth noting that the correlation is even stronger than correlation between standard metrics themselves (i.e.  $\mathrm{ATE}_{trans}$  vs.  $\mathrm{ATE}_{rot}$ ). Even  $\mathrm{ATE}_{trans}$  and  $\mathrm{ATE}_{rot}$  may rank methods differently, for example TartanVO has low translation error but the highest rotation error. In addition, we observe ORE correlates better with ATE, since both are considering the entire camera trajectory, not a local window as in RPE (see suppl.). Between translation and rotation, ORE correlates stronger with rotation.
Remark: Since ORE is a single method, we remark that rank correlations for ORE are in some sense upper bounded by the inconsistency between  $\mathrm{ATE}_{trans}$  and  $\mathrm{ATE}_{rot}$ : we can agree with only one of the two if these disagree. By removing inconsistent pairs (16% of all pairs), Kendell Coef. between ORE and ATE improves to 0.895 (see suppl.).